PaperlayerView exact revision on arXiv

Source-linked AI summary

IFC-SSL-Bench: Benchmarking Self-Supervised Learning for Annotation-Efficient Multispectral Imaging Flow Cytometry

Angelo Jovin Yamachui Sitcheu, Roxana-Teodora Fetescu, Thomas Lautenschlager, Nils Friederich, Linus Schilpp, Stephan Allgeier, Luca Deininger, Daniel Hübschmann, Ralf Mikut

arXiv:2610.31848v1cs.CV

TL;DR

IFC SSL research has focused on isolated methods, leaving comparative evaluation incomplete. IFC-SSL-Bench compares six methods across four backbones and two datasets, reporting low-annotation gains in synapse and morphology evaluations.

Abstract

from arXiv · show

Imaging Flow Cytometry (IFC) generates large multispectral datasets, yet the need for expert annotations bottlenecks the development of supervised deep learning models. While Self-Supervised Learning (SSL) offers a solution, its application to IFC remains fragmented and lacks systematic benchmarking. To address this, we present IFC-SSL-Bench, a modular framework for evaluating SSL in IFC. We benchmark six SSL approaches across four backbones on two heterogeneous datasets. By assessing downstream classification performance across annotation regimes of the training sets (1%-100%), we quantify annotation efficiency and identify optimal configurations for high-dimensional biological imaging. SSL matches the performance of fully supervised baselines trained on 100% of annotated data while using as little as 5% of annotations for morphological classification, with advantages in low-data regimes below 5%. Across both datasets, the contrastive approach MoCo-v2 paired with ConvNeXt backbones emerged as the top-performing configuration, with ConvNeXtV1-T best for immunological synapse classification and ConvNeXtV2-T for morphological classification. We identify a clear pattern in transfer dynamics: ImageNet initialization benefits morphological classification, consistent with its visual similarity to natural images, whereas random initialization better captures the more domain-specific immunological synapse classification. SSL elevates class-specific F1-scores by over 35 percentage points for structurally complex classes at 1% of annotated data. We synthesize these insights into a unified set of guidelines for deploying self-supervised workflows in high-throughput IFC, and release the benchmark as a resource for future work.

Introduction

IFC produces rich multispectral cell images, but technical variation and costly expert annotations complicate analysis. The introduction motivates a systematic benchmark of SSL methods and architectures, with the goal of guiding annotation-efficient IFC workflows.

Introduction

  • IFC combines high-throughput flow cytometry with microscopy’s spatial and morphological detail, but acquisition trade-offs and technical variation complicate analysis.The introduction also notes that supervised deep-learning models require large annotated datasets, which are costly and expertise-intensive to obtain.Source: Introduction · p. 2
  • Prior IFC studies evaluated isolated SSL methods, leaving their relative effectiveness unexplored and motivating questions about method, architecture, and annotation efficiency.The introduction frames these as questions about learning discriminative embeddings and reducing downstream annotation needs.Source: Introduction · p. 3
  • IFC-SSL-Bench is presented as a standardized, modular, reproducible, and extensible pipeline unifying data processing, pretraining, and downstream evaluation.The framework is intended to support fair comparisons across diverse SSL methods for IFC.Source: Introduction · p. 3
  • The benchmark evaluates six state-of-the-art SSL methods across four backbone architectures and two distinct SSL datasets.The authors describe this as the first structured comparison of self-supervised representation learning efficacy in IFC.Source: Introduction · p. 3
  • The work quantifies performance across annotated-data fractions and offers IFC-tailored recommendations on models, SSL methods, and initialization strategies.The authors also release the codebase under an open license to support reproducibility and future research.Source: Introduction · p. 3

Results

The Results section evaluates self-supervised learning across RBC morphology and ISF–5CH classification, including performance with limited annotations and on difficult cellular structures. It also examines how latent-space organization aligns with downstream performance and how initialization benefits vary by task.

The Benchmarking Pipeline

  • The benchmark decoupled SSL objective, backbone, and initialization effects across two IFC datasets, six SSL methods, four architectures, and annotated-data fractions from 1% to 100%.It paired SSL pretraining on unannotated images with supervised baselines and evaluated downstream representations using kNN, linear probing, XGBoost, and selected full finetuning.Source: Results · p. 4, The Benchmarking Pipeline · pp. 5–6

Supervised Baseline Performance

  • Across both datasets, supervised macro F1-scores generally rose with more annotated training data, while the θIN–θRnd performance gap generally narrowed, with architecture-dependent exceptions.On RBC, θIN usually improved performance, except ConvNeXtV2–T at 1%; on ISF–5CH, initialization effects varied by backbone, and Logicle preprocessing slightly decreased F1-score.Source: Supervised Baseline Performance · pp. 6–7

Self-Supervised Learning Resolves Complex Immunological Structures in Sparse Annotation Regimes

  • With 25% of Dann (a little less than 1,000 samples), ConvNeXtV1–T–θRnd + MoCo-v2 finetuning reached 90.8 ± 0.1% F1-score versus 89.3 ± 1.0% for the supervised baseline, and 93.0 ± 0.3% on full Dann.At 1% and 5% of Dann, the largest class-specific gains were 38.0 pp and 36.5 pp, respectively.Source: Self-Supervised Learning Resolves… · pp. 9–10
  • At 1% and 5% of Dann, the largest class-specific gains were 38.0 and 36.5 pp for B cell and T cell in one layer; at 10% and 25%, gains were 22.3 pp for T cell with signaling and 13.5 pp for Synapses without signaling, respectively.The authors suggest that self-supervised pretraining learns generalized spatial boundaries, cell-to-cell adhesion geometries, and relative morphological proportions from unannotated data.Source: Self-Supervised Learning Resolves… · p. 10

ImageNet Pretraining Enhances Performance for Morphological Cell Shape Classification

  • On the RBC dataset, θIN initialization yielded median full-Dann ΔF1 gains of 24.8 pp with kNN, 22.5 pp with linear probing, and 22.5 pp with XGBoost; top ConvNeXtV2–T–θIN with MoCo-v2 was 1.2 pp below the best supervised baseline.At 0.1% of Dann, ConvNeXtV2–T + BT with θIN followed by linear probing improved the best supervised baseline by 11.2 pp.Source: ImageNet Pretraining Enhances… · p. 10

Annotation-Efficient Finetuning: Matching Baselines with Only 5% annotated Data

  • At 5% of Dann, ConvNeXtV2–T–θIN + MoCo-v2 reached 92.3% ± 1.0 macro F1-score, equaling the 92.2 ± 1.1% full-Dann supervised baseline.At 0.1% and 1% annotated fractions, the corresponding top-performing SSL combinations improved test macro F1-score over supervised baselines by 45.7 pp and 45.9 pp, respectively.Source: Figure 5 · p. 11, Annotation-Efficient Finetuning:… · pp. 11–12

Qualitatively Mapping the Self-Supervised Latent Space

  • On ISF–5CH, class-grouped embeddings had ARI 0.078 and NMI 0.182, versus ARI 0.811 and NMI 0.792 when grouped by experiment, indicating strong batch effects alongside within-batch class organization.The analysis used k-means clustering and compared annotation-based cluster scores with experiment-based scores; low-dimensional projections were treated as qualitative rather than quantitative evidence.Source: Qualitatively Mapping the… · pp. 12–13, Table 4 · p. 14
  • ConvNeXtV1–T–θRnd + MoCo-v2 averaged ARI = 0.159 and NMI = 0.334, surpassing annotated-class grouping and the random control.Its latent space also showed higher ARI and NMI within individual experiments.Source: Qualitatively Mapping the… · p. 14
  • ViT–S/16–θIN + DINO better mitigated persistent batch variation and showed better annotated-class clustering than ConvNeXtV1–T–θRnd + MoCo-v2.The authors suggest this may reflect use of subtle visual features beyond fluorescent signals to isolate B cells despite surface-marker ambiguity.Source: Qualitatively Mapping the… · p. 14
  • ImageNet-pretrained ConvNeXtV2–T–θIN + MoCo-v2 disentangled crenated spheres from crenated spheroids, while undecidable cells formed localized pockets within defined phenotypic clusters.It also merged separate clusters of smooth discs, crenated discoids, and side cells seen with random initialization. These mappings may help identify morphological similarities among uncertain cells and support annotator consensus.Source: Qualitatively Mapping the… · p. 14
  • Across the mappings, target-domain self-supervised pretraining refined specialized features, whereas natural-image priors improved fine-grained morphological boundaries.The overlapping B-cell and T-cell class was especially complex, and its structure correlated with substantial finetuning improvements.Source: Qualitatively Mapping the… · p. 14

Discussion and Guidelines

Across two IFC datasets, the benchmark finds that SSL can substantially reduce annotation needs, while the best initialization depends on task complexity. It offers practical guidance on model choice and preprocessing, while identifying limits on cross-study transfer and the breadth of the evidence.

Weights Initialization should be tailored to Task Complexity

  • ImageNet initialization benefited fine-grained RBC morphology, whereas random initialization better resolved complex ISF–5CH phenotypes across annotation-scarcity levels.The authors frame initialization as domain-dependent rather than universally beneficial.Source: Weights Initialization should be… · p. 15

A Strong Recipe for Learning Linearly Separable Features: MoCo-v2 + ConvNeXt

  • MoCo-v2 with ConvNeXt was the strongest linear-probing configuration: ConvNeXtV2–T with ImageNet weights on RBC and ConvNeXtV1–T from scratch on ISF–5CH.ResNet50 performance varied by SSL objective, while ViT–S/16 paired with DINO was competitive across both datasets.Source: A Strong Recipe for Learning Linearly… · p. 15

Finetuning enables Annotation Efficiency

  • With roughly 1,000 annotated ISF–5CH samples, finetuning exceeded fully supervised baselines; on RBC, SSL matched the 100% baseline using 5% of annotations.The authors report this annotation efficiency across datasets differing in cell type, marker panel, class definition, and size, but did not evaluate transfer between independent studies.Source: Finetuning enables Annotation Efficiency · p. 15
  • Cross-study encoder transfer was not evaluated, and differing channel identities, counts, and intensity scaling mean encoders cannot be applied directly without channel adaptation.Released backbones are intended for studies with comparable channel configurations.Source: Finetuning enables Annotation Efficiency · p. 16

Practical Implementation and Normalization Pitfalls

  • Qualitative latent-space assessment revealed experiment-level clustering in ISF representations, although class-specific clusters remained visible within experiments.The authors advise against Logicle scaling in these visual deep-learning pipelines, reporting no downstream improvement over z-score normalization.Source: Practical Implementation and… · p. 16

Limitations and Future Work

  • The benchmark covers discriminative SSL and only two datasets; additional independent cohorts are needed to assess how broadly its conclusions apply to IFC.The study also did not evaluate explicit batch correction, channel contributions, or whether z-score normalization is optimal; three seeds limit separation of closely ranked methods.Source: Limitations and Future Work · p. 16

Methods

The methods establish a modular benchmark that separates unannotated-data pretraining from annotated downstream evaluation. It compares SSL objectives, backbones, initializations, and annotation levels across two IFC datasets using standardized preprocessing and evaluation protocols.

Data Preparation

  • IFC-SSL-Bench partitions multichannel images into unannotated pretraining and annotated evaluation sets, with channel selection, z-score normalization, and resizing to 224 × 224.The framework supports domain-specific transformations, but these were excluded from the primary benchmark.Source: Data Preparation · p. 17, Methods · p. 17
  • The benchmark uses RBC images in three channels and nine selected ISF classes, evaluating nested, stratified annotated subsets from 1% to 100%.RBC includes 70,331 annotated samples across seven morphological categories and an undecidable category; ISF uses a filtered 5,848-image subset from nine biologically relevant classes. Training, validation, and test splits for ISF are 64:16:20.Source: Data Preparation · pp. 17–18

Self-Supervised Pretraining

  • Pretraining crosses four backbones and two initialization regimes with six invariance-based SSL methods, using standardized augmentations to compare architectures and objectives.The backbones are ResNet50, ViT–S/16, ConvNeXtV1–T, and ConvNeXtV2–T; their parameter budgets are kept comparable. ImageNet first-layer weights are adapted to multispectral inputs by averaging RGB filters and replicating the mean across target channels.Source: Backbones · p. 18, Self-Supervised Learning Methods · pp. 18–19, Pretraining Data Augmentation · p. 19

Downstream Analysis

  • Downstream evaluation uses frozen-feature kNN and linear probing, XGBoost, and end-to-end finetuning of top-performing configurations.Performance is measured with macro F1-score, ARI, and NMI; XGBoost serves as a computationally efficient proxy for nonlinear information in the representations.Source: Downstream Analysis · p. 19

Training and Implementation Details

  • Experiments report means and standard deviations across three runs, using 200 pretraining epochs for RBC and 10 for the larger ISF–5CH dataset.Code and configurations are publicly available; the complete set of checkpoints is not deposited due to its size, though individual checkpoints are available upon request.Source: Training and Implementation Details · pp. 19–21

Framework Implementation Validation and Supervised Trainings

The supplied material describes supervised-baseline evaluations across initialization and data regimes, plus an ablation of Logicle transformation. It also identifies the BloodMNIST baseline table and its reporting conventions.

BloodMNIST validation

  • Table 6 reports mean ± standard deviation accuracies for supervised end-to-end baselines on BloodMNIST, with ImageNet and random initialization distinguished.The caption says the best-performing baseline for each dataset is bolded and that accuracy is reported to match the MedMNIST convention.Source: Table 6 · p. 25

Supervised training

  • Table 7 provides hyperparameters for supervised training and notes that batch size is reduced to eight for training sets below 128 samples.The dash denotes unavailable values.Source: Table 7 · p. 25

Figure S1

  • Figure S1 compares supervised-baseline macro F1 across initialization and annotated fractions, including RBC at 0.1% annotation and Logicle scaling on ISF–5CH.The stated ISF–5CH reference is ConvNeXtV1–T with random initialization, trained end-to-end on 100% of Dann.Source: Figure S1 · p. 26

Self-Supervised Learning

The benchmark evaluates six discriminative SSL methods using standardized configurations and compares their performance across backbones, initialization strategies, and annotation fractions. Supplementary figures present results across datasets and evaluation protocols.

Methods

  • The benchmark compares SimCLR, MoCo-v2, MoCo-v3, BYOL, Barlow Twins, and DINO, spanning contrastive, asymmetric teacher–student, and redundancy-reduction approaches.Implementations retain method-specific components while centralizing hyperparameter configuration for systematic comparisons.Source: Methods · p. 27
  • SimCLR maximizes agreement between augmented views; MoCo-v2 uses a momentum encoder and negative-sample queue, while MoCo-v3 removes the queue and adds an online prediction head.MoCo-v2’s queue decouples negative-sample count from batch size; both MoCo variants use EMA updates.Source: Methods · p. 27
  • DINO uses multi-crop student–teacher self-distillation, BYOL matches online predictions to EMA target representations, and Barlow Twins penalizes cross-correlations between embedding dimensions.DINO’s teacher processes global views; BYOL does not require negative pairs, while Barlow Twins pushes the cross-correlation matrix toward the identity.Source: Methods · p. 27
  • Augmentations and most training hyperparameters are standardized, while optimizer choice follows backbone normalization and reference recipes; some method settings are harmonized for compute tractability.Color jittering and solarization are excluded to preserve biologically relevant intensity information; settings include harmonized projection-head dimensions, DINO prototype dimensions, batch size, and pretraining epochs.Source: Methods · pp. 27–28
  • Supplementary tables report self-supervised augmentation parameters and pretraining hyperparameters; ISF–5CH pretraining occurred for only 10 epochs, while method-specific settings were harmonized across benchmark methods rather than reproducing each original published configuration.Source: Table 8 · p. 28, Table 9 · p. 28, Methods · p. 28

Supplementary evaluation results

  • Supplementary figures compare macro F1 across backbones, SSL methods, initialization modes, and data fractions, with separate panels for kNN, linear probing, and XGBoost.The figures cover both ISF–5CH and RBC; cell halves distinguish random from ImageNet initialization, underlining marks the leading SSL method within a backbone, and bold marks the global optimum per fraction and initialization.Source: Figure S2 · p. 29, Figure S3 · p. 30, Figure S4 · p. 31

Related papers

Loading 2610.31848v1…