PaperlayerView exact revision on arXiv

Source-linked AI summary

Model-agnostic Knowledge Augmented Machine Learning for Weakly Annotated Bioimaging Data

Jayanie Bogahawatte, Sachith Seneviratne, Simon Maksour, Dzung Do-Ha, Elizabeth Qian, Samuel Brennan, Matthew Keon, Lezanne Ooi, Saman Halgamuge

arXiv:2610.13637v1eess.IVcs.AIcs.CV

TL;DR

Weakly annotated bioimaging methods rely on learned features, while integrating biological prior knowledge remains challenging. This paper introduces model-agnostic knowledge augmentation and reports consistent performance gains across microscopy and cytomorphology datasets.

  • Problem

    Weakly supervised bioimaging methods rely on learned features and often emphasize global patterns over single-cell localization, while integrating biological prior knowledge remains challenging.

    Source: Introduction · pp. 3–4

  • Method

    The model-agnostic framework adaptively combines biologically derived descriptors with learned visual representations for annotation-efficient classification.

    Source: Introduction · p. 4, Discussion · p. 27

  • Results

    Across three independent ALS microscopy and three public cytomorphology datasets, the method consistently improved diverse ML models; learnable weighting improved average precision by 2.3% versus standard concatenation.

    Source: Introduction · p. 4, classifying SNIs based on TDP-43… · p. 13, Discussion · p. 27

  • Takeaways & Limitations

    The approach enables annotation-efficient classification of weakly labelled bioimaging data for analyzing TDP-43 mislocalization in ALS motor-neuron images.

    Source: Discussion · p. 27, Dataset 3 (D3) · p. 33

    The authors identify richer domain-informed handcrafted features as a future avenue that may improve classification accuracy.

    Source: Discussion · p. 28

Abstract

from arXiv · show

Machine learning models are increasingly used for automated feature extraction from weakly annotated bioimaging data, such as microscopy and cytomorphology images. Due to the sole dependency of automated feature extraction, existing methods yield inconsistent classification performance across weakly annotated datasets. Here, we propose a model-agnostic knowledge augmentation method that enables ML models to exploit domain knowledge beyond what automated feature extraction can capture. The proposed approach improves classification performance across seven ML architectures. When integrated with masked autoencoder-based self-supervised learning, our method achieves the strongest overall performance, improving precision by 2.9% over the next-best method on three in-house microscopy datasets and by an average of 6.8% across three public cytomorphology benchmarks. We perform nucleus-level classification based on mislocalization of TAR DNA-binding protein 43 (TDP-43) from the nucleus to the cytoplasm, a phenotype linked to Amyotrophic Lateral Sclerosis (ALS), using three independent microscopy datasets of motor neurons derived from human induced pluripotent stem cells. Accurate quantification of this phenotype at scale remains challenging and may contribute to inconsistencies across studies. Application of the best-performing model to these microscopy datasets reveals biological interpretations related to ALS, including dose-dependent effects of the proteasomal stressor, MG132, and the oxidative stressor, sodium arsenite (SA), on TDP-43 mislocalization, variable pathology in motor neurons from sporadic and familial ALS donors, and a lack of protection of the pharmacological compounds, edaravone and niclosamide, against SA-induced TDP-43 mislocalization. Together, our pipeline enables in-vitro cellular image analysis to advance understanding of ALS-associated pathology and potential treatments.

Introduction

The paper proposes a model-agnostic method that combines biological prior knowledge with learned visual representations for weakly annotated bioimaging, and reports that its MAE-based combination performs best across six datasets.

Introduction

  • Weakly supervised image-level labels may miss cell-level TDP-43 mislocalization cues, motivating integration of biological descriptors with learned features.The challenge includes variation in mislocalization across many cells within images, while nucleus-level annotation requires substantial biological expertise.Source: Introduction · pp. 3–4
  • The proposed model-agnostic method adaptively weights biologically derived descriptors and learned visual representations for weakly annotated bioimaging.The pipeline first generates single-nucleus images, then learns representations from unlabelled images using a Masked Autoencoder, and finally integrates those representations with biological descriptors.Source: Introduction · p. 4
  • The MAE-based knowledge augmentation method consistently achieves the best performance across six datasets, with evaluation spanning seven ML architectures.The evaluation covers three independent in-house microscopy datasets and three public cytomorphology datasets.Source: Introduction · pp. 4–5

Results

  • Three independent microscopy datasets support analyses of cellular stressors, ALS subtype classification, and drug discovery across varied experimental settings.The images show nuclei and TDP-43 in two channels and were acquired using different imaging techniques and experimental settings.Source: Introduction · p. 5, Results · p. 7

Microscopy Dataset 1 (D1)

  • Dataset 1 tests MG132 and sodium arsenite at low and high concentrations against an untreated baseline in control-derived motor neurons.MG132 was applied chronically for 16 hours at 1 or 10 µM; sodium arsenite was applied acutely for 30 minutes at 0.1 or 0.5 mM. Maximum-intensity projection produced one 2D image per stack.Source: Microscopy Dataset 1 (D1) · pp. 7–8

Microscopy Dataset 2 (D2)

  • Dataset 2 includes motor neurons from 21 healthy controls, 42 sporadic ALS donors, and five familial ALS donors with multiple disease-associated genotypes.Summed-intensity projection was chosen instead of maximum-intensity projection because preliminary images from the latter obscured finer structural details.Source: Microscopy Dataset 2 (D2) · p. 8

Microscopy Dataset 3 (D3)

  • Dataset 3 tests edaravone and niclosamide against sodium-arsenite-induced TDP-43 mislocalization in motor neurons from ALS-discordant monozygotic twins.Both compounds were administered at 10 µM before 500 µM sodium arsenite for one hour; focused z-stack slices were combined to create representative images.Source: Microscopy Dataset 3 (D3) · pp. 8–9

Single Nucleus Images (SNIs)

The pipeline segments microscopy images into single-nucleus images (SNIs) and classifies them by TDP-43 localization, with dataset-specific classes and limited labeled samples for training and evaluation.

Single Nucleus Images (SNIs)

  • SNIs were classified as normal (SNInorm) or TDP-43-mislocalized (SNImis); D1 also included debris, while green-high-intensity SNIs in D2 and D3 were reassigned after clustering.The green-high-intensity SNIs were temporarily treated as a separate class before unsupervised clustering reassigned them to one of the two primary classes for consistent classification.Source: Single Nucleus Images (SNIs) · p. 9
  • Training used 10 labeled SNIs per class across all datasets; evaluation used 36 per class in D1 and 40 per class in D2 and D3.Source: Single Nucleus Images (SNIs) · p. 9

Cytomorphology Datasets

The section describes three public peripheral-blood cytomorphology datasets, with single-cell images expert-annotated across morphological classes.

Cytomorphology Datasets

  • The Matek dataset comprises 18,365 single-cell images sized 400 x 400 pixels, categorized into 13 classes.The classes include leukocyte and erythroblast morphologies.Source: Cytomorphology Datasets · p. 9
  • MLL23 contains 41,621 single-cell images assigned by expert cytomorphologists to 18 morphological classes.Cells were re-imaged with a 40x objective at 288 x 288 pixels, covering a 25 x 25 μm field of view; duplicate images were removed.Source: Cytomorphology Datasets · p. 10
  • The Acevedo dataset contains 17,092 peripheral-blood cell images annotated by clinical pathologists into eight morphological groups.Images were acquired using a CellaVision DM96 system and provided as 360 x 363 pixel JPEGs; the immature-granulocyte group includes promyelocytes, myelocytes and metamyelocytes.Source: Cytomorphology Datasets · pp. 9–10

Performance analysis of existing methods on classifying SNIs based

In a label-scarce setting, existing methods showed dataset-dependent performance: MAE performed best on D1 and D3, while ResNeXt performed best on D2. No method was consistently strong across all three datasets.

Performance analysis of existing methods on classifying SNIs based on TDP-43 mislocalization

  • MAE-based methods performed strongest on D1 and D3, while ResNeXt-based methods achieved the highest scores on D2; no existing method was consistently strong across all three datasets.Performance varied substantially with dataset and experimental setting, motivating a more adaptable approach for heterogeneous microscopy datasets.Source: Performance analysis of existing… · p. 10, Table 1 · p. 11
  • Models were trained with only 10 labeled SNIs per class and evaluated on independent labeled test sets using accuracy, precision, and F1-score.For MAE, self-supervised pretraining used the training SNIs from all three datasets, followed by fine-tuning on the limited labeled data.Source: Performance analysis of existing… · p. 10, Table 1 · p. 11

Performance analysis of integrating the proposed knowledge

The proposed knowledge augmentation improved classification across all three datasets and diverse model architectures. The MAE model with knowledge augmentation achieved the best overall performance and was selected for subsequent analyses.

Performance analysis of integrating the proposed knowledge

  • Knowledge augmentation consistently improved classification performance for every method across all three datasets, demonstrating its model-agnostic effectiveness.The paper reports stable performance across distinct experimental and imaging settings.Source: Performance analysis of integrating… · p. 11
  • The MAE model integrated with knowledge augmentation was designated as the proposed method for subsequent analyses.Its selection followed its robust performance across the three datasets.Source: Performance analysis of integrating… · p. 11
  • 4.2%, 1.3%, and 3.0% precision gains over the second-best method on datasets 1, 2, and 3, respectively, made the MAE-based method with knowledge augmentation the best overall performer.The MAE encoder was finetuned jointly with knowledge augmentation.Source: Performance analysis of integrating… · p. 11, Table 2 · p. 12

classifying SNIs based on TDP-43 mislocalization

Ablation results show that combining MAE features with prior-knowledge features improves SNI classification, with learnable weighted integration performing best. Handcrafted features alone yielded comparatively limited performance.

classifying SNIs based on TDP-43 mislocalization

  • Prior-knowledge features alone yielded comparatively limited performance, suggesting handcrafted features may not capture relevant SNI morphology associated with TDP-43 mislocalization.The authors interpret the benefit of combining feature types as evidence that domain knowledge adds information beyond automated feature extraction.Source: classifying SNIs based on TDP-43… · pp. 12–13
  • The weighted knowledge augmentation variant achieved the best performance across all datasets, improving average precision 2.3% over concatenation; concatenation improved average precision 0.7% over MAE features alone.The ablation compared MAE fine-tuned features, prior-knowledge features, direct concatenation, and weighted integration across three microscopy datasets.Source: classifying SNIs based on TDP-43… · pp. 12–13, Table 3 · p. 13

Biological interpretation of pathology of TDP-43 mislocalization in ALS

Across three iPSC-derived motor-neuron datasets, pipeline-based quantification revealed stressor effects, variable ALS-associated TDP-43 mislocalization, and no significant protection from edaravone or niclosamide against SA-induced mislocalization.

Biological interpretation of pathology of TDP-43 mislocalization in ALS

  • The pipeline quantified mislocalization per microscopic image, then calculated condition- or cohort-level means across three datasets examining stressors, ALS donor cohorts, and drug compounds.D1 assessed MG132 and SA, D2 compared sALS and fALS donor lines with controls, and D3 tested edaravone and niclosamide.Source: Biological interpretation of pathology… · p. 14

D1: Proteasomal stress induces pronounced TDP-43 mislocalization

  • In healthy control 1, 1 µM MG132 raised mislocalized nuclei from 13.04% to 52.61%, while 0.5 mM SA produced a smaller increase from 10.84% to 23.26%.MG132 also increased nuclear debris dose-dependently, reaching 53.73% at 10 µM in healthy control 1 versus DMSO; SA-induced debris changes were limited under the tested conditions.Source: D1: Proteasomal stress induces… · pp. 14–15, Figure 2 · p. 18, Figure 3

D2: Elevated TDP-43 mislocalization in both sporadic and familial ALS

reveals differential drug responses

  • SA increased mislocalization in healthy and sALS motor neurons; edaravone and niclosamide did not prevent it, and their combination further increased it in healthy cells.SA raised mislocalization from 8.79% to 11.83% in healthy controls and from 9.52% to 14.98% in sALS; edaravone reduced these to 10.64% and 12.34%, respectively, without significant protection. The edaravone–niclosamide combination significantly increased mislocalization in the healthy donor compared with SA or SA plus edaravone.Source: reveals differential drug responses · pp. 22–23, Figure 5 · p. 24

publicly available cytomorphology bioimaging datasets

On three public cytomorphology datasets, knowledge augmentation improved weighted average precision across models, with the MAE-based pipeline achieving the strongest reported result. The Discussion argues that biological descriptors complement learned representations and support annotation-efficient bioimage analysis.

publicly available cytomorphology bioimaging datasets

  • With 15 labelled images per class, the study combined MAE pre-training with 51 handcrafted nucleus-shape and colour features for WBC classification.The 51 features comprised three nucleus-shape descriptors and 48 colour features from RGB, HSV, LAB, and YCrCb colour spaces.Source: publicly available cytomorphology… · p. 25
  • 6.8% higher weighted average precision than the second-best method was achieved by the MAE-based model with knowledge augmentation across the three cytomorphology datasets.The reported comparison is for the limited-label setting; knowledge augmentation also improved performance in all evaluated models.Source: Table 5 · p. 26, publicly available cytomorphology… · p. 26

Discussion

  • Biological descriptors alone were insufficient for optimal performance, but combining them with self-supervised representations improved classification, especially with learnable weighting rather than simple concatenation.The weighting strategy adaptively determines the relative contributions of biological and learned features during model optimisation.Source: Discussion · pp. 27–28
  • Improvements across seven architectures suggest knowledge augmentation addresses incomplete use of biological information, complementing rather than replacing advances in model capacity and representation learning.The Discussion frames this as broadly applicable to annotation-scarce biological imaging problems.Source: Discussion · p. 28
  • The pipeline enabled investigation of ALS-related phenotypes, including stress-induced TDP-43 mislocalization, variable pathology in fALS and sALS neurons, and no demonstrated drug protection under tested conditions.Edaravone showed a non-significant trend toward mitigating stress-induced pathology, niclosamide failed to protect, and combined treatment further increased sodium arsenite-induced mislocalization; the findings require validation in larger cohorts and other systems.Source: Discussion · p. 28
  • Future work could add richer handcrafted features and other data modalities, while the study concludes that biological prior knowledge can enhance performance across architectures and imaging modalities under annotation constraints.Patient-level clinical reports are given as an example of an additional modality.Source: Discussion · pp. 28–29

1. Image pre-processing and single nucleus images (SNIs) generation module … In-house Microscopy Image Data Collection

The pipeline extracts single-nucleus images, learns visual representations with masked-autoencoder pretraining, and combines automated features with prior-knowledge features for classification. The in-house microscopy datasets used differentiated motor neurons exposed to stressors or drug pretreatment.

Motor Neuron Differentiation for Dataset 1 and Dataset 3; Human Research Ethics

Dataset 1 (D1)

  • Dataset 1 motor neurons received MG132 for 16 hours at 10 µM or 1 µM, or vehicle, while sodium arsenite was applied for 30 minutes at 0.1 mM or 0.5 mM.Source: Dataset 1 (D1) · p. 30

Dataset 2 (D2)

  • Samples were immunostained for TDP-43, then z-stack images were acquired at 20× magnification; four fields of view were captured per well, with four replicate wells analyzed per donor.Source: Dataset 2 (D2) · pp. 30–31

Dataset 3 (D3)

  • For dataset 3, motor neurons received edaravone or niclosamide for 24 hours before a full media change and 500 µM sodium arsenite stress for 1 hour.Source: Dataset 3 (D3) · p. 31

1. Image pre-processing and single nucleus images (SNIs) generation module

  • StarDist masks and YOLOv7-refined, padded boxes produced single-nucleus images without requiring manual ground-truth annotations, preserving nearby extranuclear regions for TDP-43 localization classification.Blue-channel images were input to pretrained StarDist; its masks yielded coarse boxes used to fine-tune YOLOv7 before padded crops were extracted.Source: Dataset 3 (D3) · pp. 31–32

2. Self-supervised feature representation learning module

  • The masked autoencoder was pretrained on extracted SNIs to reconstruct masked regions, then its encoder was retained to provide features for supervised tasks.The MAE used a ViT encoder and lightweight transformer decoder; the decoder was discarded after pretraining.Source: Dataset 3 (D3) · pp. 32–33

3. Model-agnostic knowledge augmentation based classification module

  • With 10 labelled SNIs per class, learnable weighted concatenation combined encoder representations and prior-knowledge features, allowing automated and manual feature contributions to adapt during training.Microscopy features covered nuclear and surrounding-area sizes and TDP-43 intensities; cytomorphology features included three shape descriptors and 48 colour features.Source: Dataset 3 (D3) · pp. 33–34
Loading 2610.13637v1…