PaperlayerView exact revision on arXiv

Source-linked AI summary

NucShare: Lightweight Nuclei Instance Segmentation with Foundation-Model Accuracy

Wenyan Li, Xiangyang Jia, Wenlong Zhong, Xu Chen

arXiv:2609.35203v3cs.CV

TL;DR

High-accuracy nuclei segmentation often uses large encoders, while dense, varied nuclei and whole-slide images make accuracy and speed important. NucShare uses one shared decoder with lightweight task heads, then distills output predictions from a UNI2-h teacher to a ConvNeXt V2-Tiny student. NucShare-T reaches mPQ 0.524 on PanNuke, above CFR-SAM-H at 0.511; NucShare-S retains 99.1% of that at 34.7M parameters.

  • Problem

    High-accuracy nuclei segmentation often uses large encoders, while dense, varied nuclei and whole-slide images make accuracy and speed important.

    Source: §1 · p. 1

  • Method

    NucShare uses one shared decoder with lightweight task heads, then distills output predictions from a UNI2-h teacher to a ConvNeXt V2-Tiny student.

    Source: §1 · pp. 1–2

  • Results

    NucShare-T reaches mPQ 0.524 on PanNuke, above CFR-SAM-H at 0.511; NucShare-S retains 99.1% of that at 34.7M parameters.

    Source: §4 · p. 4

  • Takeaways & Limitations

    NucShare-S has 34.7M parameters, runs 27.5× faster per tile than the teacher, and uses 0.2GB of GPU memory, fitting on a 4GB card.

    Source: §3.4 · p. 4, §4 · p. 4

    On PanNuke, NucShare-S bPQ trails CFR-SAM-H, a gap the authors attribute to watershed post-processing.

    Source: §4 · p. 4

Abstract

from arXiv · show

Nuclei instance segmentation is fundamental to quantitative pathology. Yet high-accuracy methods mostly build on large vision transformers (ViTs) or heavy convolutional backbones, whose parameter count and compute cost hinder deployment in clinical and edge settings. We present NucShare, a lightweight model whose shared decoder handles nuclear pixel segmentation, horizontal-vertical distance, and classification through learnable transposed convolutions for upsampling and three lightweight heads, using 37% of independent decoders' parameters with no loss in accuracy. Output-level distillation then transfers knowledge from NucShare-T (UNI2-h) to NucShare-S (ConvNeXt V2-Tiny), reducing 690.5M parameters to 34.7M. On PanNuke they reach mPQ 0.524 and 0.519, above the prior best CFR-SAM-H at 0.511, and NucShare-S surpasses HoVer-Net and Mask R-CNN by 5.6 and 15.0 points; on zero-shot MoNuSeg it attains bPQ 0.668 against CFR-SAM-H's 0.662. Code: https://github.com/L-iMeng-L/Newidea

1. INTRODUCTION

Nuclei instance segmentation must balance accuracy and speed across densely packed, varied nuclei, yet existing efficiency strategies preserve a large teacher or rely on encoder scaling. NucShare instead shares the decoder and distills between models that differ only in their encoders.

1. INTRODUCTION

  • Nuclei instance segmentation jointly detects, segments, and classifies nuclei; dense packing, morphological diversity, staining variation, and whole-slide scale make accuracy and speed necessary.The task supports downstream analyses such as tumor grading, immune-infiltration assessment, and prognosis prediction.Source: §1 · p. 1
  • HoVer-Net uses separate decoders for complementary pixel, distance-map, and classification tasks, while distillation and encoder scaling leave accuracy dependent on large models.The paper’s alternative is to retain the encoder-derived accuracy while shrinking the decoder.Source: §1 · p. 1
  • NucShare replaces three task-specific decoders with one shared decoder and separate lightweight task heads, matching independent decoders’ accuracy at 37% of their parameters.The student shares the decoder with the teacher and differs only in its encoder.Source: §1 · pp. 1–2
  • NucShare reaches mPQ 0.524 on PanNuke, above every prior method; its distilled NucShare-S student is 20× smaller, surpasses CFR-SAM-H on PanNuke mPQ, and achieves zero-shot MoNuSeg bPQ 0.668 versus 0.662.Source: §1 · p. 2

2. METHOD

NucShare combines a shared decoder and three task-specific lightweight heads with output-level distillation from a foundation-model teacher to a smaller student.

2.1. Teacher and Student Encoders

  • The teacher and student use the same decoder and differ only in their encoders: UNI2-h for NucShare-T and ConvNeXt V2-Tiny for NucShare-S.The teacher taps four UNI2-h blocks; the student’s 27.9M-parameter encoder brings its total to 34.7M parameters.Source: §2.1 · p. 2, §2 · p. 2, Figure 1 · p. 3

2.2. Multi-Task Shared Decoder

  • A shared decoder fuses and upsamples four encoder levels into a 128-channel map for lightweight NP, HV, and NC heads.Learnable transposed convolutions preserve boundary gaps and HV gradients, while CBAM attention is reserved for the NC head.Source: §2.2 · p. 2
  • Sharing serves pixel-dense tasks that use common cues, lets their gradients regularize shared weights, and makes output differences attributable to the encoders.The shared decoder means the student inherits rather than relearns the teacher’s decoder.Source: §2.2 · p. 2

2.3. Knowledge Distillation

  • Output-level distillation trains the student from a frozen converged teacher using a convex combination of supervision and distillation losses, with α = 0.5.KL divergence matches NP and NC probability distributions; MSE matches continuous HV maps, and the teacher and student output branches are distilled independently.Source: §2.3 · p. 2, Figure 2 · p. 3

3. EXPERIMENTS

Across PanNuke and zero-shot MoNuSeg, NucShare-S performs competitively with larger models, while ablations and runtime measurements assess its design and efficiency. PanNuke evaluation uses tissue-level three-fold cross-validation and class-balanced sampling for the rare Dead class.

3.1. Datasets and Metrics

  • PanNuke comprises about 7,900 H&E images across 19 tissue types and five classes, evaluated with tissue-level three-fold cross-validation and inverse-frequency sampling for the 1.53% Dead class.MoNuSeg serves as the cross-organ zero-shot evaluation set; mPQ averages PQ over five classes, while bPQ weights frequent classes more heavily.Source: §3.1 · p. 3

3.2. Comparison with State-of-the-Art

  • 0.524 mPQ for NucShare-T exceeds CFR-SAM-H at 0.511 on PanNuke, while NucShare-S reaches 0.519 with 20× fewer parameters.NucShare-S improves on FKDNuSeg from 0.459 to 0.519 mPQ and from 0.661 to 0.683 bPQ; both NucShare models trail CFR-SAM-H in bPQ.Source: §3.2 · p. 3, Table 1 · p. 3
  • NucShare-S ranks first on Epithelial and ties the best on Connective and Dead per-class F1, with scores of 0.79, 0.64, and 0.49.It exceeds CFR-SAM-H on Dead by 4 points; these scores are unaffected by post-processing.Source: §3.2 · p. 3, Table 2 · p. 4
  • 0.668 bPQ on zero-shot MoNuSeg for NucShare-S exceeds CFR-SAM-H at 0.662 and matches the teacher.Table 3 reports zero-shot generalization on MoNuSeg.Source: §3.2 · p. 4, Table 3 · p. 4

3.3. Ablation and Distillation Analysis

  • The NucShare decoder matches independent decoders using 37% of their parameters, while adding MALA costs 3.8M parameters without gains.On one fold, bilinear upsampling and a CBAM-free NC head lose 0.0025 and 0.0022 mPQ; KD retains 99.1% of teacher mPQ versus 97.9% from scratch.Source: §3.3 · p. 4

3.4. Parameters and Inference Speed

  • NucShare-S uses 34.7M parameters and 11.6 GFLOPs per tile versus the teacher’s 690.5M and roughly 200 GFLOPs, and runs 27.5× faster per tile.At batch size 1 it occupies 0.2GB versus 2.8GB; end-to-end on a 1024² image it takes 0.14 s versus 1.03 s.Source: §3.4 · p. 4

4. CONCLUSION

NucShare combines a shared decoder with encoder-only distillation to produce compact models with strong PanNuke and zero-shot MoNuSeg results. The PanNuke bPQ score remains below CFR-SAM-H, which the authors attribute to watershed post-processing.

4. CONCLUSION

  • mPQ 0.524 on PanNuke puts NucShare-T above CFR-SAM-H at 0.511; NucShare-S retains 99.1% of that result at 34.7M parameters.NucShare-S also runs 27.5× faster per tile than the teacher, needs 0.2GB of GPU memory, and reaches bPQ 0.668 on zero-shot MoNuSeg against CFR-SAM-H at 0.662.Source: §4 · p. 4
  • On PanNuke, bPQ trails CFR-SAM-H, a gap the authors trace to watershed post-processing and leave for future work alongside INT8 quantization and TensorRT acceleration.Source: §4 · p. 4
  • The conclusion identifies Table 4 as a decoder-design ablation on PanNuke.Source: Table 4 · p. 4

COMPLIANCE WITH ETHICAL STANDARDS

The study used human-subject data retrospectively from the open-access PanNuke and MoNuSeg datasets; the attached data licenses confirmed that ethical approval was not required.

COMPLIANCE WITH ETHICAL STANDARDS

Loading 2609.35203v3…