Performance at What Cost? A Sustainability-Aware Performance Index for Cell and Nucleus Instance Segmentation

1 Introduction

• Segmentation errors can propagate into downstream measurements, while reliable boundaries support cell counting, morphometry, phenotypic profiling, and spatial analysis.The practical requirement is reliability across tissue, staining, imaging modality, morphology, object density, and image quality.Source: §1 · p. 1
• Zero-shot models avoid target-specific training, while few-shot adaptation uses limited annotated target images; performance depends on target data and support-set representativeness.Deep learning and transformer approaches extend segmentation beyond classical pipelines, including learned instance boundaries and pretrained representations [2] [6] [7].Source: §1 · pp. 1–2
• Green AI motivates evaluating predictive gains alongside computational resources, including costs of model development, target-domain adaptation, and repeated inference [13] [14] [15].Environmental impact also depends on architecture, hardware efficiency, data-centre infrastructure, and electricity carbon intensity [14].Source: §1 · p. 2
• No controlled benchmark had jointly assessed zero-shot and few-shot performance, inference and fine-tuning energy, and model complexity across specialist, generalist, and foundation models.This leaves unclear whether larger pretrained models' gains are proportionate to their additional resource demands.Source: §1 · p. 2
• The study evaluates 19 models across six CellBinDB datasets and compares zero-shot inference with frozen-encoder and full-model few-shot fine-tuning for 16 models.The datasets cover DAPI, single-stranded DNA, multiplex immunofluorescence, and H&E, with 1,044 images across 30 tissue types.Source: §1 · p. 2
• The benchmark measures GPU, CPU, and RAM energy use during inference and fine-tuning, counts model parameters, and introduces configurable SAPI to relate performance, energy, and model scale.The stated aim is to assess whether added capacity and computational expenditure yield proportionate performance improvement.Source: §1 · p. 3
• The paper frames model selection as balancing segmentation quality with computational accessibility and environmental responsibility, rather than treating the smallest model or energy efficiency as inherently preferable.Additional computational demand should be justified by improvements meaningful to the intended application.Source: §1 · p. 3

2.1 Datasets

• The benchmark spans 1,044 images, 102,480 annotated cells, 30 tissue types, and six fluorescence and H&E modalities.CellBinDB provides an independent test set with domain variation in staining, tissue, image source, and size; mIF images were inverted and the hematoxylin channel extracted from H&E images, with no other study preprocessing.Source: §2.1 · p. 3

2.2 Model zoo

• The model zoo includes 19 pretrained models from 11 families, with parameter counts from 1.4 million to approximately 700 million.Models span different encoders, instance-reconstruction strategies, and modality specializations; evaluation was restricted to each model’s supported imaging domain or domains.Source: §2.2 · pp. 3–4

2.3 Segmentation performance evaluation

• AJI+, PQ, and NSD capture complementary segmentation properties, and their dataset-level values are combined by harmonic mean into performance score P.Metrics are first calculated per image and averaged with equal image weight; the harmonic mean penalizes imbalance across instance overlap, recognition, and boundary agreement.Source: §2.3 · pp. 5–6

2.4 Energy measurements

• GPU energy was measured with NVML, while CPU-package and DRAM energy were measured with RAPL; measurements cover the complete model execution phase.The interval includes model loading, preprocessing and postprocessing, and, for fine-tuning, loss calculation, backpropagation, parameter updates, and all epochs; dataset-level transformations and metric calculations are excluded. These hardware-domain estimates omit facility overhead and are intended for controlled relative comparisons, not as exact wall-socket electricity.Source: §2.4 · pp. 6–7, §2.4.3 · p. 7

2.4.1 GPU energy

• GPU energy is the cumulative NVML counter increment across a synchronized inference or fine-tuning phase, while CPU-package and DRAM energy use summed RAPL counter increments.The selected GPU was dedicated to the experiment, and CPU and memory counters were recorded at each phase’s boundaries; component energy increments are summed as total IT energy, excluding uninstrumented infrastructure.Source: §2.4.1 · p. 7, §2.4.2 · p. 7, §2.4.3 · p. 7

2.4.4 Inference energy

• Inference energy is amortized per test image, while fine-tuning energy is amortized per support image over the complete training procedure.The fine-tuning total includes all training epochs; the SAPI fine-tuning energy term combines this support-image cost with adapted-model inference energy per test image.Source: §2.4.4 · p. 7, §2.4.5 · p. 7, §2.5 · p. 8

2.5 Sustainability-Aware Performance Index (SAPI)

• SAPI combines segmentation performance, inference and fine-tuning energy, and parameter count through a weighted multiplicative formulation.Logarithmic penalties are applied to energy and parameter count; positive weights sum to one, and fine-tuned-model energy combines post-adaptation inference with fine-tuning energy per support image.Source: §2.5 · p. 8
• SAPI weights inference energy by λ and fine-tuning energy by (1 −λ), reflecting that adaptation costs are amortized while inference energy accumulates with deployment volume.The passage describes the weighting rationale; the supplied text does not include the SAPI formula itself.Source: §2.5 · p. 8

2.6 Experimental Study Design

• Three conditions were evaluated: zero-shot inference, few-shot full-model fine-tuning, and few-shot fine-tuning with a frozen encoder, using the same protocol for applicable model–dataset combinations.Source: §2.6 · p. 8

2.6.1 Zero-shot Evaluation

• All 19 configurations were tested zero-shot on supported dataset images using native automatic pipelines, without parameter updates, threshold selection, postprocessing changes, or manual prompts.Automatic model-specific settings, such as Cellpose diameter estimation, were retained.Source: §2.6.1 · p. 9

2.6.2 Few-shot Adaptation

• Few-shot support sets used one seeded random image per tissue, with all remaining images reserved for testing and identical splits across compatible models.No validation set was created; models trained for 100 epochs with default hyperparameters and batch size 8. Sixteen configurations were fine-tuned; DeepCell NuclearSegmentation, Mesmer, and CellSAM were zero-shot only.Source: §2.6.2 · p. 9
• Both full-model and frozen-encoder adaptation were compared on the same support and test sets; freezing preserves pretrained representations and reduces trainable parameters but can restrict domain adaptation.Full-model training updates all parameters; frozen-encoder training updates decoder, prediction heads, and other designated components.Source: §2.6.2 · p. 9

2.6.3 Repeated Experiments

• Experiments were repeated ten times with seeds 0–9, independently sampling tissue-stratified support sets while keeping each dataset-and-seed split common across models and regimes.Seeds were propagated across applicable frameworks and deterministic cuDNN settings were enabled, though complete bitwise determinism was not guaranteed.Source: §2.6.3 · p. 9

2.6.4 Computational Setup

• All experiments used one compute node and GPU 0; the GPU was cooled to 45°C before each measurement to reduce thermal-state effects on energy estimates.The node had two NVIDIA Tesla V100 PCIe GPUs, two Intel Xeon Gold 6226R processors, and 501 GiB of system memory.Source: §2.6.4 · p. 9

3.1.1 Performance

• Zero-shot P ranged from 0.117 for Mesmer on H&E to 0.752 for InstanSeg on 10xGenomics_DAPI, with leaders varying by dataset.InstanSeg led 10xGenomics_DAPI and ssDNA; CellposeSAM led DAPI, CellSAM led mIF, and PathoSAM ViT-H led both H&E datasets.Source: Figure 3 · p. 10, §3.1.1 · p. 10

3.1.2 Inference energy consumption

• CellSAM used 437.65 J per image versus InstanSeg’s 8.36 J, while GPU and CPU contributions varied by model and RAM was smallest.CellSAM’s measured energy was approximately 52 times InstanSeg’s.Source: §3.1.2 · p. 10, Figure 4 · p. 11

3.1.3 Performance vs Energy vs Model size

• Across 19 models, performance–energy ρ was −0.020, while parameter-count–energy ρ was 0.747; greater size or energy did not reliably predict better zero-shot performance.Several smaller models achieved competitive performance with lower energy use.Source: §3.1.3 · p. 11, Figure 5 · p. 12

3.2.1 Performance

• Few-shot adaptation improved most model–dataset combinations, especially when zero-shot performance was weak; full-model tuning usually performed better, but exceptions and dataset variation remained.Frozen-encoder tuning occasionally matched or outperformed full-model tuning, including for CellViT-256 on both H&E datasets and InstanSeg on mIF.Source: §3.2.1 · pp. 12, 16

3.1.4 SAPIZS

• SAPIZS changed performance-only ordering: InstanSeg led four datasets, while StarDist (Fluo) led mIF and StarDist (H&E) led HE.Across fluorescence datasets, StarDist (Fluo) had the highest mean SAPIZS; across H&E datasets, StarDist (H&E) ranked first.Source: §3.1.4 · pp. 11–12, Figure 6 · p. 13

3.2.3 SAPIFT+I

• After adaptation, InstanSeg had the highest average SAPIFT+I on both fluorescence and H&E datasets; larger models’ added energy and complexity costs could lower their scores despite competitive adapted performance.Primary SAPI comparisons used performance, energy, and size weights of 3:2:1, with fine-tuning energy contributing 20% and post-fine-tuning inference energy 80%.Source: §3 · pp. 9–10, §3.2.3 · p. 16

3.2.2 Energy Consumption

• Freezing the encoder reduced fine-tuning energy for every model, with savings varying substantially by architecture; larger models generally used more energy, but size alone did not explain the differences.The largest reductions included CellposeSAM, CellViT-SAM-H, Hover-Net, and PathoSAM ViT-H.Source: §3.2.2 · p. 16, Figure 13 · p. 17

3.2.3 SAPIFT+I

• InstanSeg had the highest average SAPIFT+I across datasets, followed by MEDIAR and the two Cellpose cyto variants; no model led every dataset.The choice should reflect the target dataset and deployment scenario.Source: §3.2.3 · p. 17

4 Discussion

• Large foundation models often achieve marginally higher performance but require orders of magnitude more energy and trainable parameters; compact models can remain competitive.The model with the highest segmentation score is not necessarily the most suitable for resource-constrained deployment.Source: §4 · p. 17

4.1 Model scale did not determine segmentation performance

• Performance advantages vary across datasets and adaptation settings rather than tracking parameter count; direct energy measurements provide stronger deployment evidence than model size alone.Energy also depends on factors such as memory transfers, tiling, post-processing, framework overhead, and hardware utilization.Source: §4.1 · pp. 17, 19

4.2 Few-shot adaptation should be justified by its incremental benefit

• Few-shot adaptation can improve target-data alignment, but its incremental gains must be weighed against annotation, training-energy, memory, and implementation costs.Frozen-encoder adaptation can reduce training demands but may restrict adaptation when the target modality differs substantially from pre-training; no strategy is universally superior.Source: §4.2 · p. 19

4.3 SAPI makes the performanceresource trade-off explicit

• SAPI combines performance, energy, and parameter count, with configurable weights that make rankings reflect deployment priorities rather than a universal model ordering.Default weights assign relative importance in a 3:2:1 ratio to performance, energy, and complexity.Source: §4.3 · p. 20

4.4 Energy consumption and implications for operational carbon emissions

• Energy differences per image can accumulate across large workloads, while the study’s estimates indicate operational carbon implications rather than a complete environmental assessment.The assessment excludes embodied emissions from hardware and infrastructure and energy associated with externally developed models’ pre-training.Source: §4.4 · p. 20

4.5 Implications for sustainable biomedical image segmentation

• Higher per-image costs can constrain throughput and scientific accessibility; broad foundation-model value across multiple tasks was not directly measured.A shared model may still be advantageous if it replaces several specialized systems, but its full anticipated workload should be compared with those systems’ combined costs.Source: §4.5 · pp. 20–21
• Model choice should first meet task-specific performance requirements, then balance inference and adaptation costs against downstream value and deployment needs.Zero-shot, frozen-encoder, and full-model adaptation choices depend partly on expected retraining frequency and future inference volume.Source: §4.5 · pp. 20–21

4.6 Limitations

• The benchmark uses one hardware configuration, excludes original pre-training energy, applies a fixed adaptation protocol, covers only six CellBinDB subsets, and evaluates automatic zero-shot inference.SAPI rankings also depend on selected weights and λ, which applications should report and justify.Source: §4.6 · pp. 21–22

5 Conclusion

• Higher segmentation performance does not always justify greater computational cost, so model selection should consider modality, adaptation needs, deployment scale, and resource use.The benchmark covers zero-shot inference and few-shot adaptation across 19 pretrained models and six CellBinDB datasets.Source: §5 · p. 22
• SAPI provides a transparent way to examine performance, energy, and model-complexity trade-offs using weights adjustable to users’ priorities.The stated aim is a practical framework for selecting effective and efficient cell-segmentation models.Source: §5 · p. 22

Appendix A Supplementary Material

• Supplementary material provides additional model and software information, together with detailed quantitative and qualitative results.Quantitative results are means across ten runs unless stated otherwise; qualitative zero-shot examples use the experiment with random seed 0.Source: Appendix A · p. 26
• The main manuscript describes the experimental protocol, metric definitions, energy measurement procedure, carbon-emission calculation, and SAPI formulation.Source: Appendix A · p. 26
• Unless otherwise stated, quantitative results average ten runs, while qualitative zero-shot examples use random seed 0.Source: Appendix A · p. 26

A.1 Model and Software Details

• Table S1 lists each evaluated model, its exact pretrained checkpoint, and the software version used in the experiments.Source: §A.1 · p. 26
• For software installed from a source repository, Table S1 gives the corresponding Git commit instead of a package version.Source: §A.1 · p. 26
• The exact inference and training configurations and study-developed code are publicly available on GitHub.Source: §A.1 · p. 26

• Models were evaluated without dataset-specific adaptation and only on datasets compatible with their intended image modality.Source: §A.2.1 · p. 26
• The main manuscript reports the composite performance score P.Source: §A.2.1 · p. 26
• Supplementary Figures S1–S3 report AJI+, Panoptic Quality (PQ), and Normalized Surface Dice (NSD), respectively.Source: §A.2.1 · p. 26

• One test image per dataset was selected using fixed random seed 0.Source: §A.2.2 · p. 26
• Each supplementary figure shows the original image, ground-truth instance mask, and an overlay of that mask on the original.Source: §A.2.2 · p. 26
• The figures also show predictions from all applicable models overlaid on the original image.Source: §A.2.2 · p. 26

• One image was sampled from each tissue group for the support set, with remaining images reserved for testing.Source: §A.3 · p. 26
• Models were evaluated before and after 100-epoch fine-tuning under full-model and frozen-encoder regimes, with the procedure repeated for seeds 0–9.Source: §A.3 · p. 26
• Supplementary Figures S1–S3 show zero-shot AJI+, PQ, and NSD scores across compatible model–dataset combinations; grey cells mark modality-incompatible combinations not evaluated.Source: Figure S1 · p. 27, Figure S2 · p. 27, Figure S3 · p. 28

• Figures S10–S15 visualize the AJI+ results for each dataset.Source: §A.3.1 · p. 28

A.3.2 Panoptic Quality (PQ)

• Figures S16–S21 visualize PQ metrics for each dataset.Source: §A.3.2 · p. 28

A.3.3 Normalized Surface Dice (NSD)

• Figures S22–S27 visualize NSD metrics for each dataset.Source: §A.3.3 · p. 28
• Qualitative zero-shot figures show input images, ground-truth instance masks and overlays, and predictions from models compatible with each image modality.One image per dataset is sampled using random seed 0.Source: Figure S4 · p. 29, Figure S5 · p. 30, Figure S6 · p. 31
• Across six datasets, the NSD figures compare compatible fine-tunable models under zero-shot inference, frozen-encoder fine-tuning, and full-model fine-tuning, reporting means across ten runs.Source: Figure S22 · p. 39, Figure S23 · p. 39, Figure S24 · p. 39