FTU-Seek: Foundation Model-Guided Hard-Negative Learning for Sparse Functional Tissue Unit Segmentation

• FTUs capture epithelial, vascular, and immune organization, making their segmentation useful for extracting interpretable phenotypes from routine histopathology slides.Source: §1 · p. 3
• Sparse FTUs are unevenly distributed and surrounded by redundant easy negatives plus rare target-like regions that can trigger false positives.This makes training-set construction a WSI-scale challenge, not only a pixel-level prediction problem.Source: §1 · p. 3
• FTU-Seek ranks annotated FTU-absent patches using a classifier trained on frozen multi-depth pathology foundation-model features, then selects static per-WSI TopK hard negatives.The fixed selection occurs before segmentation training and produces a compact, workload-controlled, reproducible training set distinct from active learning and online hard-example mining.Source: §1 · p. 5
• The framework is evaluated across TLS, blood-vessel, and gland segmentation tasks against positive-only, all-tissue, random-negative, and matched random TopK strategies.These tasks span distinct sparsity and morphological profiles, enabling assessment of segmentation performance alongside retained training workload.Source: §1 · p. 5
• FTU-Seek is further applied to external TCGA cohorts to derive quantitative immune, vascular, and glandular tissue-organization phenotypes.Its fixed negative-pool construction is expected to reduce training-data workload and data-transfer burden, but wall-clock time, GPU-memory usage, and energy consumption were not directly measured.Source: §1 · pp. 5–6

2.1 Study Design and Datasets

• Three FCH cohorts comprised task-specific FTU-containing and FTU-absent patches from heterogeneous H&E WSIs, with 4:1 patient/WSI-level development–test splits established before patch extraction.TLS included 2,079 positive and 41,094 negative patches; blood vessel 4,005 and 22,048; gland 11,670 and 20,128.Source: §2.1 · p. 6
• Development sets underwent five-fold cross-validation, while independent internal test sets were held out for final segmentation evaluation using ensembles of cross-validation models and a fixed threshold of 0.5.Internal testing included 2 WSIs per task, so results were interpreted cautiously.Source: §2.1 · p. 6
• A 30-WSI held-out TLS cohort was evaluated only after segmentation configurations were prespecified from development experiments, excluding it from training, tuning, TopK selection, and threshold optimization.The cohort was constructed at the same institution to assess robustness beyond the two-slide internal test set.Source: §2.1 · p. 6
• Public TCGA diagnostic WSIs and matched clinicopathological data were downloaded from the NIH Genomic Data Commons Data Portal for downstream FTU phenotype analyses.One diagnostic WSI was included per patient.Source: §2.1 · pp. 6–7
• Exploratory TCGA analyses examined TLS phenotypes in TCGA-READ, TCGA-ESCA, and TCGA-STAD, vascular phenotypes in TCGA-LIHC, and glandular phenotypes in TCGA-PRAD against specified clinical endpoints.Cases required an eligible diagnostic WSI, matched endpoint, and valid tissue-level segmentation output, with prespecified malignancy, recurrence, and follow-up exclusions.Source: §2.1 · p. 7
• TCGA analyses were designed as exploratory applications of FTU-Seek segmentations rather than external segmentation validation or definitive clinical prediction analyses.Cohort sizes were 146, 148, and 326 patients for READ, ESCA, and STAD; 337 for LIHC; and 401 for PRAD.Source: §2.1 · p. 7
• FCH procedures received ethics approval on 14 August 2024 (No. SQ2026-050), with informed consent waived for de-identified retrospective data; public TCGA use was ethics-exempt.All procedures followed the Declaration of Helsinki.Source: §2.1 · p. 7

Annotation

• Three trained annotators manually delineated TLS, gland, and blood-vessel boundaries, with senior-pathologist review and clinical-outcome blinding; inter-observer agreement was not assessed.One annotator was assigned to each FTU type, and annotations followed predefined histopathological criteria.Source: §2.2 · p. 7

Annotation criteria

• TLS annotations excluded diffuse inflammatory infiltrates, vessels required an endothelial lining and excluded tissue-tearing artifacts, and gland boundaries followed the epithelial-stromal junction across benign and malignant morphology.Vascular annotations covered arterioles, portal veins, and hepatic veins; gland annotations included benign bilayered glands and malignant acini.Source: §2.2 · pp. 7–8

Preprocessing

• Foreground tissue was extracted by Otsu thresholding at 1 µm/pixel, divided into non-overlapping 256×256-pixel patches, and resized to 224×224 pixels.The resolution corresponded approximately to 10× magnification and was chosen to balance morphological detail with computational efficiency.Source: §2.2 · p. 8
• The same patch coordinates were used across classification, segmentation, validation, and testing, with only resizing, tensor conversion, and ImageNet normalization and no data augmentation.No random cropping, overlap, flipping, rotation, color jittering, or other augmentation was applied.Source: §2.2 · p. 8

Patch labeling

• Patches overlapping any annotated target were labeled target-containing for classifier training and hard-negative ranking, while segmentation used the original pixel-level masks.This inclusive criterion retained partial and boundary-crossing targets; non-target pixels within target-containing patches remained background during segmentation learning.Source: §2.2 · p. 8

2.3 Model Development

• FTU-Seek first trains a patch-level classifier to identify FTU-absent patches likely to be confused with FTU-containing tissue, then uses them for segmentation training.The framework comprises classifier-guided hard-negative screening followed by segmentation using the resulting compact training set.Source: §2.3 · p. 9, §2.3.1 · p. 10

2.1 Patch-Level Training Set Definition

• Patches overlapping annotated targets form P+, while non-overlapping patches form P− for classifier training and high-risk background identification.These labels establish the positive and target-absent patch pools used in Stage I.Source: §2.3.1 · p. 10

2.3.2 Classification Network

• The patch classifier independently transforms ten uniformly spaced multi-depth UNI v1 features, mean-aggregates them, and predicts target-absent versus target-containing classes.UNI v1 remains frozen; each depth-specific head expands 1024 features to 4096 and projects them back to 1024 before aggregation.Source: §2.3.2 · pp. 10–11
• Within each WSI, target-absent patches assigned high target-containing probabilities were treated as classifier-derived hard negatives; records followed stored within-WSI ranking with probability as a secondary criterion, and first K eligible records were selected.The evaluated nominal values were K ∈ {100, 300, 500, 1000}; exclusions or fewer than K eligible candidates could make the final count at most K.Source: §2.3.2 · pp. 11–12

2.3.3 Segmentation Network

• The segmentation network keeps UNI v1 frozen and progressively fuses four multi-scale Transformer features with shallow image features to produce dense binary predictions.The decoder restores resolution from token-grid scale to the original input resolution, while downstream decoding modules and the segmentation head are optimized.Source: §2.3.3 · pp. 12–14

2.3.4 Optimization Objectives and Training Protocol

• Training uses cross-entropy for patch classification and Dice–Focal loss for segmentation, with five-fold WSI-level cross-validation and within-fold hard-negative construction.The best validation model from each fold is evaluated on an independent test set; static TopK is the primary configuration to isolate classifier-guided selection.Source: §2.3.4 · pp. 14–15, §2.3.3 · p. 14

2.3.5 Comparative Patch Construction Strategies

• Comparisons use positive-only, all-tissue, matched random-K, and static FP-find TopK training, with random negatives sampled independently per WSI under the same budget as classifier-ranked negatives.This design tests whether high-risk classifier-selected negatives are more informative than randomly selected negatives.Source: §2.3.5 · pp. 15–16

Segmentation outcomes

• Segmentation accuracy was quantified using DSC to measure pixel-level agreement between predicted masks and manual annotations.DSC was the primary metric for spatial overlap and morphological delineation accuracy.Source: §2.4 · p. 16

Classification outcomes

• Patch-level classification used AUC as the principal threshold-independent metric, supplemented by sensitivity, specificity, PPV, NPV, accuracy, and F1-score.These metrics were derived from confusion-matrix counts to assess discrimination, class-specific identification, predictive reliability, and overall performance under potential class imbalance.Source: §2.4 · p. 16

Downstream phenotypes

• Segmentation outputs were converted into slide- or patient-level TLS, blood-vessel, and gland phenotypes, including density, count, size-stratified, hotspot, and morphology measures.TLS analyses additionally included clustering indices; gland phenotypes included area, perimeter, circularity, solidity, and eccentricity.Source: §2.4 · pp. 16–17

Clinical analyses

• Downstream analyses related TLS and blood-vessel phenotypes to overall survival, vascular features to MVI status in LIHC, and gland features to pathology endpoints and BCR status in PRAD.BCR was treated as a binary endpoint because reliable censoring times were unavailable for non-recurrent cases.Source: §2.4 · p. 17
• PRAD gland features were also compared across Gleason-score groups, pT stages, and R0 versus R1 resection-margin statuses.These comparisons extended glandular phenotype evaluation beyond BCR and pathology-related endpoints.Source: §2.4 · p. 17

• Frozen UNI v1 multi-depth embeddings are transformed and averaged to classify patches, whose predicted FTU-containing probabilities rank FTU-absent candidates for static TopK hard-negative selection.The classifier uses ten encoder depths, two-layer transformation heads, cross-entropy loss, AdamW, validation-AUC checkpointing, and early stopping before exporting ranked negatives.Source: §2.5 · p. 17
• The segmentation pipeline combines all FTU-containing patches with classifier-ranked FTU-absent hard negatives, using a frozen UNI v1 encoder and trainable residual feature-fusion decoder across four encoder depths.Five-fold cross-validation is performed at the WSI level, with independent test sets held out for final evaluation and inputs resized to 224 × 224 pixels.Source: §2.5 · p. 17
• Matched random TopK baselines select the same number of negative patches per WSI randomly from the FTU-absent pool.Source: §2.5 · p. 18
• Segmentation training updates only the decoder and prediction head with AdamW, mixed precision when available, and a combined objective Lseg = LDice + 20LFocal, with validation mean Dice guiding scheduling, early stopping, and checkpoint selection.The focal term uses class weights 0.25 and 0.75 with γ = 2.0; the highest-validation-mean-Dice checkpoint from each fold is evaluated on the independent test set.Source: §2.5 · p. 18
• Inference thresholds the foreground softmax probability at 0.5, maps predictions to WSI coordinates, computes Dice from WSI-aggregated pixel counts, and applies no morphological post-processing.Predictions outside the foreground tissue mask are set to zero, and Dice is not averaged across patches.Source: §2.5 · p. 18

• Segmentation and classification metrics were summarized as mean ± standard deviation across five folds, while random-baseline analyses pooled seeds 1–5 to assess sampling and training variability.The multi-seed analysis primarily assessed random-sampling baseline stability, not direct FP-find seed robustness.Source: §2.6 · pp. 18–19
• Held-out TLS comparisons used WSI-level Dice scores from 30 slides, paired confidence intervals and tests, with five-fold probability ensembles; internal two-WSI results remained descriptive.Formal paired analyses compared FTU-Seek with matched random TopK strategies and representative all-tissue or random-balanced training.Source: §2.6 · p. 19
• WSI-level Dice accumulated true-positive, false-positive, and false-negative pixels across non-overlapping patches rather than averaging patch Dice or measuring instances.Dice was reported for validation folds and independent test sets, alongside retained-patch workload relative to all-tissue training.Source: §2.6 · p. 19
• TLS survival analyses used standardized continuous features in Cox models with 95% confidence intervals and within-cohort Benjamini–Hochberg FDR correction across 60 features.Kaplan–Meier plots and data-derived cut points were exploratory; vascular analyses additionally examined OS, MVI, adjusted regression, proportional hazards, collinearity, and nonlinearity.Source: §2.6 · pp. 19–20
• Glandular features were compared across four PRAD clinicopathological endpoints with Wilcoxon tests, effect sizes, and endpoint-specific FDR correction across 40 features.All tests were two-sided; nominal p < 0.05 and exploratory FDR-adjusted p < 0.10 thresholds were used, and clinical associations were hypothesis-generating rather than confirmatory.Source: §2.6 · pp. 20–21

3.1 Dataset Characteristics

• At the instance level, development sets contained 133 TLSs, 2508 blood vessels, and 3074 glands, corresponding to mean counts of 16.62, 358.29, and 384.25 per WSI.Test sets showed the same ordering, with TLS considerably less frequent than blood vessels and glands.Source: §3.1 · p. 21, Table 1 · p. 22
• TLS was the sparsest cohort, with 3.92% of development patches and 0.65% of development tissue pixels containing annotated TLS regions.Blood vessels showed intermediate sparsity, whereas glands occupied 36.86% of development patches and 18.91% of tissue pixels.Source: §3.1 · p. 21, Table 1 · p. 22
• External TCGA cohorts enabled exploratory testing of automatically quantified FTU features against survival and clinicopathological endpoints across colorectal, esophageal, gastric, liver, and prostate cancers.Analyses included TLS overall survival, BV survival and microvascular invasion, and gland associations with Gleason score, pathological T stage, surgical margins, and biochemical recurrence.Source: §3.1 · p. 21, Table 3 · p. 24

3.2 Patch-Level Classifier Performance

• TLS classification reached a mean AUC of 95.92 ± 2.81%, sensitivity of 88.46 ± 4.74%, and specificity of 92.01 ± 6.46%, but precision varied with rarity.TLS had a high NPV of 99.48 ± 0.30% and a lower PPV of 38.97 ± 23.04%; fold AUCs reached 97.97% and 99.19% in Folds 2 and 3.Source: §3.2 · pp. 22, 23, 25, Figure 7 · p. 27
• BV classification was less separable, with a mean AUC of 90.11 ± 3.11%, sensitivity of 83.49 ± 5.82%, specificity of 81.60 ± 5.36%, and FPR of 18.40 ± 5.36%.BV-containing patches overlapped broadly with negatives, especially in Folds 2–4, consistent with vessel-like background morphology and thresholds of 0.06–0.36.Source: §3.2 · pp. 23, 25, Figure 7 · p. 27
• Gland classification achieved a mean AUC of 98.16 ± 0.61%, sensitivity of 92.49 ± 1.12%, and specificity of 95.64 ± 0.80%, with limited cross-fold overlap.The gland classifier also had a mean FPR of 4.36 ± 0.80% and fold-wise AUCs of 97.41–98.75%.Source: §3.2 · pp. 23, 26, Figure 7 · p. 27
• Patch classifiers provided effective screening across all three FTU tasks, with gland classification strongest and most stable, TLS affected by extreme sparsity, and BV by confusing background.Validation-best classifiers achieved mean AUCs of 95.92 ± 2.81% for TLS, 90.11 ± 3.11% for BV, and 98.16 ± 0.61% for glands.Source: §3.2 · pp. 22–24, Table 4 · p. 24, Figure 6 · p. 25
• Classifier-ranked hard negatives included inflammatory or stromal TLS mimics, lumen-like or collagen-rich BV mimics, and epithelial or stromal gland-boundary mimics.These examples support prioritizing FTU-absent patches with high predicted FTU-containing probabilities for segmentation training.Source: §3.2 · p. 27, Figure 8 · p. 28

• Across tasks, positive-only training was weakest or near-weakest, whereas FTU-Seek preserved much of all-tissue performance using fewer patches, with benefits strongest for sparse or ambiguous FTUs.All-tissue training generally provided the strongest reference performance but required the complete tissue-patch set.Source: §3.3 · pp. 28, 31, Table 5 · p. 29, Table 6 · p. 29
• 85.14 ± 1.61% TLS Dice was achieved by FTU-Seek Top500 using 15.8% workload, exceeding matched-random gains by 5.22, 3.81, 3.48, and 1.20 points across Top100–Top1000.Top100 achieved 85.08 ± 1.07% Dice using 6.3% of patches, and FTU-Seek matched or exceeded all-tissue performance at three of four budgets.Source: §3.3 · pp. 28, 30, Table 5 · p. 29
• 60.38±9.76% was the best BV test Dice with all-tissue training; FTU-Seek reached 57.68±6.88% at Top1000 using 50.0% workload and beat matched random at every budget.FTU-Seek performance increased with the hard-negative budget, indicating that BV segmentation required broader negative-patch coverage; BV lacked a held-out cohort.Source: §3.3 · pp. 29–31, Table 7 · p. 30
• For gland segmentation, all-tissue training achieved 86.57 ± 0.69% test Dice, while FTU-Seek Top1000 achieved 86.33 ± 0.27%; FP-find Top1000 used 67.2% workload and offered no clear overall advantage.Glandular structures were relatively abundant and morphologically distinct. No held-out cohort was included; only development-set validation and internal-test results were reported.Source: §3.3 · pp. 30–31

3.4 Multi-Seed Stability of Random-Sampling Baselines

• Across five seeds, random-sampling baselines showed limited-to-moderate variability, with magnitude depending on FTU category and negative-patch budget.Each configuration was repeated with four additional seeds beyond the original seed.Source: §3.4 · pp. 31–32
• For TLS, random Top100, Top300, Top500, and Top1000 achieved mean Dice values of 79.86 ± 3.45%, 80.88 ± 1.55%, 81.66 ± 1.24%, and 83.92 ± 0.54%.Original seed-1 FP-find results were 85.08%, 84.69%, 85.14%, and 85.12%, remaining above random-baseline averages at every budget despite the two-WSI test cohort.Source: §3.4 · p. 32
• FP-find exceeded random-baseline averages by +3.98, +0.70, +2.96, and +3.47 percentage points for vessels, but by −1.42, +0.43, +0.43, and −0.16 for glands.The results support task-dependent benefits of classifier-guided selection, especially for sparse or morphologically ambiguous targets.Source: §3.4 · p. 32

3.5 Evaluation on an Independent Held-Out TLS Cohort

• The independent held-out cohort contained 30 patients and WSIs, 1184 annotated TLS instances, 123,484 tissue patches, and a 1.68% TLS pixel-area ratio.It was excluded from training, model selection, TopK selection, and threshold tuning.Source: §3.5 · p. 32
• FTU-Seek achieved slide-level Dice values of 73.99%, 76.54%, 75.19%, and 76.69% at Top100, Top300, Top500, and Top1000, versus 71.41%, 67.59%, 70.48%, and 72.97% for matched random TopK.The largest paired improvement occurred at Top300, while Top1000 achieved the highest absolute FTU-Seek Dice.Source: §3.5 · p. 32
• Top1000 was retained as a pre-specified representative operating point based on development-set analysis, without using the held-out cohort for operating-point selection or segmentation tuning.Formal paired comparisons across the 30 held-out WSIs were reported in Table 9.Source: §3.5 · p. 32

3.6 Paired Slide-Level Analysis on the Held-Out TLS Cohort

• FTU-Seek achieved higher slide-level Dice than matched random TopK sampling across all budgets, with paired improvements of 2.58–8.96 percentage points and confidence intervals excluding zero.At Top1000, improvements were +1.57 percentage points versus all-tissue training and +5.86 versus random-balanced training; Top300 had the largest matched-random gain, whereas Top1000 achieved the highest absolute Dice.Source: §3.6 · p. 33

3.7 Qualitative Analysis

• Across TLS, gland, and blood-vessel cohorts, FTU-Seek produced segmentations more consistent with ground truth than random negative sampling, especially for sparse targets, ambiguous background, and complex boundaries.Figure 9 presents the whole-slide and magnified-region comparisons.Source: §3.7 · p. 34, Figure 9 · p. 34
• For TLS, FTU-Seek suppressed scattered background false positives while retaining principal TLS foci, indicating improved background discrimination without substantial sensitivity loss.Random negative sampling generated additional isolated predictions in background regions, particularly in enlarged regions.Source: §3.7 · p. 34
• For gland and blood-vessel segmentation, FTU-Seek yielded cleaner maps by reducing fragmented glandular responses and vessel-like false positives while preserving principal annotated structures.The qualitative benefit was attributed to prioritizing morphologically confusing negatives, including non-glandular tissue, clefts, elongated stromal structures, and other linear components.Source: §3.7 · pp. 34–35

3.8 Proof-of-Concept Downstream Phenotypic Analyses Enabled by FTU-Seek Segmentation

• FTU-Seek masks enabled exploratory TCGA phenotyping rather than clinical prediction validation, with TLS, vascular, and glandular features quantified across READ, ESCA, STAD, LIHC, and PRAD cohorts.The analyses were intended to support downstream tissue characterization and hypothesis generation.Source: §3.8 · pp. 35–37
• In TLS analyses, density ranged from 0.0048 in READ to 0.0116 in STAD, while READ TLS density and nearest-neighbor clustering showed nominal survival associations that did not meet FDR significance.READ had smaller median TLS area, STAD had higher solidity, and Kaplan–Meier groupings used data-derived cut points solely for illustration.Source: §3.8 · pp. 35–36, Figure 10 · p. 37
• In LIHC, higher blood-vessel density was associated with mortality and hotspot top mean BV density with microvascular invasion, while PRAD gland phenotypes differed by recurrence, Gleason grade, stage, and surgical-margin status.The vascular associations were exploratory and included time-varying hazards and nonlinear MVI relationships; adverse PRAD characteristics generally showed lower gland densities or areas and higher eccentricity.Source: §3.8 · pp. 36, 38, Figure 11 · p. 39, Figure 12 · p. 40

Methodological implications

• FTU-Seek concentrates training on classifier-ranked, target-like negatives, improving training-data efficiency without necessarily reducing peak GPU-memory usage or requiring additional hardware.Unlike online hard-example mining, negatives are selected before segmentation training from frozen foundation-model representations, enabling reproducible comparisons under fixed sampling budgets.Source: §4 · pp. 41–42

Task-dependent effects

• Benefits depend on target sparsity and ambiguity: TLSs gained most from small hard-negative pools, vessels benefited from broader coverage, and glands were comparatively sampling-insensitive.Sparse, imbalanced tasks contain few informative negatives, whereas diverse vessel-like morphologies require larger negative budgets and abundant, clearly separated glands show smaller gains.Source: §4 · p. 42

Limitations and future work

• Classifier misranking can omit informative negatives or waste TopK slots, while fixed budgets and allocation limit adaptability; uncertainty-aware, iterative, boundary-aware, and proportional strategies remain future directions.The study also lacks direct wall-clock, utilization, memory, and energy measurements, and its internal cohorts were small and single-institution.Source: §4 · pp. 42–44

Foundation-model role

• Frozen pathology foundation-model representations support both target-like negative identification and transferable multi-scale segmentation features while reducing trainable parameters and dependence on small annotated datasets.This extends foundation-model use beyond conventional feature extraction toward intelligent construction of segmentation training data.Source: §4 · pp. 41, 43

Downstream phenotyping

• FTU-Seek-derived masks yielded exploratory phenotypes spanning TLS organization, vascular hotspots, and gland architecture, with associations observed across tumor types, overall survival, microvascular invasion, and aggressive prostate tumors.These analyses suggest that the phenotypes retain information beyond segmentation masks, but they were hypothesis-generating rather than validated biomarkers.Source: §4 · pp. 43–45

Validation and limitations

• On 30 held-out same-institution WSIs, FTU-Seek achieved higher slide-level Dice than matched random TopK at all four negative-patch budgets.Positive paired differences and confidence intervals excluding zero support robustness within the available validation setting, but not multi-institutional generalization.Source: §4 · p. 44

Conclusions

• FTU-Seek identifies target-like negative tissue before segmentation training, reducing redundant background sampling and training workload while maintaining competitive segmentation performance.Benefits are particularly clear for sparse and imbalanced FTUs.Source: §5 · p. 45
• Clinical applicability remains unestablished because the study has few annotated WSIs, lacks direct computational benchmarking, and requires prospective multi-institutional validation.The conclusions call for independent prospective cohorts before clinical use can be established.Source: §5 · p. 45
• The static TopK strategy depends on classifier-based ranking and task-specific sampling requirements, motivating investigation of more adaptive sampling strategies.These limitations qualify the generality of the current hard-negative selection approach.Source: §5 · p. 45