PaperlayerView exact revision on arXiv

Source-linked AI summary

TAN: A Size- and Form-Adaptive Masked-Convolution Encoder for Tissue Graphs

Jan Herschel, Anselm von Gladiss

arXiv:2610.23475v1cs.CV

TL;DR

Irregular tissue superpixels can be difficult to encode without rectangular preprocessing allowing background or fill values into node features. TAN processes each superpixel at its native shape while excluding background from every operation. On BACH, its accuracy is not significantly different from UNI’s with far fewer parameters and FLOPs, while on BRACS it is close to ResNet18 but trails the histopathology foundation models.

  • Problem

    Deep encoders for irregular superpixels rely on rectangular crops and padding, resizing, or masking, which can distort scale or mix surrounding content into node features.

    Source: §1 · p. 1

  • Method

    TAN encodes each superpixel at its native shape using masked operations that exclude background throughout the encoder.

    Source: §1 · p. 1, §3.3 · p. 2

  • Results

    On BACH, TAN was not significantly different from UNI and used approximately 740× fewer parameters and 1,190× fewer FLOPs; on BRACS, it was close to ResNet18 but behind UNI and CTransPath.

    Source: §4.1 · pp. 3–4, §6 · p. 4, §4.2 · p. 4

  • Takeaways & Limitations

    TAN’s embeddings are exactly invariant to fill, while an unmasked twin reaches similar accuracy at more than twice the FLOPs.

    Source: §6 · p. 4

    Generality is limited by BRACS’s single split and class-skewed filter, and TAN’s embeddings depend on batch padding through floor pooling.

    Source: §5 · p. 4

Abstract

from arXiv · show

Tissue graphs represent histology images as graphs over tissue-conforming superpixels, which keeps homogeneous tissue regions intact instead of cutting them into rectangular patches. To embed superpixels, current encoders crop them to bounding boxes and then pad, resize, or mask them, so background or fill values can leak into the node features. This matters in practice, and especially in medical imaging, where a prediction should depend on the tissue and not on an arbitrary preprocessing choice: on BACH, changing only the fill of the masked region shifts the accuracy of frozen encoders by up to 6 percentage points, larger than the differences among the four best encoders under mean fill, and the preferred fill differs between datasets, so it cannot be fixed once and for all. We present TAN (Tissue-Aware Network), a compact convolutional encoder (0.41M parameters) that processes each superpixel at its native shape and extent and excludes background from every operation, which makes its embeddings exactly invariant to the fill by construction. Under identical masks and an identical graph classifier, TAN's accuracy on BACH is not significantly different from that of the 303M-parameter foundation model UNI (81.1% vs. 80.7-82.6%, depending on the fill), with ~740x fewer parameters, ~1,190x fewer FLOPs, and no pretraining. On BRACS, TAN is close to a ResNet18 trained under the same protocol and trails the histopathology foundation models. An ablation with TAN's architecture but standard processing of fill-padded crops shows that masking keeps accuracy similar (nominally higher on BACH, lower on BRACS, within noise) at less than half the FLOPs.

1. INTRODUCTION

Tissue graphs capture how tissue regions are arranged and connected, but encoding irregular superpixels remains challenging. TAN addresses this by processing each superpixel at its native shape while excluding background from every operation.

1. INTRODUCTION

  • Tissue graphs represent tissue regions as nodes and their spatial relationships as edges, capturing local microenvironments and global tissue architecture.Whole-slide images are often processed as rectangular patches with MIL, which treats patches as independent and ignores their locations.Source: §1 · p. 1
  • Irregular superpixels are often forced into rectangular grids through cropping, resizing, padding, or masking, which can distort scale or mix surrounding content into node features.Standard convolutions over masked regions can also let substitute fill values introduce artifacts.Source: §1 · p. 1
  • TAN processes each superpixel at its native shape and extent, excluding background from every operation.The study compares masked processing with an identical-architecture unmasked twin, a scratch-trained ResNet18, and frozen pretrained encoders on BACH and BRACS.Source: §1 · p. 1

2. RELATED WORK

Prior tissue-graph node encoders either use handcrafted descriptors or deep features derived from fixed rectangular patches. Existing masked-convolution approaches differ in how they handle invalid inputs and whether they constrain outputs to fully in-region windows.

2. RELATED WORK

  • Tissue-graph node features have used handcrafted color and texture descriptors or deep features from fixed-size patches, neither of which follows an irregular region’s shape.Pathology foundation models also take rectangular inputs, requiring a fill around an irregular superpixel.Source: §2 · p. 1
  • Partial convolution renormalizes over valid pixels but allows outputs from windows with a single valid pixel, expanding the valid mask with depth; gated convolution learns rather than guarantees exclusion.Submanifold sparse convolution keeps active sites fixed but treats inactive inputs as zero.Source: §2 · pp. 1–2
  • None of these masked-convolution approaches restricts outputs to windows lying entirely inside the tissue region.Submanifold sparse convolution also uses zero-valued inactive inputs at region boundaries.Source: §2 · pp. 1–2

3. MATERIALS AND METHODS

The study evaluates superpixel encoders on BACH and BRACS using tissue graphs and a shared downstream graph-classification protocol. TAN applies masked operations throughout its encoder, while comparison encoders and statistical tests provide controls for processing and evaluation.

3.1. Datasets

  • BACH uses 400 H&E images in four balanced classes with 15 repeated cross-validation folds, while BRACS uses a filtered official split for seven-class lesion subtyping.BRACS retains 2,584 training, 221 validation, and 321 test images; its class-skewed filtering motivates reporting macro-F1 alongside accuracy.Source: §3.1 · p. 2

3.2. Superpixel Graph Construction

  • SLIC superpixels form graph nodes, with undirected edges between regions sharing a pixel border; each encoder receives only its own superpixel’s masked content.Standard-convolution encoders use mean or white fill, whereas TAN receives the raw crop and mask without a fill value.Source: §3.2 · p. 2

3.3. TAN: Tissue-Aware Network

  • TAN uses four stages of masked convolution, mask-aware normalization, activation, and masked pooling, followed by attention pooling to produce a 1024-dimensional node embedding.Only outputs whose full 3×3 neighborhood lies within the mask carry data-dependent image content; the mask is downsampled alongside features.Source: §3.3 · p. 2, Figure 1 · p. 2

3.4. Comparison Encoders

  • TAN-unmasked matches TAN’s architecture but processes fill-padded pixels with standard operations, while ResNet18-scratch serves as an architecture control.Frozen ImageNet, CTransPath, and UNI encoders provide pretrained comparisons.Source: §3.4 · p. 3

3.5. Downstream Classifier and Protocol

  • A three-layer graph attention network classifies each graph from its node embeddings using mean and max pooling.Frozen backbones are not adapted; only the graph classifier is trained on their embeddings.Source: §3.5 · p. 3

3.6. Statistical Methodology

  • BACH comparisons use a Nadeau–Bengio corrected resampled t-test with Holm–Bonferroni correction, whereas BRACS results are reported as seed means and standard deviations without tests.The BACH test accounts for shared training and test data across repeated cross-validation folds.Source: §3.6 · p. 3

4. RESULTS

On BACH, TAN is not significantly different from UNI under either fill; on BRACS, it is close to ResNet18 but below the histopathology foundation models.

4.1. Results on BACH

  • TAN is numerically first under mean fill and second to UNI under white fill on BACH, with no significant difference from UNI under either condition.The comparison uses corrected p = 1.0; the test has limited power, so non-significance does not establish equivalence.Source: §4.1 · p. 3, Table 1 · p. 4

4.2. Results on BRACS

  • On BRACS, TAN’s accuracy is close to ResNet18-scratch and ImageNet-pretrained ResNet18, but below UNI and CTransPath under both fills.The gaps to UNI and CTransPath are 7.5 and 4.7 pp under mean fill, and 3.6 and 1.7 pp under white fill.Source: Table 2 · p. 4, §4.2 · p. 4

4.1. Results on BACH

  • TAN-unmasked matches TAN on BACH under mean fill and is 1.8 pp lower under white fill; masking does not improve accuracy there.The white-fill difference is not significant after correction (p = 1.0).Source: §4.1 · p. 4
  • TAN uses 0.10 GFLOPs per superpixel versus 0.22 for TAN-unmasked, while TAN-unmasked changes by 1.6 pp across fills.TAN’s embeddings were bit-identical under four backgrounds on 100 BRACS superpixels; frozen ResNet18 embeddings differed by a median of 64% of their norm between white and mean fill.Source: §4.1 · p. 4
  • TAN has ≈740× fewer parameters and uses ≈1,190× fewer FLOPs than UNI; frozen encoders’ BACH accuracies shift by up to 6 pp across fills.For the frozen encoders, fill effects exceed the differences among the four best encoders under mean fill.Source: §4.1 · p. 4

5. LIMITATIONS

The evaluation uses weakly supervised superpixel labels and RoIs rather than complete whole-slide images, and BRACS generality is constrained by its split and superpixel setup.

5. LIMITATIONS

  • Superpixels lack their own labels and tissue-structure annotations; training uses the image class, and experiments use RoIs rather than complete WSIs.The authors note that weak supervision may explain low absolute accuracies, which are not comparable to image-level results.Source: §5 · p. 4
  • BRACS has one class-skewed split, while superpixels were matched in pixels rather than tissue area and their size was not tuned.Further datasets, resolutions, and superpixel-size studies are needed to evaluate generality and identify an appropriate size.Source: §5 · p. 4
  • TAN’s embeddings depend on batch padding through floor pooling; padding to multiples of 16 should remove this dependence.Source: §5 · p. 4

6. CONCLUSION

TAN is a compact encoder that excludes background from its operations, making embeddings invariant to fill; its results are close to different baselines on BACH and BRACS.

6. CONCLUSION

  • TAN has 0.41M parameters and excludes background from every operation, making its embeddings exactly invariant to fill.Source: §6 · p. 4
  • On BACH, TAN’s accuracy is not significantly different from UNI’s, with ≈740× fewer parameters, ≈1,190× fewer FLOPs, and no pretraining.Source: §6 · p. 4
  • On BRACS, TAN is close to ResNet18 but behind UNI and CTransPath; whether pretraining or a better superpixel size closes the gap remains open.Source: §6 · p. 4

Compliance with Ethical Standards

The study analyzes publicly available, de-identified BACH and BRACS human-subject data retrospectively and reports that no ethical approval was sought.

Compliance with Ethical Standards

Related papers

Loading 2610.23475v1…