PaperlayerView exact revision on arXiv

Source-linked AI summary

HACT-Net: A Hierarchical Cell-to-Tissue Graph Neural Network for Histopathological Image Classification

Pushpak Pati, Guillaume Jaume, Lauren Alisha Fernandes, Antonio Foncubierta, Florinda Feroce, Anna Maria Anniciello, Giosue Scognamiglio, Nadia Brancati, Daniel Riccio, Maurizio Do Bonito, Giuseppe De Pietro, Gerardo Botti, Orcun Goksel, Jean-Philippe Thiran, Maria Frucci, Maria Gabrani

arXiv:2007.00584v1cs.CV

TL;DR

The paper addresses the limited representation of histopathological tissue by cell-only or pixel-based approaches. It introduces HACT, which links cell- and tissue-level graphs through spatial hierarchies, and HACT-Net for breast cancer subtype prediction. HACT-Net outperformed the evaluated CNN and GNN approaches, while offering entity-based multi-level tissue modeling aligned with pathological assessment.

Abstract

from arXiv · show

Cancer diagnosis, prognosis, and therapeutic response prediction are heavily influenced by the relationship between the histopathological structures and the function of the tissue. Recent approaches acknowledging the structure-function relationship, have linked the structural and spatial patterns of cell organization in tissue via cell-graphs to tumor grades. Though cell organization is imperative, it is insufficient to entirely represent the histopathological structure. We propose a novel hierarchical cell-to-tissue-graph (HACT) representation to improve the structural depiction of the tissue. It consists of a low-level cell-graph, capturing cell morphology and interactions, a high-level tissue-graph, capturing morphology and spatial distribution of tissue parts, and cells-to-tissue hierarchies, encoding the relative spatial distribution of the cells with respect to the tissue distribution. Further, a hierarchical graph neural network (HACT-Net) is proposed to efficiently map the HACT representations to histopathological breast cancer subtypes. We assess the methodology on a large set of annotated tissue regions of interest from H\&E stained breast carcinoma whole-slides. Upon evaluation, the proposed method outperformed recent convolutional neural network and graph neural network approaches for breast cancer multi-class subtyping. The proposed entity-based topological analysis is more inline with the pathological diagnostic procedure of the tissue. It provides more command over the tissue modelling, therefore encourages the further inclusion of pathological priors into task-specific tissue representation.

I. INTRODUCTION

Breast cancer subtyping is clinically important but difficult because lesions carry different invasive risks and pathologist assessments can vary. Existing pixel-based and cell-graph approaches miss either biological context or tissue-level hierarchy, motivating HACT and HACT-Net.

  • Atypia and ductal carcinoma in situ carry higher risks of transition to invasive carcinoma than benign lesions, making accurate class discrimination important for treatment planning.Source: §I · p. 1
  • Pathologists assess tissue-part morphology and spatial distribution before evaluating cellular phenotype, topology, and tissue distribution within regions of interest.Source: §I · p. 1
  • Patch-based CNNs face a context-resolution trade-off, assume patches share the region label, and do not directly encode biological entities or pathological context.Source: §I · p. 1
  • Cell-graphs represent cellular morphology and topology but discard tissue distributions such as stroma, tumor microenvironment, and lumen structure.Source: §I · pp. 1–2
  • HACT combines a cell-graph, tissue-graph, and cell-to-tissue hierarchy, while HACT-Net learns from these levels to predict cancer types using local and global structural attributes.Source: §I · p. 2

II. METHODS

The method builds hierarchical tissue representations from cell-level graphs and processes them with graph neural networks. The representation follows a pipeline from stain-normalized regions of interest to HACT construction and classification.

II. METHODS

  • HACT-Net combines cell- and tissue-level graph processing, merging learned cell embeddings with corresponding tissue embeddings through cell-to-tissue hierarchies.Source: §II · p. 2

A. Representation

  • An undirected graph is represented by nodes, edges, an adjacency matrix, and node feature vectors that define local neighborhoods.Source: A. Representation · p. 2
  • Cell-graph nodes represent detected nuclei, whose shape, texture, and normalized spatial-location features initialize cellular node embeddings.Source: 1) Cell-graph (CG) · p. 2

1) Cell-graph (CG):

The cell-graph models nuclei as nodes with morphology, texture, and location features, connecting nearby cells to approximate cellular interactions.

2) Cell-graph (CG):

  • Each nucleus is represented by 18 handcrafted shape, texture, and normalized spatial-location features used as initial cell-graph node embeddings.Source: 1) Cell-graph (CG) · p. 2
  • Cell-graph edges connect spatially nearby nuclei using k-nearest neighbors followed by distance pruning, with k = 5 and dmin = 50 pixels.Source: 1) Cell-graph (CG) · p. 2

2) Tissue-graph (TG):

The tissue-graph represents spatially organized tissue parts using super-pixels and adjacency relationships. Its construction captures local tissue attributes and tumor–stroma context within the proposed methodology.

2) Tissue-graph (TG):

  • Tissue parts are over-segmented and hierarchically merged into homogeneous super-pixels using texture and color similarity, producing 26-dimensional representations.Source: 2) Tissue-graph (TG) · p. 2
  • The methodology block diagram links preprocessing, HACT representation of a region of interest, and HACT-Net classification.Source: Figure 1 · p. 3
  • A region adjacency graph connects adjacent super-pixels, encoding tissue-part interactions and tumor–stroma microenvironment context.Source: 2) Tissue-graph (TG) · pp. 2–3

3) HierArchical-Cell-to-Tissue (HACT) representation:

HACT jointly represents cell- and tissue-level graphs with a spatial assignment matrix that maps cells to tissue regions.

3) HierArchical-Cell-to-Tissue (HACT) representation:

  • HACT combines a cell graph, a tissue graph, and a cell-to-tissue assignment matrix into one hierarchical representation.The matrix maps each cell node to the super-pixel-based tissue node containing it, enabling pooling from the finer cell graph to the coarser tissue graph.Source: 3) HierArchical-Cell-to-Tissue (HACT)… · p. 3

B. HACT Graph Neural Networks (HACT-Net)

HACT-Net processes cell and tissue graphs hierarchically, transferring contextualized cell information into tissue nodes before producing a graph-level classification embedding.

B. HACT Graph Neural Networks (HACT-Net)

  • HACT-Net applies GIN layers to cell and tissue graphs, combines mapped cell embeddings with tissue-node features, and predicts cancer subtypes from a graph-level embedding.The learned graph-level representation concatenates aggregated tissue-node embeddings from all tissue-level layers before MLP classification.Source: B. HACT Graph Neural Networks (HACT-Net) · pp. 3–4
  • Figure 2 visualizes cell-graph nodes and edges, tissue-graph nodes and edges, and blue cell-to-tissue hierarchies in HACT.The visualization omits some hierarchies for clarity.Source: Figure 2 · p. 3
  • Cell-level GIN updates produce contextualized cell-node embeddings that are used as additional features for corresponding tissue nodes.The mapping set S(v) contains the cell-graph nodes assigned to tissue node v.Source: B. HACT Graph Neural Networks (HACT-Net) · p. 3
  • Tissue-level GIN propagation encodes cellular and tissue information within T_TG hops of each tissue node.Source: B. HACT Graph Neural Networks (HACT-Net) · p. 3

A. Dataset

BRACS is a large, pathologist-annotated breast carcinoma dataset designed to evaluate subtype classification under realistic slide-level separation and tissue variability.

A. Dataset

  • BRACS contains 2,080 annotated regions of interest from 106 H&E-stained breast carcinoma whole-slide images across five diagnostic categories.The categories are Normal, Benign, Atypical, ductal carcinoma in situ, and Invasive, with annotations established by consensus of three pathologists.Source: A. Dataset · p. 4
  • BRACS is more than four times larger than BACH and includes challenging atypical hyperplasia, variable ROI dimensions, artifacts, and single or multiple glandular regions.Source: A. Dataset · p. 4
  • Train, validation, and test regions are split at the WSI level across four random splits to prevent correlated regions from the same slide crossing partitions.The paper states that image-level splitting can overestimate evaluation results and reduce generalizability to unseen data.Source: A. Dataset · p. 4

B. Implementation

The implementation benchmarks HACT-Net against graph and convolutional baselines using matched classifiers, weighted F1-based model selection, and multi-scale CNN comparisons.

B. Implementation

  • HACT-Net is compared with standalone CG-GNN, TG-GNN, Concat-GNN, and single- and multi-scale CNN approaches.Concat-GNN concatenates cell- and tissue-graph embeddings, while the multi-scale CNNs use late fusion over specified magnification combinations.Source: B. Implementation · p. 4
  • The dataset was pending approval for release to the research community, limiting immediate public availability.Source: B. Implementation · p. 4
  • Graph models use four GIN layers with 32-dimensional hidden states, while classifiers use two-layer MLPs with 64 hidden neurons and five output classes.The graph models are trained with cross-entropy loss, Adam optimization, batch size 16, and initial learning rate 10^-3.Source: B. Implementation · p. 4
  • Weighted F1-score measures classification performance, and the validation-best model is selected as the final model for each approach.Source: B. Implementation · p. 4

C. Discussion

HACT-Net combines cell-level and tissue-level topological information through a hierarchical representation, improving breast cancer subtype classification and aligning analysis with pathological structure.

C. Discussion

  • Tissue graphs encode lumen, apical snouts, necrosis, and tumor-associated stroma that cell graphs cannot access.These tissue components provide high-level microenvironmental information alongside cellular morphology and topology.Source: C. Discussion · p. 4, Figure 3 · p. 5
  • HACT-Net outperforms TG-GNN, CG-GNN, and Concat-GNN, indicating that complementary cell/tissue information and local cell-to-tissue modeling improve structure-function mapping.The comparison is reported in Table II and supports hierarchical fusion over graph-level concatenation.Source: C. Discussion · p. 6, Table II · p. 6
  • Atypical cases are hardest to classify because of high intra-class variability and ambiguity with benign and DCIS, while invasive cases are best detected.Tissue information improves atypical discrimination, whereas nuclei information is more informative for Normal, Benign, and DCIS.Source: C. Discussion · p. 6
  • HACT-Net uses stromal microenvironment, nuclei, and tissue-distribution properties to improve performance across almost all subtypes without bias toward one class.The class-wise gains over CG-GNN, TG-GNN, and Concat-GNN support the value of hierarchical learning.Source: C. Discussion · p. 6
  • Entity-based hierarchical representations may support interpretability beyond cell graphs by identifying nuclei, tissue parts, and cell-to-tissue interactions.This follows the multilevel structure of pathological assessment more closely than prior cell-graph-only analyses.Source: C. Discussion · pp. 6–7

IV. CONCLUSION

The paper concludes that HACT representation and HACT-Net strengthen histopathological structure-to-function mapping through multilevel topological modeling and hierarchical learning.

IV. CONCLUSION

  • HACT representation combines cell graphs, tissue graphs, and cell-to-tissue hierarchies, while HACT-Net learns from this structure for breast cancer subtyping.The representation provides control over tissue encoding and supports pathological context in the model.Source: §IV · p. 7
  • The method achieves superior classification performance against state-of-the-art CNNs and GNNs on breast cancer subtyping.The conclusion attributes this result to enriched multilevel topology and hierarchical learning.Source: §IV · p. 7
  • HACT representations can scale to differently sized regions of interest while incorporating local and global context.The entity-based representation also supports further inclusion of pathological priors.Source: §IV · p. 7
  • Hierarchical modeling may enable digital-pathology interpretability methods to analyze tissue organization beyond cell graphs.The conclusion points toward interpreting the hierarchical nature of tissue.Source: §IV · p. 7
Loading 2007.00584v1…