PaperlayerView exact revision on arXiv

Source-linked AI summary

SlideBank: A Persistent Hierarchical Evidence Bank for Consistent Whole-Slide Reasoning

Beidi Zhao, Gexin Huang, Ciro Zhang, Anqi Li, Yusheng Tan, Chen Zhou, Gang Wang, Zu-hua Gao, Xiaoxiao Li

arXiv:2609.00342v1cs.AI

TL;DR

WSI reasoning must locate sparse, heterogeneous morphology while preserving its connection to visual evidence. SlideBank builds a training-free persistent evidence bank, routes questions to grounded multiscale signals, and combines linked evidence across levels. It achieves competitive benchmark performance, improved repeated-query stability, and efficiency gains through evidence reuse.

  • Problem

    WSI evidence is sparse, heterogeneous, and distributed across gigapixel images, while existing representations are difficult to access semantically without losing visual traceability.

  • Method

    SlideBank performs question-independent coarse-to-fine exploration, converts selected views into morphological observations, grounds pathology signals to regions and coordinates, and retrieves evidence through concept-to-signal routing.

  • Results

    SlideBank shows competitive performance on WSI-VQA and SlideBench-BCNB, including 52.77% on WSI-VQA with Patho-R1 and 50.92% average accuracy on SlideBench-BCNB with Quilt-LLaVA.

  • Takeaways & Limitations

    Structured evidence organization supports reusable, traceable WSI reasoning with improved stability across repeated queries and efficiency gains from evidence reuse.

Abstract

from arXiv · show

Whole-slide images (WSIs) are challenging for vision-language reasoning because diagnostically relevant morphology is sparse, heterogeneous, and distributed across gigapixel-scale images and multiple spatial resolutions. Existing WSI models and pathology agents can aggregate slide features or actively acquire evidence, but the information retained after exploration is often difficult to access semantically while preserving its connection to the original visual evidence. We introduce SlideBank, a training-free framework that represents each WSI as a persistent, concept-indexed, and spatially grounded evidence bank. SlideBank performs question-independent coarse-to-fine exploration to identify informative regions and multi-scale views, converts them into explicit morphological observations, and grounds pathology signals to their supporting patches and WSI coordinates. At inference time, questions are routed to relevant signals and evidence scales, and the linked global, regional, and patch evidence is integrated through confidence-based cross-level consensus. Experiments on WSI-VQA and SlideBench-BCNB show that with Patho-R1, SlideBank reaches 52.77% on WSI-VQA and with Quilt-LLaVA, it reaches 50.92% average accuracy on SlideBench-BCNB, while structured signal-guided retrieval consistently outperforms random evidence sampling. Reusing the same bank across repeated queries further achieves over 99% rephrasing consistency and substantially reduces amortized inference cost through persistent evidence reuse.

1 Introduction

SlideBank addresses the difficulty of locating and semantically organizing sparse, heterogeneous WSI evidence by building a persistent, spatially grounded representation. It separates slide exploration from question answering so diverse queries can retrieve traceable multi-scale evidence.

  • Motivation: WSIs contain billions of pixels, while decisive findings may occupy tiny, dispersed, heterogeneous regions across architectural and cellular scales.Thumbnail compression or uniform patch sampling can discard diagnostically relevant evidence.
  • Motivation: Existing methods aggregate latent slide features or acquire evidence actively, but often retain flat observations or transient states that are difficult to access across questions.The paper identifies explicit semantic organization of already acquired evidence as comparatively underexplored.
  • Approach: SlideBank converts each WSI into a concept-indexed, spatially grounded hierarchical evidence bank without task-specific training.Question-independent coarse-to-fine exploration selects informative anchors and complementary architecture- and cell-level views, which are converted into explicit morphological observations.
  • Approach: Questions are routed to relevant pathology concepts and signals, then answered using linked global-, anchor-, and patch-level evidence combined through confidence-based cross-level consensus.The representation preserves links from pathology findings to supporting regions, multi-scale views, and WSI coordinates.
  • Evaluation: SlideBank is evaluated on two public WSI question-answering benchmarks for accuracy, efficiency, and cross-turn consistency, including a controlled comparison with random evidence sampling.The contribution framing emphasizes concept-to-signal retrieval as the comparison mechanism.

2 Related Work

Prior WSI systems combine visual-language modeling, active navigation, and hierarchical representations, but usually retain evidence as latent features or transient states. SlideBank instead materializes multiscale visual fields, descriptions, pathology signals, and coordinates in a reusable, inspectable bank.

  • Vision-language models and agentic reasoning: Early pathology vision-language models focus on image-text alignment or instruction following from localized histology fields, while WSI systems aggregate features because native-resolution slides exceed model limits.Examples include CONCH, Quilt-LLaVA, and PathChat.
  • Vision-language models and agentic reasoning: Recent pathology agents use planning, region selection, and multiscale navigation to acquire diagnostically relevant evidence, including question-independent scans and online memory.The related work includes HistoSelect, BEACON, AdaptivePath, and PathNavigate.
  • Persistent evidence representation: Existing methods primarily represent acquired evidence as model-internal features or transient reasoning states, whereas SlideBank materializes image fields, morphological descriptions, signals, and WSI coordinates.This makes the evidence bank independently inspectable and retrievable across multiple questions about the same slide.
  • Persistent evidence representation: Hierarchical modeling offers nested visual representations, but SlideBank adds an interpretable concept-to-signal routing interface for question-side categories and localized image-derived findings.The bank stores signal status and links to anchors, multiscale views, descriptions, and coordinates.

3 Method

SlideBank separates one-time WSI exploration from question-conditioned reasoning by constructing a persistent hierarchical bank, routing questions to grounded signals, and integrating evidence across global, anchor, and patch levels. The same bank is reused across later questions to avoid repeated exploration and evidence drift.

  • Overview: SlideBank separates slide exploration from question answering because gigapixel WSIs exceed current VLM image limits and relevant findings occupy small slide regions.Its three components are bank construction, evidence-augmented reasoning, and persistent reuse across queries.
  • Evidence bank construction: Question-independent coarse-to-fine exploration builds a slide-specific bank containing global context, retained anchors, multiscale views, grounded signals, and WSI locations.Global thumbnails support contextual cues and coordinate normalization, while category-aware sampling allocates more samples to suspicious regions.
  • Evidence bank construction: Selected patches receive morphology descriptions and confidence or quality-control metadata, which are aggregated into anchor observations while retaining patch images, magnifications, descriptions, and coordinates.Low-confidence or invalid observations are discarded, and anchor descriptions summarize complementary morphology across magnifications.
  • Signal grounding and retrieval: Free-form observations are normalized into localized pathology signals with statuses, confidence, and links to supporting views, enabling concept-based retrieval despite varied wording.The bank associates coarse pathology concepts with relevant signal sets and stores records across anchors.
  • Signal grounding and retrieval: A deterministic question router selects concepts and target signals, prioritizes strongly supported records, and retrieves question-specific global, anchor, patch, and signal evidence.Present records are preferred over uncertain or absent records, with confidence ranking among records sharing a status.
  • Evidence-augmented reasoning: Global, anchor, and patch branches independently score answer options, aggregate local candidates, and combine predictions through level-weighted consensus with greater emphasis on localized morphology.The resulting candidates and spatial links preserve a trace from the final answer to supporting WSI regions.
  • Persistent reuse: SlideBank constructs the evidence bank once and reuses it across turns, routing each current question independently while carrying only bounded dialogue context.This avoids repeated WSI exploration and ensures identical retrieved evidence for a question asked alone or within a conversation.

4 Experiments

Experiments evaluate SlideBank on two WSI benchmarks, testing standard question answering, hierarchical evidence, signal-guided retrieval, concept sensitivity, qualitative traceability, repeated-query stability, and amortized efficiency. SlideBank improves benchmark performance, maintains high rephrasing consistency, and reduces per-query runtime as evidence is reused.

  • Experimental Setup: SlideBank is evaluated on WSI-VQA and SlideBench-BCNB using standard accuracy and repeated-query metrics.The evaluation compares zero-shot VLMs, WSI-trained models, and agentic systems under shared question sets and answer spaces.
  • Standard WSI Question Answering: 52.77% on WSI-VQA and 50.92% average accuracy on SlideBench-BCNB demonstrate strong standard reasoning performance.With Quilt-LLaVA, tumor typing reaches 89.82% on SlideBench-BCNB; with Patho-R1, WSI-VQA performance exceeds its thumbnail-only counterpart by 8.49 percentage points.
  • Effect of Evidence Organization and Access: 52.77% on WSI-VQA and 50.36% on SlideBench-BCNB are achieved by combining global, anchor, and patch evidence.Anchor and patch evidence outperform global-only evidence in the reported ablation.
  • Effect of Evidence Organization and Access: Signal-guided retrieval improves accuracy from 50.13% to 52.77% on WSI-VQA and from 49.62% to 50.36% on SlideBench-BCNB versus random sampling.The comparison keeps the constructed evidence bank and other settings unchanged while replacing structured retrieval with 32 randomly sampled patches.
  • Sensitivity and Qualitative Example: Accuracy peaks at 52.8% with three pathology concepts, after rising from 49.9% with one concept and declining as more concepts are added.Performance remains above random sampling across two to six concepts, motivating three concepts as the default.
  • Sensitivity and Qualitative Example: SlideBank preserves an explicit trace from grade-related signals through linked regions and multi-magnification patches to the final prediction.The qualitative example illustrates how routed signals connect predictions to supporting visual evidence.
  • Persistent Evidence Reuse Across Queries: 99.47% and 99.21% consistency across semantically equivalent formulations accompanies accuracy drops of 1.38 and 3.83 percentage points for Quilt-LLaVA and Patho-R1.The evidence bank is reused across question sequences averaging 29 questions per WSI.
  • Persistent Evidence Reuse Across Queries: Average runtime decreases from 94.3 s for a single query to 5.9 s per query over repeated questions, approaching 2.73 s online reasoning cost.The one-time construction cost is progressively amortized as more questions are asked about the same slide.

5 Conclusion

SlideBank is a training-free evidence-bank framework that organizes persistent, concept-indexed, spatially grounded pathology evidence for whole-slide reasoning. Its exploration, morphology description, and evidence reuse support interpretable retrieval across scales and repeated queries.

  • SlideBank converts each WSI into a persistent, concept-indexed, and spatially grounded evidence bank.
  • The framework separates slide exploration from question answering and retrieves question-relevant evidence through explicit links to supporting regions and patches.
  • Evaluation: Structured evidence organization yields competitive performance, improved repeated-query stability, and efficiency gains from evidence reuse.Figure 4 compares amortized runtime under evidence-bank reuse and runtime with consistency among high-consistency methods; Table 3 reports rephrase consistency in multi-turn WSI-VQA.
  • Evidence exploration: Coarse-to-fine exploration selects anchors, fine-detail patches, and independently chosen low-magnification context views.Anchor localization and fine-detail selection use grid-based scoring, while architectural context selection targets transition fronts, heterogeneous structure, necrosis interfaces, and ductal layout.
  • Evidence representation: Multi-scale view descriptions store observable morphology with confidence and quality-control information while excluding diagnostic conclusions, grading, and treatment statements.

B Pathology Concept and Signal Ontology

SlideBank uses a pathology ontology that connects clinical question concepts to locally assessable morphologic signals. This mapping makes retrieval interpretable by tracing answers to signals and their supporting image regions.

  • A concept represents the clinical intent of a question, whereas a signal represents an atomic morphologic finding assessable in a local image region.
  • Questions are assigned to concepts, which select relevant signal sets used to retrieve multi-scale evidence for answer generation.
  • The concept-to-signal pipeline allows each answer to be traced from the question concept through morphologic signals to supporting image regions.

B.1 Concept Vocabulary

The ontology covers morphology-centered breast-pathology questions alongside metadata-oriented and morphology-based proxy tasks. Its many-to-many concept-signal mapping supports shared findings and fallback to broader evidence when dedicated signals are unavailable.

  • The concept vocabulary covers common morphology-centered breast-pathology questions, metadata-oriented tasks, and morphology-based proxy tasks.
  • Concepts and signals are linked many-to-many, so one concept may depend on several findings and one finding may support multiple concepts.
  • Concepts without dedicated signals are answered using global and multi-scale regional evidence.
  • Signals are affirmative, locally assessable morphologic findings rather than diagnostic labels, and each is linked to the regions where it is observed.
  • Each evidence region records a signal as present, unknown, or absent with confidence and supporting views, while retrieval prioritizes relevant linked evidence.

B.3 Multi-scale Evidence Organization

SlideBank organizes evidence across global, anchor, and patch scales while preserving each signal’s spatial origin and morphologic interpretation. This structure supports traceable retrieval and answer generation.

  • Global views capture slide-level context, anchor regions localize informative areas, and patch views preserve detailed architectural or cellular morphology.
  • Each signal links to supporting anchor and patch views, preserving both spatial origin and morphologic interpretation for downstream reasoning.
  • The organized evidence lets answers be traced back to the corresponding signal and image region.

C.1 Dataset Statistics

The evaluation uses WSI-VQA and SlideBench-BCNB, with question counts, slide counts, task coverage, and answer-option formats specified for each dataset.

  • WSI-VQA contains 388 four-option questions associated with 85 WSIs.Questions cover diagnosis, morphology, grade, size, receptor status, margins, and staging.
  • SlideBench-BCNB contains 1,058 breast-cancer WSIs and 3,042 questions across tumor type, histological grading, and molecular subtype.The task totals are 1,058 tumor-type, 926 grading, and 1,058 molecular-subtype questions.
  • Tumor-type and grading questions use three answer candidates, whereas molecular-subtype questions use four.

C.2 Baselines

The baselines span general-purpose and pathology-tuned VLMs, WSI-specific multimodal models, multi-agent collaboration, and training-free slide navigation. Evaluation protocols specify shared questions and answer candidates, while single-image VLM baselines receive only a 1024×1024 thumbnail.

  • All methods are evaluated on the same questions and answer candidates, following each baseline’s released inference protocol unless otherwise specified.
  • The VLM baselines include general-purpose models, medical- or pathology-tuned models, and WSI-specific multimodal models.The listed models include Qwen2.5-VL-7B, Qwen3-VL-8B, Qwen3.5-4B, LLaVA-Med, Quilt-LLaVA, PathGen-LLaVA, Patho-R1, WSI-LLaVA, and TITAN.
  • Single-image VLM baselines receive one 1024×1024 WSI thumbnail and the multiple-choice question without additional high-resolution fields.
  • MedAgents tests inference-time multi-agent collaboration through specialist discussion and consensus using GPT-4o with the WSI thumbnail and question.
  • PathAgent is a training-free agentic baseline that iteratively navigates slides, extracts morphology at selected regions and magnifications, and integrates observations into an interpretable answer.

D Additional Implementation Details

Implementation details fix the hierarchical fusion weights, construct one persistent evidence bank per slide for multi-turn evaluation, and use deterministic answer parsing without model-based adjudication.

  • Fusion weights remain fixed across datasets and backbones at (𝜆𝐺, 𝜆𝐴, 𝜆𝑃) = (0.2, 0.4, 0.4), without dataset-specific tuning.
  • Multi-turn evaluation uses 382 source questions and 2,292 turns from 80 WSIs, with five rephrasings per question and shuffled answer options.The evidence bank is constructed once per slide and remains fixed throughout each conversation.
  • Answer parsing deterministically extracts or matches a valid option, and unresolved outputs count as incorrect without additional language-model adjudication.
Loading 2609.00342v1…