Source-linked AI summary
Decoupled I/O-Dominant Pipelines for Large-Scale Whole-Slide Image Embedding Extraction
Mayanka Chandrashekar, Xi Zhang, Ethan Seefried, Tirthankar Ghosal, John Gounley, Heidi Hanson
TL;DR
Large-scale WSI embedding extraction is constrained by patch data movement and orchestration, not only model computation. The paper decouples generation, inference, and ingestion into independently scalable stages, then shows that storage constraints dominate beyond moderate concurrency. The resulting persistent vector database supports reuse of embeddings with associated metadata.
Problem
Millions of patches make large-scale WSI embedding extraction an I/O- and orchestration-dominated workload, while existing pipelines tightly couple stages and treat embeddings as transient outputs.
Method
A decoupled pipeline separates patch generation and staging, embarrassingly parallel embedding inference, and sharded vector-database ingestion.
Results
Storage and data movement govern end-to-end performance, with scaling degrading as shared filesystem contention increases despite independent execution.
Takeaways & Limitations
Persistent metadata-coupled embeddings can be reused across downstream analyses without repeatedly accessing raw WSIs.
Abstract
from arXiv · showhide
Whole-slide images (WSIs) are central to computational pathology but are prohibitively large, making patch-based processing the practical unit for foundation model inference. At scale, however, generating and handling massive numbers of patches on quickly introduces significant I/O and orchestration overhead, often dominating end-to-end performance. We present a decoupled, I/O-aware pipeline for large-scale WSI embedding extraction that decomposes the workflow into three stages: (1) patch generation and staging, (2) embarrassingly parallel embedding inference, and (3) sharded vector database ingestion. This design isolates data movement from compute, enabling efficient patch delivery, scalable multi-node inference with minimal communication. The resulting system produces a distributed vector database where embeddings are persistently coupled with rich metadata (e.g., patient, slide, and patch attributes), enabling efficient filtering, retrieval, and downstream reuse. This representation database is compact and reusable for tasks such as retrieval, classification, and few-shot learning, particularly benefiting low-resource environments. We show that decoupling I/O, computation, and ingestion enables high-throughput WSI embedding extraction at scale. By characterizing the scaling envelope, we demonstrate that storage dominates beyond moderate concurrency, reframing WSI embedding extraction as a data-centric systems problem rather than a purely compute-bound workload.
I. INTRODUCTION
Large-scale WSI embedding extraction becomes dominated by patch data movement, storage behavior, and orchestration rather than accelerator throughput. The paper addresses this with a decoupled pipeline and a persistent metadata-aware embedding database.
- Tens-of-gigapixel WSIs require patch-based decomposition for practical foundation-model inference.
- At large scale, generating, moving, and processing millions of patches can shift runtime bottlenecks from accelerator throughput toward I/O and pipeline design.
- Existing tightly coupled workflows interleave patch generation, inference, and output handling, increasing shared-storage contention and limiting scalability.
- The proposed architecture separates patch generation and staging, massively parallel inference, and sharded vector-database ingestion for independent optimization.
- Embeddings are persistently associated with patient, slide, and patch metadata in a reusable database supporting downstream indexing, filtering, and retrieval.
- The evaluation reports high-throughput scalable execution and characterizes resource bottlenecks on supercomputing infrastructure.
II. BACKGROUND, MOTIVATION AND RELATED WORK
Prior work advances WSI representation quality, HPC data movement, and vector databases largely as separate concerns. The paper identifies the missing integration: an I/O-dominant, decoupled embedding-extraction pipeline with reusable embeddings.
- WSI representation-learning research has progressed from weak supervision to self-supervised, transformer-based, hierarchical, and foundation-model approaches.
- These model-focused studies generally improve representation quality without addressing computational bottlenecks that slow large-scale deployment.
- Typical WSI pipelines interleave patch generation, inference, and output handling, limiting scalability through shared I/O contention and preventing independent stage optimization.
- HPC techniques address data movement for large contiguous datasets and structured communication patterns, unlike WSI workloads producing massive numbers of small independent files.
- Existing WSI-scale HPC efforts focus primarily on training or preprocessing rather than embedding extraction and its I/O-dominant behavior.
- Vector databases support scalable indexing and retrieval but remain decoupled from upstream embedding generation, leaving embeddings as transient outputs.
- The identified gap is jointly addressing I/O-dominant embedding extraction, pipeline decoupling, and embedding reuse at large WSI scale.
III. WSI PIPELINE AND PARALLEL EXECUTION MODEL
The pipeline decouples WSI processing into stages parallelized for storage, compute, and write-throughput constraints. Rank-local execution minimizes communication, while persistent sharded storage supports reusable embeddings and metadata.
- III. WSI PIPELINE AND PARALLEL EXECUTION MODEL: The design separates patch generation, embedding inference, and vector-database ingestion, assigning each stage its dominant bottleneck.
- Stage 1: Patch Generation: Stage 1 uses MPI spatial decomposition and deterministic cyclic partitioning over patch coordinates for balanced rank-local extraction.
- Stage 1: Patch Generation: Patch extraction avoids steady-state communication and defers synchronization to a barrier followed by rank-0 metadata consolidation.
- Stage 1: Patch Generation: Small-file writes and filesystem contention dominate patch-generation performance despite communication-free extraction.
- Stage 2: Embedding Inference: Stage 2 performs SPMD batched foundation-model inference independently across tasks, typically assigning one task per GPU without collective communication.
- Stage 3: Vector Database Ingestion: Stage 3 uses streaming, batched, shard-parallel ingestion constrained primarily by write throughput and indexing overhead.
- Stage 3: Vector Database Ingestion: The resulting vector database preserves embedding, spatial, and metadata relationships as a persistent resource for downstream tasks without repeated raw-WSI access.
A. Dataset
The evaluation uses 4,185 H&E WSIs from the CCDI MCI dataset, spanning diverse slide sizes and tissue content. The embedding workflow distributes indexed patches and writes rank-local results before consolidation and sharded ingestion.
- A. Dataset: The dataset contains 4,185 H&E whole-slide images from the Childhood Cancer Data Initiative Molecular Characterization Initiative.
- Embedding Extraction: Each inference rank loads the patch index, constructs a local dataset, and processes its cyclic subset with a DataLoader.
- Embedding Extraction: Batched inference applies preprocessing and extracts embeddings, with attention tensors optionally produced jointly when supported by the model.
- Embedding Extraction: Ranks serialize local results and completion markers, after which rank 0 merges and sorts outputs by coordinates.
- Vector Database Ingestion: The ingestion algorithm assigns embedding and metadata subsets to K shards, inserts them in parallel, and builds a local index per shard.
- A. Dataset: The slides span workloads from tens to more than 10^5 patches per WSI, covering small- and large-scale regimes.
B. Models
The evaluation uses three foundation models with differing resolution, embedding dimensionality, and computational characteristics on Frontier’s heterogeneous CPU-GPU and parallel-filesystem platform.
- HIPT, H-Optimus-0, and Virchow2 differ in patch resolution, embedding dimensionality, and computational characteristics.
- The experiments run on Frontier, where each node combines a 64-core AMD EPYC CPU with eight AMD Instinct MI250X GPUs.
- Orion provides high aggregate bandwidth but faces metadata and small-file access constraints under high concurrency, which are relevant to millions of concurrent patch files.
- Abundant GPU capacity leaves effective throughput governed by filesystem bandwidth and metadata scalability in these experiments.
D. Experimental Methodology
The methodology evaluates all three pipeline stages across scaling regimes and measures whether performance is limited by computation or data movement. Results emphasize file-count overhead and shared-filesystem contention during patch generation.
- Experimental Methodology: The evaluation covers patch generation, embedding extraction, and sharded vector database ingestion across distributed and single-GPU execution regimes.
- Experimental Methodology: Four experimental setups vary patch count, GPU count, kernel behavior, and production-scale orchestration to isolate embedding-stage bottlenecks.
- Experimental Methodology: Runtime, throughput, speedup, efficiency, file statistics, GPU utilization, I/O fraction, insertion rate, and memory footprint distinguish compute scaling from storage-limited behavior.
- Experimental Methodology: Across stages, each component is embarrassingly parallel in isolation, but end-to-end performance is limited by data movement and storage behavior.
- Patch Generation: 25% fewer files yields proportional patch-generation throughput improvements despite negligible change in total bytes written.
- Patch Generation: 1.69× speedup at 224 versus 56 MPI ranks achieves 42.3% efficiency, while shared-filesystem contention causes sublinear scaling.
B. Embedding Inference Analysis
Embedding inference transitions from compute-dominated behavior to I/O-dominated execution as workload and concurrency increase. GPU kernels remain efficient, but data-delivery latency and storage contention limit throughput and scaling.
- Patch Sweep: At small patch counts, throughput rises with workload size, but larger workloads plateau as I/O wait time increases.
- Patch Sweep: GPU utilization remains modest across scales because accelerators stall on input data latency rather than lacking compute capacity.
- Strong Scaling: Throughput increases with GPU count across HIPT, H-Optimus-0, and Virchow2, but efficiency falls below 30–50% at higher concurrency.
- Strong Scaling: Communication-free execution does not ensure scalability because shared storage bandwidth and metadata contention saturate the I/O path.
- Kernel-Level Analysis: Kernel profiling finds high arithmetic intensity without approaching memory-bound ceilings, indicating efficient GPU execution.
- Kernel-Level Analysis: Short kernel durations expose inter-batch latency, leaving GPUs stalled while waiting for upstream data delivery.
- Production-Scale Behavior: At production scale across 4,185 WSIs, throughput improves with patch count, but storage contention and slide heterogeneity create substantial run-to-run variance.
C. Vector Database Ingestion Analysis
Vector database ingestion gains substantial throughput from shard-level parallelism but remains constrained by shared filesystem behavior. Its streaming design avoids memory accumulation while higher concurrency increasingly exposes storage contention.
- Scaling Results: 22K to 188K rows/s delivers an 8.5× ingestion speedup, with efficiency peaking at 86.7% at 4 nodes before declining.
- Scaling Results: Low concurrency underutilizes bandwidth, whereas high concurrency makes metadata and read contention dominant during I/O-bound ingestion.
- Scaling Results: Each rank independently processes embedding files and writes to a separate shard, but shared storage limits scalability as concurrency rises.
- Patch-Generation Context: Increasing patch size reduces generated file count while increasing average file size and effective write throughput in Stage 1.
- Patch-Generation Context: Strong scaling remains sublinear across 56, 112, and 224 MPI ranks, with rising write throughput consistent with I/O-dominated behavior.
- Embedding Context: Embedding-inference strong scaling shows declining efficiency and rising I/O fraction as GPU concurrency increases.
- Memory Footprint: Approximately 0.6 GB per rank remains stable across configurations, consistent with streaming ingestion rather than significant in-memory accumulation.
- Operating Point: The 4-node configuration is a practical operating point balancing shard parallelism against manageable filesystem contention.
VI. DISCUSSION
Large-scale WSI embedding extraction exhibits I/O-dominated scaling: storage contention and data movement constrain end-to-end performance even when stages are independently parallelized. The discussion motivates decoupling I/O-heavy, compute-heavy, and write-heavy stages to reduce cross-stage interference.
- End-to-end performance is governed by data movement and storage constraints, despite ideal parallel complexity within individual pipeline stages.
- Patch generation: Patch generation scales sublinearly because small-file creation and metadata operations create filesystem contention across millions of patches.The nominal work distribution is O(N/R), but per-patch overhead becomes comparable to data movement.
- Embedding inference: Throughput plateaus and I/O wait increases as patch counts grow, marking a transition from compute-bound to storage-limited execution.GPU utilization remains modest despite sufficient computing capacity.
- Embedding inference: Performance degrades with increasing GPU count despite independent execution, showing that shared storage bandwidth—not communication—limits strong scaling.
- Vector database ingestion: Vector database ingestion improves with shard parallelism but loses efficiency at higher concurrency because of read contention and write amplification.The ingestion stage remains bounded by storage-system behavior.
- Performance model: The performance model separates compute time, N/R, from data-transfer time, NB/BWstorage, which additional compute capacity cannot reduce.N is total patches, R is effective compute rate, B is average bytes per patch, and BWstorage is effective storage bandwidth.
- Design implication: Scalable WSI pipelines should prioritize data orchestration, storage access patterns, and intermediate representations alongside compute optimization.The decoupled design isolates I/O-heavy, compute-heavy, and write-heavy stages for independent optimization.
VII. CONCLUSION
The paper presents a decoupled pipeline that separates patch staging, parallel inference, and sharded vector-database ingestion for large-scale WSI embedding extraction. Its findings identify data movement and orchestration, rather than communication or peak accelerator throughput, as the central scaling constraint, while persistent metadata-linked embeddings support reuse.
- The pipeline separates patch generation and staging, embarrassingly parallel inference, and sharded vector database ingestion.
- Large-scale WSI embedding extraction is constrained by data movement and orchestration rather than collective communication or peak accelerator throughput.Throughput plateaus, GPU utilization remains modest, I/O fractions rise under scaling, and vector-database ingestion is costly.
- Embeddings are persistently stored with spatial and slide-level metadata in a vector database for downstream reuse without repeated raw-WSI access.This makes embedding extraction a one-time data-generation stage whose outputs can support subsequent analyses.