Source-linked AI summary
TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning
Fernando Pérez-García, Rachel Sparks, Sébastien Ourselin
TL;DR
Medical-image deep learning must address scarce labels, high computational costs, spatial metadata, and other modality-specific processing challenges. TorchIO provides a modular, PyTorch-style library for loading, preprocessing, augmentation, and patch-based sampling, supporting standardized and reproducible workflows compatible with other frameworks. Its main scope boundary is that most transforms are not differentiable and modality-specific coverage is richer for MRI than for CT or ultrasound.
Problem
Medical-image deep learning involves scarce labels, high computational costs, voxel metadata, and modality-specific processing requirements that general-purpose image frameworks may not implement.
Method
TorchIO is an open-source, PyTorch-style library providing medical-image loading, preprocessing, augmentation, and patch-based sampling for deep-learning workflows.
Results
TorchIO provides modular processing that supports reproducibility, traceability, standardization, and compatibility with higher-level medical-imaging frameworks.
Takeaways & Limitations
Researchers can use TorchIO to standardize medical-image processing while focusing on deep-learning experiments and combining its components with other frameworks.
Takeaways & Limitations
Most TorchIO transforms are not differentiable, and its modality-specific transform coverage is richer for MRI than for CT or ultrasound.
Abstract
from arXiv · showhide
Processing of medical images such as MRI or CT presents unique challenges compared to RGB images typically used in computer vision. These include a lack of labels for large datasets, high computational costs, and metadata to describe the physical properties of voxels. Data augmentation is used to artificially increase the size of the training datasets. Training with image patches decreases the need for computational power. Spatial metadata needs to be carefully taken into account in order to ensure a correct alignment of volumes. We present TorchIO, an open-source Python library to enable efficient loading, preprocessing, augmentation and patch-based sampling of medical images for deep learning. TorchIO follows the style of PyTorch and integrates standard medical image processing libraries to efficiently process images during training of neural networks. TorchIO transforms can be composed, reproduced, traced and extended. We provide multiple generic preprocessing and augmentation operations as well as simulation of MRI-specific artifacts. Source code, comprehensive tutorials and extensive documentation for TorchIO can be found at https://torchio.rtfd.io/. The package can be installed from the Python Package Index running 'pip install torchio'. It includes a command-line interface which allows users to apply transforms to image files without using Python. Additionally, we provide a graphical interface within a TorchIO extension in 3D Slicer to visualize the effects of transforms. TorchIO was developed to help researchers standardize medical image processing pipelines and allow them to focus on the deep learning experiments. It encourages open science, as it supports reproducibility and is version controlled so that the software can be cited precisely. Due to its modularity, the library is compatible with other frameworks for deep learning with medical images.
1. Introduction
Medical imaging creates challenges for deep-learning workflows, including sparse labels, volumetric computational costs, and spatial metadata that must be handled correctly. TorchIO addresses these needs with a PyTorch-style library for preprocessing, augmentation, sampling, and reproducible pipeline development.
- Challenges: Medical images differ from typical RGB images because labels are scarce, volumes are computationally expensive, and voxel metadata encodes physical position and size.These challenges require specialized processing beyond general-purpose computer-vision pipelines.
- TorchIO: TorchIO is an open-source, PyTorch-compatible library for efficient loading, preprocessing, augmentation, and patch-based sampling in medical-image deep learning.Its design targets integration into training workflows and reduces duplicated implementation effort.
- TorchIO: TorchIO supports modular, reproducible, and traceable processing pipelines that can be used alongside higher-level medical-imaging frameworks.The library is described as compatible with frameworks such as MONAI and intended to standardize processing methods.
- Challenges: Ignoring origin, orientation, or spacing can misalign MRI and segmentation volumes despite identical image dimensions.Figure 1 uses two 181 × 181 images to demonstrate this spatial-metadata failure.
- Challenges: Data augmentation enlarges training datasets artificially, while medical-image patching and preprocessing help address limited data, memory demands, and normalization needs.Medical-image augmentation must preserve spatial relationships and account for metadata during transformations.
2. Methods
TorchIO is a Python library for loading and augmenting medical images in deep-learning workflows. Its compact, modular, PyTorch-style design supports reusable processing while preserving interoperability with other frameworks.
- TorchIO focuses on loading and augmenting medical images for deep learning.
- The unified library makes medical image properties explicit and avoids coding processing pipelines from scratch.This can accelerate research by reducing duplicated preprocessing work.
- TorchIO excludes network architectures, loss functions, and training workflows to preserve modularity between training and preprocessing.
- TorchIO follows PyTorch’s philosophy of simplicity and usability while reusing PyTorch classes and infrastructure where possible.The library can also be used with TensorFlow or Keras.
- Its features are organized into data structures, input/output, and preprocessing and augmentation transforms.These interfaces and the codebase are represented in Figure 2.
2.1. Data
TorchIO organizes medical images, subjects, datasets, and patch-based pipelines around spatial metadata and PyTorch data-loading infrastructure. It supports parallel preparation, patch sampling, queueing, and dense-volume inference.
- TorchIO uses NiBabel and SimpleITK to read and write images, combining them for broader format support.
- The library supports up to 4D single-channel or multichannel images, including CT, fMRI, and dMRI data.
- Each Image stores a 4D tensor, an affine transform mapping voxel indices to world coordinates, and optional acquisition metadata.
- ScalarImage and LabelMap distinguish image types such as CT scans and segmentations.
- Images use lazy loading, so data are loaded only when an operation requires them.
- A Subject groups related Image instances and can store fields such as age, diagnosis, or ethnicity.
- SubjectsDataset stores subjects and applies transforms after loading when queried; PyTorch DataLoader can process different subjects in parallel on CPUs.
- Patch-based training: Patch-based training uses samplers to extract subvolumes, while GridSampler and GridAggregator support dense inference across the entire volume.UniformSampler samples centers uniformly, WeightedSampler uses a probability distribution image, and GridSampler uses a set stride.
2.2. Transforms
TorchIO provides modular medical-image transforms that account for spatial metadata, support preprocessing and augmentation, and can be composed or extended.
- Transform design: TorchIO transforms follow a PyTorch-like API while using medical-imaging libraries to account for image size, resolution, location, and orientation.The API accepts multiple image representations and returns the same type, including tensors, SimpleITK images, NumPy arrays, Pillow images, dictionaries, Subjects, and Images.
- Transform design: Transforms are organized as spatial, intensity, preprocessing, or augmentation operations, with randomly parameterized transforms subclassing RandomTransform.Spatial transforms modify bounds or spacing and resample images, whereas intensity transforms modify voxel values only in ScalarImage objects.
- Preprocessing: Preprocessing supports spatial alignment through resampling and reorientation, plus intensity normalization and MRI histogram standardization.Normalization can use foreground masks to define voxels for statistics, while histogram standardization addresses scanner- or sequence-related intensity differences.
- Augmentation: Augmentation includes spatial and intensity operations, MRI-specific k-space artifact simulation, and a scope boundary: domain-specific techniques are mostly MRI-related.Examples include affine and elastic deformation, blur, noise, spikes, motion, ghosting, and bias-field artifacts; users are encouraged to contribute physics-based US or CT techniques.
- Composability and extensibility: Transforms can be composed linearly or as a directed acyclic graph, including probabilistic choices, and extended with arbitrary callable objects or more complex custom transforms.OneOf can select affine or elastic augmentation with specified probabilities before a shared intensity rescaling step.
- Reproducibility and extensibility: TorchIO supports reproducible, traceable processing and standardization of medical-image pipelines while remaining compatible with other deep-learning frameworks.The library is designed to help researchers focus on deep-learning experiments and supports open-science practices.
3. Results
TorchIO provides installable and interactive interfaces for applying and visualizing medical-image transforms, while its features are illustrated through reproducible workflow examples.
- Code availability: TorchIO code, documentation, tutorials, and releases are publicly available, with version 0.18.0 identified for precise citation.The package is distributed through PyPI and can be installed on Windows, macOS, or Linux with pip install torchio.
- Community adoption: More than 900 GitHub stars and 5000 monthly PyPI downloads were reported for TorchIO as of August 2021.These figures describe the library’s reported community usage at that time.
- Additional interfaces: The torchio-transform CLI applies transforms without Python and supports visualization, experimental assessment, and shell-script preprocessing or augmentation.The CLI can be used either to inspect pipelines or to preprocess and augment datasets when storage is available for faster loading.
- Additional interfaces: A 3D Slicer GUI visualizes transform effects without coding, allowing users to assess preprocessing and augmentation before network training.The example applies RandomBiasField, RandomGhosting, RandomMotion, RandomAffine, and RandomElasticDeformation.
- Workflow examples: TorchIO features are demonstrated by reproducing workflows for synthetic high-resolution MRI generation and patch-based CT segmentation.The examples combine subjects, spatially paired image-label transforms, intensity processing, artifact simulation, patch queues, and error-map-guided sampling.
- Workflow examples: The CT example uses patch-based training for scans too large for GPU memory and generates context and full-resolution patches with different fields of view.Error maps computed from labels and predictions guide preferential sampling of patches with larger errors.
4. Discussion
TorchIO is a PyTorch-style library for efficient medical-image loading, preprocessing, augmentation, and patch-based sampling during CNN training. Its modular design supports standardized, reproducible workflows and interoperability, while most transforms remain non-differentiable.
- TorchIO efficiently loads, preprocesses, augments, and samples medical imaging data during CNN training in a PyTorch-style library.
- The toolkit helps standardize medical image processing pipelines and lets researchers focus on deep learning experiments.
- TorchIO supports interoperability with higher-level frameworks such as MONAI, combining its MRI transforms and patch-based sampling with MONAI training components.
- Most TorchIO transforms are not differentiable because tensors are converted to SimpleITK images or NumPy arrays within most transforms.
- Future work includes extending preprocessing and augmentation to modalities such as CT and US and improving compatibility with related tools.