Source-linked AI summary
Deep Reinforcement Learning for Organ Localization in CT
Fernando Navarro, Anjany Sekuboyina, Diana Waldmannstetter, Jan C. Peeken, Stephanie E. Combs, Bjoern H. Menze
TL;DR
Organ localization is an important CT pre-processing task, but existing approaches can require large annotated datasets. This paper uses deep reinforcement learning with tailored box-manipulation actions, and reports effective localization across organs and imaging conditions, including an overall IoU of 0.63.
Problem
Organ localization in CT is needed for downstream image analysis, but existing deep-learning approaches require large annotated training datasets.
Source: §1 · p. 1Method
A deep reinforcement-learning agent learns to localize organs by sequentially deforming a 3D bounding box with 11 tailored actions.
Source: §1 · p. 3, §3 · pp. 4–5Results
Takeaways & Limitations
RL can support organ localization in limited-data regimes compared with supervised approaches requiring substantially more annotated data.
Source: §1 · p. 3, §4 · p. 7, §5 · p. 9The step-size hyper-parameter α is fixed, although organ-specific values could improve localization performance and are left for future work.
Source: §4 · p. 7
Abstract
from arXiv · showhide
Robust localization of organs in computed tomography scans is a constant pre-processing requirement for organ-specific image retrieval, radiotherapy planning, and interventional image analysis. In contrast to current solutions based on exhaustive search or region proposals, which require large amounts of annotated data, we propose a deep reinforcement learning approach for organ localization in CT. In this work, an artificial agent is actively self-taught to localize organs in CT by learning from its asserts and mistakes. Within the context of reinforcement learning, we propose a novel set of actions tailored for organ localization in CT. Our method can use as a plug-and-play module for localizing any organ of interest. We evaluate the proposed solution on the public VISCERAL dataset containing CT scans with varying fields of view and multiple organs. We achieved an overall intersection over union of 0.63, an absolute median wall distance of 2.25 mm, and a median distance between centroids of 3.65 mm.
1. Introduction
The paper frames organ localization as a challenging but important CT pre-processing task and proposes deep reinforcement learning with actions tailored to deformable 3D bounding boxes. It reports that RL can learn organ localization with limited annotated data.
1. Introduction
- Organ localization supports downstream diagnosis, treatment, segmentation, lesion detection, and registration, while reducing reliance on time-consuming and error-prone manual localization.The task remains difficult because organs vary in shape, size, field of view, and pathology.Source: §1 · p. 1
- Existing deep-learning localization methods require large annotated datasets, motivating the exploration of RL for more data-efficient organ localization.Prior approaches include 2D regression and 3D object-proposal methods, while earlier alternatives include computationally expensive atlas registration and feature-dependent machine learning.Source: §1 · p. 2
- RL is presented as the first deep-reinforcement-learning approach for organ localization, learning a policy from the agent’s own experiences.The agent sequentially deforms a bounding box to fit the organ of interest.Source: §1 · pp. 2–3
- The method introduces 11 actions tailored to organ localization, including translations, global scaling, and direction-specific box deformations.These actions address the need to fit organs with varying sizes and shapes rather than only locate a point.Source: §1 · pp. 2–3
- The authors report that RL can learn organ localization under a limited-data regimen compared with CNNs.This claim is stated as a central result of the introduction.Source: §1 · p. 3
2. Background
The paper formulates organ localization as a partially observable reinforcement-learning problem in which an agent selects actions from CT-derived states and receives rewards. Deep Q-learning approximates action values with a neural network trained from replayed experiences.
2. Background
- Reinforcement learning models an agent interacting with an environment through states, actions, and rewards to maximize future reward.The organ-localization problem is adapted from this Markov decision process formulation.Source: §2 · p. 3
- Deep Q-learning approximates the optimal action-value function with a nonlinear neural network when the state space is too complex to explore directly.The value function maps states and actions to expected future reward using Bellman-style recursive estimation.Source: §2 · p. 3
- For multi-organ localization, the network receives current box voxel values together with the last four states and outputs 11 action q-values.The architecture therefore uses recent state history while assigning one value to each legal action.Source: Figure 2 · p. 4
- Experience replay stores agent transitions and samples mini-batches to optimize the network by minimizing a loss based on target and predicted q-values.A target network updated periodically stabilizes rapid policy changes.Source: §2 · p. 4
3. Method
The method formulates organ localization as a reinforcement-learning problem in which an agent iteratively transforms a bounding box within a 3D CT scan. It combines a tailored action space, IoU-based rewards, sequential state input, and task-specific termination rules.
3. Method
- The method models organ localization as a Markov decision process with actions, states, and a reward function.The environment is the 3D CT scan, and the section defines each MDP component for localization.Source: §3 · p. 4
- The state contains voxel values inside the current 3D bounding box, with the last 4 states supplied to stabilize search and encode recent actions.The box position is continuous, although CT intensities are extracted in voxel space.Source: §3 · p. 4
- The 11-action space combines translations, global scaling, and directional deformations so the box can represent varied organ locations, sizes, and aspect ratios.Translations preserve size and aspect ratio; global scaling preserves aspect ratio, while thinner, flatter, and taller actions alter it.Source: §3 · pp. 4–5
- The reward is binary, assigning +1 or −1 according to whether an action improves intersection over union.This avoids the very small reward changes that direct IoU differences can produce near convergence.Source: §3 · p. 5
- Training terminates when IoU reaches threshold τ, whereas testing terminates when oscillation occurs.The distinct stopping rules separate threshold-based training termination from oscillation-based inference termination.Source: §3 · p. 5
- The network receives voxel values from the last 4 steps and outputs a distribution over the available actions.The architecture was adopted from prior landmark-detection work and tailored to organ localization.Source: §3 · p. 5
4. Experiments and Results
Experiments evaluate localization across seven organs and varied CT settings, using per-organ and overall metrics plus comparisons with prior methods. The results indicate effective localization with limited annotated data, while performance varies with organ size and fixed step size.
Dataset and evaluation
- The evaluation uses seven organs from VISCERAL CT scans spanning whole-body and thorax fields of view, with and without contrast enhancement.The experiments use 70 training scans and 20 testing scans after isotropic resampling to 3mm3.Source: §4 · pp. 5–6
Training localization agent
- The training procedure uses one agent per organ, 30 epochs, ε-greedy exploration, guided exploration, and an IoU stopping threshold of 0.85.Training each agent takes 13–20 hours depending on the number of steps required.Source: §4 · pp. 6–7
Evaluating the localization performance
- 2.35 mm median absolute wall distance and 3.36 mm median centroid distance quantify the proposed method’s overall localization accuracy.Table 1 reports per-organ and global IoU, wall distance, and centroid distance.Source: Table 1 · p. 6, §4 · p. 7
- Larger organs localize better than smaller organs, which the authors associate with the fixed step size α = 0.1.They suggest organ-specific α values or multi-scale strategies as potential improvements for small-organ localization.Source: §4 · p. 7
Limited-data ablation
- Training with 10% of the data increased wall distance relative to full-data training, yet matched or outperformed systems trained with hundreds of annotated scans.The authors attribute this low-data behavior to sequential state-to-action learning and many possible paths through the environment.Source: §4 · pp. 7–8
Comparison with other methods
- Humpire-Mamani et al. (2018) achieves the lowest wall distance for most organs, but requires approximately 2000 annotated scans, making scaling data-intensive.The comparison uses reported results from different datasets because no common CT organ-localization benchmark exists.Source: Table 2 · p. 7, §4 · p. 8
Visualizing the localization
- Most qualitative predictions achieve IoU values between 0.65 and 0.9, often covering the organ or closely matching the ground-truth box.Figure 3 compares dotted ground-truth locations with solid predicted boxes across sample volumes.Source: §4 · p. 8, Figure 3 · p. 8
5. Conclusion
The paper presents deep reinforcement learning as a first approach to organ localization and introduces actions tailored to organ size and shape. It concludes that RL can work with substantially less annotated data while localizing seven diverse organs across different fields of view.
5. Conclusion
- Deep reinforcement learning is presented as a first approach to organ localization, with a new action set tailored to diverse organ shapes and sizes.Source: §5 · p. 9
- RL can train effectively in limited-data regimes, requiring approximately 20 times less data than supervised learning approaches.The method is evaluated on seven organs with diverse shapes, sizes, appearances, and fields of view.Source: §5 · p. 9
- Future work proposes multi-agent deep reinforcement learning and multi-scale organ localization, especially for small organs.Source: §5 · p. 9