Source-linked AI summary
ReLayNet: Retinal Layer and Fluid Segmentation of Macular Optical Coherence Tomography using Fully Convolutional Network
Abhijit Guha Roy, Sailesh Conjeti, Sri Phani Krishna Karri, Debdoot Sheet, Amin Katouzian, Christian Wachinger, Nassir Navab
TL;DR
Retinal OCT segmentation must handle pathological fluid and difficult, variable image interpretation, while earlier approaches often rely on multistage heuristics. ReLayNet uses an end-to-end fully convolutional encoder-decoder with weighted logistic and Dice losses, and it demonstrates superior comparative performance, including pathology cases. Its evaluation assumes all image volumes are aligned left to right because local eye-side information is unavailable.
Problem
Earlier retinal OCT methods are not end-to-end, often require heuristics and multistage processing, and may omit fluid-filled regions in pathological settings.
Method
ReLayNet is an end-to-end fully convolutional encoder-decoder trained with combined weighted logistic and Dice losses.
Results
ReLayNet exhibited superior performance against five state-of-the-art methods and reliably segmented OCT scans despite severe pathology affecting normal retinal layering.
Takeaways & Limitations
ReLayNet is presented as suitable for clinical applications because it segments a single B-scan in approximately 0.01 seconds.
Takeaways & Limitations
Evaluation assumes all image volumes are aligned left to right because the dataset lacks local anatomical information about left and right eyes.
Abstract
from arXiv · showhide
Optical coherence tomography (OCT) is used for non-invasive diagnosis of diabetic macular edema assessing the retinal layers. In this paper, we propose a new fully convolutional deep architecture, termed ReLayNet, for end-to-end segmentation of retinal layers and fluid masses in eye OCT scans. ReLayNet uses a contracting path of convolutional blocks (encoders) to learn a hierarchy of contextual features, followed by an expansive path of convolutional blocks (decoders) for semantic segmentation. ReLayNet is trained to optimize a joint loss function comprising of weighted logistic regression and Dice overlap loss. The framework is validated on a publicly available benchmark dataset with comparisons against five state-of-the-art segmentation methods including two deep learning based approaches to substantiate its effectiveness.
1. Introduction
SD-OCT provides high-resolution cross-sectional views of retinal layers, supporting assessment of diabetic retinal pathology. ReLayNet addresses difficult, variable OCT interpretation by segmenting retinal layers and fluid pockets end to end.
- SD-OCT acquires non-invasive, high-resolution cross-sectional scans with sufficient tissue penetration for retinal imaging.The modality is reported at 6µm resolution and 0.5–2 mm depth of penetration.
- Diabetes affects about 415 million people, and diabetic retinopathy can damage retinal vessels and cause fluid accumulation between retinal layers.
- Motion artifacts, tissue inclination, low signal-to-noise ratio, and highly myopic eyes make retinal OCT acquisition and interpretation challenging.
- ReLayNet performs end-to-end segmentation of multiple retinal layers and delineates fluid pockets in eye OCT images.The paper presents it as a deep learning framework called ReLayNet, short for Retinal Layer segmentation network.
2. State of the art
Earlier retinal OCT segmentation methods commonly use multistage graph, dynamic-programming, or handcrafted procedures and may omit fluid regions. ReLayNet instead applies a fully convolutional encoder-decoder design with data-driven training and composite loss.
- Existing methods: Graph-construction and dynamic-programming methods estimate retinal boundaries using intensity, learned edges, constraints, or CNN predictions.
- Existing methods: Other approaches use diffusion maps, shape regularizers, or parallel active contours for retinal layer segmentation.
- Limitations of prior work: Earlier methods are not end-to-end, often require preprocessing, post-processing, domain knowledge, and modeling approximations, and may overlook fluid-filled regions.
- Deep segmentation: Fully convolutional semantic segmentation predicts labels for all image pixels together, combining contextual and appearance information through encoder-decoder designs and skip connections.
- ReLayNet: ReLayNet is presented as a fully convolutional, data-driven end-to-end method using unpooling, skip connections, and a weighted logistic-plus-Dice loss.The paper describes the method as the first fully convolutional deep learning approach for retinal OCT layer and fluid segmentation.
3.1. Problem statement
The problem is formulated as assigning every OCT-image pixel to one of 10 tissue or fluid-related classes.
- Each pixel x = (r, c) in retinal OCT image I is assigned a label l from a K-class label space.
- The classification problem uses K = 10 classes: seven retinal layers, two regions surrounding the retina, and accumulated fluid.
3.2. Network architecture
ReLayNet uses an encoder-decoder architecture that transfers multiscale representations through skip connections and restores spatial resolution using pooling indices during unpooling.
- The network comprises contracting encoder blocks and expansive decoder blocks, with matched encoder features concatenated into decoder blocks before classification.
- Encoder block: Encoder blocks apply convolution, batch normalization, ReLU activation, and max pooling to progressively reduce feature-map dimensions.The convolution kernels are rectangular 7 × 3, and max pooling reduces feature-map dimensions by half.
- Pooling and unpooling: Pooling saves intermediate indices that decoder unpooling reuses to place upsampled activations at appropriate spatial locations.
- Decoder block: Decoder blocks perform unpooling, feature concatenation, convolution, batch normalization, and ReLU activation.
- Classification layer: A final 1 × 1 convolution maps 64 channels to 10 classes, followed by softmax probabilities for each pixel.
3.3. Training
ReLayNet is trained with a composite objective combining weighted multi-class logistic loss and Dice loss, while pixel weighting addresses class imbalance and difficult tissue transitions. Training uses sliced OCT B-scans, with whole-scan end-to-end segmentation during testing.
- Weighting scheme: Figure 4 presents an OCT training B-scan, its ground-truth labels, and a heat map of the corresponding training weights.The label colors match those used in Figure 1.
- Loss functions: Weighted multi-class logistic loss penalizes deviations between estimated class probabilities and ground-truth labels at each pixel.The pixel weight ω(x) modifies each location’s contribution to the loss.
- Loss functions: Dice loss evaluates spatial overlap between the predicted segmentation and the ground truth.The paper uses a differentiable approximation of Dice loss.
- Weighting scheme: Pixel weights emphasize tissue-transition regions, where diffuse boundaries, speckle noise, and limited OCT resolution make segmentation difficult.The weighting scheme also compensates for strong imbalance between retinal layers and the dominant background class.
- Training and testing procedure: Training slices OCT B-scans width-wise into non-overlapping lines, whereas testing segments the whole B-scan end-to-end.Figure 5 illustrates the overall training and testing flow.
3.4. Optimization
Optimization estimates ReLayNet’s weights and biases by minimizing the overall cost, including weighted logistic loss, Dice loss, and weight decay. The cost gradients are computed by backpropagation and optimized with stochastic mini-batch gradient descent with momentum.
- 3.4. Optimization: Training estimates all layer weights and biases Θ = {W(·), b(·)} to minimize the overall cost function.Θ* denotes the parameter set that minimizes this cost.
- 3.4. Optimization: The overall cost is optimized using stochastic mini-batch gradient descent with momentum and backpropagation.The parameter derivative is computed by applying the chain rule to backpropagated gradients.
3.5. OCT B-scan slicing and data augmentation
Because full-width OCT images exceed available GPU RAM and can produce noisy, divergent training with small batches, ReLayNet trains on sliced B-scan lines with augmentation. At test time, its resolution-preserving design enables seamless whole-B-scan segmentation without slicing artifacts.
- 3.5. OCT B-scan slicing and data augmentation: Full-width OCT training is limited by GPU RAM, forcing smaller batches that can yield noisy gradients and divergent loss curves.The authors address this training constraint with width-wise slicing.
- 3.5. OCT B-scan slicing and data augmentation: OCT B-scans are sliced width-wise into non-overlapping lines for training.The sliced data are augmented with random horizontal flips, slight spatial translations, and cropping.
- 3.5. OCT B-scan slicing and data augmentation: At test time, the whole B-scan is segmented because ReLayNet preserves resolution.This produces seamless segmentation without slicing-induced artifacts.
4. Experimental setup
The experiments evaluate ReLayNet on a publicly available Duke SD-OCT dataset and compare it with five retinal OCT segmentation methods using standard segmentation metrics.
- Dataset: The Duke SD-OCT dataset contains 110 annotated 512 × 740 B-scans from 10 patients with diabetic macular edema.Each patient contributes 11 scans centered on the fovea and extending laterally from it.
- Training setup: Subjects 1–5 provide 55 training B-scans, while subjects 6–10 provide 55 testing B-scans under the standard split.Training uses mini-batches of 50 augmented slices, with λ1 = 1, λ2 = 0.5, λ3 = 0.0001, ω1 = 10, and ω2 = 5.
- Comparative methods: ReLayNet is compared with CM-GDP, CM-KR, CM-LSE, CM-Unet, and CM-FCN.CM-Unet depth was reduced because of the limited training data.
- Evaluation metrics: Performance is assessed with Dice overlap score, contour error, and mean absolute difference in layer thickness, reported in pixels.Pixel reporting is used because individual scan resolutions are unavailable in the Duke dataset.
- Model configurations: Table 1 organizes ReLayNet and baseline configurations by architecture depth, loss function, skip connections, and loss weighting.These attributes define the principal architectural and training differences among the compared models.
- Qualitative evaluation: Figure 6 compares expert annotations, ReLayNet, and five comparative predictions on a foveal DME scan containing a small fluid mass.CM-GDP and CM-LSE do not provide fluid predictions.
5. Experimental observation and discussion
ReLayNet shows strong qualitative and quantitative performance across retinal-layer and fluid segmentation, with ablations supporting skip connections, joint losses, and the selected architecture depth.
- 5.1. Qualitative comparison: ReLayNet successfully segments a small fluid pool that CM-Unet, CM-FCN, and CM-KR fail to capture in a pathological foveal scan.The comparison covers DME with thin foveal layers and accumulated fluid masses.
- 5.1. Qualitative comparison: ReLayNet and CM-Unet produce layer segmentations comparable in quality to another human expert, whereas CM-GDP smooths detail and mispredicts several layers.CM-GDP over-predicts NFL-IPL and under-predicts lower retinal layers; CM-GDP and CM-LSE also omit fluid modeling.
- 5.2. Quantitative comparison: ReLayNet achieves the highest segmentation efficacy in 9 of 10 classes, with DS above 0.9 for six specified retinal-layer classes.For the difficult OPL layer, ReLayNet reaches DS 0.84 versus expert-observer DS 0.74.
- 5.2. Quantitative comparison: ReLayNet obtains fluid DS 0.77, exceeding CM-Unet by 0.10 and CM-FCN by 0.49, while CM-FCN records 0.28 DS.These results quantify the fluid-segmentation advantage over the compared deep models.
- 5.2. Quantitative comparison: ReLayNet has consistently superior MAD-LT across layers and the best CE for every layer except ONL-ISM.CM-LSE and CM-GDP outperform ReLayNet on ONL-ISM CE because they do not estimate fluid, while ReLayNet beats CM-KR by 0.39 pixels.
- 5.3. Importance of ReLayNet contributions: The authors attribute the overall gains to Dice loss and unpooling, alongside the encoder–decoder design and skip connections.The reported interpretation distinguishes ReLayNet from CM-Unet and CM-FCN.
- 5.3. Importance of ReLayNet contributions: Removing skip connections reduces fluid DS by 0.09 and worsens ONL-ISM MAD-LT and CE by 0.6 and 0.08 pixels.The ablation compares ReLayNet with BL-1, which lacks skip connections.
- 5.3. Importance of ReLayNet contributions: The weighted logistic and Dice losses are complementary, while removing weighting lowers fluid DS by 0.06 and degrades several layer metrics.The unweighted BL-8 baseline performs worst for multiple MAD-LT and CE measurements.
6. Conclusion
ReLayNet is an end-to-end fully convolutional framework for segmenting seven retinal layers and fluid masses, validated on expert-annotated OCT B-scans. It outperformed state-of-the-art and incremental baselines across standard metrics, including pathological scans, while achieving approximately 0.01-second single-B-scan test time.
- ReLayNet segments seven retinal layers and fluid masses in retinal OCT B-scans using an end-to-end fully convolutional framework.The model is trained and validated on a publicly available benchmark of expert-annotated B-scans from 10 patients.
- Its combined training objective minimizes weighted logistic loss and Dice loss.
- Approximately 0.01 seconds are required to segment a single B-scan, supporting the framework’s suitability for clinical applications.
- ReLayNet outperformed five state-of-the-art methods and eight incremental baselines using Dice loss, retinal thickness estimation, and deviation from layer contours.
- The framework reliably segmented retinal structures even when severe pathology disrupted the normal layered retinal structure.