Source-linked AI summary
Two for One: Privileged Learning for Resource-Constrained Screening
Lisa Y. W. Tang
TL;DR
When bilateral observations are unavailable at deployment, monocular models cannot directly use contralateral-eye information, while bilateral models require both images. The paper transfers bilateral teacher representations to a student that uses one eye at inference. The framework improved reported left-eye measures, with DR results also showing improvements across both eyes.
Problem
Monocular models cannot directly exploit contralateral-eye information, whereas bilateral models require both images at inference, motivating transfer of bilateral training information to a single-eye model.
Source: §1 · pp. 1–2Method
A teacher trained on paired fundus images guides a student trained to match its bilateral representation while using monocular inputs.
Source: §1 · p. 2, §2 · p. 2, §2.1 · p. 3Results
Left-eye AUROC improved from 0.931 to 0.966 and balanced accuracy from 0.834 to 0.879; DR AUROC improved across both eyes, with the largest left-eye change from 0.789 to 0.836.
Source: §3 · pp. 3–4Takeaways & Limitations
The student can operate from a single usable eye when the contralateral image is unavailable or ungradable, supporting use where only one fundus image is available.
Source: §1 · p. 2, §4 · p. 4The relatively small test set limits statistical power and estimate precision, and external validation is needed to assess robustness across populations, acquisition conditions, and imaging systems.
Source: §3 · p. 4
Abstract
from arXiv · showhide
Machine learning systems are often developed under data-rich conditions but deployed when only incomplete information is available. A natural example arises in colour fundus photography (CFP), where both eyes are commonly imaged because they can provide complementary information for ocular disease assessment. Prior approaches that exploit bilateral information either fuse both eyes in a single model or use knowledge distillation to compress a dual-eye teacher into a lightweight student that still requires both images at test time. However, bilateral acquisition may not always be feasible. We thus propose a privileged learning framework in which a teacher model has access to bilateral CFP images during training and learns representations informed by information from both eyes, while the student is trained to transfer this knowledge from the bilateral teacher using only a single-eye image as input. At inference, the student operates exclusively on this single-eye input and does not require the contralateral image. Experimental results show consistent improvements in area under the receiver operating characteristic curve over conventional monocular training, with the largest gains observed for left-eye images for the detection of age-related macular degeneration (0.931 vs. 0.966) and diabetic retinopathy (0.789 vs. 0.836).%, while requiring only a single-eye image at inference. By enabling models to benefit from bilateral information during training while requiring only a single-eye image at inference, our approach may support more reliable deployment in resource-constrained clinical and screening environments.
1 Introduction
Bilateral retinal images can offer complementary information, but a model that requires both at inference may be unusable when only one eye is gradable. The paper transfers bilateral training information to a student that predicts from one eye alone.
1 Introduction
- Bilateral acquisition may fail in routine screening, so requiring both images at inference could exclude cases where the available monocular image is sufficient.Images may be ungradable because of opacity, pupil dilation, defocus, or fixation issues, and quality can differ between eyes.Source: §1 · p. 1
- Bilateral inputs offer complementary information, while monocular models can operate with one usable eye but cannot directly use the contralateral eye.Requiring two usable images at inference creates a practical trade-off between complementary information and robustness to incomplete acquisition.Source: §1 · p. 1
- The student learns from a bilateral teacher during training but requires only a single-eye image at inference.The contralateral-eye image is treated as privileged information because it is available during training but not required at inference.Source: §1 · p. 2
- Earlier bilateral fusion and dual-eye distillation approaches still require paired images at inference, motivating a focus on deployment flexibility.The cited dual-eye distillation approach uses a student that fuses information from both eyes at test time.Source: §1 · p. 2
- The framework aims to transfer bilateral information available during development to a student that can use either eye independently.This combines richer training observations with monocular inference rather than imposing bilateral acquisition requirements on deployment.Source: §1 · p. 2
2 Methods
The framework trains a bilateral teacher and a monocular student, with feature-level distillation added to the student's classification objective. Evaluation uses patient-level splits on ODIR-5K for AMD and DR detection.
2 Methods
- The student’s objective combines both networks’ binary cross-entropy losses with a feature-level distillation loss.The teacher’s bilateral representation guides the student’s feature alignment.Source: §2 · p. 2
2.1 Model Architectures
- Unlike the monocular baseline, the privileged student matches its features to the detached bilateral teacher representation while sharing the baseline’s architecture and inputs.Detaching the teacher representation means distillation gradients update the student only; inference uses the student on a single eye.Source: §2.1 · p. 3
2.2 Experimental Setup
- Experiments use ODIR-5K to detect AMD and DR, with patient-level splitting to keep each patient’s images within one subset.The held-out test set is 20% of patients; validation supports early stopping and threshold calibration, and evaluation includes AUROC, AUPRC, balanced accuracy, precision, recall, and FPR.Source: §2.2 · p. 3
3 Results & Discussion
The privileged framework improves AUROC and balanced accuracy, with task-specific trade-offs in precision, recall, and false-positive rate. The authors caution that the estimates and generalizability are limited by the test set and single dataset split.
3 Results & Discussion
- The authors report 28 positive AMD cases (4.7%) and 189 positive DR cases (32.2%) in the held-out test set.They identify the relatively small test set and single dataset split as limitations, and call for external validation across populations, acquisition conditions, and imaging systems.Source: §3 · pp. 3–4
- 0.931 vs. 0.966 for left-eye AMD AUROC and 0.789 vs. 0.836 for left-eye DR AUROC favor the privileged framework.The framework also improves balanced accuracy across both tasks; Table 1 summarizes the task results and trade-offs.Source: §3 · pp. 3–4, Table 1 · p. 4
- For left-eye AMD, recall rises from 0.679 to 0.786 while FPR rises from 0.011 to 0.027 and precision falls from 0.760 to 0.595.The authors note that higher sensitivity may be desirable in screening when false negatives can be more costly than false positives.Source: §3 · pp. 3–4
- For left-eye DR, higher precision and lower FPR come at the expense of recall.The largest reported left-eye gains include AUROC of 0.789 vs. 0.836 and balanced accuracy of 0.576 vs. 0.749.Source: §3 · p. 4
4 Conclusions
The conclusion presents privileged distillation as a way to use paired-image training while making DR screening possible from a single fundus image. It identifies observed trade-offs and asymmetries as directions for future research.
4 Conclusions
- A bilateral teacher guides a student that processes monocular images for retinopathy detection.The framework is evaluated against a baseline that treats each eye independently.Source: §4 · p. 4
- The authors describe the approach as practical for DR screening when only one fundus image is available.Source: §4 · p. 4
- Observed trade-offs and asymmetries motivate future research into multimodal and asymmetric medical imaging.Source: §4 · p. 4
A FEATURE QUALITY VIA LINEAR PROBING.
Linear probing tests whether the privileged student learns higher-quality features by evaluating frozen backbones with newly trained linear classifiers.
A FEATURE QUALITY VIA LINEAR PROBING.
- Both the baseline and privileged student backbones were frozen, and their classification heads were replaced with fresh, randomly initialized linear layers.Only the new linear layers were trained.Source: A FEATURE QUALITY VIA LINEAR PROBING · p. 5
- The new linear classifiers were trained on the training set for 20 epochs, with the backbones kept entirely frozen.Source: A FEATURE QUALITY VIA LINEAR PROBING · p. 5
- Higher test-set AUROC from the privileged backbone under this protocol indicates greater linear separability and quality of the student’s learned features.Source: A FEATURE QUALITY VIA LINEAR PROBING · p. 5
A.1 RESULTS
Linear probing found improved AUROC for the privileged student’s frozen backbone on AMD and DR, indicating that distillation benefits extend beyond the original classification head.
A.1 RESULTS
- The privileged frozen backbone improved AUROC by +2.0 percentage points for AMD (0.815 vs. 0.836) and +2.9 percentage points for DR.Source: §A.1 · p. 5
- For AMD, the privileged backbone achieved AUROC 0.836 versus 0.815 for the baseline when evaluated with a new linear classifier.Source: §A.1 · p. 5
- The probing results suggest that privileged-information distillation benefits the learned backbone representations, not only the original classification head.Source: §A.1 · p. 5