Predicting Radiologist Expertise from 3D Gaze Patterns During CT Interpretation

1 Introduction

• Objective expertise assessment can support training, and radiologists’ visual search behavior differs consistently between experts and novices, but gaze differences depend on task and modality.Eye tracking enables quantitative measurement of spatial and temporal attention during diagnostic reading, while handcrafted statistical measures may not capture the complexity of visual expertise.Source: §1 · p. 1
• Most gaze-based skill-assessment studies use 2D views or video clips, leaving their generalization to 3D CT unclear; only one study had examined 3D CT scanpath prediction.Scanpath-prediction models learn sequential patterns from expert gaze recordings and can generate expert-like trajectories conditioned on medical images.Source: §1 · p. 2
• The paper introduces a DINOv2-based transformer that integrates radiologist gaze into volumetric CT representation learning and compares it with adapted approaches on 182 reading sessions.Per-patch gaze signals enter through a learnable log-space self-attention bias and gaze-weighted pooling; the sessions include synchronized eye tracking from five expert and novice radiologists.Source: §1 · p. 2

2.1 Data Collection

• Three expert radiologists and two novice readers analyzed 40 LIDC–IDRI scans, including 24 cancer and 16 non-cancer cases.The cancer cases contained 47 nodules: 20 small-to-intermediate and 27 large.Source: §2.1 · p. 2
• Gaze and audio were recorded during routine CT review, with calibration, verbal session markers, and synchronized fixation on a reference point.Readers used a Tobii Eye Tracker 4C at 90 Hz and could zoom, adjust window level and width, and navigate orthogonal planes.Source: §2.1 · pp. 2–3

2.2 Data Preprocessing

• Gaze coordinates were synchronized to video frames, drift-corrected, and mapped through affine alignment to pixel locations in the corresponding CT slices.Viewer metadata including CT plane, slice number, window level, and window width were extracted to retrieve and window the original NIfTI slices.Source: §2.2 · p. 3
• Fixations were extracted with I-DT and projected into per-slice heatmaps smoothed with a session-calibrated Gaussian kernel.The duration and dispersion thresholds were 100 ms and 1° of visual angle, respectively; heatmaps were normalized.Source: §2.2 · p. 3

3 Methods

• Windowed CT slices are converted to triplicated pseudo-RGB images, and each slice’s heatmap is average-pooled to the patch grid and ℓ1-normalized into patch gaze weights.Source: §3 · p. 4
• Each slice is patch-embedded with a prepended CLS token and positional embeddings, then processed by transformer blocks with a gaze-derived additive log-space attention bias.The CLS key receives zero bias; the bias is shared across heads within a layer and is identical across layers.Source: §3 · pp. 4–5
• The bias scales each key’s attention weight by w_j^α, amplifying heavily fixated patches and suppressing patches receiving little gaze.Source: §3 · p. 5
• Final patch tokens are pooled using normalized gaze weights into slice embeddings, and uniformly sampled slices are mean-pooled into a session embedding during training.The training subset contains K slices sampled uniformly from the D available slices.Source: §3 · p. 5
• A two-layer MLP maps the session embedding to logits for the novice and expert classes.Source: §3 · p. 5

Dataset and implementation

• The evaluation used 182 sessions from five radiologists, with 115 samples for training and a held-out test set of 67 samples (39 expert, 28 novice).Radiologists C and E were held out; models used stratified 5-fold cross-validation, and fold predictions were averaged at inference.Source: §4 · pp. 5–6
• Training sampled eight lung-windowed CT slices per volume and used Adam, binary cross-entropy, and validation ROC-AUC checkpoint selection for 100 epochs.The DINOv2 ViT-B/14 model had 86.8 M trainable parameters and was trained for approximately 17 hours.Source: §4 · p. 6

Evaluation

• Four adapted CNN skill-assessment models used gaze-derived modalities and fixation-ordered CT scans with gaze instead of operator-acquired standard-plane images.The comparison included Late, Intermediate, Hybrid, and Tensor Fusion CNNs, using either SGM and GTI or FO-CT+Gaze, SGM, and GTI; pupillary images were omitted.Source: §4 · p. 6
• Scanpath-prediction baselines adapted Lou et al.’s mammography model to CT using session-level or 20-slice chunk-level MIPs, and fine-tuned CT-Searcher on expert data.Expertise predictions were thresholded using NSS for Lou et al.’s model and MultiMatch vector similarity for CT-Searcher.Source: §4 · pp. 6–7

Ablation study

• DINOv2 achieved inter-patient separation of 0.9381±0.0261 with representational variance of 0.6814; Med3D, UniMISS, and SwinUNETR showed the reported limitations.Med3D had cross-CT similarity 0.9998 and variance 0.0002; UniMISS had inter-patient separation of 0.9801 ± 0.0193; SwinUNETR had between-patient discrimination of 0.9349 ± 0.0668 and consecutive similarity of 0.9999.Source: §4 · p. 7
• Frozen DINOv2 embeddings produced Mean Val AUC: 0.4455 ± 0.045 and AUC on test: 0.5234; subsequent ablations varied attention mode and pooling strategy.The tested attention modes were Gaze-Bias, Fixation-Mask, and None, paired with Gaze-Weighted or CLS pooling.Source: §4 · p. 7

• AUC 0.91, F1 0.86, and specificity 0.93 were the proposed model’s highest values, and it led on almost all reported metrics.Source: §5 · p. 7
• Adapted Sharma et al. fusion models performed comparably on CT, while TF-CNN with SGP and GTI had sensitivity 0.92 but specificity 0.61.Novices were frequently misclassified as experts.Source: §5 · pp. 7–8
• Adding CT image information did not consistently improve performance: IF-CNN ROC-AUC rose 0.73 to 0.74 and LF-CNN rose 0.75 to 0.76.The authors suggest limited added value beyond other features or overfitting from increased complexity on the small dataset.Source: §5 · p. 8
• Lou et al. ’s computationally expensive three-encoder approach achieved ROC-AUC 0.56 with chunk-level processing, while CT-Searcher achieved ROC-AUC 0.88.CT-Searcher was designed for scanpath tasks rather than skill classification; its results suggest scanpath representations can encode skill-relevant information.Source: §5 · p. 8
• AUC 0.91 with Gaze-Bias plus Gaze-weighted pooling outperformed attention-only AUC 0.89 and pooling-only AUC 0.86.The authors attribute the gains to gaze guiding feature extraction and determining which regions contribute most during aggregation.Source: §5 · p. 8, Table 2 · p. 8
• Gaze-based assessment could support trainee feedback and inform detection systems tailored to less experienced readers; future work will expand beyond five radiologists for clinical validation.Source: §5 · p. 8