PaperlayerView exact revision on arXiv

Source-linked AI summary

OCT-FedSIR: Toward Trustworthy Federated Ophthalmic Learning under Annotation Noise

Sina Gholami, Abdulmoneam Ali, Tania Haghighi, Rashadul H. Badhon, Behafarin Emam, Sally S. Y. Ong, Atalie C. Thompson, Theodore Leng, Ahmed Arafa, Jennifer I. Lim, Minhaj Nur Alam

arXiv:2609.14734v1cs.LGcs.AIcs.DC

TL;DR

Ophthalmic federated learning must contend with unreliable annotations that can be hard to distinguish from legitimate clinical differences across clients. OCT-FedSIR uses spectral characterization and selective relabeling; across 117 experimental conditions, it achieved 86.73% mean accuracy, exceeding RoFL and FedCorr.

  • Problem

    In ophthalmic FL, structured annotation errors and legitimate differences in clinical case mix can be difficult to distinguish, although unreliable labels can harm learning and correction can damage originally correct labels.

    Source: Introduction · pp. 2–3

  • Method

    OCT-FedSIR combines class-balanced spectral estimation, Stage-I logit adjustment, and complementary descriptors to identify unreliable clients and guide selective relabeling using reliable-client spectral information.

    Source: Introduction · p. 3

  • Results

    Across 117 experimental conditions, OCT-FedSIR achieved 86.73% overall mean accuracy, 6.79 percentage points higher than RoFL and 7.98 points higher than FedCorr.

    Source: Discussion · p. 13

  • Takeaways & Limitations

    Under the controlled conditions evaluated, retaining corrected clients was generally more effective than discarding them, so annotation reliability and data value should not be treated as equivalent.

    Source: Discussion · p. 15

    The framework depends on sufficient reliable class support for spectral references, and performance may deteriorate when reliable clients are too few or diagnostic classes are absent or sparsely represented.

    Source: Discussion · p. 15

Abstract

from arXiv · show

Federated learning enables collaborative model development without centralizing patient data, but annotation reliability at participating institutions cannot always be assumed. In ophthalmic imaging, differences in disease prevalence and class composition can resemble changes caused by corrupted supervision. We introduce OCT-FedSIR, a reliability-aware spectral framework for federated OCT classification under client-dependent annotation noise and heterogeneous data distributions. OCT-FedSIR combines class-balanced spectral estimation, Stage-I logit adjustment, complementary spectral descriptors, selective spectral relabeling, and noise-aware federated optimization. We evaluated the framework on the Kermany, University of Illinois Chicago, and Wake Forest datasets under symmetric and structured asymmetric noise and three levels of non-IID heterogeneity. Across 117 experimental conditions, OCT-FedSIR achieved a mean accuracy of 86.73%, compared with 79.94% for RoFL and 78.75% for FedCorr. It correctly separated clients with original and corrupted annotations across all evaluated conditions, while the original FedSIR identification procedure was less robust, particularly under asymmetric noise. Spectral relabeling recovered 77.2% of corrupted annotations with 91.3% correction precision and a 3.5% false-correction rate. Retaining corrected clients outperformed spectral pruning by 9.30 percentage points on average. These findings show that annotation noise can often be identified and corrected without discarding informative client data.

Introduction

Ophthalmic federated learning must distinguish unreliable supervision from legitimate clinical variation: differences in case mix can resemble annotation corruption. OCT-FedSIR uses spectral client characterization and selective relabeling to address this challenge while evaluating across multiple datasets and noise settings.

Introduction

  • Federated ophthalmic learning keeps images local, but prior work has largely focused on data heterogeneity rather than annotation reliability.FedSIR [31] uses changes in feature geometry to identify label-corrupted clients and guide selective relabeling.Source: Introduction · pp. 2–3
  • Annotation errors can be structured, and correction risks both missed corruptions and harmful changes to originally correct labels.Reliable noisy-label learning therefore requires assessing recovery and preservation, not just downstream accuracy.Source: Introduction · pp. 2–3
  • Legitimate differences in prevalence, class balance, and client distributions can alter feature geometry and resemble annotation corruption.The challenge is distinguishing clinical heterogeneity from unreliable supervision before modifying labels or client contributions.Source: Introduction · p. 3
  • OCT-FedSIR combines class-balanced spectral estimation, Stage-I logit adjustment, and complementary similarity descriptors to distinguish annotation disruption from local class-composition variation.It uses spectral information from reliable clients to guide selective relabeling rather than automatically excluding unreliable clients.Source: Introduction · p. 3
  • The study measures sample-level recovery and harmful correction, and compares retaining corrected unreliable clients with pruning them.This tests whether useful local information can be recovered rather than discarded.Source: Introduction · pp. 3–4
  • Evaluation spans Kermany multiclass OCT disease classification, UIC ordinal DR classification, and Wake Forest binary GA classification under symmetric and structured asymmetric noise and varying non-IID heterogeneity.The study examines whether spectral characteristics distinguish annotation disruption from legitimate client heterogeneity and whether corrected clients should be retained.Source: Introduction · p. 4

Results

OCT-FedSIR was evaluated across three ophthalmic classification tasks under symmetric and asymmetric label noise and varying client heterogeneity. Results cover overall accuracy, client identification, annotation recovery, dataset-specific robustness, pruning, and component ablations.

Results

  • Experiments covered Kermany OCT, UIC ordinal DR, and WF binary GA classification, with symmetric and structured asymmetric noise across three heterogeneity levels.The evaluation compared conventional and noisy-label federated methods across 117 condition-level mean accuracies.Source: Results · pp. 4–5, Overall performance · p. 5

Overall performance

  • Across 117 conditions, OCT-FedSIR achieved 86.73% mean accuracy, compared with 79.94% for RoFL and 78.75% for FedCorr.The overall mean summarizes performance across datasets, noise mechanisms, noise levels, and heterogeneity settings.Source: Table 1 · p. 5, Overall performance · pp. 5–6

Spectral identification and annotation recovery

  • OCT-FedSIR achieved 100% identification accuracy, corrupted-client recall, and clean-reference precision; relabeling recovered 77.2% of corrupted labels with 91.3% precision and a 3.5% false-correction rate.FedSIR was less robust under asymmetric noise, chiefly by assigning additional noisy clients to the clean component.Source: Spectral identification and annotation… · pp. 6–7

Dataset-specific robustness

  • OCT-FedSIR led UIC conditions; Kermany results remained strong under severe asymmetric noise, while WF method differences were narrower under symmetric and clearer under asymmetric noise.The reported macro-F1 trends generally followed the accuracy trends across tasks.Source: Kermany · pp. 11–12

Retention versus pruning of clients with corrupted annotations

  • Across all 117 conditions, retaining corrected clients achieved 86.73% mean accuracy, compared with 77.43% for pruning, a 9.30-point difference.The retention advantage was also reported across Kermany, UIC, and WF dataset-specific averages.Source: Retention versus pruning of clients… · p. 12

Ablation study

  • Removing relabeling reduced mean accuracy by 7.26 percentage points, compared with reductions of 1.82 for KD, 1.05 for LA, and 0.52 for DaAgg.The one-component-at-a-time analysis found the complete framework most sensitive to removing relabeling.Source: Ablation study · p. 12, Table 4 · p. 13

Discussion

Across 117 experimental conditions, OCT-FedSIR improved classification and distinguished simulated clean from corrupted clients, while retaining corrected clients generally outperformed pruning. Its results remain bounded by the controlled corruption setting, class-support requirements, and the need for independent-site validation.

Discussion

  • Across 117 conditions, OCT-FedSIR averaged 86.73% accuracy, 6.79 percentage points above RoFL, 7.98 above FedCorr, and 5.03 below all-clean FedAvg.Source: Discussion · p. 13
  • OCT-FedSIR separated simulated clean and corrupted clients across evaluated settings, while FedSIR clean-client F1 fell to 68.5%–74.6% under structured asymmetric noise.The controlled corruption setup may yield cleaner separation than independent clinical institutions, where scanner, prevalence, image-quality, referral, and practice differences also alter feature geometry.Source: Discussion · pp. 13–14
  • Relabeling recovered a mean 77.2% of corrupted annotations, with correction precision of 86.7%–95.0% and a 3.5% mean false-correction rate.Labels changed only when representative and residual-subspace criteria agreed on the alternative class.Source: Discussion · p. 14
  • At α = 0.1, performance declined in several severe-noise settings; sparse class support can destabilize spectral references, and dataset comparisons also reflect differing tasks, cohorts, and architectures.Source: Discussion · p. 14
  • OCT-FedSIR remained robust in controlled asymmetric-noise experiments, whose designed category transitions should not be treated as estimates of real clinical error probabilities.Perturbations targeted selected diagnosis pairs for Kermany, neighboring DR severity categories for UIC, and GA-to-NGA errors for WF.Source: Discussion · p. 14
  • Retaining corrected clients exceeded spectral pruning by 9.30 percentage points on average, although pruning won under the most heterogeneous Kermany setting with severe asymmetric corruption.A corrupted client may still contain correctly labeled samples and clinically informative variation.Source: Discussion · pp. 14–15
  • Removing spectral relabeling reduced mean accuracy from 86.73% to 79.47%, the largest ablation decrease; KD, LA, and DaAgg removals reduced it by 1.82, 1.05, and 0.52 points.Because components operate jointly, these ablations show associations rather than isolated causal effects.Source: Discussion · p. 15
  • OCT-FedSIR depends on reliable class support for spectral references; independent-site validation is needed before clinical use, and robustness alone establishes neither clinical utility nor formal privacy.Future work should assess information leakage from model updates and shared spectral statistics.Source: Discussion · p. 15

Methods

The study evaluates federated OCT classification across three datasets and controlled client-level annotation corruption under varying data heterogeneity. Its protocol compares methods under matched client partitions, noise assignments, and training conditions.

Study design

  • OCT-FedSIR uses spectral client identification, periodic relabeling from clean-client references, and noise-aware federated optimization with LA, KD, and DaAgg.Source: Study design · p. 16

Datasets

  • The experiments cover multiclass OCT disease classification, ordinal DR severity classification, and binary GA classification across Kermany, UIC, and WF data.Kermany's original held-out test set remained separate; UIC and WF partitions were made at the patient level.Source: Kermany · p. 16, Wake Forest · p. 17

Federated client construction

  • Training data were distributed across 30 Kermany, 20 UIC, and 5 WF clients using class-wise Dirichlet allocations, with α values of 2.0, 0.5, and 0.1 controlling heterogeneity.Smaller α values produce more heterogeneous, increasingly uneven local class proportions; partitions were fixed across competing methods.Source: Federated client construction · p. 17

Client-dependent label-noise simulation

  • Across configurations, 60%–90% of clients were assigned corrupted labels, and ground-truth client status was withheld from OCT-FedSIR during training and identification.The clean/noisy assignments and corrupted labels were held constant across methods within each experimental condition.Source: Client-dependent label-noise simulation · p. 18
  • Noise was sampled within each represented class; symmetric corruption reassigned labels uniformly among other classes, while asymmetric corruption followed predefined dataset-specific transitions.Symmetric ratios ranged from 0.3 to 0.9 and asymmetric ratios from 0.4 to 0.9; the asymmetric mappings reflected related Kermany diagnoses, neighboring UIC severity classes, or GA-to-NGA transitions.Source: Client-dependent label-noise simulation · pp. 18–19, Figure 5 · p. 19

Class balancing for spectral estimation

OCT-FedSIR balances observed classes for spectral estimation, identifies likely clean clients, and uses spectral evidence to guide relabeling and federated optimization while retaining clients.

Class balancing for spectral estimation

  • Class balancing oversamples observed classes to equalize their counts without adding absent classes, and is used for feature extraction and spectral estimation rather than model optimization.Balancing uses potentially corrupted observed labels, not ground-truth clean labels.Source: Class balancing for spectral estimation · pp. 19–20

FL formulation

  • OCT-FedSIR estimates a clean-client subset, builds class-specific spectral references from those clients, and adjusts local supervision and server aggregation by estimated client status.The framework contrasts with conventional FedAvg, which does not explicitly account for annotation reliability.Source: FL formulation · p. 21

OCT-FedSIR for noisy-label ophthalmic FL

  • A two-component GMM classifies clients using spectral descriptors, designating the component with lower cross-class similarity as clean without using ground-truth client status.The descriptors summarize mean off-diagonal similarity, mean squared magnitude, and maximum.Source: Stage-I: spectral client identification · p. 22
  • Clean-client class-specific references combine representative and residual spectral directions through weighted projector averaging, with 12 residual directions used in the experiments.When fewer than 12 residual directions are available, all available directions are used.Source: Class-specific representative and… · p. 23
  • A label change is accepted only when representative- and residual-subspace predictions agree; otherwise, the observed annotation is retained.The agreement rule is intended to reduce the risk of changing uncertain clinical labels.Source: Figure 7 · p. 23, Spectral relabeling of noisy clients · p. 24
  • Distance-aware aggregation reduces the contribution of noisy-client models that deviate substantially from clean-client models without removing them from the federation.Predicted clean clients are aggregated by sample size to update the clean reference model.Source: Distance-aware aggregation (DaAgg) · p. 25

Network architectures

The study uses lightweight, dataset-specific ImageNet-pretrained convolutional networks, replacing each classifier with a layer matched to the target classes.

Network architectures

  • SqueezeNet1.1, ShuffleNetV2, and RegNetY-400MF were used for Kermany, UIC, and WF, respectively.Each model’s classifier was replaced with a linear layer matching the dataset’s class count, and the backbone and classifier were fine-tuned jointly.Source: Network architectures · p. 26
  • The architectures were selected to limit local computation, memory use, and communication cost in federated learning.Within each dataset, all federated methods used the same architecture and initialization.Source: Network architectures · p. 26
  • All three backbones were initialized with ImageNet-pretrained parameters before task-specific fine-tuning.The ImageNet classifier was replaced with a target-class linear layer.Source: Network architectures · p. 26

Training protocol and reproducibility

Experiments used shared seeds and initialization, a fixed communication-round protocol, and consistent image preprocessing across the three datasets.

Training protocol and reproducibility

  • Experiments used ImageNet-pretrained initialization and were repeated with three independent random seeds.Seeds were set consistently for model training, client partitioning, label corruption, data-loader workers, and batch generators.Source: Training protocol and reproducibility · p. 26
  • OCT-FedSIR trained for 100 communication rounds, with one local epoch and batch size 128 per client per round.Its noise-aware stage used Adam with weight decay 5 × 10^-4.Source: Training protocol and reproducibility · p. 26
  • Images were resized to 128 × 128 pixels and normalized using channel-wise mean 0.5 and standard deviation 0.5.Training additionally used random horizontal flipping; evaluation did not use random augmentation.Source: Training protocol and reproducibility · p. 26

Evaluation and statistical analysis

The study evaluates final-round classification, client identification, and sample-level relabeling, with competing methods compared under matched experimental conditions.

Evaluation and statistical analysis

  • Final-round global-model classification was summarized using accuracy and macro-F1, while held-out test performance was excluded from training and model-selection decisions.The global model was evaluated after each round to characterize training trajectories.Source: Evaluation and statistical analysis · p. 26
  • Client identification was assessed against simulated client status using identification accuracy, clean-client precision, recall and F1, noisy-client recall, and misclassification counts.Ground-truth status was used only for retrospective evaluation, not by the GMM during identification.Source: Evaluation and statistical analysis · pp. 26–27
  • Relabeling evaluation tracked edit rate, corrupted-label recovery, correction precision, false corrections to originally correct labels, and residual label noise.The metrics compare original reference labels, simulated corrupted labels, and post-correction labels.Source: Evaluation and statistical analysis · p. 27
  • Results were generally reported as mean ± standard deviation across three independent runs, with competing methods sharing the same partitions, corrupted labels, and initialization within each condition.The matched setup enables paired, directly comparable evaluations.Source: Evaluation and statistical analysis · p. 27

Use of generative artificial intelligence

ChatGPT was used only for language editing, and the authors reviewed and approved all revised text.

Use of generative artificial intelligence

Author information

The paper lists authors and institutional affiliations, assigns contributions across the research and manuscript work, and names Minhaj Nur Alam as the contact author.

Authors and affiliations

  • The authors are affiliated with UNC Charlotte, Wake Forest School of Medicine, Stanford University, and the University of Illinois Chicago.The listed authors are Sina Gholami, Abdulmoneam Ali, Tania Haghighi, Rashadul H. Badhon, Behafarin Emam, Ahmed Arafa, Minhaj Nur Alam, Sally S.Y. Ong, Atalie C. Thompson, Theodore Leng, and Jennifer I. Lim.Source: Authors and affiliations · pp. 30–31

Contributions

  • Contributions span study design, data collection and analysis, visualization, interpretation, manuscript preparation and review, supervision, funding, and project administration.Minhaj Nur Alam led supervision and funding acquisition; Sina Gholami and Minhaj Nur Alam handled project administration.Source: Contributions · p. 31

Corresponding author

Ethics declarations

UIC and Wake Forest data were collected under approved institutional protocols; the authors report no competing interests and provide data- and code-access information.

Ethics declarations

  • UIC and Wake Forest dataset collection followed protocols approved by the respective IRBs and the Declaration of Helsinki ethical principles.Source: Ethics declarations · p. 31

Competing interests

Data availability

  • The Kermany dataset is publicly available; UIC and Wake Forest datasets can be requested from the corresponding author subject to reasonable conditions.Source: Data availability · p. 31

Code availability

Supplementary material

Supplementary tables report classification accuracy for Kermany, UIC, and Wake Forest under symmetric and asymmetric label noise, with results averaged over three seeds.

Supplementary material

  • Kermany supplementary tables report classification accuracy under symmetric and asymmetric label noise, as mean ± standard deviation across three independent seeds.The tables are S1 and S2.Source: Table S1 · p. 33, Table S2 · p. 34
  • UIC supplementary tables report classification accuracy under symmetric and asymmetric label noise, as mean ± standard deviation across three independent seeds.The tables are S3 and S4.Source: Table S3 · p. 35, Table S4 · p. 36
  • Wake Forest supplementary tables report classification accuracy under symmetric and asymmetric label noise, as mean ± standard deviation across three independent seeds.The tables are S5 and S6.Source: Table S5 · p. 37, Table S6 · p. 38

Related papers

Loading 2609.14734v1…