Federated LoRA Adaptation of BiomedCLIP Across Four International Chest X-Ray Cohorts

Sanjaya Poudel; Nirajan Kunwor; Manish Dhakal; Debesh Jha; Sunil Kumar Gaire

Source-linked AI summary

Federated LoRA Adaptation of BiomedCLIP Across Four International Chest X-Ray Cohorts

Sanjaya Poudel, Nirajan Kunwor, Manish Dhakal, Debesh Jha, Sunil Kumar Gaire

arXiv:2609.02101v1Sep 2, 2026cs.LGcs.AIcs.CV

TL;DR

Federated PEFT of biomedical vision-language models raises questions about adapting across heterogeneous institutions without sharing data and about aggregating compact LoRA updates correctly. The paper benchmarks BiomedCLIP with federated LoRA across four chest-radiograph cohorts, finding improved performance over the frozen backbone and isolated training, with SVD-based aggregation essential and FedProx adding no measurable benefit in single-seed runs.

Problem
Federated PEFT of biomedical vision-language models remains insufficiently examined across heterogeneous, geographically distributed chest-radiograph cohorts and aggregation settings.
Method
The study federates BiomedCLIP LoRA adapters across four public chest-radiograph cohorts using FlexLoRA’s SVD-based product-space aggregation without pooling raw images.
Results
Federated LoRA improves shared-class AUC over frozen BiomedCLIP on all four cohorts, raising mean AUC from 0.687 to 0.802 and approaching the centralized 0.812 reference without pooling data.
Takeaways & Limitations
Biomedical vision-language models can be adapted collaboratively across heterogeneous institutions while data remain local, with aggregation performed in product space.
Takeaways & Limitations
The study cannot exclude within-patient leakage and identifies multiple seeds, per-finding evaluation, additional cohorts and backbones, and differential privacy as future work.

Abstract

from arXiv · show

Federated learning (FL) lets institutions train a shared model without exchanging data, and Low-Rank Adaptation (LoRA) makes this practical at scale by communicating only compact low-rank updates. Biomedical imaging is a compelling setting for this combination: patient data are archived behind privacy regulations, and institutions differ widely in scanners, protocols, and compute. Such heterogeneity raises the question of how federated LoRA updates should be aggregated, increasingly pressing as multimodal vision-language models become central to medical image analysis. We benchmark federated Parameter-efficient fine-tuning (PEFT) of BiomedCLIP for chest radiograph classification across four public cohorts on three continents (USA, Vietnam, Spain). Federated LoRA adaptation improves shared-class AUC on all four cohorts over the unadapted BiomedCLIP backbone (mean 0.687 to 0.802), showing that the gains come from federated adaptation rather than from the pretrained model's zero-shot ability. Relative to isolated single-cohort training, federation improves the weaker cohorts while largely preserving the strongest and approaches a centralized reference (0.812) that pools all data. The singular value decomposition (SVD)-based product-space aggregation introduced by FlexLoRA is essential to this gain (naive factor averaging drops mean AUC by 0.097), whereas a drift-correcting optimizer (FedProx) shows no benefit over FedAvg in our single-seed runs, consistent with LoRA's low-rank updates already limiting client drift. Biomedical vision-language models can thus be adapted collaboratively across heterogeneous, geographically distributed institutions without centralizing data.

1 Introduction

This study benchmarks federated LoRA-based PEFT of BiomedCLIP for chest radiograph classification across geographically distinct cohorts, focusing on aggregation under heterogeneous biomedical data. It finds that SVD-based product-space aggregation enables gains over frozen and isolated baselines, while FedProx adds no clear benefit.

Study scope: Federated PEFT of BiomedCLIP is benchmarked across four geographically distinct chest X-ray cohorts, making the benchmark itself the primary contribution.The cohorts span NIH ChestX-ray14 and CheXpert in the USA, VinDr-CXR in Vietnam, and PadChest in Spain.
Main findings: Federated LoRA adaptation improves shared-class AUC over the frozen BiomedCLIP backbone on all four cohorts, increasing the mean from 0.687 to 0.802.This improvement is attributed to adaptation rather than BiomedCLIP’s pretrained zero-shot ability.
Main findings: Federation improves weaker cohorts while largely preserving the strongest, raising mean shared-5 test AUC from 0.776 to 0.802 and approaching the centralized 0.812 reference without pooling data.CheXpert improves by 0.038, VinDr by 0.016, NIH ties, and PadChest decreases slightly.
Aggregation: SVD-based aggregation is essential because naive factor averaging drops mean shared-5 AUC by 0.097, reaching the frozen-backbone level.FlexLoRA reconstructs client updates, averages them in product space, and redistributes low-rank factors through truncated SVD.
Optimization: FedProx provides no meaningful benefit over FedAvg under LoRA, with mean AUC values of 0.799 and 0.802, respectively.The comparison is based on single-seed runs and is consistent with LoRA limiting client drift.

2 Related Work

Prior work established federated learning and parameter-efficient adaptation for medical imaging, while biomedical vision-language models remain comparatively underexplored in federated PEFT. This paper addresses that gap with a systematic federated benchmark for a biomedical VLM.

Federated learning: Federated learning supports collaborative medical-image model training without sharing patient data, but prior chest radiograph studies mostly use convolutional networks rather than multimodal foundation models.FedAvg, FedProx, SCAFFOLD, and FedBN address optimization under heterogeneous data.
Parameter-efficient fine-tuning: Parameter-efficient fine-tuning updates only a small subset of parameters, with LoRA the most widely adopted approach in this literature.FedIT combined LoRA with federated averaging, and later work refined low-rank-update aggregation for heterogeneous clients.
Biomedical vision-language models: Federated PEFT of biomedical vision-language models remains largely unexplored despite strong aligned image-text representations from medical VLMs.The paper positions its federated BiomedCLIP benchmark as addressing this gap.

3 Methods

The study federates BiomedCLIP LoRA adapters across four cohorts, exchanging adapter weights while keeping raw images local. It aggregates client updates in product space with truncated SVD to recover a rank-constrained global update.

Federated setup: Four cohorts iteratively train BiomedCLIP LoRA adapters, exchange only adapter weights, and keep raw images at their source cohorts.The study compares isolated training, one-shot aggregation, and five-round multi-round federation.
Model and adaptation: LoRA adapters on fused qkv projections update approximately 0.25% of parameters, reducing each client’s per-round payload to approximately 1.13 MB.The adapters use rank r = 8, scaling factor α = 16, and dropout 0.1.
Model and adaptation: Clients optimize LoRA adapters with an image–text contrastive loss using normalized image and text embeddings, symmetric cross-entropy losses, and prompts listing present findings or no findings.The temperature is τ = 0.07, corresponding to a logit scale of approximately 14.3.
Federated aggregation: Client LoRA updates are reconstructed as products, averaged in product space, and projected back to rank r using truncated SVD.Independent factor averaging is not equivalent to averaging the corresponding products.
Federated aggregation: 94.6% of averaged-update singular-value energy is retained by the top-r components across twelve adapted blocks.The reported range is 92.9–95.4%, so the rank-r projection discards only a small fraction of the aggregated update.
Optimization: FedAvg uses equal client weights with SVD-based aggregation, while FedProx adds a proximal regularizer with coefficient µ = 0.01.The comparison uses otherwise identical hyperparameters.

4 Experiments

Experiments evaluate federated BiomedCLIP across four public chest-radiograph datasets using standardized cohort partitions, multi-round training, and shared evaluation labels. Performance is measured with macro-AUC on native and cross-cohort shared findings, with bootstrap confidence intervals.

Datasets: The benchmark uses NIH ChestX-ray14 and CheXpert from the USA, VinDr-CXR from Vietnam, and PadChest from Spain.Native label sets contain 14 findings for NIH, CheXpert, and VinDr, and 30 frequent findings for PadChest.
Preprocessing and splits: CheXpert is restricted to frontal images with uncertain labels treated as positive, while PadChest is restricted to frontal adult radiographs.PadChest preprocessing includes 8-bit conversion using 1st–99th percentile normalization.
Training protocol: Five communication rounds use one local epoch per round, Adam optimization, learning rate η = 2 × 10^-4, and batch size 32.After each round, client LoRA states are aggregated and redistributed.
Evaluation: Evaluation reports macro-averaged AUC on held-out client test sets in full-class and shared-5 modes.Shared-5 covers Atelectasis, Cardiomegaly, Consolidation, Effusion, and Pneumothorax after canonical label mapping.
Evaluation: At test time, each finding is scored by cosine similarity between the image embedding and a single class text prompt.Bootstrap 95% confidence intervals use 300 test-set resamples and reflect test-set variability.

5 Results

Federated evaluation progressed from isolated and one-shot adapter training to five-round collaboration, with SVD-based aggregation supporting the strongest shared-5 performance. Federation improved weaker cohorts, approached the centralized reference, and produced lower but consistent full-class results.

Single-client baselines: Single-client baselines train each cohort independently and evaluate shared-5 macro AUC across all four held-out test sets.Table 2 organizes models trained on one cohort by rows and held-out test sets by columns.
Aggregation strategy comparison: 0.781 mean shared-5 AUC from SVD-based one-shot aggregation exceeded naive factor averaging (0.684) by 0.097.SVD reconstructs and re-factorizes the product-space update; naive factor averaging performs at the frozen-backbone level.
Multi-round federation: 0.802 mean shared-5 test macro AUC was achieved by five-round FedAvg, exceeding the best single-client baseline (0.776) and one-shot aggregation (0.781).FedProx reached 0.799 in the same single-seed comparison.
Multi-round federation: 0.812 was reached by a centralized pooled-data model, which federation approached while keeping data local.The centralized and federated models used different training budgets, so the comparison is not directly comparable.
Multi-round federation: 0.734 mean macro AUC was obtained on native full-class label sets, below the shared-5 FedAvg result of 0.802.Native evaluation covers 14 classes for NIH, CheXpert, and VinDr, and 30 for PadChest.
Multi-round federation: +0.038 on CheXpert and +0.016 on VinDr were FedAvg’s cohort-level gains over the strongest single-client baselines, while NIH tied and PadChest decreased by −0.028.The federated mean increased from 0.776 to 0.802.

6 Discussion

The discussion identifies federation’s main benefit as helping weaker cohorts while largely preserving the strongest, whereas FedProx adds little in the reported single-seed setting. The study’s conclusions remain bounded by limited experimental replication, evaluation scope, and possible within-patient leakage.

Federation gains: Federation improves weaker cohorts, with gains concentrated on CheXpert and VinDr while largely preserving PadChest.The discussion frames this as extending prior multi-site findings to vision-language PEFT.
Optimizer comparison: FedProx performed similarly to FedAvg in single-seed runs, providing little observed benefit under LoRA federation.The authors suggest low-rank updates and a frozen text encoder may limit client drift, but describe this explanation as empirical rather than mechanistic.
Limitations: Single-seed training on one accelerator leaves comparative gaps subject to caution because reported confidence intervals reflect test-set, not training, variance.The study also evaluates five shared findings via macro-AUC without calibration.
Limitations: Within-patient leakage cannot be excluded for the image-level VinDr split.The authors list multiple seeds, per-finding evaluation, more cohorts and backbones, and differential privacy as future work.

7 Conclusion

The study evaluates federated LoRA adaptation of BiomedCLIP across four chest-radiograph cohorts on three continents. It reports improved performance over the frozen backbone, near-centralized performance without pooling data, essential SVD aggregation, and no measurable FedProx benefit in the single-seed setting.

7 Conclusion: Federated LoRA adaptation of BiomedCLIP improved over the frozen backbone across four cohorts, raising mean shared-5 AUC from 0.687 to 0.802.The study covered four cohorts on three continents and approached a centralized reference of 0.812 without pooling data.
7 Conclusion: SVD-based aggregation was essential, while FedProx added no measurable benefit in the reported single-seed setting.The conclusion supports data-localizing adaptation of biomedical vision-language models.