PaperlayerView exact revision on arXiv

Source-linked AI summary

Pushing the (Decision) Boundaries: Dynamically Calibrating Differentially Private Noise to Explainability in Federated Learning

Michael Khavkin, Kichang Lee, Jaeho Jin, JeongGil Ko, Eran Toch

arXiv:2609.03851v1cs.LG

TL;DR

Differential privacy in federated learning can degrade explanation fidelity, while prior feature-importance adaptations do not calibrate noise to explanation quality during training. XCal-FL uses closed-loop explainability and decision signals to adapt DP noise during cross-silo image-classification training, achieving stronger predictive performance, explanation fidelity, and privacy-budget efficiency across medical imaging tasks.

  • Problem

    Differentially private FL can distort learned representations and degrade explanation fidelity, while prior static feature-importance methods do not adapt noise to explanation quality during training.

  • Method

    XCal-FL dynamically calibrates local-training DP noise from logit changes, counterfactual margins, and saliency concentration while tracking adaptive privacy accounting.

  • Results

    Across medical imaging tasks, XCal-FL improves F1-score by over 10% and ROAD fidelity by up to 5× over static DP-FL, with up to 25% higher privacy-utility efficiency and threefold higher privacy-explainability efficiency.

  • Takeaways & Limitations

    Explanation fidelity is a distinct privacy-trade-off dimension because it changes non-linearly with cumulative privacy loss rather than following predictive performance alone.

  • Takeaways & Limitations

    XCal-FL is designed for image classification and relies on Grad-CAM, so other attribution methods or non-vision domains may require adaptation and retuning.

Abstract

from arXiv · show

Federated Learning (FL) with Differential Privacy (DP) is increasingly adopted to preserve data confidentiality in distributed machine learning. However, DP noise distorts learned representations and degrades explanation fidelity, limiting differentially private FL where trustworthy explanations are required, such as assistive clinical diagnosis. Prior work adapted DP noise with static feature-importance signals, restricting explainability to post hoc analysis and precluding noise calibration to explanation quality during training. We propose XCal-FL, a closed-loop, explainability-driven local training algorithm for image classification in cross-silo FL that dynamically calibrates DP noise from three complementary signals: (1) prediction logit variations, measuring causal influence on model confidence, (2) counterfactual margins, capturing decision-boundary sensitivity, and (3) saliency concentration, quantifying spatial coherence of model attention, while enforcing formal DP guarantees via adaptive privacy accounting. Experiments on three medical imaging datasets across varying FL configurations show that XCal-FL yields more accurate and interpretable global models, improving predictive performance by over 10\% and explanation fidelity by up to 5$\times$ over static-noise FL, and outperforming state-of-the-art adaptive DP methods in fidelity. XCal-FL also achieves higher privacy-budget efficiency, turning each unit of cumulative privacy loss into larger gains in both accuracy and explanation fidelity. Our analysis further reveals that, unlike predictive performance, which scales roughly linearly with privacy loss, explanation fidelity exhibits non-linear dynamics. These findings suggest explainability is a distinct dimension of the privacy trade-off that cannot be inferred from utility alone, with implications for training and privacy-budget allocation in decision-critical applications.

1 Introduction

Differentially private federated learning protects distributed sensitive data but can over-perturb informative features and degrade explanation fidelity. XCal-FL addresses this gap with closed-loop, explainability-driven noise calibration and reports improved predictive performance, explanation fidelity, and privacy-budget efficiency across medical imaging tasks.

  • Motivation: Standard DP-FL uniformly perturbs gradient features, over-perturbing discriminative features and wasting noise on less informative ones; feature-importance methods remain fixed rather than training-adaptive.These limitations motivate calibrating noise to explainability and decision-related signals during training.
  • Closed-loop noise calibration: XCal-FL dynamically calibrates DP noise during local training from logit variations, counterfactual margins, and saliency concentration while preserving formal (ε,δ)-DP guarantees.The adaptive privacy cost is tracked throughout training.
  • Privacy-explainability dynamics: XCal-FL evaluates privacy-explainability trade-offs directly, finding that predictive performance scales approximately linearly with cumulative privacy loss while explanation fidelity follows non-linear dynamics.The paper argues that utility alone cannot capture the effect of DP on explanation quality.
  • Empirical evaluation on medical imaging: The evaluation covers blood cell classification, pneumonia detection, and melanoma detection under cross-silo FL with 3, 10, or 50 clients and varying data heterogeneity.Explanation fidelity is measured with the ROAD metric.
  • Empirical evaluation on medical imaging: Over 10% higher F1-score and up to 5× higher ROAD explanation fidelity than static DP-FL are reported across three medical imaging datasets and varying FL configurations.The method also outperformed AGP by over 10% in F1-score and over 75% in explanation fidelity on average.
  • Related work: Existing adaptive DP-FL methods generally adjust clipping, noise, or budget allocation without integrating explainability during training, whereas AGP uses fixed-magnitude spatial noise allocation.XCal-FL instead dynamically regulates the noise multiplier from measured model responses.

4 XCal-FL: Explainability-driven FL Training with Differentially Private Noise Calibration

XCal-FL calibrates client-side DP noise during local training using a composite explainability signal, while tracking privacy expenditure and adapting noise around a reference level. The procedure combines decision sensitivity, causal influence, and spatial concentration to preserve informative representations and explanation fidelity.

  • 4.1 Main Idea: XCal-FL dynamically calibrates DP noise from explainability signals during client training while preserving the target (ε,δ)-DP guarantee through adaptive accounting.The method partitions the privacy budget between signal privatization and gradient perturbation, then tracks cumulative gradient privacy cost with an RDP accountant.
  • 4.1 Main Idea: A masked salient region supplies the feedback: logit decreases measure predictive influence, counterfactual margins assess class separation, and concentration measures whether attention is localized.Grad-CAM identifies the top-q region before forward and masked-forward passes produce the decision-level indicators.
  • 4.1 Main Idea: The composite signal combines logit change, counterfactual margin, and saliency concentration to measure causal influence, decision-boundary sensitivity, and spatial specificity.The three components are weighted into a unified explainability score; no single component is sufficient on its own.
  • 4.2 XCal-FL Training Procedure: The explainability signal is privatized, smoothed, and mapped to a bounded noise band, with each mini-batch noise level charged to the privacy accountant.The adaptive procedure adjusts noise between σ_min and σ_max around σ_ref while enforcing the allocated gradient privacy budget.
  • 4.2 XCal-FL Training Procedure: The implementation uses fixed-size mini-batch sampling rather than the Poisson subsampling assumed by RDP accounting, so reported privacy parameters are approximate.This is identified as an implementation caveat for interpreting the privacy guarantee.
  • 4.2 XCal-FL Training Procedure: Signal-dependent regularization reduces noise when explanations are coherent and increases it when the gradient signal is unreliable, linking noise calibration to learning fidelity.The paper argues that predictive performance and explanation fidelity respond differently to cumulative privacy loss because spatial coherence is more perturbation-sensitive.

5 Empirical Evaluation on Medical Imaging

The evaluation tests XCal-FL on diverse medical imaging tasks and federated configurations using CNNs, controlled adaptive-noise settings, and predictive and explainability metrics. The reported results compare adaptive training with static-noise FL and related configurations under matched experimental conditions.

  • 5.1 Experimental Setup: The study evaluates blood cell, chest X-ray pneumonia, and dermoscopic melanoma classification with EfficientNet-B0 and ResNet-18 models in federated settings.The datasets provide varied medical imaging tasks, while the two CNN architectures test the method across model types.
  • 5.1 Experimental Setup: The experiments simulate both class-label shift and covariate shift across clients, including systematically varying image brightness, over 30 communication rounds.The covariate-shift construction preserves the underlying label structure while changing client illumination conditions.
  • 5.1 Experimental Setup: The configuration allocates 10% of ε to the explainability signal and 90% to gradients, with b = 0.2 producing σ_step ∈ [0.8σ_ref, 1.2σ_ref].The experiments also use δ = 10^-5 and mask the top q = 0.2 saliency fraction.
  • 5.2 Evaluation Metrics: F1-score measures predictive performance alongside cumulative privacy loss, while explanation quality is assessed through quantitative and qualitative analyses.The evaluation uses F1-score because medical classification tasks may have imbalanced class or feature distributions.

6 Results

XCal-FL improved predictive performance and explanation fidelity over static-noise training across medical imaging tasks and privacy settings. Its privacy-efficiency results show that predictive performance and explainability follow different dynamics as privacy loss accumulates.

  • Predictive Performance: 0.716 F1 versus 0.654 with static noise in blood cell classification under 2.77 ≤ σ ≤ 4.16, while XCal-FL also outperformed AGP across configurations.Both adaptive and static DP models remained below centralized and non-DP FL performance in this setting.
  • Model Explainability: 80.57%, 7.23%, and 42.66% ROAD scores were reached for blood cell, pneumonia, and melanoma tasks, with XCal-FL producing larger confidence drops than static training.Blood cell confidence fell by over 80% when the most salient regions were removed, compared with a 15.56% drop for static training.
  • Client and Distribution Effects: ROAD scores ranked IID highest, followed by label shift and covariate shift, while increasing client counts produced small positive explainability effects under IID and label shift.For label shift, ResNet-18 performance increased from 0.70 at N=3 to 0.86 at N=50.
  • Privacy-Utility Efficiency: XCal-FL improved privacy-to-utility efficiency across datasets, including ΔF1/Δε ratios of 0.223 versus 0.178 for blood cell classification and 0.209 versus 0.106 for melanoma.The melanoma result was nearly twice the static baseline efficiency.
  • Privacy-Explainability Efficiency: 82.12 versus 10.57 ΔROAD/Δε in blood cell classification showed higher privacy-explainability efficiency for XCal-FL, with corresponding advantages in pneumonia and melanoma.The adaptive model reached a local ROAD maximum after 20% of normalized privacy expenditure, whereas the static baseline peaked after roughly 80%.
  • Performance and Explainability Dynamics: F1 trajectories were approximately monotonic and near-linear, whereas ROAD trajectories were nonmonotonic, so privacy-explainability trade-offs cannot be inferred from privacy-utility trade-offs alone.Explanation fidelity may peak before predictive performance stabilizes, allowing earlier termination when explainability is the primary objective.

7 Ablation Study: Explainability Signal Weights

The ablation study found that component weighting affected explanation quality more strongly than predictive performance. The balanced configuration using logit change, counterfactual margin, and saliency concentration generally provided the strongest overall behavior.

  • Ablation Design: The ablation varied isolated components, balanced baselines, relative weights, and concentration scaling while reporting macro F1 and ROAD across datasets.The component definitions are α for logit change, β for counterfactual margin, and γ for saliency concentration.
  • Predictive Performance: Under strict privacy, balanced weighting produced the strongest melanoma F1 performance, while amplified saliency concentration degraded performance at ε = 0.05.At relaxed privacy levels ε ≥ 5, all configurations converged to comparable F1 scores of 0.75–0.88.
  • Explainability Signal Weights: Balanced weighting (α = β = γ = 1) achieved the highest ROAD scores at most noise levels, supporting inclusion of all three explainability components.The pure logit-change configuration was competitive at ε = 0.5 but did not generalize across noise levels.
  • Explainability Signal Weights: Explainability differences across configurations were more pronounced than predictive-performance differences, with the balanced configuration reaching a 110% ROAD change at ε = 50.The counterfactual-margin-dominant configuration produced lower and more variable ROAD scores.

8 Execution Runtime Complexity Analysis

XCal-FL adds explainability-signal computation to standard DP-SGD, increasing runtime relative to conventional training. The reported overhead is potentially manageable in cross-silo settings but challenging for resource-constrained clients.

  • Runtime Overhead: Two additional inference-mode forward passes per mini-batch incur up to 80% overhead relative to standard DP-SGD.The analysis used a 64GB RAM machine with an NVIDIA RTX 2080 Ti across three datasets and two model architectures.
  • Runtime Overhead: XCal-FL’s overhead is attributed to computing an explainability-driven signal in addition to the standard DP-SGD training step.Standard DP-SGD costs approximately 3F + O(P) per mini-batch, where F is a training-mode forward pass and P is the parameter count.

9 Discussion and Limitations

XCal-FL combines explainability and decision-related signals for adaptive DP noise while preserving formal privacy guarantees. Its behavior reveals non-linear explanation dynamics and important scope boundaries for attribution method, deployment scale, and privacy granularity.

  • Discussion: XCal-FL combines logit change, counterfactual margins, and saliency concentration to calibrate DP noise for informative regions, decision boundaries, and compact attention.The three signals provide complementary contributions to explainability-driven noise control.
  • Discussion: The realized noise multiplier remains within [σmin, σmax], while Rényi accounting enforces the allocated (ε, δ)-DP guarantee at every training step.Updates exceeding the allocated privacy budget are prevented, matching standard DP-SGD calibrated to the same guarantee.
  • Discussion: Predictive performance improves approximately monotonically with privacy loss, whereas ROAD explanation scores peak early and then stabilize in some configurations.These non-linear explanation dynamics motivate evaluating explainability separately from standard privacy-utility behavior.
  • Limitations: XCal-FL is limited to image classification settings using Grad-CAM, because replacing the attribution method may change signal characteristics and require retuning.Applicability to non-vision domains remains future work.
  • Limitations: Scaling gains are moderate and sometimes inconsistent under covariate shift, while two additional forward passes can add up to 80% overhead relative to standard DP-SGD.The method is presented as best suited to cross-silo deployments; it also provides record-level rather than patient-level privacy guarantees.

10 Conclusion

XCal-FL calibrates differential privacy noise using explainability signals to balance privacy, predictive utility, and explanation fidelity. Across medical imaging tasks, it outperforms fixed-noise and adaptive-DP baselines while improving privacy-budget efficiency for decision-support models.

  • Conclusion: XCal-FL achieves over 10% higher performance and more than 75% higher attribution-explanation fidelity than state-of-the-art adaptive-DP methods in FL.It also reaches up to 25% higher privacy-utility efficiency and up to threefold higher privacy-explainability efficiency.

Ethical Considerations

The study uses anonymized public medical datasets and reports no collection, processing, or storage of personally identifiable information. An anonymous repository provides implementation and configuration materials for reproducing the experiments, while generative AI tools were used only for text revision.

  • Ethical Considerations: The experiments rely exclusively on publicly available, anonymized datasets, with no personally identifiable information collected, processed, or stored.The authors state that they complied with each provider’s terms of use.
  • Ethical Considerations: An anonymous repository contains the client-side training procedure, server-side aggregation, model architectures, and configuration files for reproducing the reported experiments.The repository supports reproduction of the proposed XCal-FL procedure.
  • Ethical Considerations: Generative AI tools were used to revise wording and correct language, and the authors manually verified the resulting text and retained responsibility for its accuracy and integrity.The stated use concerns editing rather than experimental analysis.

Appendix

The appendix describes the server-side adaptive explainability-driven DP-FL procedure built on FedAvg. It specifies training, privacy, explainability, and client-aggregation parameters used to produce the final global model.

  • Appendix: The server-side algorithm builds on FedAvg and coordinates adaptive explainability-driven client training across sampled clients.Each round distributes the current global model, collects locally trained private models and cumulative privacy loss, then aggregates the updates.
  • Appendix: The procedure parameterizes training with learning rate and batch size, privacy with clipping and noise-accounting variables, and explainability with method-specific controls.The privacy parameters define the noise range and failure probability δ.
  • Appendix: At each round, the server samples m clients, broadcasts global parameters, receives private local models with cumulative εcum, and computes weighted aggregation.The resulting global model is broadcast for the subsequent round.
  • Appendix: The algorithm initializes global model parameters w0 and outputs the final global parameters wT after the configured training rounds.The appendix identifies wT as the final global model output.

B Privacy Guarantee Under Adaptive Noise Calibration

XCal-FL partitions privacy loss between explainability signals and noisy gradients, then uses adaptive composition and a privacy filter to preserve the target (ε,δ)-DP guarantee.

  • XCal-FL splits the total privacy budget into (εs,δs) for explainability signals and (εg,δg) for gradient perturbations, with (ε,δ)=(εs+εg,δs+δg).
  • The step-specific noise multiplier is computed from privatized signals and public calibration bounds, so its adaptive use incurs no additional privacy loss through post-processing.
  • The adaptive gradient mechanism admits, clamps, or halts noisy updates so cumulative gradient privacy cost never exceeds (εg,δg).
  • If calibrated noise remains below the reference multiplier, a client may exhaust its gradient budget early and halt noisy updates before completing all T steps.
  • Sequential composition combines the signal and filtered gradient mechanisms to establish the overall target (ε,δ)-DP guarantee.

C Description of Medical Imaging Datasets

The evaluation uses medical imaging datasets and compares XCal-FL configurations across predictive performance and ROAD explanation fidelity under varied privacy and federated settings.

  • Description of Medical Imaging Datasets: XCal-FL evaluation spans blood-cell classification, pneumonia detection from chest X-rays, and melanoma detection from dermoscopic images across medical imaging tasks.
  • Predictive Performance: Balanced component weights generally preserve predictive performance, with Blood Cells reaching F1=0.72 at ε=0.5 and Chest X-ray exceeding 0.61 at ε=5 and ε=50.
  • Explainability: Explainability differences are larger than predictive differences: Blood Cells favors Balanced or Logit Change-dominant weights, whereas Chest X-ray favors dampened saliency concentration.
  • Evaluation Setup: At ε=0.5, balanced XCal-FL configurations are compared with FL+DP across three datasets, model architectures, client counts, and IID or non-IID distributions.
  • Explainability: Amplifying saliency concentration performs poorly on Chest X-ray, producing the lowest ROAD scores across most noise levels.

F Detailed Complexity Analysis

XCal-FL adds local explainability computation to each client’s DP-SGD step, increasing runtime while leaving communication cost unchanged.

  • Runtime analysis measures single-client round duration and overhead relative to standard static-noise DP-SGD on an RTX 2080 Ti system.
  • The added signal computation requires two inference-mode forward passes, yielding theoretical overhead 2·Finf/(3F+O(P)).
  • XCal-FL incurs up to 80% overhead over static DP-SGD, depending on dataset and architecture.The measured overhead is reported across three datasets and two architectures.
  • The overhead is purely local and does not increase communication cost, which remains the dominant bottleneck in cross-silo federated training.
Loading 2609.03851v1…