PaperlayerView exact revision on arXiv

Source-linked AI summary

Does Federated Training Degrade Explainability? A Comparative Analysis of Grad-CAM Faithfulness in Centralized vs. Federated Deepfake Detection

Muhammad Uzair Shahid, Muhammad Zabil Mehboob

arXiv:2610.39483v1cs.AIcs.CRcs.CVcs.LG

TL;DR

This paper asks whether federated training affects the faithfulness of gradient-based explanations needed for forensic review. It evaluates Grad-CAM for federated deepfake detection across client heterogeneity levels, finding that faithfulness is preserved or exceeds the centralized baseline at lower heterogeneity but breaks down at the extreme. The results make heterogeneity a key boundary for interpreting explanation reliability in this setting.

  • Problem

    Whether privacy-preserving federated training preserves trustworthy gradient-based explanations for visual deepfake detection has not been studied, despite their use in forensic review.

    Source: A. Motivation · pp. 1–2

  • Method

    The study evaluates Grad-CAM faithfulness for EfficientNet-B4 deepfake detectors trained centrally or across five simulated clients with Dirichlet-controlled heterogeneity.

    Source: B. Federated Data Partitioning · p. 5, F. Grad-CAM Explanation Generation · p. 5

  • Results

    At α ∈{0.5, 1.0}, federated models produce higher mean AOPC than the centralized baseline; at α = 0.1, AOPC turns negative (d = 0.315).

    Source: F. Summary of Findings · p. 8

  • Takeaways & Limitations

    Explanation faithfulness is not inherently damaged by federation in this setting, but extreme client heterogeneity is associated with spatially incoherent and potentially causally misleading explanations.

    Source: §VI · p. 9

    The study uses a single-GPU simulation with five clients, one EfficientNet-B4 backbone, one Grad-CAM method, and controlled Dirichlet partitions that do not fully capture real deployment heterogeneity.

    Source: E. Limitations · p. 9

Abstract

from arXiv · show

Automated detection has become the essential safeguard against realistic synthetic media, but deploying it at scale requires training on sensitive data from multiple organizations, putting privacy and the transparency demanded by forensic review in direct conflict. Federated learning resolves the privacy side by sharing only model updates rather than raw data, but whether this comes at the cost of trustworthy gradient-based explanations, which analysts rely on to decide whether or not to trust a flagged clip, has not been studied. We present the systematic study of Grad-CAM explanation faithfulness under federated deepfake detection, evaluating an EfficientNet-B4 detector on FaceForensics++ across five federated clients under varying degrees of data heterogeneity. We find that the relationship between federation and explanation quality is non-monotonic: at near-IID (Independent and Identically Distributed Data) and moderate heterogeneity, federated models match or exceed the faithfulness of a centralized baseline, but at extreme heterogeneity, explanations break down entirely, becoming spatially incoherent and, on average, causally misleading. This degradation is not a general cost of federated training but a threshold effect tied to client heterogeneity, and we offer practical guidance for identifying when privacy-preserving deepfake detection can be deployed safely and when its explanations can no longer be trusted. Code and data partitions are available for reproducibility.

I. INTRODUCTION

Realistic deepfakes call for automated detection, but forensic use requires both privacy-preserving training and explanations that help reviewers assess decisions. The introduction motivates studying whether federated training affects explanation faithfulness and identifies the lack of quantitative comparisons for visual deepfake detection.

A. Motivation

  • Forensic reviewers need facial-region explanations alongside predictions, while pooling sensitive video data raises privacy concerns that rule out naive centralized aggregation.Federated learning shares model updates rather than raw samples, addressing the privacy demand without resolving whether explanations remain trustworthy.Source: A. Motivation · p. 1
  • Federated aggregation across clients with different manipulation mixes may alter learned features and gradients, potentially weakening the link between Grad-CAM heatmaps and model decisions.The introduction notes that analysts may trust flagged clips when heatmaps look plausible, making this potential effect consequential for review.Source: A. Motivation · p. 2

B. Research Gap

  • Federated-learning studies rarely assess gradient-based explanation quality, while deepfake explanations and faithfulness metrics have mainly been evaluated under centralized training.The gap spans federated optimization, deepfake detection, and XAI faithfulness evaluation.Source: B. Research Gap · p. 2
  • Existing federated-XAI applications show explanations can be generated without exposing client data, but they do not quantitatively compare explanation faithfulness for visual deepfake detection.The cited examples concern non-visual applications.Source: B. Research Gap · p. 2

C. Hypothesis

The paper hypothesizes that client heterogeneity may reduce the spatial faithfulness of federated Grad-CAM explanations and presents a controlled comparison to investigate that relationship. Prior federated-learning work characterizes heterogeneity’s effects on optimization and accuracy, but leaves downstream explanation faithfulness largely unexamined.

C. Hypothesis

  • H1 predicts that federated training will reduce Grad-CAM spatial faithfulness relative to centralized training, with degradation increasing as client heterogeneity rises.The hypothesis uses Dirichlet concentration α to control heterogeneity.Source: C. Hypothesis · p. 2

D. Contributions

  • The study measures AOPC and Explanation IoU for centralized and federated EfficientNet-B4 models across four heterogeneity levels on FaceForensics++.Its contributions also include released partitions, configurations, maps, and evaluation scripts, plus practitioner guidance on when explanation quality may need support.Source: D. Contributions · p. 2

A. Federated Learning and Non-IID Data

  • Federated-learning research has examined convergence, communication, accuracy, and heterogeneity, but the effects of altered gradients on post-hoc explanation faithfulness remain little studied.Prior work includes FedAvg, FedProx, convergence analyses, and benchmarks of heterogeneous settings.Source: A. Federated Learning and Non-IID Data · pp. 2–3

B. Deepfake Detection and Explainability

Deepfake detectors can perform well, yet forensic and security users need explanations to verify their predictions. Prior explanation methods and faithfulness tests have largely been studied with centralized models, leaving their behavior under federated training unclear.

B. Deepfake Detection and Explainability

  • Existing deepfake-XAI studies use centralized training, leaving open whether federated aggregation changes explanation-map spatial consistency or faithfulness.Prior work includes network dissection, Grad-CAM with SHAP, and attention maps, but does not study this federated effect.Source: B. Deepfake Detection and Explainability · p. 3

C. XAI Faithfulness Measurement

  • AOPC measures confidence drops as important regions are progressively removed, while sanity checks and robustness analyses test whether saliency maps reflect model behavior.These faithfulness tools have been validated almost exclusively under centralized training, and their application to federated visual forensics remains unexplored.Source: C. XAI Faithfulness Measurement · p. 4

D. XAI within Federated Systems

Prior federated-XAI work demonstrates that explanations can be generated without exposing client data, but it has not established their faithfulness for visual deepfake detection. The section also describes the study’s dataset preparation and reproducible split.

D. XAI within Federated Systems

  • Federated XAI has produced explanations in supply-chain credit assessment and intrusion detection, but these non-visual studies do not establish faithfulness under federated visual deepfake detection.Quantitative comparisons of centralized and federated explanations for visual deepfake detection have not been reported.Source: D. XAI within Federated Systems · p. 4

E. Research Gap

  • The combined gap is whether non-IID federated optimization affects explanation faithfulness in visual forensic models, a question not addressed across the four research pillars.Prior work separately studies federated optimization, centralized deepfake explanations, centralized faithfulness metrics, and federated XAI in non-visual domains.Source: E. Research Gap · p. 4

A. Dataset and Pre-processing

  • FaceForensics++ c23 provides 6,000 clips across real and five manipulated classes, collapsed to binary real-versus-fake labels for this study.Extracted face crops were split reproducibly into training, validation, and test sets; the test set contains 12,037 images and was held out from model selection and tuning.Source: A. Dataset and Pre-processing · p. 4
  • Frames were sampled at stride 10, face crops were resized to 224 × 224, and training required at least 14,000 accepted crops.Training images received random flips and mild color jitter, while evaluation images received only resizing and normalization.Source: A. Dataset and Pre-processing · pp. 4–5

B. Federated Data Partitioning

The study partitions training data across five simulated clients at four Dirichlet heterogeneity levels, uses EfficientNet-B4, and compares federated results with a centralized baseline.

B. Federated Data Partitioning

  • Five clients receive Dirichlet-partitioned data at α = 0.1, 0.3, 0.5, or 1.0; lower α concentrates client data in fewer manipulation categories.At α = 1.0, client distributions closely approximate uniform coverage across classes.Source: B. Federated Data Partitioning · p. 5

C. Model Architecture

  • The detector uses an ImageNet-pretrained EfficientNet-B4 backbone, selected for its FaceForensics++ track record and compatibility with gradient-based attribution.The paper introduces no new architecture; its contribution is analyzing explanations under federation.Source: C. Model Architecture · p. 5

D. Centralized Baseline Training

  • The centralized baseline trains on the full training split for 10 epochs and uses its peak-validation-accuracy checkpoint for performance and faithfulness comparisons.Training uses Adam with learning rate 10−4, batch size 32, and cross-entropy loss.Source: D. Centralized Baseline Training · p. 5

E. Federated Training Protocol (FedAvg)

Federated training uses sequential FedAvg across five clients, while Grad-CAM and AOPC quantify spatial explanations and their faithfulness.

E. Federated Training Protocol (FedAvg)

  • Federated training proceeds as sequential FedAvg, with one local epoch per client per round and sample-weighted model aggregation.The simulation keeps only one client model in GPU memory at a time; α = 0.1 runs for 30 rounds, while α ∈ {0.3, 0.5, 1.0} run for 10.Source: E. Federated Training Protocol (FedAvg) · p. 5

F. Grad-CAM Explanation Generation

  • Grad-CAM is generated for the fake-class score at the final convolutional layer, producing normalized [0, 1] heatmaps aligned to test-image IDs.The method computes the target-score gradient with respect to convolutional feature maps and applies a ReLU-weighted linear combination.Source: F. Grad-CAM Explanation Generation · p. 5

1) Area Over the Perturbation Curve (AOPC):

  • AOPC measures the confidence drop as CAM-ranked pixels are progressively masked; larger values indicate more faithful explanations, while negative values can indicate misleading regions.At each of 10 steps, the top k · 10% salient pixels are replaced with the per-channel image mean.Source: 1) Area Over the Perturbation Curve… · pp. 5–6

2) Explanation IoU:

The section measures spatial overlap between explanations and tests whether AOPC differences vary with heterogeneity.

2) Explanation IoU:

  • Explanation IoU compares binary masks of the top 20% CAM activations, measuring spatial agreement between models and tracking within-run convergence across rounds.The masks are binarized at the 80th percentile.Source: 2) Explanation IoU · p. 6

H. Statistical Testing

  • A one-sided Mann–Whitney U test compares centralized and federated AOPC per α, while Cohen’s d measures effect size and Spearman ρ tests monotonic degradation with heterogeneity.The U test uses the alternative that centralized AOPC exceeds federated AOPC.Source: H. Statistical Testing · p. 6

IV. RESULTS

Detection accuracy declines as heterogeneity increases, while explanation faithfulness varies non-monotonically and fails at the most extreme setting.

B. Explanation Faithfulness

  • Mean AOPC was 0.1657 at α = 1.0, 0.1080 at α = 0.5, 0.0492 at α = 0.3, and −0.0104 at α = 0.1, versus centralized 0.0694.Fig. 1 plots mean AOPC for centralized training and the four α configurations, with standard-deviation error bars.Source: Figure 1 · p. 6, B. Explanation Faithfulness · p. 6

A. Detection Accuracy

  • Accuracy decreased monotonically with heterogeneity, from 97.31% centrally to 78.42% at α = 0.1, a drop of nearly 19 percentage points.The α = 1.0, 0.5, and 0.3 results were 96.68%, 95.31%, and 94.43%, respectively.Source: A. Detection Accuracy · p. 6, §IV · p. 6

B. Explanation Faithfulness

  • At α = 0.1, the federated heatmaps become diffuse and spatially incoherent, unlike the centralized model’s coherent focal activations.The figure shows the centralized model focusing on core facial features while the α = 0.1 model disperses attention toward peripheral regions.Source: Figure 2 · p. 7
  • Federated AOPC was higher at α ∈ {0.5, 1.0} without significant central superiority, but fell significantly below baseline at α = 0.3 and 0.1.The reported effects were small at α = 0.3 (d = 0.068) and small-to-medium at α = 0.1 (d = 0.315); only the latter had negative AOPC.Source: B. Explanation Faithfulness · p. 7

C. Heterogeneity Effect on Explanation Degradation

Explanation faithfulness varies with client heterogeneity: moderate-to-low heterogeneity preserves or improves AOPC relative to the centralized model, while extreme heterogeneity degrades it. Explanation IoU stabilizes by round 10 in the less heterogeneous settings but remains low at α = 0.1.

C. Heterogeneity Effect on Explanation Degradation

  • AOPC degradation ranks perfectly with heterogeneity (Spearman ρ = −1.0, p < 0.001), but faithfulness degrades primarily at α = 0.1.At moderate-to-low heterogeneity, federated models preserve or improve explanation faithfulness relative to the centralized model.Source: C. Heterogeneity Effect on Explanation… · p. 7

D. Explanation Stability Across Communication Rounds

  • Explanation IoU rose between rounds 5 and 10 for α ∈ {0.3, 0.5, 1.0}, then plateaued by round 10.For α = 0.3, 0.5, and 1.0, IoU changed from 0.446 to 0.534, 0.500 to 0.547, and 0.487 to 0.513, respectively.Source: D. Explanation Stability Across… · p. 7, Figure 3 · p. 7
  • At α = 0.1, IoU rose only from 0.127 at round 10 to 0.132 at round 30, supporting non-convergence rather than slow learning.The extended run showed essentially flat metrics at this most heterogeneous setting.Source: D. Explanation Stability Across… · p. 7
  • At α = 0.1, the centralized heatmap focuses on core facial features while the federated heatmap disperses attention to diffuse peripheral regions.The qualitative comparison in Fig. 2 corroborates the spatial-coherence collapse at extreme heterogeneity.Source: D. Explanation Stability Across… · p. 7

E. Adebayo et al. Sanity Check

The parameter-randomization test found that centralized Grad-CAM maps were sensitive to learned model parameters. Across the broader findings, federation did not uniformly weaken explanation faithfulness: results were favorable or near baseline at lower heterogeneity, but failed at α = 0.1.

E. Adebayo et al. Sanity Check

  • The parameter-randomization test yielded mean Explanation IoU below the prespecified 0.3 threshold, indicating sensitivity to learned parameters.This check was conducted on the trained centralized model before faithfulness evaluation.Source: E. Adebayo et al. Sanity Check · p. 7

F. Summary of Findings

  • Detection accuracy dropped monotonically with heterogeneity, from 0.63 pp at α = 1.0 to 18.89 pp at α = 0.1.At α = 0.3 and 0.5, degradation remained within the anticipated 3–8 pp range.Source: F. Summary of Findings · p. 7
  • At α = 0.1, AOPC turned negative (d = 0.315), while α = 0.3 showed a small below-baseline effect (d = 0.068).At α = 0.5 and 1.0, mean AOPC was higher than the centralized baseline; p = 1.0.Source: F. Summary of Findings · p. 8

A. Federation Does Not Simply Break Explanations

  • At α = 1.0 and 0.5, federated mean AOPC was 0.1657 and 0.1080, respectively, above the centralized baseline of 0.0694.Neither setting showed a statistically significant centralized-model advantage (p = 1.0).Source: A. Federation Does Not Simply Break… · p. 8
  • The proposed implicit-regularization account is tentative; explaining the moderate-heterogeneity AOPC gains would require ablations beyond this study.The authors state that the observed effect repeats across settings but that its mechanism is unsettled.Source: A. Federation Does Not Simply Break… · p. 8

B. The Collapse at Extreme Heterogeneity Is a Different Kind of Failure

At extreme heterogeneity, explanations collapse alongside model performance, with negative AOPC and near-zero IoU. The results suggest a sharp heterogeneity threshold and motivate checking explanation faithfulness—not accuracy alone—before deployment.

B. The Collapse at Extreme Heterogeneity Is a Different Kind of Failure

  • At α = 0.1, mean AOPC fell to −0.0104 and Explanation IoU to 0.132, indicating misleading attributions and non-convergence.IoU moved only from 0.127 to 0.132 over 20 additional rounds, ruling out slow convergence as the explanation.Source: B. The Collapse at Extreme… · p. 8
  • At α = 0.1, clients mainly receive one manipulation category, leaving FedAvg to combine conflicting local updates; accuracy falls to 78.42%, 18.89 points below centralized.The near-zero IoU and negative AOPC indicate unreliable explanations in addition to reduced classification accuracy.Source: B. The Collapse at Extreme… · p. 8

C. What the Monotonicity Result Tells Us

  • AOPC degradation follows a perfect rank ordering across four heterogeneity settings (Spearman ρ = −1.0, p < 0.001), with the sharp decline concentrated at the extreme.The authors suggest α or an estimated heterogeneity measure could help flag systems for further explanation evaluation.Source: C. What the Monotonicity Result Tells Us · p. 9

D. Practical Implications for Federated Forensic Pipelines

  • At α ≥ 0.3, federated training does not compromise and may strengthen faithfulness; at α ≈ 0.1, explanations cannot be trusted.The authors recommend routine AOPC checks because degradation at α = 0.3 can appear while accuracy remains 94.43%; options for extreme heterogeneity include FedProx or explanation-aware aggregation.Source: D. Practical Implications for… · p. 9

E. Limitations

The conclusion finds that Grad-CAM faithfulness under federated deepfake detection depends on client heterogeneity: moderate settings match or exceed centralized results, while extreme heterogeneity produces misleading explanations. These findings are bounded by a single-GPU simulation with five clients, one detector backbone, one explanation method, and a controlled heterogeneity model.

E. Limitations

  • The study’s conclusion is limited by a sequential single-GPU simulation with N = 5 clients, one EfficientNet-B4 backbone, and Grad-CAM as the sole explanation method.Its Dirichlet partition approximates, but does not fully capture, structured heterogeneity in real federated nodes.Source: E. Limitations · p. 9

VI. CONCLUSION

  • At α ∈{0.3, 0.5, 1.0}, federated EfficientNet-B4 models match or exceed centralized AOPC faithfulness, with Explanation IoU above 0.5.Detection accuracy remains within the range prior federated learning work would predict.Source: §VI · p. 9
  • At α = 0.1, accuracy drops by nearly 19 percentage points, Explanation IoU falls to 0.132, and mean AOPC becomes negative, with 30 rounds failing to resolve the instability.The authors report that the minimally changed IoU between rounds 10 and 30 suggests the issue is not simply slow learning.Source: §VI · p. 9
  • The authors revise their hypothesis: moderate heterogeneity can improve faithfulness over the centralized baseline, shifting the question toward the threshold where harm begins.The conclusion frames heterogeneity, rather than federation itself, as the relevant condition for explanation degradation.Source: §VI · pp. 9–10
  • Accuracy alone is insufficient for deployment decisions in explanation-dependent forensic applications; the authors recommend including AOPC or a comparable causal perturbation metric in validation.Explanation faithfulness can deteriorate even when accuracy remains well above operational thresholds.Source: §VI · p. 10
  • Proposed follow-ups include explanation-aware aggregation, adaptive communication based on Explanation IoU, and testing transformer-based detectors with attention-based explanations.The authors note that the α = 0.1 non-convergence might respond to more rounds or adaptive aggregation, but they could not fully explore this.Source: E. Limitations · p. 9, §VI · p. 10

Related papers

Loading 2610.39483v1…