PaperlayerView exact revision on arXiv

Source-linked AI summary

Exploiting Explanations for Model Inversion Attacks

Xuejun Zhao, Wencan Zhang, Xiaokui Xiao, Brian Y. Lim

arXiv:2104.12669v3cs.CVcs.CYcs.LG

TL;DR

Explanations may expose sensitive information to inversion attacks, while the privacy risk of using them to reconstruct active users’ test images has been underexplored. The paper develops multimodal, spatially aware inversion attacks using target explanations and surrogate-model attention transfer. It reports improved inversion performance over prediction-only attacks and increased risk even for non-explainable targets.

  • Problem

    The privacy risk of using model explanations to invert active users’ test instances remains underexplored, despite explanations containing sensitive information.

    Source: §1 · p. 2, §2 · p. 3

  • Method

    The authors develop multimodal, spatially aware transposed-CNN inversion attacks using target explanations, plus surrogate explanation attention transfer for non-explainable targets.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    The attacks achieve significantly higher inversion performance than prediction-only methods, and surrogate explanations bring performance close to inverting an explainable target model.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Explanations create privacy risks for image inversion, highlighting a need for defenses that balance explainability and privacy.

    Source: §1 · p. 2

    The study defers privacy defenses to future work.

    Source: §1 · p. 2

Abstract

from arXiv · show

The successful deployment of artificial intelligence (AI) in many domains from healthcare to hiring requires their responsible use, particularly in model explanations and privacy. Explainable artificial intelligence (XAI) provides more information to help users to understand model decisions, yet this additional knowledge exposes additional risks for privacy attacks. Hence, providing explanation harms privacy. We study this risk for image-based model inversion attacks and identified several attack architectures with increasing performance to reconstruct private image data from model explanations. We have developed several multi-modal transposed CNN architectures that achieve significantly higher inversion performance than using the target model prediction only. These XAI-aware inversion models were designed to exploit the spatial knowledge in image explanations. To understand which explanations have higher privacy risk, we analyzed how various explanation types and factors influence inversion performance. In spite of some models not providing explanations, we further demonstrate increased inversion performance even for non-explainable target models by exploiting explanations of surrogate models through attention transfer. This method first inverts an explanation from the target prediction, then reconstructs the target image. These threats highlight the urgent and significant privacy risks of explanations and calls attention for new privacy preservation techniques that balance the dual-requirement for AI explainability and privacy.

1. Introduction

AI applications need both explanations to support understanding and privacy to protect personal information, but explanations may give attackers more information for reconstructing private images. This study develops explanation-aware inversion attacks and examines their risks, including when the target model does not provide explanations.

1. Introduction

  • Model explanations support users in understanding decisions, while privacy protects personal information such as anonymity and reputation.The paper notes this dual requirement in applications including driver drowsiness detection and classroom facial engagement prediction.Source: §1 · p. 1
  • Explanations significantly improve image reconstruction over prediction-only inversion, whose reconstructions are poor and similar across different faces.The Figure 1 demonstration uses emotion prediction as the target task and face reconstruction as the attack task.Source: §1 · p. 1, Figure 1 · p. 1
  • The study investigates inversion of active users’ test instances, exploiting saliency-map explanations and attention transfer from surrogate models when targets provide no explanations.The authors report that surrogate-explanation performance is close to inverting an explainable target model.Source: §1 · p. 2
  • The authors develop spatially aware, multimodal transposed-CNN attacks and analyze privacy risks across explanation types and factors affecting inversion performance.The examined explanation types include Gradients, Grad-CAM, and LRP.Source: §1 · p. 2
  • The study defers privacy defenses to future work, identifying co-optimization of explainability and privacy as a needed direction.The authors characterize this as their first study of explanation privacy risk for inversion attacks.Source: §1 · p. 2

2. Related Works

Prior work studies explanation-based privacy attacks on training data, model confidentiality, and model extraction, while inversion of active users’ test instances remains underexplored. This paper focuses on image saliency maps and inversion attacks against explainable and non-explainable models.

2. Related Works

  • Image explanations include gradients, activation maps, and class activation maps; Grad-CAM can provide class-specific maps without retraining the CNN.The work evaluates original Gradients and Grad-CAM, and defers study of proxy explanations such as LRP.Source: §2 · p. 2
  • Attention transfer has been used in training and is adapted here to align an attack model’s intermediate explanation with a surrogate target model’s explanation.Related approaches include student-teacher transfer and GAIN’s saliency-mask objective.Source: §2 · pp. 2–3
  • Inversion attacks can reconstruct original data from target predictions, but effective attacks have often relied on white-box access or auxiliary knowledge not readily available in practice.Prior improvements used deep architectures and information such as feature embeddings or blurred images.Source: §2 · p. 3
  • Explanation-based privacy research has addressed membership inference and model extraction, but inversion of active users’ test instances remained unexplored.This work investigates saliency-map explanations against both explainable and non-explainable models.Source: §2 · p. 3

3. XAI-Aware Model Inversion Attack

The section defines attacks that reconstruct private images from model outputs, then develops XAI-aware architectures that exploit one or multiple explanations. It also extends inversion to non-explainable targets by transferring explanations from a surrogate model.

3.1. Threat Model

  • Attackers with black-box API access and an independent dataset reconstruct private inputs from prediction and explanation outputs, without privileged background knowledge.The threat model considers sensitive information recoverable from the reconstructed image.Source: §3.1 · p. 3

3.2. Model Inversion with Target Explanations

  • Multi-modal inversion models combine predictions with 2D explanations, using flattened inputs, CNN encoders, U-Net, or a Flatten+U-Net hybrid.U-Net bypass connections retain spatial information from the explanation.Source: §3.2 · p. 4

3.3. Model Inversion with Multiple Explanations

  • Alternative CAMs and constituent CAMs concatenate class-specific or neuron-specific saliency maps into 3D inputs for inversion.Alternative CAMs use explanations for multiple classes; constituent CAMs use partial CAMs derived from Grad-CAM.Source: §3.3 · p. 4

3.4. Model Inversion with Surrogate Explanations

  • For non-explainable targets, an explanation inversion model reconstructs a surrogate explanation from the target prediction before image inversion.Training combines image- and explanation-reconstruction losses; the reconstructed explanation is available at prediction time and can feed the XAI input methods.Source: §3.4 · pp. 4–5

4. Experiments

The experiments cover image-based identification, emotion, and digit-recognition settings across three datasets. They compare explanation-aware attacks with prediction-only and earlier inversion baselines under a split-data protocol.

4.1. Experiment Setup

  • Experiments span iCV-MEFED, CelebA, and MNIST, covering face emotion or identity tasks and handwritten-digit recognition.The datasets contain 115 identities and 24,120 instances, 1,000 identities and 30,000 instances, and 70,000 digits, respectively.Source: §4.1 · p. 5
  • Each dataset is split evenly between target-model training and attack data, with the attack data further divided 80/20 for training and testing.The target model is tested on the attack dataset.Source: §4.1 · p. 5
  • The comparison includes four saliency-map types, multiple CAM variants, surrogate-CAM transfer, Fredrikson’s attack, and a prediction-only transposed CNN.Target networks were kept small so CAM explanations would remain at 16×16 pixels.Source: §4.1 · p. 5

4.2. Evaluation Metrics

The paper evaluates both image similarity and recovery of task-relevant information. Its metrics range from pixel-level comparisons to perceptual similarity and sensitive-label prediction.

4.2. Evaluation Metrics

  • Pixelwise Similarity is 1 − MSE after scaling images to a unit square and normalizing pixels to [0,1].The resulting similarity is image-size invariant and increases with image similarity.Source: §4.2 · pp. 5–6
  • SSIM supplements MSE with a perception-based image-quality measure, averaged over Gaussian-kernel comparisons at calibrated σ values.Smaller σ gives a more precise comparison; values were calibrated separately for iCV-MEFED, CelebA, and MNIST.Source: §4.2 · p. 6
  • Attack Accuracy measures whether a CNN trained on original images can predict sensitive labels from reconstructed images.The evaluation model uses identity labels for face datasets and digit labels for MNIST, and is not trained on reconstructions.Source: §4.2 · p. 6
  • Attack Embedding Similarity measures how closely reconstructed and original images match in the attack model’s embedding space.It is computed as e^−|z−zr|² and is bounded between 0 and 1.Source: §4.2 · p. 6

4.3. Experiment Results

The experiments compare explanation inputs, inversion architectures, explanation types, surrogate explanations, and explanation-quality factors across image datasets. They assess both reconstruction performance and attack accuracy, with the quality-factor analysis focused on iCV-MEFED emotion predictions and re-identification.

4.3.1 Attacking with different XAI Input Methods

  • Inversion performance rose with spatial awareness, from Prediction only through Flatten, CNN, and U-Net to Flatten+U-Net.The study attributes the strongest architecture to its combination of raw-pixel information and semantic features.Source: §4.3.1 · pp. 6–7, Figure 4 · p. 7

4.3.2 Attacking multi-explanation target models

  • Attack performance increased from CAM to Σ-CAM to ∂-CAM, while access to unrestricted alternative explanations could permit very accurate inversion.The authors recommend limiting access to numerous alternatives; ∂-CAM retains individual-kernel information.Source: §4.3.2 · p. 7, Figure 5 · p. 7

4.3.3 Attacking target models with different XAI types

  • Inversion performance increased from Prediction only through LRP, Gradient, and CAM to Gradient ⊙Input, with constituent CAMs leaking more than aggregated CAM.The authors connect the ordering to how much information each explanation retains about the input image.Source: §4.3.3 · p. 7, Figure 4 · p. 7

4.3.4 Attacking non-explainable target models

  • With non-explainable targets, reconstructed surrogate explanations (rs-CAM) improved inversion over Prediction only, though they performed slightly below CAM.The finding also held for attacks trained on out-of-distribution data, albeit more weakly.Source: §4.3.4 · pp. 7–8, Figure 5 · p. 7

4.3.5 Impact of explanation quality on attack accuracy

  • More relevant and more typical CAM explanations improved attack performance in the iCV-MEFED emotion and re-identification analysis.Flatten+U-Net could still achieve high accuracy with less relevant explanations.Source: §4.3.5 · p. 8, Figure 6 · p. 8
  • Target-model prediction confidence had a slight positive effect on attack performance, whereas target-model accuracy did not affect it.The authors distinguish their test-instance inversion objective from prior work targeting a class-prototypical image.Source: §4.3.5 · p. 8, Figure 6 · p. 8

5. Conclusion

The conclusion presents explanation-aware inversion methods as evidence of increased privacy risk, including when targets do not provide explanations. It reports stronger attacks with richer explanations and identifies balancing explainability and privacy as a next challenge.

5. Conclusion

  • The proposed explanation-aware inversion methods demonstrate increased privacy risk from exploiting model explanations.The authors frame this work as a first step toward balancing explainability and privacy.Source: §5 · p. 8
  • The core inversion model combines a Flatten input layer with U-Net to use explanation detail, latent semantic features, and multi-scale spatial information.For non-explainable targets, a surrogate model and explanation-inversion stage produce a surrogate explanation for the XAI-aware inversion model.Source: §5 · p. 8
  • Attack accuracy increased up to 33x on iCV-MEFED emotion and 2.4x on CelebA ID with target explanations, and up to 2.15x without them.Multi-explanations reached up to 39x for contrastive explanations and 42x for detailed constituent explanations on the iCV emotion task.Source: §5 · p. 8

A.1. XAI-Aware Inversion Attack Models

The supplement outlines inversion architectures that combine target predictions with image explanations, including methods designed to preserve spatial information. It also lists network-layer settings for target and attack models.

A.1. XAI-Aware Inversion Attack Models

  • Supplementary Tables 1–8 provide network-layer settings for target and inversion attack models, including convolution, pooling, fully connected, and transposed-convolution layers.The tables cover iCV-MEFED, CelebA, MNIST, and several iCV-MEFED attack architectures.Source: §A.1 · p. 12, Table 1 · p. 13, Table 2 · p. 13
  • The model family progresses from a prediction-only baseline to explanation inputs processed by flattening, CNN dimension reduction, U-Net spatial connections, or combined designs.The final variant combines Flatten+U-Net with multiple explanations represented as a 3D tensor.Source: Figure 1 · p. 12
  • Detailed architectures concatenate explanation-derived representations with the target prediction; U-Net variants add bypass connections between corresponding convolutional layers.CAM explanations smaller than the reconstructed image omit bypass connections for later larger layers, while the described gradient variant uses all bypass connections.Source: Figure 2 · p. 12

B. Supplementary results

This supplementary section adds image-similarity metrics, the Fredrikson inversion baseline, and further demonstration instances for each dataset.

B. Supplementary results

  • The supplementary results add image-similarity metrics beyond those reported in the main paper.The section also includes the Fredrikson inversion baseline and more demonstration instances for each dataset.Source: B. Supplementary results · p. 15
  • The Fredrikson inversion model is included as an additional baseline in the supplementary results.Source: B. Supplementary results · p. 15
  • The supplement provides more demonstration instances for each dataset.Source: B. Supplementary results · p. 15

B.1. Attacking with different XAI Input Methods

This subsection reports PSNR as an alternative image-similarity metric and presents inversion performance across XAI input methods and explanation types.

B.1. Attacking with different XAI Input Methods

  • PSNR is calculated as log10(MAX2/MSE), where MAX is the image pixel dynamic range and equals 255 for 8-bit greyscale images.The supplement uses PSNR in place of 1 −MSE for the corresponding main-paper result.Source: §B.1 · p. 15
  • The supplementary figure reports PSNR in place of 1 −MSE for the corresponding main-paper result.Source: §B.1 · p. 15, Figure 3 · p. 15
  • Supplementary Figure 3 compares inversion performance across XAI input methods and XAI types for the iCV-MEFED emotion target task.Its error bars indicate 90% confidence intervals.Source: Figure 3 · p. 15

B.2. Attacking non-explainable target models

Supplementary results compare inversion methods across datasets and settings, including out-of-distribution attacks and image reconstructions. They report stronger performance when explanations are exploited, with OOD-trained attacks still improving on prediction-only attacks.

B.2. Attacking non-explainable target models

  • OOD-trained attack models improve on prediction-only inversion, though they perform less strongly than models trained on IID data.The supplementary figure also compares attack performance on OOD data and CIFAR data.Source: §B.2 · p. 15, Figure 5 · p. 15
  • CIFAR inversion performance is reported as similar to results for iCV-MEFED, CelebA, and MNIST.Source: §B.2 · p. 15, Figure 5 · p. 15
  • Inversion performance improves in the order Fredrikson < Prediction only < rs-CAM < CAM.Fredrikson performs worst because it produces one class-level reconstruction, whereas the other approaches reconstruct differently per instance.Source: Figure 4 · p. 15, §B.2 · p. 15
  • For emotion prediction and face reconstruction, reconstructions are poor and similar across faces with prediction-only inputs, but improve with single or multiple explanations.The demonstration uses six emotions from one iCV-MEFED identity; the Fredrikson baseline is omitted because it yields the same reconstruction for each emotion class.Source: Figure 6 · p. 16

Related papers

Loading 2104.12669v3…