PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating explainable artificial intelligence methods for multi-label deep learning classification tasks in remote sensing

Ioannis Kakogeorgiou, Konstantinos Karantzalos

arXiv:2104.01375v2cs.LGcs.CV

TL;DR

Deep neural networks achieve strong remote-sensing performance, but their black-box decisions hinder interpretability and can obscure dataset or model shortcomings. The paper evaluates ten XAI methods quantitatively and qualitatively on multi-label models trained with BigEarthNet and SEN12MS, finding that Occlusion, Grad-CAM, and Lime are the most interpretable and reliable, with trade-offs in resolution and computational cost.

  • Problem

    Deep neural networks' complex, nonlinear black-box operation limits understanding of remote-sensing decisions and can conceal biases or shortcomings in datasets and model performance.

  • Method

    The study compares ten XAI methods using quantitative metrics and qualitative analyses of predictions from DenseNet and ResNet models trained on BigEarthNet and SEN12MS.

  • Results

    Occlusion, Grad-CAM, and Lime were the most interpretable and reliable methods across the evaluated multi-label remote-sensing tasks.

  • Takeaways & Limitations

    XAI provides insights into deep black-box model decisions and benchmark-dataset composition, while no evaluated method stands out as best overall.

  • Takeaways & Limitations

    None of the evaluated methods provides high-resolution outputs; Lime and Occlusion are computationally expensive compared with Grad-CAM.

Abstract

from arXiv · show

Although deep neural networks hold the state-of-the-art in several remote sensing tasks, their black-box operation hinders the understanding of their decisions, concealing any bias and other shortcomings in datasets and model performance. To this end, we have applied explainable artificial intelligence (XAI) methods in remote sensing multi-label classification tasks towards producing human-interpretable explanations and improve transparency. In particular, we utilized and trained deep learning models with state-of-the-art performance in the benchmark BigEarthNet and SEN12MS datasets. Ten XAI methods were employed towards understanding and interpreting models' predictions, along with quantitative metrics to assess and compare their performance. Numerous experiments were performed to assess the overall performance of XAI methods for straightforward prediction cases, competing multiple labels, as well as misclassification cases. According to our findings, Occlusion, Grad-CAM and Lime were the most interpretable and reliable XAI methods. However, none delivers high-resolution outputs, while apart from Grad-CAM, both Lime and Occlusion are computationally expensive. We also highlight different aspects of XAI performance and elaborate with insights on black-box decisions in order to improve transparency, understand their behavior and reveal, as well, datasets' particularities.

1. Introduction

Deep neural networks perform strongly in remote sensing but remain difficult to interpret because of their complex, nonlinear, black-box operation. This study evaluates diverse XAI methods for multi-label remote-sensing models to improve understanding of predictions and expose dataset or model shortcomings.

  • Black-box deep neural networks limit transparency and interpretability despite modeling complex patterns and achieving higher performance.
  • XAI methods can make black-box decisions more human-interpretable and support inspection of individual predictions beyond standard accuracy metrics.
  • XAI analysis may reveal biases, multi-label dataset issues, and spurious or artifactual correlations learned by models.
  • Prior remote-sensing XAI studies examined applications including crop yield, land-use classification, and interpretable land-cover modeling, but lacked a comparative analysis.
  • This study quantitatively and qualitatively evaluates widely used XAI methods across multi-label BigEarthNet and SEN12MS experiments using DenseNet and ResNet models.Evaluation uses Max-Sensitivity, AUC-MoRF, File Size, and Computational Time, alongside analyses of correct, incorrect, and competing-label predictions.

2. Materials and Methods

The study evaluates widely used XAI methods for individual predictions in multi-label remote sensing, using quantitative metrics and deep-learning models across benchmark datasets. It defines attribution approaches, model-specific and model-agnostic methods, perturbation-based evaluation, and the SEN12MS comparison setup.

  • Explainable AI methods: The study compares widely used XAI methods that attribute a class prediction's relevance, contribution, or importance to input features.The methods explain individual predictions and include gradient-based, activation-based, perturbation-based, surrogate-model, and baseline-based approaches.
  • Gradient-based methods: Saliency and Input × Gradient use input gradients, but both require differentiable models; Input × Gradient additionally multiplies the input by its gradient.Guided Backpropagation restricts propagated gradients through ReLU functions to non-negative values.
  • Activation-based methods: Grad-CAM averages output gradients across feature-map channels, multiplies them by channel activations, sums across channels, and applies ReLU to produce coarse positive-contribution regions.Its output can be bilinearly upsampled to the input-image resolution, while Guided Grad-CAM combines Grad-CAM with Guided Backpropagation.
  • Model-agnostic methods: Occlusion replaces contiguous input patches with a baseline and monitors prediction decreases, whereas Lime fits a local surrogate model using interpretable features from the original input.Occlusion is model-agnostic but requires a baseline; Lime is also model-agnostic and requires choices such as segmentation, similarity function, and sample count.
  • Baseline-based methods: Integrated Gradients follows gradients from a baseline to the input, while DeepLift attributes output differences between the input and baseline through recursive backpropagation.Integrated Gradients can be time-consuming because of numerical integration samples, and DeepLift applies only to deep-learning models and requires a baseline.
  • Evaluation metrics: Max-Sensitivity assesses explanation stability under small input changes, while AUC-MoRF assesses reliability by progressively removing features ranked most relevant by the explanation.Lower Max-Sensitivity is preferred, and the AUC-MoRF score is minimized when relevant-feature removal causes the desired prediction decrease efficiently.
  • Datasets and models: The study evaluates models on BigEarthNet and SEN12MS, using ResNet-50 for SEN12MS because no multi-label classification model was available for comparison.DenseNet-121 achieved state-of-the-art overall performance on BigEarthNet, while DenseNet-121 and ResNet-50 obtained overall F1 scores of 74.35% and 74.62% on SEN12MS.

3. Experimental Results and Evaluation

Quantitative and qualitative evaluations consistently favored Occlusion, Lime, and Grad-CAM for reliable localization, but exposed trade-offs involving resolution, file size, and computation time. These methods also performed well across single-label, competing-label, and challenging prediction cases, with Grad-CAM showing particularly strong class discrimination.

  • Quantitative Evaluation: Occlusion, Lime, and Grad-CAM achieved the lowest Max-Sensitivity scores across both datasets, below 0.20 for BigEarthNet and below 0.11 for SEN12MS.Input × Gradient had the highest scores: 0.59 for BigEarthNet and 0.30 for SEN12MS; SmoothGrad generally reduced sensitivity.
  • Quantitative Evaluation: Occlusion, Lime, and Grad-CAM produced the fastest MoRF performance decreases, with AUC-MoRF below 16.38 for BigEarthNet and below 28.87 for SEN12MS.Input × Gradient was closest to the random baseline and reached AUC-MoRF values of 26.15 and 38.8, respectively.
  • Quantitative Evaluation: Occlusion, Lime, and Grad-CAM generated the smallest explanation files, below 3.24 KB for BigEarthNet and below 7.91 KB for SEN12MS.Their lower resolution provides rougher, more concise localization, whereas finer-grained explanations contain more potentially confusing information.
  • Quantitative Evaluation: Occlusion and Lime scaled poorly to larger images, while Grad-CAM maintained low computation time regardless of image size.Using a small Occlusion stride increased computation time to 8.73 seconds for BigEarthNet and 29.71 seconds for SEN12MS.
  • Explaining Single Class Correct Predictions: In correctly predicted single-class cases, the methods generally localized relevant regions, with Lime, Occlusion, and Grad-CAM showing Max-Sensitivity below 0.10 for Urban Fabric.For Water, several methods identified the water region, while DeepLift, Guided Backpropagation, and Guided Grad-CAM were less interpretable; SmoothGrad produced smoother explanations.
  • Explaining Correct Predictions for Multiple Competing Classes: For competing labels, Occlusion, Lime, and Grad-CAM provided accurate localization, whereas Guided Backpropagation focused on the same regions for Urban Fabric and Broad-leaved Forest.For Urban Fabric and Industrial Units, Grad-CAM and Integrated Gradients also included pixels from the competing Industrial Units class.
  • Explaining Correct Predictions for Multiple Competing Classes: Grad-CAM was the only method that highlighted the Savanna region in a Forest–Savanna case, making it the most class-discriminative method in that example.The model predicted Forest and Savanna with sigmoid probability scores of 0.97 and 1.00, respectively.

4. Discussion

Occlusion, Grad-CAM, and Lime were the most interpretable and reliable methods, while other methods showed class-insensitive or poorly localized explanations. The study also exposed trade-offs among reliability, resolution, computation, and dataset-label quality.

  • Occlusion, Grad-CAM, and Lime best explained competing multi-label decisions while locating corresponding image regions.
  • Guided Backpropagation was least reliable because it highlighted striking primitives such as edges independently of the predicted class.
  • Saliency, Input × Gradient, Integrated Gradients, and DeepLift sometimes poorly localized spatially distributed classes such as Savanna.
  • Grad-CAM offers reliability, interpretability, scalability, and speed but lacks high-resolution outputs, whereas Lime and Occlusion are computationally expensive.
  • High prediction confidence did not guarantee interpretable localization, as Savanna achieved 1.00 sigmoid probability and 84.65% F1 despite poor explanations.
  • Marine Water explanations revealed label-distribution and annotation issues affecting Beaches, Dunes, Sands, whose F1 score was 63.30%.

5. Conclusions

The study evaluated XAI methods for black-box multi-label remote-sensing predictions using quantitative metrics and extensive qualitative experiments. Occlusion, Grad-CAM, and Lime were strongest overall, but no method combined all desirable properties.

  • The study evaluated XAI methods on state-of-the-art deep-learning predictions in BigEarthNet and SEN12MS using quantitative and qualitative analyses.
  • Occlusion, Grad-CAM, and Lime were the most interpretable and reliable methods, with the lowest reported Max-Sensitivity and AUC-MoRF scores.
  • None of the evaluated methods provided high-resolution outputs while also avoiding the practical trade-offs identified in the study.
  • XAI provided insights into black-box decisions and benchmark-dataset composition and shortcomings.

Supplementary Material

The supplementary material contains the paper title and authors, additional prediction-explanation figures, training-set co-occurrence information, and per-class or model-level metric tables.

  • The supplementary material identifies the paper as Evaluating Explainable Artificial Intelligence Methods for Multi-label Deep Learning Classification Tasks in Remote Sensing.
  • It lists Ioannis Kakogeorgiou and Konstantinos Karantzalos as the authors.
  • Figures S1–S9 show DenseNet prediction explanations for classes including Urban Fabric, Water, Forest, Savanna, Coniferous Forest, Urban/Built-up, and Marine Waters.
  • Figure S10 presents BigEarthNet training-set class co-occurrence.
  • Tables S1–S3 report per-class F1 scores and quantitative metrics for DenseNet121 and ResNet50 across the benchmark datasets.
Loading 2104.01375v2…