Source-linked AI summary
Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey
Vanessa Buhrmester, David Münch, Michael Arens
TL;DR
Black-box Deep Neural Networks can be difficult to understand and may learn vulnerabilities or biases from their data, creating concerns for robustness, fairness, and explanation rights. This survey organizes and compares explainers for computer-vision DNNs by their mechanisms and properties, reviews selected methods and human understanding, and identifies open problems. It highlights practical limitations in explanation complexity and the difficulty of measuring understanding and faithfulness.
Problem
Deep Neural Networks are difficult to understand, may be vulnerable or biased, and their decisions can be hard for users to verify in settings requiring trust and fairness.
Method
The survey provides a taxonomy of DNN explainers for computer-vision tasks, comparing their mechanisms and technical properties while reviewing selected methods and studies of human understanding.
Results
The survey identifies explainers that expose input contributions, learned features, neuron relationships, or decision structure, and reports that greater explanation complexity increases response times and lowers user satisfaction.
Takeaways & Limitations
Explainability can support inspection of model reasoning and is relevant to trust, fairness, robustness, and users’ right to an explanation.
Takeaways & Limitations
Explanation complexity and the time needed to understand decisions remain difficult to measure, while decision-tree explanations of DNNs can become expensive and hard to comprehend.
Abstract
from arXiv · showhide
Deep Learning is a state-of-the-art technique to make inference on extensive or complex data. As a black box model due to their multilayer nonlinear structure, Deep Neural Networks are often criticized to be non-transparent and their predictions not traceable by humans. Furthermore, the models learn from artificial datasets, often with bias or contaminated discriminating content. Through their increased distribution, decision-making algorithms can contribute promoting prejudge and unfairness which is not easy to notice due to lack of transparency. Hence, scientists developed several so-called explanators or explainers which try to point out the connection between input and output to represent in a simplified way the inner structure of machine learning black boxes. In this survey we differ the mechanisms and properties of explaining systems for Deep Neural Networks for Computer Vision tasks. We give a comprehensive overview about taxonomy of related studies and compare several survey papers that deal with explainability in general. We work out the drawbacks and gaps and summarize further research ideas.
1. INTRODUCTION
AI systems increasingly support high-impact classification and regression tasks, but complex deep models are difficult for humans to understand and can learn misleading or discriminatory patterns. Explainability is therefore presented as a means to inspect model reasoning, support trust and fairness, and identify weaknesses.
- AI technologies are increasingly used for practical classification and regression tasks, including medicine, insurance, advertising, surveillance, hiring, and credit scoring.
- Deep Learning models are difficult for humans to trace because they contain millions of optimized parameters and require extensive computation.VGG-19 has about 144 million parameters, while ResNet has about 5 · 10^7 trainable parameters and about 10^10 floating-point operations per image.
- High-performing models can rely on spurious correlations, such as weather for tank classification or snow for distinguishing dogs from wolves.
- Small, human-imperceptible input changes can alter neural-network predictions, exposing weaknesses in model robustness.The DeepFool example changes a correctly classified whale image into a turtle prediction after adding a very small perturbation.
- Convolutional Neural Networks may favor texture over shape, motivating efforts to improve shape bias for results that align more closely with human image interpretation.
- Understanding feature–prediction links is framed as important for trust, fairness, and detecting discrimination or serious mistakes in automated decisions.
2. Overview about explaining systems of DNNs
The survey organizes DNN explainers by their mechanisms and properties, including model integration, locality, and model dependence. It reviews representative computer-vision explainers and identifies challenges in human understanding, completeness, and broader data coverage.
- Taxonomy: Explainers are classified as ante-hoc, which build interpretability into a model, or post-hoc, which explain predictions from the model’s outcome.Post-hoc methods are easier to apply across models but provide less information about the model as a whole.
- Taxonomy: A second taxonomy distinguishes local explanations of individual predictions from global explanations of the complete input-to-output logic, and model-specific from model-agnostic methods.The survey also discusses interpretability, explainability, comprehensibility, completeness, fidelity, trust, usability, scalability, and related properties.
- Selected explainers: Deconvnet propagates information backward through convolutional networks using deconvolution and unpooling to visualize input patterns associated with highly activated hidden units.Its limitations include dependence on max-pooling and the lack of a direct theoretical criterion connecting predictions to the generated patterns.
- Selected explainers: LRP redistributes prediction activity through layers using a conservation-based decomposition of relevance values, interpretable as a deep Taylor decomposition for deep ReLU networks.The survey presents this as avoiding gradients while assigning relevance back to input variables.
- Selected explainers: LIME is a post-hoc, local, interpretable, model-agnostic explainer that highlights image superpixels or other input features relevant to a prediction.The survey notes that LIME is itself a black box and has reported performance concerns.
- Evaluation and open problems: Human-interpretability studies find that greater explanation complexity increases verification time and lowers user satisfaction, while the survey notes that explanation understanding and coverage remain difficult to measure.The survey also identifies limited treatment of vectors, matrices, complex spatio-temporal data, sequences, and networks compared with images, text, and tabular data.
3. Conclusion
The survey focuses on state-of-the-art explainers for computer-vision DNNs, organizing them by mechanisms, applications, data, representations, and pros and cons while identifying open problems.
- The paper presents a taxonomy of explaining systems for Deep Neural Networks, focusing especially on Computer Vision.
- It compares explainers according to their technical mechanisms, applications, data, and advantages and disadvantages.
- The survey also compares related surveys and studies that analyze or evaluate explaining systems and quantify machine understanding.
- It summarizes open problems and outlines ideas for further research.