Analyzing Classifiers: Fisher Vectors and Deep Neural Networks

Sebastian Bach; Alexander Binder; Grégoire Montavon; Klaus-Robert Müller; Wojciech Samek

Source-linked AI summary

Analyzing Classifiers: Fisher Vectors and Deep Neural Networks

Sebastian Bach, Alexander Binder, Grégoire Montavon, Klaus-Robert Müller, Wojciech Samek

arXiv:1512.00172v1Dec 1, 2015cs.CV

TL;DR

The paper addresses the black-box nature of Fisher Vector and deep neural network image classifiers, particularly how context contributes to their predictions. It extends Layer-wise Relevance Propagation to Fisher Vectors and uses heatmaps to compare classifier reasoning, quantify context, and identify biases. The analysis finds stronger object-focused reasoning in DNNs, while FV models rely more on contextual or potentially artefactual cues.

Problem
The paper asks how Fisher Vector and deep neural network classifiers use image context and whether their decisions depend on object-relevant or undesirable features.
Method
The authors extend Layer-wise Relevance Propagation to Fisher Vector classifiers and apply heatmaps and context measures to compare FV and DNN decisions on PASCAL VOC.
Results
The analysis finds that DNNs rely more on exact object shape and object-focused features, whereas FV models benefit from contextual and potentially artefactual correlations.
Takeaways & Limitations
Heatmaps can support data collection and model selection by checking whether classifiers rely on appropriate features and guiding dataset or model changes when artefacts matter.

Abstract

from arXiv · show

Fisher Vector classifiers and Deep Neural Networks (DNNs) are popular and successful algorithms for solving image classification problems. However, both are generally considered `black box' predictors as the non-linear transformations involved have so far prevented transparent and interpretable reasoning. Recently, a principled technique, Layer-wise Relevance Propagation (LRP), has been developed in order to better comprehend the inherent structured reasoning of complex nonlinear classification models such as Bag of Feature models or DNNs. In this paper we (1) extend the LRP framework also for Fisher Vector classifiers and then use it as analysis tool to (2) quantify the importance of context for classification, (3) qualitatively compare DNNs against FV classifiers in terms of important image regions and (4) detect potential flaws and biases in data. All experiments are performed on the PASCAL VOC 2007 data set.

2 Introduction

The paper studies how context affects Fisher Vector and deep neural network image classifiers, using Layer-wise Relevance Propagation to compare their decisions and expose contextual biases.

The study asks how much context classifiers use and whether they can exploit it for prediction.
Fisher Vectors process an entire image into one feature map and score, so context naturally enters training and testing.
Larger image patches and limited test-time regions are expected to increase context dependence.
The paper extends relevance propagation to Fisher Vectors, defines context measures, compares FV and neural-network classifiers on PASCAL VOC, and detects contextual biases without bounding boxes.
The paper reviews related work, introduces Fisher-vector explanations and context measures, evaluates them experimentally, and concludes with implications.

3 Related Work

Prior explanation methods targeted particular model architectures, while Layer-wise Relevance Propagation provides prediction-focused relevance scores and is extended here to Fisher Vector classifiers.

Earlier approaches included gradient-based sensitivity analysis and architecture-specific explanations for bag-of-words features.
Layer-wise Relevance Propagation computes partial prediction contributions for intermediate and input representations, including image regions or pixels.
This paper extends LRP, previously applied to bag-of-visual-words classifiers and deep neural networks, to Fisher Vector classifiers.

4 Fisher Vectors in a Nutshell

Fisher Vector representations use a Gaussian Mixture Model to encode local descriptors through soft assignments and first- and second-order statistics, then normalize and classify the aggregate descriptor.

Fisher Vectors are rich image or video representations built from local descriptors and provide state-of-the-art performance among feature-extraction algorithms.
A Gaussian Mixture Model fitted to training descriptors supplies a soft vocabulary of visual prototypes.
Each local descriptor contributes soft mapping weight, deviation-from-mean, and variance statistics for all mixture components.
The single-descriptor embedding concatenates outputs for all K components into a (1 + 2D)K-dimensional vector.
Image-level embeddings are averaged and then power- and ℓ2-normalized before linear-SVM prediction.

5 Explaining Classification Decisions

Layer-wise Relevance Propagation decomposes classifier outputs into input-level relevance, enabling Fisher Vector heatmaps and a relevance-ratio measure for quantifying contextual support.

LRP addresses the pixel-level opacity created by nonlinear mappings in predictors such as linear SVMs over Fisher Vectors.
The method propagates relevance from outputs toward inputs, assigning positive, neutral, or negative contributions while conserving total relevance.
Layer-wise Relevance Propagation: For neural networks, relevance is recursively distributed among input neurons using propagation formulas based on their contributions to activations.
LRP for Fisher Vector Classifiers: For Fisher Vectors, the authors modify LRP because derivative-based features can be negative, then propagate SVM relevance through normalization and local-feature mappings.
Pixel-wise Heatmaps: Local-feature relevance is uniformly assigned to covered pixels, producing a visualizable heatmap of the classification decision.
Measuring Context: The outside-inside relevance ratio compares positive relevance outside versus inside annotated bounding boxes: high values indicate contextual reliance, whereas low values indicate object focus.

6 Experimental Evaluation

Experiments on PASCAL VOC 2007 validate LRP explanations for Fisher vectors and use them to compare contextual reliance, feature strategies, and model quality against DNNs.

6.2 Are Fisher Explanations Meaningful ?: LRP heatmaps for Fisher vectors produce more meaningful region orderings than random sequences under perturbation-based evaluation.Meaningful regions are those whose perturbation causes a sharp average prediction decline.
6.3 Shallow vs. Deep Features: The DNN achieved 25% higher absolute accuracy than the Fisher model on sheep, whose models used markedly different image evidence.The Fisher model emphasized high-resolution wool texture and contextual regions, whereas the DNN emphasized the sheep’s contour and largely ignored context and wool texture.
6.3 Shallow vs. Deep Features: The DNN’s superior sheep predictions were attributed largely to modeling exact object shape, while the Fisher model benefited from weak correlations among class, texture, and context.For the Fisher model, grass contributed positively and a human face negatively to the prediction.
6.4 Test Error and Model Quality: Similar boat accuracy concealed different strategies: the DNN focused on the boat, while the Fisher model focused on water and would likely degrade for boats outside water.The average heatmaps confirmed Fisher emphasis on the image bottom and DNN emphasis on the middle, where boats typically appeared.
6.4 Test Error and Model Quality: Similar horse accuracy also concealed a Fisher-model artifact: relevance concentrated on a copyright tag, whose removal considerably reduced Fisher prediction quality but scarcely changed the DNN heatmap.The copyright-tag dependence shows why test error alone did not predict model quality in this case.
6.5 Quantitative analysis of context use: Across the 20 classes, Fisher vectors generally used more context than DNNs, with especially high contextual reliance for boat and airplane.Water and sky supported Fisher predictions for boat and airplane, while bicycle, car, motorbike, and sheep involved comparatively little Fisher context.

7 Conclusion

The paper uses LRP heatmaps to compare Fisher Vector and DNN decisions, finding that FV models rely more on contextual or artefactual cues while DNNs focus more on objects. These explanations also motivate heatmaps in data collection and model selection.

LRP was extended from neural networks to Fisher Vector models and validated as a way to identify pixels supporting classifier decisions.The resulting heatmaps support comparative analysis of FV and DNN reasoning.
FV models compensate for limited depth by using contextual information, including weakly correlated artefacts, so their generalization can be overstated when similar context appears in test images.This conclusion links contextual reliance to the features extracted by the classifier architecture.
DNNs focus on the actual object rather than context, with their higher predictive accuracy and deep multitask properties favoring natural image features over low-level cues such as copyright text.The paper attributes this focus to both predictive accuracy and feature composition supported by deep multitask learning.
Heatmaps can support data collection and model selection by revealing whether classifiers use appropriate features and by motivating model changes or dataset extensions when artefacts drive decisions.The proposed use is to make artefactual features less able to support classification.

C Details on Relevance Decomposition for Fisher Vectors

The Fisher Vector relevance decomposition propagates prediction relevance from FV dimensions to local descriptors and then to pixels. Because signed mappings can make direct redistribution unstable, the analysis compares decomposition methods and selects an epsilon-stabilized version for subsequent heatmaps.

Relevance values for FV dimensions are mapped to local descriptors through a function relating each descriptor to its output-space dimensions.This reformulation adapts Bag-of-Feature relevance propagation to Fisher Vector mappings with positive and negative contributions.
Pixel relevance is computed by uniformly distributing each local descriptor’s relevance across the pixels covered by that descriptor’s receptive field.The resulting pixel-level scores form a visualizable heatmap.
Signed local mappings can cause near-zero denominators when contributions cancel, which can exaggerate otherwise insignificant descriptor weights during relevance redistribution.This numerical problem motivates an adapted decomposition based on the LRP treatment of such cases.
The epsilon-stabilized decomposition produced the best Fisher Vector heatmaps despite violating relevance conservation, so Equation 22 with epsilon = 100 was used for later evaluations.Figure 7 compares heatmap quality using the area-above-curve and sign-switching measures.

C.1 ϵ-Stabilized Decomposition

The ϵ-stabilized decomposition prevents near-zero divisions in Fisher Vector relevance propagation, but can violate LRP’s relevance-conservation constraint.

Introducing a numerical stabilizer ϵ > 0 prevents near-zero divisions in the decomposition formula.
The stabilizer can absorb or generate varying amounts of relevance, violating LRP’s conservation constraints.

C.2 Absolute Value Decomposition

The absolute-value decomposition resolves weight-cancellation divisions without adding parameters or losing relevance, but discards information encoded in mapping signs.

Taking the absolute value of a descriptor’s mapping contribution removes remaining division-by-zero cases caused by weight cancellation.
The method requires no additional parameters and avoids the relevance loss associated with the ϵ-stabilized alternative.
Its drawback is that the mapping signs’ encoded information is lost.
Relevance-guided data deterioration evaluates heatmaps by removing the most relevant input components first and measuring the classifier’s output change.
The heatmap-quality measure A is obtained from the area between the original and altered-input prediction curves, with higher A indicating better representation of classifier decisions.

D.1 Local Feature Replacement

Local feature replacement adapts relevance-guided deterioration to feature-extraction pipelines, avoiding pixel-level recomputation and using GMM-generated replacements near the data manifold.

Pixel-level perturbation is computationally expensive for dense, multiscale local-feature pipelines because changing one pixel recomputes many overlapping descriptors.
Pixel perturbations can also damage image integrity or introduce class-related artifacts through blurring or random colors.
The proposed method replaces local descriptors rather than pixels, treating descriptors in the orderless set L as independently exchangeable events.
The GMM distribution λ generates dimensionality-reduced replacement descriptors intended to remain close to the data manifold.
Algorithm 1 orders descriptors by relevance, replaces them iteratively, updates the Fisher Vector, and evaluates the transformed representation with the classifier.
The experiments replace the first 10,000 ordered descriptors in batches of 100, yielding 100 measurement points; images contain roughly 15,000–100,000 features, averaging 69,000.
The procedure was repeated five times, recording 28,870 perturbation traces on true-positive predictions after threshold optimization for EER.
The neural-network setup used multi-label hinge-loss training, reflecting that PASCAL VOC images can contain multiple classes.

Training data:

Training images were prepared without bounding-box information, using rescaling and boundary padding so the neural network could use contextual content.

Bounding boxes were not used to generate training data, and images were rescaled so their largest side measured 256 pixels.
The smaller image side was padded at its boundary after rescaling.

Testing data:

Testing used a single center crop for heatmaps and the reported 72.12 mAP, while a sliding-window strategy reached 75.9 mAP with only 12–35 patches per image. The evaluation also counted multiple correctly detected classes in individual images and excluded several-hundred-window approaches because of computation time.

Test strategy: The heatmaps were computed using one center crop only, matching the single-crop evaluation setup.This alignment was chosen to preserve comparability between explanations and scores.
Test strategy: Mean average precision depends on how each test image is scored, with averaging over many crops being common.Thus, test-time crop aggregation changes the reported ranking performance.
Test strategy: 72.12 mAP was obtained by resizing the largest image side to 256 pixels and using one 227 × 227 center crop per image.This setup matched the heatmap computation for comparability.
Test strategy: 75.9 mAP resulted from resizing the smallest image side to 256 pixels and averaging sliding-window predictions with stride 20 pixels.This strategy used only 12–35 test patches per image.
Computational boundary: Approaches using several hundred test windows were not considered because their substantially increased computation time made them impractical for this evaluation.The paper therefore prioritized the context-focused analysis over maximizing the ranking score.
Evaluation data: 25,774 true-positive predictions were collected across all 20 classes with five repetitions per case.A single image could contain multiple correctly detected object classes simultaneously.