Source-linked AI summary
Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks
Anh Nguyen, Jason Yosinski, Jeff Clune
TL;DR
Existing activation maximization does not systematically represent neurons responding to multiple feature types and can create unnatural mixtures. The paper introduces multifaceted feature visualization and center-biased regularization to synthesize facets separately and improve image structure. The resulting visualizations have more natural colors, more global structure, details, and context, while providing a more comprehensive understanding of neuron functions.
Problem
Existing activation maximization lacks a systematic way to visualize the multiple facets to which neurons respond and can produce unnatural, incoherent image mixtures.
Method
MFV clusters images that activate a neuron, initializes activation maximization from cluster means, and combines it with regularization methods including center-biased regularization.
Results
MFV produces higher-quality visualizations with more natural colors, global structure, details, and context, while revealing multiple facets of each neuron.
Takeaways & Limitations
Separately visualizing facets provides a clearer and more comprehensive description of each neuron’s function.
Takeaways & Limitations
Center-biased regularization biases optimization toward central objects but does not guarantee that an image contains only one object.
Abstract
from arXiv · showhide
We can better understand deep neural networks by identifying which features each of their neurons have learned to detect. To do so, researchers have created Deep Visualization techniques including activation maximization, which synthetically generates inputs (e.g. images) that maximally activate each neuron. A limitation of current techniques is that they assume each neuron detects only one type of feature, but we know that neurons can be multifaceted, in that they fire in response to many different types of features: for example, a grocery store class neuron must activate either for rows of produce or for a storefront. Previous activation maximization techniques constructed images without regard for the multiple different facets of a neuron, creating inappropriate mixes of colors, parts of objects, scales, orientations, etc. Here, we introduce an algorithm that explicitly uncovers the multiple facets of each neuron by producing a synthetic visualization of each of the types of images that activate a neuron. We also introduce regularization methods that produce state-of-the-art results in terms of the interpretability of images obtained by activation maximization. By separately synthesizing each type of image a neuron fires in response to, the visualizations have more appropriate colors and coherent global structure. Multifaceted feature visualization thus provides a clearer and more comprehensive description of the role of each neuron.
1. Introduction
Deep Visualization uses synthetic images to reveal what DNN neurons detect, but prior activation maximization struggled with unrealistic images and multifaceted neurons. The paper introduces MFV and center-biased regularization to visualize facets separately and improve image coherence.
- Activation maximization synthesizes images that maximally activate selected output or hidden neurons, revealing the features they detect.
- Gradient-only activation maximization can produce unrecognizable images, so optimization must incorporate priors favoring natural-looking images.
- Prior visualizations produced unnatural colors, repeated fragments, incoherent structure, and lacked systematic methods for visualizing a neuron’s different facets.
- MFV systematically reveals multiple facets by separately synthesizing the different types of stimuli that activate each neuron.
- MFV reveals multifaceted neurons at all network levels, with higher-level neurons more multifaceted than lower-level neurons.
- Separately synthesized facets improve image quality, producing more natural colors and globally consistent structure instead of inappropriate mixtures.
2. Methods
The methods combine activation maximization with natural-image regularization, cluster-based facet initialization, and center-biased optimization. These procedures target both multifaceted responses and the repeated fragments that impair global coherence.
- The experiments use a CaffeNet variant trained on the 1.3-million-image ILSVRC 2012 ImageNet dataset.
- Activation maximization: Activation maximization updates pixels to increase a target neuron’s activation while regularization biases results toward natural images.
- Multifaceted feature visualization: MFV clusters high-level representations of activating images, averages images near each cluster centroid, and uses those means to initialize separate activation-maximization runs.
- Center-biased regularization: Repeated fragments arise because gradients jointly intensify existing objects and draw new fragments, making coherent global structure difficult to obtain.
- Center-biased regularization: Center-biased regularization gives center pixels more optimization iterations than edge pixels, encouraging a sharp, centrally located object.
- Center-biased regularization: Center-biased images contain fewer duplicated fragments and more closely resemble training images, but the method does not guarantee a single object.
3. Results
Multifaceted feature visualization produces distinct synthetic images for different stimuli that activate the same neuron, revealing richer representations across network layers. It also improves visual coherence and color realism, although some hidden-layer interpretations remain unresolved.
- Multifaceted class neurons: Multifaceted visualization synthesizes different colors, perspectives, object counts, and contexts for images activating the same neuron.Examples include bell peppers, convertible cars, movie theaters, pool tables, grocery stores, and fishing reels.
- Multifaceted class neurons: Regularized multifaceted reconstructions can separate grocery-store facets into synthetic images resembling apples, artichokes, or oranges.The network may classify these images as both grocery store and the corresponding produce class.
- Hidden-layer neurons: Facet diversity increases from conv3 upward, progressing from changes in pose and color to different object counts and object types.Lower layers show fewer noticeable facet differences, while higher layers become more invariant to changes in the input.
- Hidden-layer neurons: Fully connected hidden neurons often combine multiple high-level concepts, such as turtles, scuba divers, whales, and sharks forming a possible “something underwater” representation.The authors note that facets may be impure or may reflect genuinely abstract feature detectors.
- Limitations and open questions: The authors could not determine whether some fc6 and fc7 neurons are true amalgams using Deconv and Layer-wise Relevance Propagation.They identify this distinction as requiring future research.
- Visualization quality: The visualizations show more natural colors and globally consistent objects than previous activation maximization methods.The comparison is subjective, but the authors report improved recognizability, detail, global consistency, and contextual structure.
4. Discussion and Conclusion
The paper argues that multifaceted feature visualization reveals richer neuron representations and improves activation-maximization image quality, while leaving automatic facet-count determination unresolved.
- Multifaceted visualization shows that neurons at all network levels detect multiple feature facets, with higher-level neurons being more multifaceted than lower-level ones.
- The method suggests that supervised DNNs learn objects’ global structure, details, context, and multiple facets, rather than only sparse discriminative features.
- The number of visualized facets depends on the manually chosen k in k-means, and automatically identifying the true facet count remains unanswered.
- MFV produces more natural colors and globally coherent images by synthesizing each facet separately rather than mixing incompatible visual properties.
Supplementary Information for: Multifaceted Feature Visualization: Uncovering the Different Types of Features
The supplementary information accompanies the paper on multifaceted feature visualization and its investigation of feature types learned by neurons.
- The supplement is associated with the study of different feature types learned by neurons in deep neural networks.
S1. Activation maximization initialized with interpolated images
Interpolated-image initialization usually leads activation maximization toward one facet rather than a hybrid, though the selected facet can differ from the seed and depends on the example.
- The experiment tests whether initializing from the mean of two facet types yields a hybrid visualization or causes optimization to favor one facet.
- The procedure clusters roughly 1,300 training images with k = 10, selects images from different clusters, interpolates between them, and uses each interpolation to initialize activation maximization.
- Starting from a single real image can preserve the initial facet while producing different details in the synthesized visualization.
- In other cases, optimization converges to a different facet than the one represented by the initial image.
- Across examples, optimization often reconstructs one facet or another rather than a hybrid of both.
S2. Comparison between different priors
The supplementary comparisons show how regularization and initialization affect activation-maximization images, with total variation, center bias, and mean-image seeding improving different aspects of visualization quality.
- S2. Comparison between different priors: Unregularized activation maximization produces unrecognizable, overly high-frequency images, motivating comparisons among image priors.
- S2. Comparison between different priors: Interpolated-image initialization often reconstructs one facet rather than a hybrid, illustrating a limitation of using intermediate seeds to combine facets.
- S2.1. Total variation: TV regularization is applied as a separate minimization step after each activation update, using Split Bregman optimization for image smoothing.
- S2.1. Total variation: 100 TV-minimization iterations per activation-maximization iteration empirically produced slightly better results than the prior method’s objective formulation.
- S2.1. Total variation: TV alone produced fairly good visualizations and was not qualitatively improved by adding jitter.
- OPTIMIZATION SCHEDULE AND PARAMETERS: Center-biased regularization reduces repeated fragments and biases optimization toward a central object, but does not guarantee a single object.
- S2.4. Initialization with mean images: Mean-image initialization can turn a blurry general layout into a coherent scene and substantially improve visualization colors.
- S2.4. Initialization with mean images: MFV images often already contain centered objects, so adding center-biased regularization produces no substantial qualitative improvement in that setting.
S3. What are the hidden units in fully connected layers for?
Hidden fully connected neurons often respond to multiple, semantically different image types, but their precise functions remain difficult to determine. Multifaceted visualizations and relevance analyses reveal both this diversity and the ambiguity of what features drive it.
- Hidden-layer neurons: Many fc6 and fc7 neurons fire for substantially different image types, including turtles, scuba divers, sharks, bells, faces, and trucks.One example neuron consistently produces a hybrid visualization across different initialization concepts.
- Layer differences: Unlike fc6 and fc7 units, fc8 neurons more often represent a shared semantic concept across distinct facets.For example, a restaurant neuron responds to different views of a restaurant, while some class neurons still respond to unrelated objects.
- Interpretation limits: Deconvolution and Layer-wise Relevance Propagation do not resolve whether disparate responses reflect object semantics, shared shape patterns, or distributed representations.Deconvolution is noisy, while LRP highlights outlines spanning underwater objects, bells, heads, and wheels.
- Optimization behavior: Optimization commonly converges to the same hybrid or dominant visualization across starting points, despite the underlying neuron responding to varied image types.Examples include a dog-plus-bathtub hybrid and more homogeneous arch or quail-like bird visualizations.
- Interpretation limits: The evidence leaves the precise function of hidden fully connected neurons an open question requiring further research.The authors specifically conclude that it remains unclear what these neurons represent.