Source-linked AI summary
CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization
Zijie J. Wang, Robert Turko, Omar Shaikh, Haekyu Park, Nilaksh Das, Fred Hohman, Minsuk Kahng, Duen Horng Chau
TL;DR
Beginners struggle to connect CNNs’ low-level mathematical operations with their high-level model structure. CNN Explainer addresses this gap through interactive, linked visualization views, and a qualitative study found improved understanding, engagement, and usability, while identifying guidance needs for less experienced learners.
Problem
Beginners often struggle to understand how CNN inputs are transformed into class predictions and how mathematical operations relate to model structure.
Method
CNN Explainer integrates a high-level CNN architecture overview with interactive, on-demand views of low-level operations and transitions between abstraction levels.
Results
A qualitative observational study found that participants considered CNN Explainer useful and usable, remained engaged, and gained better understanding of CNN concepts.
Takeaways & Limitations
CNN Explainer offers a browser-based approach for helping non-experts learn CNNs and broadening access to modern AI education.
Takeaways & Limitations
Beginners with less CNN knowledge needed more instructions, and some initially found the visualization and animation difficult to understand.
Abstract
from arXiv · showhide
Deep learning's great success motivates many practitioners and students to learn about this exciting technology. However, it is often challenging for beginners to take their first step due to the complexity of understanding and applying deep learning. We present CNN Explainer, an interactive visualization tool designed for non-experts to learn and examine convolutional neural networks (CNNs), a foundational deep learning model architecture. Our tool addresses key challenges that novices face while learning about CNNs, which we identify from interviews with instructors and a survey with past students. CNN Explainer tightly integrates a model overview that summarizes a CNN's structure, and on-demand, dynamic visual explanation views that help users understand the underlying components of CNNs. Through smooth transitions across levels of abstraction, our tool enables users to inspect the interplay between low-level mathematical operations and high-level model structures. A qualitative user study shows that CNN Explainer helps users more easily understand the inner workings of CNNs, and is engaging and enjoyable to use. We also derive design lessons from our study. Developed using modern web technologies, CNN Explainer runs locally in users' web browsers without the need for installation or specialized hardware, broadening the public's education access to modern deep learning techniques.
1 INTRODUCTION
CNN Explainer addresses beginners’ difficulty connecting CNN layer operations with overall model structure through integrated, multi-level visual explanations. It is designed for non-experts and runs in a browser without installation.
- Motivation: Beginners struggle to understand how image inputs become class predictions because CNNs combine diverse layers, operations, and structural connections.The paper identifies the interplay between low-level mathematics and high-level network integration as a central learning challenge.
- Motivation: Existing CNN visualizations typically explain either high-level structure or low-level mathematics, leaving their connection unaddressed.CNN Explainer aims to bridge this gap.
- System: CNN Explainer integrates an Overview, Elastic View, and Interactive Formula to connect model-wide structure with detailed computations.The views transition across abstraction levels while preserving context for inspecting selected neurons and inputs.
- Access: The tool is designed for non-experts and is publicly available as a browser-based demonstration.The paper states that it runs locally in modern web browsers without installation or specialized hardware.
2 BACKGROUND FOR CONVOLUTIONAL NEURAL NETWORKS
CNNs transform input images into class predictions through multiple specialized layers. Their learned parameters and nonlinear operations support feature extraction while pooling reduces representation size and parameter count.
- Image Classification: Image classification maps an input image X to an output class Y, such as mapping a cat image to the label “cat”.CNNs use multiple computational layers to learn better image representations.
- CNN Architecture: CNNs contain convolutional, downsampling, and activation layers, each performing a predetermined function on its input.The different layer types contribute distinct operations within the network.
- Feature Extraction: Convolutional layers extract increasingly complex features, from early edges to later semantic features such as car headlights.Backpropagation learns kernel weights and biases from collections of input images.
- Layer Operations: Pooling downsampled inputs decrease the number of parameters and help avoid overfitting, while activation functions introduce non-linearity.Convolutional kernels slide across inputs, compute element-wise dot products, add bias, and produce activation maps.
3 RELATED WORK
Prior educational visualizations often isolate model structure, mathematics, or algorithmic behavior rather than connecting both CNN abstraction levels. CNN Explainer addresses this gap within broader algorithm-visualization research.
- Visualization for Deep Learning Education: Existing deep-learning educational tools include TensorFlow Playground for dense networks and GAN Lab for generative adversarial networks.These systems illustrate interactive visualization approaches for learning modern machine-learning concepts.
- Visualization for Deep Learning Education: Most educational resources explain either high-level model structures and training or low-level mathematics, but not both together.CNN Explainer combines both levels using real image data rather than synthetic 2D data.
- Algorithm Visualization: Algorithm visualizations use graphical representations, interaction, and animation to teach dynamic algorithm behavior.Prior studies report mixed effectiveness, while engagement is associated with better learning outcomes.
- Visualization for Deep Learning Education: Deep-learning visualization research has provided relatively little design and evaluation of visual educational tools for complex models.CNN Explainer draws on algorithm-visualization guidelines and reports findings related to prior visualization research.
- Visual Analytics for Experts: Many visual analytics systems target experts analyzing predictions, learned features, hidden states, training processes, or model vulnerabilities.Examples include Summit, LSTMVis, DGMTracker, and DeepEyes.
4 FORMATIVE RESEARCH & DESIGN CHALLENGES
The authors used instructor interviews, a student survey, and literature review to identify CNN-learning challenges and derive design requirements. These requirements emphasize connecting model structure, layer mathematics, and accessible interaction.
- Instructor Interviews: Four instructors reported relying on toy illustrations and valuing interactive tools with real image inputs for teaching CNNs.Their interviews also informed the subsequent student survey.
- Student Survey: The student survey retained 19 qualifying respondents from 43 responses and asked about learning challenges and helpful visualization features.Participants had previously studied CNNs and included Ph.D. students, M.S. students, undergraduates, and a faculty member.
- Research Process: The formative research combined instructor interviews, a student survey, and literature review to synthesize five design challenges.The stated goal was to help students understand CNN concepts well enough to design their own models.
- Design Challenges: Students faced challenges involving intricate model structure and complex layer operations, especially the mathematics behind CNNs.The survey and interviews highlighted connections between layers and the difficulty of their computations.
- Design Challenges: The central design challenge was connecting low-level mathematical operations with their role in high-level model structure.Prior learning-tool research likewise described difficulty translating network equations into mental models of transformations.
- Design Requirements: Effective visual learning tools should clarify mappings between algorithms and visual encodings while actively engaging learners.The authors also considered browser accessibility and limited the first prototype to inference after training rather than backpropagation or training.
5 DESIGN GOALS
CNN Explainer’s design goals target beginners’ difficulty connecting CNN structure with mathematical operations through integrated, interactive, accessible views.
- G1 provides a visual summary of CNN layers, outputs, and connections so users can track data from input images to class predictions.
- G2 provides interactive formula views so users can examine the mathematical operations underlying CNN layers.
- G3 uses focus-plus-context displays and smooth transitions to connect low-level mechanisms with high-level CNN structure.
- G4 combines explanations, annotations, and visualization customization to make the system understandable and engaging.
- G5 uses web browsers to make CNN learning accessible without installation or coding and releases the code for future development.
6 VISUALIZATION INTERFACE OF CNN EXPLAINER
CNN Explainer visualizes forward propagation through a Tiny VGG model using an overview, elastic explanation views, and interactive formula views connected by smooth transitions and instructional support.
- The system visualizes forward propagation through a trained Tiny VGG model and integrates Overview, Elastic Explanation, and Interactive Formula Views.Tiny VGG uses fewer convolutional layers than VGGNet and classifies 10 image classes.
- Visual Encoding: The model uses persistent diverging color scales to distinguish positive and negative activation, weight, and bias values.
- Overview: The Overview presents neurons grouped into sequential layers, activation maps, and connections between consecutive layers.
- Elastic Explanation Views: Elastic Explanation Views focus attention on selected neurons while retaining structural context and visualizing intermediate computations.The convolutional view animates a kernel sliding across input neurons, with flowing-dashed edges becoming solid after the animation.
- Flatten Layer: The Flatten Elastic Explanation View shows how an n-dimensional tensor becomes a 1-dimensional tensor before fully connected classification.It traverses pixels in row-major order and links flatten-layer neurons to source components.
- Interactive Formula View: Interactive Formula Views explain convolution, ReLU, pooling, and softmax through animated or hover-controlled mappings between inputs, outputs, and equations.
- Transitions and Learning Support: Users move from architectural inspection to elastic explanations and then mathematics, supported by annotations and a tutorial article.The interface reveals mathematical operations, weights, and biases through further interaction after structural views establish context.
- Model: The trained Tiny VGG model achieves 70.8% top-1 accuracy on the validation dataset.Batch size and learning rate are fine-tuned with 5-fold cross-validation.
7 USAGE SCENARIOS
Usage scenarios show CNN Explainer supporting individual exploration of layer connectivity and classroom teaching through interactive demonstrations and hyperparameter experimentation.
- 7.1 Beginner Learning Layer Connectivity: The Flatten Elastic Explanation View helps the learner understand how activation-map pixels are unwrapped into a 1-dimensional layer between max pooling and output.
- 7.2 Teaching Through Interactive Experimentation: An instructor uses the Convolutional Elastic Explanation View to demonstrate convolution inside a complete image-classification CNN and answer questions with real inputs.
- 7.2 Teaching Through Interactive Experimentation: The Hyperparameter Widget lets instructors demonstrate how stride and other hyperparameters change the convolutional sliding window and output dimensions.The web-based tool replaces hand-drawn toy-matrix illustrations for this classroom demonstration and supports later student experimentation.
8 OBSERVATIONAL STUDY
An observational study with 16 participants examined CNN EXPLAINER’s usability, usefulness, learning support, and interaction design. Participants generally found it easy to use, engaging, and helpful for connecting CNN operations with model structures, while the study also identified guidance and explanatory gaps for beginners.
- Participants and procedure: 16 participants explored CNN EXPLAINER in one-on-one sessions after a brief CNN overview and tool tutorial.Participants then freely explored the browser-based tool and completed a feature checklist and exit questionnaire.
- Transitions between different views: 9/16 participants said transitions between high-level structure views and low-level explanations helped them understand how layer computations transform data.Participants valued moving from the overview to detailed formula and explanation views while retaining connections to the overall model.
- Transitions between different views: The fisheye transition kept neighboring layers visible while reducing the opacity of more distant layers, preserving structural context during layer-specific inspection.Participants could focus on a selected layer while recognizing that the same operation occurs elsewhere in the network.
- Animations for enjoyable learning experience: Animations supported navigation, linked visual elements, and engagement, but animation alone did not make convolution easy for every participant to understand.Participants often needed annotations or the interactive formula view alongside the sliding-window animation to interpret the operation.
- Limitations: Beginners needed more guidance, and some participants wanted explanations of why architectural choices such as ReLU and max pooling help CNN performance.The authors identify step-by-step tutorials, additional annotations, and visualizations of alternative functions as potential improvements.
9 DISCUSSION AND FUTURE WORK
CNN Explainer’s future work centers on expanding coverage beyond inference-time CNN explanations, validating educational effectiveness quantitatively, and generalizing its visualization principles to other layers and models.
- Explaining training process and backpropagation: CNN Explainer currently visualizes how a pre-trained CNN transforms input images into class predictions, while training and backpropagation remain planned extensions.Preliminary and observational studies found learner interest in understanding the training process and backpropagation in greater detail.
- Generalizing to other layer types and neural network models: Its Interactive Formula Views could be adapted to explain additional layer types, such as Leaky ReLU, or combinations of layers, such as Residual Blocks.
- Generalizing to other layer types and neural network models: Transitions between abstraction levels could generalize to neural networks such as long short-term memory networks and Transformers.These models also require learners to understand layer operations within complex network structures.
- Integrating algorithm visualization best practices: CNN Explainer applies visualization-with-explanations and customizable-visualization principles, while future tools could incorporate practices such as interactive pop quizzes.
- Quantitative evaluation of educational effectiveness: The study evaluated usefulness and usability qualitatively; quantitative studies could assess engagement and content understanding through interaction logs.
10 CONCLUSION
CNN Explainer is an open-source interactive visualization system for non-experts learning about CNNs. It runs in modern web browsers and aims to lower barriers to understanding and appropriately applying AI technologies.
- CNN Explainer helps non-experts learn about CNNs through an interactive visualization system developed to support first steps into complex deep learning concepts.
- The system runs in modern web browsers and is open-sourced, broadening public access to education about modern AI techniques.
- The authors report design lessons from iterative development and an observational user study, and hope to inspire further visual learning tools for AI.