Source-linked AI summary
Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention
Jinkyu Kim, John Canny
TL;DR
Self-driving controllers need interpretable rationales, but neural networks can be opaque and attention maps may include spurious regions. The paper combines end-to-end visual attention with causal filtering, showing across three driving datasets that attention preserves control accuracy while producing more concise saliency explanations.
Problem
Neural vehicle controllers need easy-to-interpret rationales, yet end-to-end neural-network behavior can be opaque.
Method
The model predicts steering from images with visual attention, then tests clustered attention regions by masking them and retains regions that causally affect output.
Results
Attention does not degrade control accuracy, and causal filtering removes non-influential features to reduce explanation complexity across three real-world driving datasets.
Takeaways & Limitations
Causal visual saliency provides a more succinct way to expose which image features influence a deep vehicle controller's behavior.
Takeaways & Limitations
The evaluation uses specific driving datasets and CNN configurations, while exploration of other convolutional architectures is out of scope.
Abstract
from arXiv · showhide
Deep neural perception and control networks are likely to be a key component of self-driving vehicles. These models need to be explainable - they should provide easy-to-interpret rationales for their behavior - so that passengers, insurance companies, law enforcement, developers etc., can understand what triggered a particular behavior. Here we explore the use of visual explanations. These explanations take the form of real-time highlighted regions of an image that causally influence the network's output (steering control). Our approach is two-stage. In the first stage, we use a visual attention model to train a convolution network end-to-end from images to steering angle. The attention model highlights image regions that potentially influence the network's output. Some of these are true influences, but some are spurious. We then apply a causal filtering step to determine which input regions actually influence the output. This produces more succinct visual explanations and more accurately exposes the network's behavior. We demonstrate the effectiveness of our model on three datasets totaling 16 hours of driving. We first show that training with attention does not degrade the performance of the end-to-end network. Then we show that the network causally cues on a variety of features that are used by humans while driving.
1. Introduction
The paper addresses opaque neural vehicle controllers by combining visual attention with causal filtering to produce concise, interpretable steering explanations. It evaluates this approach on large real-world driving datasets while preserving control accuracy.
- Neural-network vehicle controllers can be cryptic because their architectures and hidden activations lack an obvious relation to estimated functions.
- Visual attention maps provide instantly displayable image regions that users can interpret as potentially influencing vehicle control.
- The method clusters attention outputs into blobs, masks each blob, and retains only regions whose removal causally affects network output.
- The model uses convolutional feature extraction, a coarse-grained visual-attention decoder, and a fine-grained causal-saliency refinement step.
- Over 1,200,000 video frames from three real-world driving datasets, totaling approximately 16 hours, are used to demonstrate effectiveness.
- The paper reports that attention heat maps do not degrade control accuracy and that causal filtering reduces explanation complexity by removing non-influential features.
2. Related Works
Related work spans mediated-perception controllers, end-to-end image-to-control learning, intermediate interpretable features, and visualization methods. The paper positions attention with causal filtering as a way to expose influential spatial regions while addressing opacity and spurious saliency.
- Mediated-perception approaches recognize human-designated features such as lane markings and cars before applying rule-based control.
- End-to-end methods learn steering and related controls directly from vehicle-camera video and human-driver control recordings, but their behavior can remain opaque.
- Intermediate-feature approaches predict interpretable quantities such as lane curvature and distances, but one prior system degraded control accuracy by 40% or more.
- Deconvolution and contribution-based methods visualize activations or pixel effects, but their treatment of spatially extended features lacks formal measures.
- Attention-based methods directly identify image areas passed to the recognition network, while causal filtering removes spurious areas that do not affect output.
3. Method
The model predicts driving control from raw images using convolutional features, soft visual attention, and a fine-grained causal-saliency decoder. Attention weights form image maps, while masking clustered regions tests which areas affect prediction performance.
- 3.1. Preprocessing: The model predicts inverse turning radius from raw images, then relates it to steering angle through Ackermann steering geometry.Training requires timestamped vehicle speed and steering-angle measurements.
- 3.1. Preprocessing: Raw images are resized to 80×160×3 and normalized in HSV colorspace before entering the network.Height is cropped to match the target aspect ratio before nearest-neighbor down-sampling.
- 3.2. Encoder: Convolutional Feature Extraction: A convolutional encoder produces spatial feature vectors in a W×H×D feature cube, allowing attention to select image regions without pooling-based spatial loss.Each spatial slice is a D-dimensional vector, and the feature cube is provided to the LSTM attention model.
- 3.3. Coarse-Grained Decoder: Visual Attention: The soft attention decoder computes normalized weights over feature locations and combines weighted features into a context vector used for inverse-turning-radius prediction.The LSTM generates attention weights conditioned on the current feature cube and previous hidden state; the weighted context is passed to the output layer.
- 3.3. Coarse-Grained Decoder: Visual Attention: A penalized loss encourages attention to cover different image parts across time while training the attention model by standard back-propagation.The model uses a penalty coefficient λ for this temporal coverage regularization.
4. Result and Discussion
Across three large driving datasets, the model maintains competitive steering prediction while visual attention highlights driving-relevant regions and causal filtering removes spurious attention blobs to simplify explanations.
- 4.2. Training and Evaluation Details: Over 16 hours and more than 1,200,000 video frames from Comma.ai, Udacity, and HCE support training and evaluation.The datasets contain front-view video and time-stamped vehicle-dynamics measurements.
- 4.2. Training and Evaluation Details: The experiments omit data augmentation because the Comma.ai and HCE images came from a single front-view camera.The authors note that augmentation using off-center views may improve performance.
- 4.3. Effect of Choosing Penalty Coefficient λ: The model attends to road elements including lane markings, guardrails, and vehicles ahead while predicting steering-related curvature.Attention maps are evaluated with mean absolute error, and larger λ encourages attention to wider image regions.
- 4.4. Effect of Varying Smoothing Factors: αs = 0.05 yields the best steering-estimation performance among the tested smoothing factors.The smoothing procedure reduces effects from human-factor variation and measurement noise in steering angle and velocity.
- 4.5. Quantitative Analysis: The attention model achieves MAE values of 1.18–4.15 and does not degrade control accuracy compared with alternatives.Alternatives include an identical base CNN with a fully connected network without attention and a CNN-LSTM without attention.
- 4.6. Effect of Causal Visual Saliencies: Causal filtering removes spurious attention blobs, producing simpler and more accurate visual-saliency maps.The reported spurious-source proportions are 62% for Comma.ai and 58% for HCE.
5. Conclusion
The paper presents an interpretable visualization for deep self-driving controllers using visual attention and causal filtering. Across more than 16 hours of real driving data, attention preserves control accuracy, highlights interpretable features, and reduces explanation complexity.
- 5. Conclusion: The method combines a visual attention model with an additional causal-filtering layer for visualizing deep vehicle controllers.The system was tested on three large-scale real driving datasets containing over 16 hours of video frames.
- 5. Conclusion: Attention does not degrade control accuracy compared with an identical base CNN without attention.Raw attention maps highlight interpretable image features, while causal filtering removes features that do not significantly affect the output.
- 5. Conclusion: Causal filtering reduces explanation complexity by removing image features that do not significantly affect the controller output.