Source-linked AI summary
Interactive Analysis of Global Explanations using Aggregated Class Activation Maps for Network Data
Igor Cherepanov, David Sessler, Alex Ulmer, Felix Wagner, Throsten May, Jörn Kohlhammer
TL;DR
Divergent patterns within network-traffic classes make comprehensive explanations and descriptive rules difficult to derive. The paper aggregates CNN-based local explanations into interactive global visualizations, which experts used successfully to identify impactful bytes and compare class patterns.
Problem
Divergent patterns within network-traffic classes limit clear explanations and complicate extracting descriptive rules for network analysis and intrusion detection.
Method
The system aggregates local CAMs from a 1D-CNN into global class explanations that experts can visualize, compare, and refine interactively.
Results
Experts completed impactful-byte identification tasks well and positively rated the system, while interactive analysis revealed distinct patterns and potential subclasses.
Takeaways & Limitations
Global explanations support network experts in recognizing class patterns, identifying influential features, and extracting descriptive rules from traffic data.
Takeaways & Limitations
Applying the system in companies requires deeper guidance for network experts and their involvement in creating training datasets, while new application classes require retraining.
Abstract
from arXiv · showhide
Recent machine learning (ML) advances have demonstrated that deep learning (DL) achieves impressive results in different application domains, including the classification of computer network traffic to corresponding applications. However, the data frequently contains diverging patterns within a single predicted class. This presents a significant challenge to the ability to provide a clear and comprehensive explanation and emphasizes the necessity for tools capable of detecting and analyzing these patterns. Furthermore, the capacity to extract descriptive rules for classes is a crucial requirement in network traffic analysis and intrusion detection, particularly when leveraging advanced tools like next-generation firewalls. We provide a visual-interactive system that explains predictions of classes for network traffic. Global explanations derived from multiple samples of a given class contribute to understanding model predictions. Visualization of global explanations enables recognition of different patterns that offer experts a more comprehensive overview of its characteristics. We introduce a prototype that facilitates visual exploration and refinement of global explanations, enabling network experts to detect and refine new patterns for specific applications. These explanations support the identification of misleading features and the formulation of new rules for the management of networks. Our approach also aims at enabling ML experts to acquire new insights, including the possibility of separating or merging classes and the development of more accurate and reliable DL models. Our proposed prototype was evaluated by experts in machine learning and network analysis.
1. Introduction
The paper addresses the explainability challenges of complex ML models in network traffic classification by visualizing class-specific feature contributions. It aggregates local CAMs into global explanations that help experts identify patterns, compare classes, and extract descriptive network rules.
- Motivation: Complex, high-performing ML models create a growing need to understand their decisions, especially when data complexity or dimensionality obscures patterns from humans.The introduction frames explainability as necessary for integrating AI into complex systems while supporting expert goals.
- Network traffic analysis: The approach targets network traffic monitoring by visualizing class-specific feature contributions to support payload-based rules for Suricata, Snort, and next-generation firewalls.The paper focuses on traffic-data classification and rule creation for advanced network-security systems.
- Class activation maps: One-dimensional CNNs learn hidden network-traffic patterns without manual feature extraction and can calculate a class activation map directly for each classified sample.CAMs provide local explanations of individual predictions during classification.
- Global explanations: The prototype visualizes local CAMs after aggregation as global explanations, enabling experts to identify within-class patterns and key points of interest.Global explanations also support comparisons across different classes.
- Expert analysis: Global explanations support descriptive-rule extraction, reveal new insights into network data, and make model decision-making more comprehensible to increase trust.The visualizations help experts analyze patterns and compare classes while understanding the model’s decisions.
2. Related Work
The related work reviews network traffic analysis and network-data classification, including explainable AI techniques. It also contrasts rule-based inspection with deep learning and motivates explanation methods for complex models.
- Related work: The review is organized around network traffic analysis, network-data classification, and XAI techniques in this domain.These topics are covered in Sections 2.1 and 2.2.
- Network traffic analysis: Network traffic analysis supports security by identifying threats such as unauthorized data flows and enabling prevention strategies.A central focus is precisely identifying different traffic types.
- Network traffic analysis: Deep packet inspection classifies applications using packet headers, payloads, predefined patterns, and regular expressions, but evolving applications require frequent pattern updates.DPI also struggles to recognize new applications, whereas DL autonomously learns features without manual rules.
- Explainable network-data classification: DL models improve feature learning but lack interpretability, limiting trust in classifications, especially for cybersecurity applications.Class activation maps were introduced as an explanation technique based on CNNs.
3. Data-User-Task
This section defines PCAP data, identifies network and machine-learning experts as target users, and derives their analysis tasks and requirements for the proposed approach.
- PCAP Data: PCAP files store captured network packets in binary format and support examining traffic patterns, identifying anomalies, troubleshooting issues, and detecting security threats.They are generated by monitoring tools such as Wireshark and tcpdump.
- Target Users: Network experts—including administrators, cybersecurity professionals, and malware analysts—use the approach to identify patterns and formulate rules for next-generation firewalls.Their work includes comprehensive network management, examination, threat detection, and security-policy enforcement.
- Target Users: ML developers and AI researchers use visualized classification explanations to understand model decisions and communicate with network experts to improve AI systems.Their tasks include providing explanatory AI models and acquiring potential knowledge from model decisions.
- User Tasks: The approach addresses network optimization, security-policy compliance, explainable model decisions, misleading features, and AI-model optimization.These tasks span QoS and resource management, regulating application flows, correcting model biases, and identifying improvement areas.
- Requirements: Derived requirements include overviewing class explanations, identifying influential bytes, comparing byte-wise sequences, visualizing byte-impact distributions, and refining class explanations to detect patterns.The requirements map to tasks T1–T5 and support analysis of class-specific explanations.
4. Visual-interactive XAI System
The system uses a 1D-CNN to classify variable-length network packets and generates global class explanations by aggregating local CAMs. Its interactive visualizations support pattern exploration across classes and byte-level impact distributions.
- Model and preprocessing: The model uses 1D convolutions to capture relationships between neighboring network-data points and was selected because it performs well on network data.
- Model and preprocessing: Variable-length packets are converted to fixed-size inputs by truncating longer packets and padding shorter ones.
- Model and preprocessing: 97% F1-score was achieved by the trained 1D-CNN on the authors’ balanced dataset with new application classes.The architecture used feature maps of 16, 32, five layers with 64, and 128, 1500 inputs, stride 1, and kernel size 7.
- Global explanations: Local CAMs explain individual predictions, while their aggregation produces global explanations for specific classes because packet attributes retain consistent positions and lengths.Mean aggregation represents the overall average impact distribution, while the median is less influenced by extreme values.
- Interactive visualization: The interface supports class selection, CAM tooltips, adjustable cell sizes, and a scarf-plot view for examining impact and variability across many byte positions.Cells larger than 10 pixels encode aggregated impact and variability; smaller cells disable variability and become stripes.
5. Usage Scenario
The usage scenarios show how the visual-interactive system supports network analysts and ML professionals through global explanations, data visualization, filtering, and model analysis. Analysts can isolate distinct traffic patterns, formulate class specifications, and identify misleading features, while ML professionals can examine model behavior and reliability.
- System Use: The system supports network analysts’ investigations with global explanations of network traffic, visual data representation, filtering, and integrated classification.It is also intended to support collaboration with ML experts to enhance model reliability.
- Network Analyst Scenario: Global explanations highlight impactful bytes whose packet-level analysis helps analysts develop characteristic specifications for classes.A long yellow bar indicates that the corresponding byte strongly influences a particular class.
- Network Analyst Scenario: Multiple modes in a byte’s local CAM-value distribution can be separated by filtering to a single mode, isolating packets with distinct impact characteristics.Selecting one mode produces a more similar distribution of impact values for the analyzed packets.
- Network Analyst Scenario: The tool helps analysts detect semantic discrepancies and misleading features, such as masked IP addresses or excluded Ethernet headers, to refine model accuracy and reliability.These findings identify features with which analysts may disagree during significant-byte analysis.
- ML Professional Scenario: ML professionals can use global explanations to understand influential features, model behavior, strengths, limitations, biases, and errors in network-traffic predictions.They can also experiment with model parameters by comparing models trained on two classes with models trained on additional classes.
6. Evaluation
The evaluation used task-based sessions and think-aloud protocols with machine-learning and network-analysis experts to assess the visualization’s usability and domain-specific utility. Experts accurately interpreted impactful bytes and global CAM patterns, praised the system’s intuitive overview, and rated usability above 80 on the SUS.
- Evaluation Method: Six experts from machine learning and network analysis/cybersecurity completed domain-tailored tasks, with both groups also answering shared questions about visual coding and interaction.The study included three experts in each group; sessions began with 15 minutes of training and used think-aloud interaction.
- Network and Cybersecurity Experts: All experts accurately located the most influential bytes and recognized that encrypted sections did not reveal influential bytes.ML experts also correctly identified the most influential byte in global CAM impact distributions, and found histograms helpful for examining value distributions.
- Network and Cybersecurity Experts: Network experts said the explanations could support discussions with ML specialists by indexing the specific byte considered incorrect by the model.None had previously used a packet-classification system, although they had seen Cisco’s classification solution, which did not provide an explanation.
- Machine-Learning Experts: ML experts used global CAMs to examine classification errors, similar classes, and features that were important for some samples but unimportant for others.They observed that the rear payload exhibited no patterns and was shown in a less significant color, but preferred not to exclude it without a domain expert.
7. Discussion, Limitations and Future Work
The evaluation found that network experts valued the application and rated it positively, but their limited prior exposure to ML techniques creates a need for deeper guidance. Future work will automate suggestions of impactful bytes, rule generation, and distribution analysis.
- Limitations: Network experts, who had limited prior exposure to ML models and explanatory techniques, recognized the application’s value and gave the evaluation and SUS score positive ratings.The paper notes that this group encountered substantial novel information about ML techniques and would require deeper, more detailed guidance.
- Future Work: The planned system will automatically suggest impactful bytes, including the most impactful, strongly separated peaks, and completely unimportant bytes.These suggestions are intended to satisfy analyst criteria for interpreting byte-level impact.
- Future Work: Future work will investigate automatic rule generation from classifications and explanations and automatically analyze distributions for faster detection.The planned analyses target network-management rules and rapid identification of distributional patterns.
8. Conclusion
The paper introduces a visual-interactive system that connects experts, algorithms, and data by aggregating CNN local explanations into global explanations for interpreting network application classes.
- The system serves as an interface between experts, algorithms, and data.
- It aggregates local explanations from a CNN model for network data into global explanations of application classes.
- The paper summarizes related work, defines user groups and tasks, and derives system requirements.