Source-linked AI summary
Length Matters: Clustering System Log Messages using Length of Words
Keiichi Shima
TL;DR
System-log template mining must classify increasingly diverse messages while producing useful templates efficiently. The paper proposes one-pass clustering based on word-length sequences, reports online template mining across three datasets, and identifies threshold selection and lost state distinctions as limitations.
Problem
Existing template-mining methods often inspect word frequencies, character properties, or prior software knowledge, while two-pass processing can delay clustering and complicate adaptation to changing message sources.
Method
The paper clusters syslog messages by the sequence of word lengths and designs the algorithm for online, one-pass template mining.
Results
The algorithm was implemented in Python and applied with SHISO to three syslog datasets, producing online template-mining results and supporting message-group analysis.
Takeaways & Limitations
Word-length sequences can cluster messages for online template mining, with the proposed mechanism producing a similar number of templates to prior work with less mining complexity.
Takeaways & Limitations
The method does not use word frequency, which can merge state messages that would be useful as separate templates, and its behavior depends on threshold settings.
Abstract
from arXiv · showhide
The analysis techniques of system log messages (syslog messages) have a long history from when the syslog mechanism was invented. Typically, the analysis consists of two parts, one is a message template generation, and the other is finding something interesting using the messages classified by the inferred templates. It is important to generate better templates to achieve better, precise, or convincible analysis results. In this paper, we propose a classification methodology using the length of words of each message. Our method is suitable for online template generation because it does not require two-pass analysis to generate template messages, that is an important factor considering increasing amount of log messages produced by a large number of system components such as cloud infrastructure.
1 Introduction
Syslog messages are widely used but remain difficult to analyze because their bodies are often free-form and systems may use different formats. Existing template-inference methods address this with frequency, source-code, character-class, or word-vector information, but several require two-pass processing or prior knowledge.
- Message formats: Syslog collects system-status messages across systems, but its original protocol left message-body structure undefined, so messages are typically free-form text.The newer specification organizes semantic structure, yet adoption is limited and software environments may require handling both old and new formats.
- Template inference: SLCT infers templates by counting frequent words and their positions across log messages in a two-pass process.For example, it represents fixed keywords in “interface eth0 up” by their word positions.
- Template inference: LogCluster relaxes SLCT’s fixed-position constraint by allowing variable-length wildcard sequences between fixed words, while still requiring two-pass processing.It can group messages such as “interface eth0 up” and “interface HQ Link up” under one template.
- Template inference: Source-code-based inference uses knowledge of installed software but requires preparation and template updates when software is added or changed.This approach is useful when the target operating system’s software is known in advance.
- Similarity-based methods: Character-class and word-vector methods compare messages using character-type composition or word properties to estimate similarity and form clusters.The character-class method weights five word classes, while SHISO uses Euclidean distance between word vectors and a threshold.
2 Analysis of Messages in the Wild
Syslog messages exhibit recurring word-length and word-position patterns because programs print them in pre-formatted styles. These patterns can distinguish many templates, but different messages can still have similar length-based distances.
- Word-length properties: Syslog messages contain fixed and variable components, and their word lengths provide a possible classification cue without requiring knowledge of each word’s semantic type.Fixed components tend to retain the same length, while variable components often have context-specific length ranges.
- Word-length properties: Each message group shows a distinctive word-length distribution pattern, with process names often fixed in the first position and identifiers commonly spanning 3–5 digits.IPv4 and IPv6 addresses can produce wider ranges, while host names may be mostly stable but include short and long outliers.
- Dataset analysis: The dataset analysis used messages from WIDE project hypervisors, with the number of words excluding date and host information examined across messages.The most common message group contained 11 words, and 27 templates were created half-manually.
- Template distinction: Distance between template messages distinguishes two templates in most cases, according to the similarity matrix for the templates listed in Table 1.The analysis therefore treats distance as a useful but incomplete discriminator.
- Template distinction: Some completely different template messages nevertheless have a small distance, motivating analysis of word positions in addition to word-length distributions.Cosine similarity can regard such messages as similar when they share limited distributional structure.
- Sub-clustering: Grouping messages only by word count is too primitive, so word-length values are used to form sub-clusters while other indices remain necessary in some cases.Messages printed in predefined styles usually have the same number of words, but that criterion alone does not separate all templates.
3 LenMa: Length Matters Clustering
LenMa represents messages by word-length vectors and clusters online by combining cosine similarity with positional word matching. It updates cluster vectors incrementally and creates a new cluster when no existing cluster exceeds the threshold.
- Length-based representation: LenMa uses each message’s word lengths as similarity parameters.Messages are converted into vectors whose elements are the lengths of successive words.
- Similarity calculation: Messages with similar word-length vectors are treated as candidates for the same cluster.The method uses vector similarity to identify messages that may belong together.
- Similarity calculation: Cosine similarity compares the word-length vector of an incoming message with an existing cluster.The cluster and message vectors contain the lengths of corresponding words.
- Online clustering procedure: Cluster word-length and template vectors are updated whenever a new message is integrated.Length values are retained when positions agree and replaced when the incoming message differs; variable words are represented by wildcards.
- Online clustering procedure: LenMa processes each incoming message by creating its vectors, comparing only clusters with the same number of words, and applying threshold Tc.If no cluster exceeds Tc, the message starts a new cluster; otherwise, the most similar cluster is updated.
- Online clustering procedure: The method adds positional similarity Sp because cosine similarity alone cannot always identify the proper cluster.Sp counts shared words occurring at the same positions, and insufficient positional similarity rejects the candidate.
4 Implementation
The authors implemented LenMa in Python and evaluated it on three syslog datasets. Processing time increased with the number of inferred templates, while template growth and processing time stabilized after most templates were found; kernel boot messages could still create unwanted templates.
- Implementation and datasets: LenMa was implemented in Python and applied to three syslog datasets from public, hypervisor, and laboratory server sources.The laboratory dataset includes hypervisors and service hosts such as web servers.
- Implementation and datasets: The method assumes the first three message components are a date, host name, and process name, although syslog messages do not always follow that format.This is introduced as a syslog-specific heuristic based on the authors’ experience.
- Processing behavior: Processing cost depends linearly on the number of inferred templates, and the threshold affects the final template count.A smaller threshold produces fewer templates containing more wildcard marks.
- Processing behavior: Processing time increased with template count but stabilized once the number of templates became stable.Figure 7 measures the time required to process 10000 messages and plots it against the number of inferred templates.
- Observed templates: Kernel boot messages generated many one-time patterns that increased the total number of inferred templates in datasets #2 and #3.The authors report that cleansing raw messages may be needed to avoid unwanted template generation.
5 Using Clustered Syslog Messages for Analysis
LenMa clusters syslog messages into minute-level groups, enabling recurring and unique message-pattern analysis. Most one-minute groups matched two frequently observed patterns, while some unique patterns indicated unusual SSH activity or node rebooting.
- 132480 one-minute groups were reduced to 25 message-group clusters using template-appearance counts and a χ2 test.The groups covered 60 minutes × 24 hours × 92 days of dataset #3.
- The two frequent patterns appeared 16235 and 115299 times, together matching almost 91 of 92 days.
- Unique patterns included uncommon SSH incoming activity and a pattern observed when a target-group node rebooted.
- The clustered templates can be used with existing anomaly-detection or message-clustering methods.
6 Remaining Issues
The method has limitations shared with online template mining and specific to its design. Its results depend on threshold selection and may merge semantically distinct messages or require dataset-specific tuning.
- The proposed algorithm ignores word-frequency information, so distinct stable keywords can be merged into one template.For example, “interface eth0 up” and “interface eth1 down” may become “interface * *”.
- Threshold selection is important: loose values can separate messages with the same meaning into different groups.The paper used Tc = 0.9 and Tp = 3 for three message sources.
- The method achieved a similar number of templates to SHISO on standard Linux server syslog messages, but suitable values may differ for other datasets.
7 Conclusion
The paper proposes clustering syslog messages by word-length sequences for online, one-pass template mining. It reports comparable template counts to prior work while avoiding the complexity and delay of two-pass frequency analysis.
- The method clusters messages using each message’s unique sequence of word lengths.
- The proposed method is designed for online, one-pass template mining rather than two-pass frequency-based template generation.
- It could produce a similar number of templates as past works with less mining-processing complexity.