Source-linked AI summary
HSMLog: Small Language Model-Assisted Hardware Security Module Log Anomaly Detection with Behavioral Analysis
Chia-Hsuan Wu, Dar-Hsin Dustin Wu, Rui Fang, Yi-Ting Lee, Chia-Chih Lin, Ming-Syan Chen
TL;DR
HSM anomalies depend on relationships across stateful events, keys, objects, sessions, and temporal context, while existing approaches may separate detection from HSM-specific evidence review. HSMLog combines SLM-based contextual and policy-guided detection with retrieval-grounded candidate analysis, achieving 98.97% precision, 96.00% recall, 98.66% anomalous-event coverage, and a 97.46% F1 score in the studied setting.
Problem
HSM anomaly detection must interpret relational, state-dependent behavior and organization-specific policies rather than isolated log events.
Method
HSMLog uses structured HSM event windows for SLM contextual and policy-guided detection, followed by retrieval-grounded review using policies, prior suspicious-key records, and related log context.
Results
98.97% precision, 96.00% recall, 98.66% anomalous-event coverage, and a 97.46% F1 score were achieved on industrial HSM background logs augmented with practitioner-informed anomaly scenarios.
Takeaways & Limitations
HSMLog demonstrates effective anomaly alerting and incident triage in the studied industrial HSM setting.
Takeaways & Limitations
Evaluation is limited to one HSM environment and structured log schema, and Stage 2’s usefulness to HSM engineers was not directly evaluated.
Abstract
from arXiv · showhide
Hardware Security Module (HSM) logs capture security-critical behavior, but anomalies emerge from relationships across event sequences, keys, object states, sessions, and temporal patterns rather than isolated events. Existing methods separate detection from HSM-specific evidence validation and reporting. In this paper, we present HSMLog, a two-stage framework for HSM log anomaly detection with retrieval-grounded behavioral analysis. In Stage 1, a small language model (SLM) identifies candidate alerts from sliding windows of structured HSM events and performs policy-guided assessment using HSM-specific operational rules. In Stage 2, retrieved policies and historical suspicious-key records strictly predating the alert window, together with candidate-related log context, support conservative candidate review and incident analysis. Evaluated on real industrial HSM background logs augmented with anomaly scenarios co-defined with industrial partners, HSMLog achieves 98.97% precision, 96.00% recall, 98.66% anomalous-event coverage, and a 97.46% F1 score, demonstrating effective anomaly alerting and incident triage in the studied setting.
I. INTRODUCTION
HSM logs encode stateful, security-critical operations whose meaning depends on relationships among keys, objects, sessions, policies, and outcomes. HSMLog addresses these challenges with a two-stage framework combining contextual and policy-guided detection with retrieval-grounded candidate review.
- Research gap: Existing approaches may miss organization-specific policies, contextual workflow deviations, or repeated operations involving the same protected asset, requester, or invalid state transition.Operational HSM data also commonly lacks exhaustive event-level labels.
- Approach: HSMLog combines SLM-based contextual detection and policy-guided assessment with retrieval-grounded candidate review and behavioral analysis.The framework uses structured HSM events and merges positive windows into candidate incidents before evidence retrieval.
- Evaluation: The study evaluates HSMLog on real industrial HSM logs using five practitioner-informed anomaly categories and comparisons with representative baselines.The analysis covers case detection, anomalous-event coverage, component contributions, retrieval-supported review, and observation-window sensitivity.
- Motivation: HSM audit records require joint interpretation of targets, sessions, roles, lifecycle states, mechanisms, returned handles, and result codes.An identical result may be benign or suspicious depending on surrounding operations and object relationships.
B. Operational HSM Monitoring Challenges
HSM anomalies are sparse, relational, and state-dependent, so HSMLog detects candidates from structured event windows and then reviews them against temporally prior evidence and related context.
- Monitoring challenge: HSM operations can be expected in one workflow but suspicious when their target, state, authority, or outcome conflicts with surrounding activity or policy.This makes anomaly evidence relational and state-dependent rather than fully specifiable in advance.
- Stage 1: HSMLog represents the audit stream as canonicalized events and analyzes local windows using contextual and organization-specific policy decisions.The window contains consecutive canonicalized events, while the two decisions capture contextual anomaly detection and policy-guided assessment.
- Candidate construction: Related positive windows are merged into candidate incidents to avoid duplicated alerts.This produces candidate units for the second-stage review process.
- Stage 2: For each candidate, Stage 2 retrieves applicable policies, suspicious-key records strictly predating the incident interval, and candidate-related log context.The retrieved evidence supports conservative review and behavioral analysis for retained candidates.
- Review outcome: Retained candidates receive retrieval-grounded behavioral analyses intended to identify suspicious local behavior and provide operational support for engineer investigation.Candidates are retained only after evidence review.
III. HSMLOG APPROACH
HSMLog combines structured, behavior-preserving event representation with a two-stage workflow that detects contextual or policy-level anomalies and then performs temporally constrained evidence review.
- Stage 1: Stage 1 classifies overlapping structured event windows as Anomaly or Normal and assesses them against applicable organization-specific policies.Positive windows from either contextual detection or policy-guided assessment are merged into candidate incidents.
- Stage 2: Stage 2 retrieves HSM policies, strictly prior suspicious-key records, and candidate-related events for conservative review and behavioral analysis.It introduces no new detections, and the temporal restriction prevents an incident from supporting itself.
- Representation: HSMLog converts interdependent raw audit records into compact representations that preserve relationships among operations, keys, objects, sessions, and results.The representation is designed for SLM inference rather than treating fields independently.
- Representation: The representation retains timestamps, sessions, roles, operations, key identifiers, mechanisms, attributes, search templates, returned handles, and result codes.List and dictionary attributes are normalized to text for processing.
- Behavioral context: Linked handles and labels, requester context, and operation metadata enable reasoning about object-state progression, cross-object access, repeated failures, and session-dependent usage.These fields connect operation intent and outcome within the event stream.
C. Sliding Windows and Case-Aware Training Protocol
HSMLog preserves local behavioral context through overlapping windows and uses conservative case-aware labeling for SLM classification. Its preprocessing canonicalizes raw records into compact events before windowed analysis.
- Sliding windows: Overlapping local event windows keep related behavior visible when benign activity is interleaved and avoid assuming a single application session.This supports contextual analysis across the canonical event stream.
- Case-aware training: A window is labeled Anomaly only when it fully contains an annotated case, while partial overlaps remain Normal.This requires complete behavioral evidence but can place anomalous events near case boundaries in Normal-labeled windows.
- Preprocessing: Raw records are canonicalized into compact events and organized into overlapping windows for SLM classification.The preprocessing is behavior-preserving and displays only selected fields for clarity.
- SLM classification: The SLM classifies each canonicalized window as Anomaly or Normal while assessing sequence coherence across events, sessions, keys, objects, and outcomes.The instruction also covers object-state progression, access relationships, mechanism usage, and result-code patterns.
- Behavioral detection: Contextual analysis can identify behavior that appears benign in isolation but becomes suspicious through repeated failures, cross-object access, or lifecycle inconsistencies.The detection therefore relies on HSM-specific context rather than operation names alone.
E. Stage 1: Policy-Guided SLM Assessment
HSMLog combines policy-guided assessment with contextual evidence and retrieval-grounded review to form conservative candidate incidents and support behavioral analysis.
- HSMLog supplies applicable protected rules and log context for policy-guided assessment, then combines the decision with contextual evidence.
- Overlapping positive windows are merged into a single candidate incident to avoid duplicate alerts.
- Stage 2 retrieves relevant policies, strictly earlier suspicious-key records, and candidate-related events for each candidate incident.
- Stage 2 filters only weak, unsupported candidates, introduces no new detections, and uses no groundtruth labels.
- Retained incidents are organized into analyses covering affected keys or objects, suspicious progression, supporting evidence, and investigation-relevant context.
A. Dataset Construction
The study combines real operational HSM backgrounds with independently instantiated practitioner-informed anomaly scenarios and trains the contextual detector using fixed observation windows and weighted classification loss.
- Training contains 27,319 events from 352 normal-background sessions and 180 labeled anomalous cases across four contextual-detector categories.Policy Violation is assessed only through policy-guided assessment at inference.
- The final test stream contains 16,573 events across 100 near-balanced cases in five categories, with session-, key-, handle-, and window-disjoint anomaly instances.
- Stage 1 uses a predefined observation window of w = 20 to preserve context for operations separated or interleaved by routine activity.
- Fine-tuning uses 54,600 window instances and response-only supervision, with anomaly-response tokens receiving weight 50 versus 1 otherwise.
- The five practitioner-informed categories cover cryptographic failure probing, object handle probing, lifecycle violation, policy violation, and low-and-slow metadata probing.Together, the scenarios test relationships among keys or objects, sessions, states, and results.
D. Baselines
HSMLog is compared with seven generic log-anomaly baselines under common session-disjoint splits and case-aware incident evaluation, including an unseen policy-driven category.
- The comparison includes DeepLog, LogRobust, PLELog, LogBERT, NeuralLog, LogLLM, and MIDLog.
- These baselines span sequential prediction, robust representation, parsing-free semantic encoding, weak supervision, self-supervision, and language-model-based analysis.
- Policy Violation is an unseen policy-driven test category, and baselines assess it without organization-specific policies.
- All methods use the same session-disjoint data splits, case-aware incident protocol, fixed configurations, and prespecified decision thresholds.
- Case detection rate measures matched anomaly cases, while event coverage measures anomalous events covered by matched incidents; precision, recall, and F1 operate over alert incidents.
V. RESULTS AND DISCUSSION
HSMLog achieves strong overall alerting results, while ablations show that contextual detection and policy-guided assessment complement each other and retrieval review mainly improves precision.
- 98.97% precision, 96.00% recall, 98.66% EC, and 97.46% F1 are reported for HSMLog.HSMLog improves F1 by 4.75 points and recall by 7 points over PLELog, the strongest baseline by F1 at 92.71%.
- 98.85% EC from NeuralLog exceeds HSMLog by 0.19 points, but NeuralLog has 89.00% recall and 91.75% F1.HSMLog detects seven more cases while improving precision from 94.68% to 98.97%.
- Contextual SLM alone achieves 78.00% recall, 68.70% EC, and 81.25% F1, while policy-guided assessment alone achieves 100.00% precision and 21.00% recall.
- Combining contextual detection and policy-guided assessment raises recall to 96.00%, EC to 98.66%, and F1 to 96.97%.This improves recall by 18 points over contextual SLM alone.
- Retrieval review raises precision from 97.96% to 98.97% and F1 from 96.97% to 97.46%, without changing recall or EC.The reported gains reflect Stage 2’s bounded triage role rather than a second detector.
C. Observation-Window Trade-off
HSMLog’s observation-window analysis exposes a trade-off between incident detection performance and anomalous-event coverage, while Stage 2 remains a bounded, evidence-grounded review mechanism rather than an additional detector.
- Observation-Window Trade-off: w = 10 yields higher recall and F1, but w = 20 provides higher anomalous-event coverage.The paper fixes w = 20 to preserve broader candidate-related evidence for retrieval-grounded review and behavioral analysis.
- Observation-Window Trade-off: Beyond w = 20, recall drops as unrelated background dilutes localized evidence.
- Retrieval-Grounded Behavioral Analysis: Stage 2 organizes policies, pre-incident suspicious-key context, and candidate event sequences into behavioral analyses of affected assets and suspicious progression.
- Retrieval-Grounded Behavioral Analysis: The behavioral analysis supports post-detection explanation and triage rather than functioning as a free-form additional detector.
- Relation to Existing Methods: HSMLog connects SLM-based structured-window detection and policy-guided assessment with retrieval-grounded review and operational investigation.
VII. LESSONS LEARNED AND LIMITATIONS
The paper emphasizes preserving stateful HSM context, incorporating deployment-specific policy knowledge, and keeping review conservative and evidence-grounded. Its evaluation uses industrial background logs with practitioner-informed scenarios in one HSM environment, with Stage 2 usefulness to engineers left for future study.
- Lessons Learned: HSM monitoring should preserve key/object, session, state, and timing context.
- Lessons Learned: Learned contextual detection benefits from deployment-specific policy knowledge, while post-detection review should remain conservative and evidence-grounded.
- Limitations: Anomaly cases were practitioner-informed scenarios co-defined with industrial partners, not confirmed production incidents.
- Limitations: Evaluation is limited to one HSM environment and structured log schema, and Stage 2’s usefulness to HSM engineers was not directly evaluated.
- Evaluation Context: 98.97% precision, 96.00% recall, 98.66% anomalous-event coverage, and 97.46% F1 were achieved in the studied industrial HSM setting.