PaperlayerView exact revision on arXiv

Source-linked AI summary

From Alert Floods to Precedence Forests: Zero-Prior-Knowledge Incident Triage with LOGOS

Radhika Niranjan Mysore

arXiv:2610.02297v1cs.DCcs.NIcs.SE

TL;DR

Enterprise operators often lack the traces, topology maps, baselines, or incident histories used by observability and AIOps systems, leaving raw logs as a common diagnostic artifact. LOGOS links co-occurring entities and events, then orders faults by temporal precedence to form compact failure forests. Across enterprise and open-source incidents, it sharply reduced log noise and supported zero-shot LLM root-cause identification, while remaining short of causal diagnosis.

  • Problem

    Enterprise troubleshooting often lacks rich domain artifacts and healthy baselines, leaving open how far raw text logs alone can bound the diagnostic search space.

    Source: §1 · p. 1

  • Method

    LOGOS links entities and events through co-occurrence, tracks temporal onset across subsystem faults, and prunes ubiquitous entities to build precedence forests.

    Source: §1 · pp. 1–2

  • Results

    99.8% median background-noise reduction in 4.5 minutes with 0.76 mean recall; zero-shot LLM root-cause accuracy was 80% enterprise and 100% open-source.

    Source: §1 · p. 2, §9 · p. 17

  • Takeaways & Limitations

    The compressed failure graphs support direct operator triage and zero-shot LLM root-cause identification without pre-built domain artifacts or historical baselines.

    Source: §9 · p. 17

    LOGOS captures structural and temporal correlation rather than causality, so automated remediation requires domain reasoning; silent failures or disconnected identifier schemes can fracture the forest.

    Source: §8 · p. 17

Abstract

from arXiv · show

Commercial observability platforms rely on domain artifacts like distributed traces, topology maps, and baseline metrics. However, when troubleshooting proprietary software, enterprise operators are left with only raw, unannotated text logs. We explore the extreme boundary of log-only diagnosis: To what extent can we isolate failure propagation using strictly raw text logs? We present LOGOS, an unsupervised system that exploits entity-event co-occurrence and temporal precedence to collapse millions of raw log lines into a compact precedence forest. Evaluated across 25 production enterprise outages and 12 open-source issues, LOGOS operates with zero prior knowledge---requiring no seed queries, observed symptoms, or pre-defined incident boundaries. In a median wall-time of 4.5 minutes, LOGOS eliminates a median 99.8% of background noise, achieves 0.76 mean recall, and detects failure cascades with a 16-hour median diagnosis-verified lead time---consolidating alert floods 124x to enable 80% enterprise (100% open-source) zero-shot LLM root-cause accuracy.

1 Introduction

LOGOS addresses log-only diagnosis where enterprise operators lack domain artifacts, incident histories, and reliable baselines. It links log events and entities, traces temporal precedence, and compresses alerts into a precedence forest.

1 Introduction

  • Without healthy baselines, traces, or topology maps, proprietary cross-vendor failures leave enterprise operators with raw logs and few reliable diagnostic inputs.Existing log-only studies rely on clean, stationary baselines, while many AIOps pipelines assume curated playbooks or historical incident archives.Source: §1 · p. 1
  • LOGOS links events sharing identifiers and entities co-occurring across events in a bipartite graph, then uses transitive closure to isolate related events from background noise.Higher-severity events are treated as faults, while participating identifiers represent originating faults or downstream victims.Source: §1 · p. 1
  • LOGOS elevates subsystem logs that wrap third-party events and extracts embedded severity keywords because vendor logs may omit severity or mislabel faults as INFO.Exact event severity cannot be determined without domain knowledge, so LOGOS identifies fault candidates heuristically.Source: §1 · p. 2
  • Progressive onset tracking starts at the first window where correlated faults span at least two subsystems, then expands the cascade until a window adds no new faults.The resulting component is separated into independent subgraphs; ubiquitous entities are pruned before parent-child precedence trees are built.Source: §1 · p. 2
  • 0.76 mean recall, median 99.8% background-noise reduction, and 80% enterprise (100% open-source) zero-shot LLM root-cause accuracy were achieved across 25 enterprise outages and 12 open-source incidents.LOGOS ran in a median wall-time of 4.5 minutes, consolidated alerts 124×, and detected cascades with a 16-hour median diagnosis-verified lead time.Source: §1 · p. 2

2 Background and Motivation

The background motivates LOGOS with the manual cost of enterprise log diagnosis, the limits of flat entity rankings, and the structure of raw log telemetry. Preserving the entity-event graph avoids the memory costs of projecting it into a dense entity graph.

2 Background and Motivation

  • Across 25 selected log-diagnosable outages, third-party vendor issues accounted for 16 of 25 incidents, and root causes rarely occupied the symptom’s subsystem.The 25 incidents were selected from 56 log-centric issues based on complete log bundles and verified ground truth; diagnosing those 56 averaged about 33 calendar days.Source: §2 · p. 2
  • 12 of 25 incidents (48.0%) had a root-cause entity surfaced by at least one Top-10 frequency or severity heuristic, but flat lists did not reconstruct propagation across subsystems.Operators therefore still had to investigate entities as isolated starting points.Source: §2 · pp. 2–3, Table 2 · p. 3
  • Enterprise bundles span days to weeks and contain heterogeneous telemetry, but LOGOS operates strictly on raw text logs.Log lines include timestamps, emitting subsystems, native severity, and free-form text; native severity levels can be unreliable.Source: §2 · p. 3
  • Individual log lines contain multiple co-occurring identifiers, averaging 2.57 to 4.37 IDs per line across enterprise, OpenStack, and TrainTicket data.LOGOS abstracts text into parameterized templates, retaining dynamic identifiers as links across events and subsystems.Source: §2 · p. 3
  • O(M) bipartite processing succeeded on all 25 enterprise bundles, while O(M2) graph-centrality baselines ran out of memory on 9 of 25 bundles (36%).Unipartite projection created cliques with up to 68.21% overall density.Source: §2 · p. 3

3 Upgrade Incident Case Study

The upgrade case shows LOGOS reconstructing a cross-entity failure sequence from raw logs despite the product defect itself being unrecorded. Its forest narrowed a multi-million-line bundle to a compact diagnostic structure, while topology-only LLM reasoning produced a plausible but speculative diagnosis.

3 Upgrade Incident Case Study

  • A latent defect left configuration updates unpersisted for two years, so the post-upgrade reboot restored stale disk state and triggered network errors and VM migration failures.The log bundle did not explicitly record the defect itself, but did record its downstream infrastructure impact.Source: §3 · p. 4
  • LOGOS elevated benign-tagged [INFO] events to faults using embedded severity keywords or central-wrapper logs, then used entity co-occurrence to isolate related events from background noise.For example, the management_node line was elevated because it contained “alert,” while boot_init lines were elevated as central-subsystem logs interfacing with unstandardized third-party components.Source: §3 · p. 4
  • Under 48 minutes, LOGOS processed the bundle from ingestion through precedence-forest construction, versus over a month of out-of-band engineering debugging.The incident involved 20.5 million raw log lines, and the forest reduced 7.9 million lines from the evaluated seven-day period to 34,927 nodes, a ∼227× reduction.Source: §3 · pp. 4–5
  • The precedence forest divided the incident excerpt into three trees: a connected restore branch, an isolated retry root, and a separate parsing root.Tree A links the database failure and backup-restore sequence to virtsw_local_binding; Tree B is add_virtual_switch_to_host, and Tree C is config_parse_exception.Source: §3 · pp. 4–5
  • Within the evaluated seven-day window, config_parse_exception strictly preceded the other faults, while add_virtual_switch_to_host retries coincided with boot_init faults.The underlying incident root cause lay outside that window.Source: §3 · p. 5
  • Given only forest topology and entity names, Gemini proposed a configuration issue near the actual cause, but inferred cross-tree causal links and possible unseen hardware failure.The product defect was absent from the log bundle, so the forest captured subsequent propagation rather than the latent root cause; the authors identify explicit tree-boundary awareness and event text as needed augmentations.Source: §3 · p. 6

4 Design

LOGOS uses a two-stage pipeline to transform raw logs into compact precedence forests. It elevates fault candidates, detects and expands correlated cross-subsystem components over time, then prunes hubs and assigns temporal precedence edges.

4 Design

  • The two-stage design parses raw lines into entity identifiers and events, elevates fault candidates, then constructs compact precedence forests from those candidates.Severity elevation promotes events with embedded severity keywords or originating from central wrapper logs; events below [WARN] are excluded from the elevated stream.Source: §4 · p. 6

4.1 Graph Construction

  • A cascade onset is flagged when the maximal connected component spans at least two distinct subsystems; LOGOS then expands across up to 15 consecutive windows.The timeline is discretized into uniform two-hour windows, and expansion stops when the component no longer grows or the walk cap is reached; forests are emitted at onset and convergence or termination.Source: §4.1 · pp. 6–7
  • Within the temporal correlation bound Tc = 2h, faults sharing entities are connected, and the component spanning the most impacted entities becomes the maximal failure subgraph.LOGOS uses Union-Find to merge adjacent faults only when their temporal separation is within Tc.Source: §4.1 · pp. 6–7
  • After filtering ubiquitous hub entities, LOGOS records each entity’s first and last appearances, orders entities by earliest appearance, and links each to its nearest earlier co-occurring entity.The resulting directed acyclic forest encodes chronological fault propagation across impacted entities.Source: §4.1 · pp. 7–8

5 Implementation

LOGOS is implemented as a CPU-only Python pipeline with source-specific parsing integrations and SQLite-backed intermediate storage. Its shard and batch design bounds ingestion memory independently of raw log scale.

5 Implementation

  • LOGOS is a CPU-only pipeline of approximately 3,150 lines of Python 3.10+, using thread pools and pre-compiled regular expressions for parallel ingestion.Parsed event templates and extracted entities are persisted in SQLite for progressive onset tracking.Source: §5 · p. 8
  • Each log source uses a one-time integration module with regex parsing, severity-elevation rules, entity extraction patterns, and a hub-node blocklist.The parsing tuple includes timestamp, subsystem, severity, and text; examples of extracted identifiers include config_store UUIDs and TrainTicket TraceIDs/SpanIDs.Source: §5 · pp. 8–9
  • Per-subsystem SQLite shards receive parsed events in 5,000-item write batches, bounding ingestion memory to a flat RSS footprint regardless of raw log scale.Severity and timestamp indexes support fault-window queries, while raw logs remain on disk for lazy parameter resolution.Source: §5 · p. 9

6 Evaluation

The evaluation examines LOGOS on enterprise and open-source incidents, assessing root-cause diagnosis, early failure detection, alert and search-space reduction, and baseline performance through case studies and parameter sensitivity. It also evaluates ablation effects and the scaling of ingestion and graph construction across 37 benchmark datasets.

6.1 Forest Recall and Accuracy

  • Across 25 enterprise incidents, LOGOS reached 60.0% Top-10 reachability and 0.76 recall; on open-source incidents, it achieved 83.3% Rank-1 and 100.0% Top-10 reachability.Graph baselines completed only 16 enterprise bundles, achieving 0.60 raw recall and at most 31.2% Top-10 reachability; LOGOS reached 0.72 recall on those same 16 bundles. On open-source incidents, centrality baselines had 33.3% Top-10 reachability, while earliest-temporal ordering reached 91.7%.Source: Table 5 · p. 9, Table 6 · p. 9, §6.1 · p. 10

6.2 Interpretability of Precedence Forests

  • Precedence forests yielded 10/25 enterprise Full Diagnoses and 20/25 (80%) overall accuracy, plus 12/12 (100%) open-source accuracy.The 1,600-node prompt retained 0.46 enterprise recall and 1.00 open-source recall; uncapped database annotations could credit Partial Diagnoses when the prompt omitted ground-truth entities. Across 60 enterprise trials, forests achieved at least Partial Diagnosis in 95.0% of runs versus 35.0% for raw-log prompting.Source: Table 7 · p. 11, §6.2 · p. 11

6.3 Reduced and Early Alerting

  • LOGOS achieved a median diagnosis-verified lead time of 16 hours, consolidated a median 1,658 raw alerts into 12 forests (124× reduction), and reduced the search space by 99.8%.Lead time was assessed across the 20 incidents with a Full or Partial Diagnosis; alert counts cover 22 incidents with telemetry, while LOGOS metrics span all 25 enterprise incidents.Source: Table 8 · p. 12, §6.3 · p. 12

6.4 Case Studies: Early Detection

  • At their earliest maximum-diagnosis checkpoints, forests produced Full Diagnoses in two of five case studies and Partial Diagnoses in the other three; raw alerts achieved one Partial Diagnosis and four misses.Raw-alert prompting also constructed confident false narratives in Cases 1 and 5, whereas LOGOS reached a Full Diagnosis 18 hours early in Case 1 and 26 hours early in Case 2.Source: §6.4 · pp. 12–13, Table 9 · p. 13

6.5 Sensitivity and Ablations

  • Widening Tc raised Recall@1600 from 0.61 to 0.66 while overall recall remained 0.87; expanding Tw lowered Top-10 reachability from 80.0% to 60.0%.At Tw=2.0h, alert volume was 26.2 alerts/incident versus 48.4 at Tw=1.0h. Raising Kwalk to 25 reduced alerts from 31.6 to 26.0 alerts/incident and increased overall recall from 0.87 to 0.90; the Tc=2 h bound filtered a median 85.7% of background events while preserving 0.76 mean recall.Source: §6.5 · p. 14
  • Forest structural tagging raises Full Diagnoses from 5/25 to 10/25 and Full + Partial from 15/25 to 20/25, with recall unchanged at 0.76 / 0.46.Tags [ROOT_CANDIDATE] and [DEAD_END] guide LLM reasoning; Partial Diagnoses remain flat at 10/25.Source: §6.5 · p. 15
  • 0.96 to 0.49 recall follows disabling subsystem-level severity promotion; disabling the log-text keyword override alone drops recall to 0.65.Both measurements use the raw 168h candidate pool across all N = 25 incidents, counting the incident whose ground truth is not isolated as zero recall; true faults can otherwise be dropped before graph construction.Source: §6.5 · p. 15
  • 0.59 to 0.76 overall recall follows progressive onset tracking (Tw = 2h), while capped recall rises from 0.31 to 0.46 and Top-10 reachability from 52.0% to 60.0%.Tracking isolates the true onset window instead of using a flat 7-day log sweep.Source: §6.5 · p. 15
  • Disabling hub-node pruning raises raw graph recall from 0.76 to 0.77 but lowers Full Diagnoses from 10/25 to 8/25; earliest-preceding parent selection raises leaf nodes from 24.2% to 97.5%.Pruning removes a median 2.8% of entities; unpruned hubs become spurious root candidates, while nearest-preceding parent selection retains intermediate propagation structure. Parent assignment is recall-invariant at 0.76 uncapped / 0.46 capped.Source: §6.5 · p. 15

6.6 Compute and Memory Overheads

  • Ingestion wall-time scales linearly with N (r = 0.99), with median times of 2.9 minutes for enterprise datasets and 0.4–5.4 seconds for open-source datasets.The largest dataset (156.4M lines) takes 6.4 hours; peak memory remains between 18.7 MB and 479.9 MB through fixed-size batch processing.Source: §6.6 · p. 15
  • Graph-construction wall-time and peak memory correlate with input scale N (r = 0.92–0.93); median enterprise wall-time is 1.6 minutes and median peak memory is 4.17 GB.Graph construction takes 0.96 seconds for open-source datasets, peaks at 28.9 minutes on the largest corpus, and reaches 48.05 GB in the single worst case, within host memory capacity.Source: §6.6 · p. 15

7 Related Work

LOGOS targets diagnosis where domain artifacts and healthy baselines are absent, deriving failure structure from noisy logs alone. It differs from systems that depend on topology, schemas, or historical incident resources.

Domain-Augmented Methods

  • LOGOS derives structural dependencies post-hoc from noisy logs rather than relying on topology maps, metrics, or active probing.This places it at the log-only extreme of diagnostic methods.Source: §7 · pp. 15–16
  • Unlike SkyNet, which requires known topology and hierarchy maps to group alerts, LOGOS correlates events independently of system topology.Both address alert reduction using passive monitoring telemetry.Source: §7 · p. 16

Generative AI and Agentic AIOps

  • LOGOS complements AIOps frameworks by providing compact precedence forests when playbooks do not exist and raw logs exceed LLM context windows.Unlike frameworks requiring playbooks or historical ticket corpora, it operates without historical domain knowledge.Source: §7 · p. 16

Statistical & Performance Inference Methods

  • The paper does not benchmark directly against agentic LLM-RCA systems because their private incident corpora and implementations are unavailable for faithful reproduction.Synthesizing the required resources would test a better-instrumented regime than LOGOS targets.Source: §7 · p. 16
  • LOGOS assigns precedence from single instances of entity co-occurrence and uses dynamic identifiers, rather than aggregating repeated transactions or masking identifiers as wildcards.It can stitch fragmented, unmodeled local identifiers transitively without trace headers or pre-authored schemas.Source: §7 · pp. 16–17

8 Limitations and Open Problems

LOGOS is a single-pass triage aid with limits in parsing, cascade selection, subsystem coverage, and causal interpretation. Its forest can also fracture when logging or identifier links are missing.

Maximal Cascade Selection

  • Selecting only the maximal connected component yields zero recall in 3 of 25 incidents with roots in secondary subgraphs; emitting all subgraphs recovers them with 2.2× net alert reduction.The reported 0.76 mean recall also reflects partial recall within incidents.Source: §8 · p. 17

Third-Party Subsystem Boundary

  • Accuracy is 100% (9/9) on native infrastructure faults but 69% (11/16) on third-party components, which account for all 5 diagnostic misses.Custom vendor parsing and targeted severity elevation are identified as engineering levers.Source: §8 · p. 17

Log Parsing

  • Vendor-specific parsing adds a one-time engineering cost, and an uncaptured severity keyword in 1 of 25 incidents limited detectable ground truth to 24/25 and capped mean enterprise recall at 0.96.Incomplete parameter masking can fragment templates, although shared entities can overcome that fragmentation through transitive closure.Source: §8 · p. 17

Interactivity

  • LOGOS runs as a single batch pass without iterative refinement or operator-guided narrowing, providing initial bounded context rather than replacing interactive triage.Source: §8 · p. 17

Ground-Truth and Scoring Boundaries

  • Strict entity-identifier matching can score a correct root-failure mechanism as only a Partial Diagnosis when specific entity tokens are absent.Ground-truth entities and subsystems were reconstructed from vendor incident reports and escalation logs.Source: §8 · p. 17

The Gap to Self-Healing

  • LOGOS captures structural and temporal correlation, not causality; automated remediation therefore requires domain reasoning before corrective action.Source: §8 · p. 17

Telemetry Gaps

  • Without text logs or shared identifier links, transitive closure cannot cross the gap; silent failures require topology maps or metric anomalies to bridge it.Source: §8 · p. 17

9 Conclusion

LOGOS shows that raw logs alone can substantially reduce enterprise incident search space while retaining recall. Its compact failure graphs also support zero-shot LLM diagnosis and operator triage.

9 Conclusion

  • LOGOS eliminates a median 99.8% of background noise in a median wall-time of 4.5 minutes while retaining 0.76 mean recall.The conclusion describes this as achievable aggressive search-space reduction on raw text logs alone.Source: §9 · p. 17
  • Operating on 2-hour log windows, LOGOS detects cascades with a 16-hour median diagnosis-verified lead time and consolidates alerts 124×.The compressed graphs support zero-shot LLM root-cause accuracy of 80% for enterprise and 100% for open-source cases.Source: §9 · p. 17
  • LOGOS is being integrated into an enterprise cloud provider’s AIOps pipeline to automate incident triage and accelerate ticket resolution.The conclusion presents direct operator triage as another use for the compressed failure graphs.Source: §9 · p. 17
Loading 2610.02297v1…