PaperlayerView exact revision on arXiv

Source-linked AI summary

Causal Inference-Based Root Cause Analysis for Online Service Systems with Intervention Recognition

Mingjie Li, Zeyan Li, Kanglin Yin, Xiaohui Nie, Wenchi Zhang, Kaixin Sui, Dan Pei

arXiv:2206.05871v1cs.SEcs.AI

TL;DR

Root cause analysis in online service systems requires identifying a small set of indicators from extensive monitoring data, but causal-inference-based solutions are limited. The paper formulates RCA as intervention recognition and proposes CIRCA, which combines causal-graph construction, regression-based testing, and descendant adjustment. CIRCA is theoretically reliable in simulation and improves real-world top-1 recommendation recall by 25% over the best baseline.

  • Problem

    Root cause analysis in online service systems must identify underlying faults from extensive monitoring data, while causal-inference-based RCA has been little studied.

  • Method

    CIRCA formulates RCA as intervention recognition and uses an architecture-informed causal graph, regression-based hypothesis testing, and descendant adjustment.

  • Results

    25% higher top-1 recommendation recall than the best baseline is reported on a real-world dataset, with simulation supporting theoretical reliability.

  • Takeaways & Limitations

    The intervention-recognition formulation provides a causal-inference-based direction for RCA, while CIRCA shows practical value over baseline methods.

  • Takeaways & Limitations

    CIRCA relies on assumptions that real applications may violate, and descendant adjustment requires verification on more real-world datasets.

Abstract

from arXiv · show

Fault diagnosis is critical in many domains, as faults may lead to safety threats or economic losses. In the field of online service systems, operators rely on enormous monitoring data to detect and mitigate failures. Quickly recognizing a small set of root cause indicators for the underlying fault can save much time for failure mitigation. In this paper, we formulate the root cause analysis problem as a new causal inference task named intervention recognition. We proposed a novel unsupervised causal inference-based method named Causal Inference-based Root Cause Analysis (CIRCA). The core idea is a sufficient condition for a monitoring variable to be a root cause indicator, i.e., the change of probability distribution conditioned on the parents in the Causal Bayesian Network (CBN). Towards the application in online service systems, CIRCA constructs a graph among monitoring metrics based on the knowledge of system architecture and a set of causal assumptions. The simulation study illustrates the theoretical reliability of CIRCA. The performance on a real-world dataset further shows that CIRCA can improve the recall of the top-1 recommendation by 25% over the best baseline method.

1 INTRODUCTION

The paper frames root cause analysis in online service systems as intervention recognition, addressing causal-model and incomplete-distribution challenges with CIRCA. CIRCA combines architecture-informed graph construction with regression-based testing and descendant adjustment, outperforming baselines in simulation and real-world evaluation.

  • Motivation and problem: Root cause analysis in online service systems is formulated as intervention recognition: finding the underlying intervention from observational and faulty-system behavior.The paper maps faults to interventions and identifies root cause indicators through causal inference.
  • Challenges: Causal inference-based RCA is challenging because existing approaches do not solve intervention recognition and observational knowledge may be incomplete.Sage’s design assumes no intervention, while distributional non-overlap can obscure normal behavior around a failure.
  • Approach: CIRCA constructs a metric-level causal Bayesian network from system architecture and causal assumptions, extending coverage beyond a few service-latency metrics.The method uses domain knowledge to build the graph among monitoring metrics.
  • Approach: CIRCA addresses incomplete faulty-data distributions with regression-based hypothesis testing and incomplete fault-free distributions with descendant adjustment.Descendant adjustment is intended to reduce bias caused by poor understanding of the system’s normal status.
  • Results: 25% higher top-1 recommendation recall than the best baseline is reported on a real-world dataset, while simulation evaluates CIRCA’s theoretical reliability.The introduction reports both simulation and real-world evaluation as evidence for the method’s reliability and practical value.

2 PROBLEM FORMULATION

The paper maps online-service root cause analysis onto causal inference by distinguishing observational and interventional distributions. It defines intervention recognition and root causes within a structural causal model and its causal Bayesian network.

  • 2.1 Preliminary: The structural causal model represents each metric as a function of its parent variables and an exogenous variable, with a directed graph connecting parents to children.The graph is defined as G = (V, E), where edges point from parent metrics to child metrics.
  • 2.1 Preliminary: The causal formulation separates observational knowledge L1(V)=P(V) from interventional knowledge P(V|do(m)), which fixes selected variables to given values.The do-operator denotes intervention, and the second causal layer describes the resulting distribution.
  • 2.1 Preliminary: The framework assumes a directed acyclic graph, independent exogenous parent sets, and faithfulness, under which the graph can be treated as a causal Bayesian network.Faithfulness requires interventions to produce observable distributional changes.
  • 2.2 Root Cause Analysis and Causal Inference: The paper maps a fault to an unexpected intervention, fault-free data to the observational distribution, and faulty data to an interventional distribution.This mapping connects operational RCA terminology to causal-inference objects.
  • 2.2 Root Cause Analysis and Causal Inference: Intervention recognition is defined as finding the intervention m from the observational distribution L1 and the corresponding interventional distribution P_m.The definition treats the intervention as the object to be recovered from the two distributions.
  • 2.2 Root Cause Analysis and Causal Inference: The root cause is defined as the set of intervened variables M, with each member called a root cause variable.This definition makes the causal intervention variables the targets of RCA.

3 INTERVENTION RECOGNITION CRITERION

The paper positions intervention recognition at the interventional layer of the causal ladder and derives a criterion for identifying intervened variables from their conditional distributions. The criterion enables root cause indicators to be found using the causal Bayesian network without requiring counterfactual knowledge.

  • Causal-layer characterization: Intervention recognition is equivalent to second-layer interventional knowledge under the faithfulness assumption, so counterfactual knowledge is not necessary.The paper places the task at Layer 2 and states that Layer 3 knowledge is unnecessary.
  • Intervention Recognition Criterion: The causal Bayesian network bridges observational data and interventional knowledge, enabling the criterion to identify root cause indicators.The paper uses the CBN as the basis for converting causal structure into an RCA test.
  • Intervention Recognition Criterion: A variable is intervened if and only if it no longer follows the distribution defined by its parent variables in the causal Bayesian network.This conditional-distribution change is the Intervention Recognition Criterion under faithfulness.

4 CAUSAL INFERENCE-BASED ROOT CAUSE ANALYSIS

CIRCA builds a causal graph for online service-system metrics from architecture knowledge and causal assumptions, then uses regression-based testing and descendant adjustment to identify root-cause indicators.

  • Regression-based Hypothesis Testing: CIRCA applies regression-based hypothesis testing because faulty data provide incomplete distributions, using pre-fault data to estimate each metric’s expected distribution.A regression model is trained for each variable as a proxy for its structural equation, and residual assumptions support anomaly scoring.
  • Structural Graph Construction: CIRCA constructs a structural graph by combining system architecture with causal assumptions among four service dimensions: Traffic, Errors, Latency, and Saturation.Traffic is treated as causing the other dimensions, Errors as their effect, and Saturation as a cause of Latency.
  • Structural Graph Construction: The graph extends service-level meta-metric relations across dependent services and plugs monitoring metrics into their corresponding dimensions.A monitoring metric may be derived from multiple meta metrics, while some meta metrics may have no available monitoring metric.
  • Descendant Adjustment: Descendant adjustment raises a metric’s anomaly score using descendants’ scores, preferring abnormal parent metrics as root-cause indicators over their effects.Metrics below the three-sigma threshold are excluded from root-cause indicators before descendant scores are propagated to ancestors.

5 EXPERIMENTS

The experiments evaluate CIRCA through simulation and real-world studies, measuring top-k recall and analysis duration against graph-construction and scoring baselines. CIRCA achieves strong performance, with regression-based testing and descendant adjustment contributing positively, while incomplete causal knowledge remains a limitation.

  • 5.2 Simulation Study: RHT-PG approaches ideal performance and outperforms baselines with p<0.001 in t-tests for AC@k, supporting CIRCA’s theoretical reliability in simulation.The gap between RHT and RHT-PG enlarges as graph size increases, reflecting the restriction that a broken CBN cannot guarantee correct RCA.
  • 5.2.2 Performance Evaluation: RHT and RHT-PG achieve the best results across weak, mixed, and strong faults, whereas anomaly-detection and DFS-based methods degrade for strong faults.Strong-fault root cause metrics may be less abnormal than other metrics, making anomaly-based ranking less reliable.
  • 5.3 Real-World Evaluation: CIRCA achieves the best results on the real-world dataset compared with baseline methods, while recommending root cause metrics within seconds with the structural graph.Random walk methods perform best with PCTS but require much time to construct the graph.
  • 5.3 Component and Graph Evaluation: Removing CIRCA components shows that both regression-based hypothesis testing and descendant adjustment positively affect performance, while the structural graph improves AC@5 for DFS and CIRCA.PCTS fits random-walk methods better than the structural graph.
  • 5.4 Discussion: CIRCA remains stable across different delay and reference-window hyperparameters and outperforms baseline methods, but incomplete Layer 1 knowledge constrains RCA.The authors prefer CIRCA to pure RHT for deployment and say descendant adjustment requires verification on more real-world datasets.

6 RELATED WORKS

Prior RCA methods use graph traversal, random walks, PageRank, or invariant networks, but they face explainability, anomaly-detection sensitivity, or limited metric coverage. This paper instead constructs a metric-level causal graph from system knowledge and treats RCA as intervention recognition.

  • Existing RCA methods traverse abnormal subgraphs, use random walks or PageRank, or learn invariant networks.
  • DFS-based methods are sensitive to anomaly-detection results, while random-walk and PageRank methods lack explainability.
  • Causal discovery from observational data may produce graphs that are not explainable to human operators.
  • Unlike MicroHECL and Sage, the proposed structural graph uses system architecture knowledge and intuitive assumptions to cover more monitoring metrics.

7 CONCLUSION AND FUTURE WORK

The paper formulates online-service root cause analysis as intervention recognition and develops CIRCA to apply this causal criterion. CIRCA is theoretically reliable in simulation and outperforms baselines on a real-world dataset, while future work must address hidden metrics and violated assumptions.

  • The paper formulates RCA as intervention recognition, linking root cause analysis with causal inference through the Intervention Recognition Criterion.
  • CIRCA outperforms baseline methods in simulation and on a real-world dataset, supporting its theoretical reliability and practical potential.
  • CIRCA combines structural graph construction, regression-based hypothesis testing, and descendant adjustment.
  • Future work must address incomplete Layer 1 knowledge, hidden meta metrics, and assumptions that real applications may violate.

A PROOF OF THEOREM 3.1

The proof establishes that identifiable intervention recognition recovers intervention information from Layer 2 knowledge and therefore belongs to the second layer of the causal ladder. It derives this through equivalence classes of interventions and causal Bayesian-network identities.

  • Identifiable intervention recognition finds potential interventions that produce the same observed distribution under an intervention.
  • The proof derives identifiable intervention recognition from Layer 2 knowledge and shows that its knowledge encodes Layer 2.
  • Interventions are grouped into equivalence classes when they induce the same distribution over variables, enabling inverse mapping from distributions to intervention classes.
  • Using causal Bayesian-network separation and parent-related conditions, the proof reduces conditional interventional distributions to the form required for identifiable recognition.
  • Identifiable intervention recognition distinguishes an intervention from alternatives and yields the same answer as intervention recognition.

B PROOF OF THEOREM 3.4

The proof of Theorem 3.4 shows that a variable is intervened on exactly when its distribution conditioned on its parents differs from the observational conditional distribution. This supplies the criterion used to identify root-cause metrics.

  • When a variable itself is intervened on, faithfulness requires the conditional-distribution difference used by the criterion.
  • When a variable is not intervened on, the interventional and observational parent-conditioned distributions are equal.
  • A variable is intervened on if and only if its parent-conditioned interventional distribution differs from its observational counterpart.

C IMPLEMENTATION DETAILS

The implementation combines Python with R’s pcalg package for the PC algorithm and uses process-based parallelism for error isolation. Baselines are reproduced through available implementations or reimplementations when source code is unavailable.

  • C IMPLEMENTATION DETAILS: The PC algorithm is implemented with the R package pcalg, while most other code is written in Python.The authors use their implementation of SPOT and reimplement ENMF in Python from the authors’ MATLAB implementation.
  • C IMPLEMENTATION DETAILS: Process-based parallel programming is used specifically to isolate errors during execution.
  • C IMPLEMENTATION DETAILS: Unavailable baseline methods are implemented according to the authors’ understanding.

C.2 Simulation Data Generation

Simulation data are generated from a vector autoregression model with randomly constructed connected DAG structures. Edge weights, autoregressive parameters, and Gaussian-noise scales are sampled or fixed according to the stated experimental design.

  • C.2 Simulation Data Generation: The simulation datasets follow a vector autoregression model with β=0.1 for βx(t−1).Non-zero adjacent-matrix entries A_ij are sampled uniformly from (−2.0, −0.5) ∪ (0.5, 2.0).
  • C.2 Simulation Data Generation: The zero-mean Gaussian-noise standard deviations are sampled from an exponential distribution.
  • C.2 Simulation Data Generation: The simulation graph is built by first generating a tree for a connected DAG, then inserting additional edges randomly.Algorithm 3 specifies the graph using N_node nodes and N_edge edges, with parent nodes selected randomly from earlier nodes.
  • C.2 Simulation Data Generation: In the empirical study, metrics outside the final structural graph are dropped because the available knowledge does not cover them.Baseline methods use only labeled metrics for fair comparison.

C.4 Regression Method Selection

The study compares regression methods for RHT and selects SVR because nonlinear relationships improve performance over linear regression, while RF and MDN perform worse under limited normal-status understanding.

  • C.4 Regression Method Selection: SVR with a sigmoid kernel improves RHT performance over linear regression because real-world variable relations are seldom linear.
  • C.4 Regression Method Selection: RHT combined with RF or MDN performs worse than simple linear regression because the normal status is insufficiently understood.
  • C.4 Regression Method Selection: SVR is selected for the empirical study after comparing the regression alternatives.Table 5 reports RHT performance with different regression methods on D_O.
Loading 2206.05871v1…