PaperlayerView exact revision on arXiv

Source-linked AI summary

ObliVul: Alert-Conditioned Safety Obligation Modeling and Bidirectional Counterfactual Validation for Code Vulnerability Detection

Heyang Tan, Chengxin Gao, Xin Wen, Jiaxin Li, Rui Cao

arXiv:2610.11801v1cs.SE

TL;DR

Static analysis produces many candidate alerts, while existing approaches have difficulty aligning recovered safety semantics with the program structures needed to verify an individual alert. ObliVul constructs alert-centered safety-obligation graphs and validates them with bidirectional counterfactual interventions. Experiments report more stable vulnerable/fixed distinction and fewer persistent false positives on fixed versions.

  • Problem

    Existing methods struggle to align safety semantics with program nodes, dependencies, and paths, making it difficult to verify whether obligations affect an individual alert.

    Source: §1 · pp. 1–2

  • Method

    ObliVul extracts alert-local CPG evidence, maps LLM-recovered safety obligations to program structures in an LSOG, validates risk responses through protection-adding and protection-removing interventions, and aggregates alert evidence with VAFI.

    Source: §1 · pp. 2–3, §3.2 · p. 6, §3.3.1 · p. 7

  • Results

    Experiments on PrimeVul pair-function show that ObliVul reduces persistent false positives on fixed versions and more stably distinguishes vulnerable from fixed versions.

    Source: §6 · p. 19

  • Takeaways & Limitations

    ObliVul can provide risk scores, audit priorities, and local evidence for review in the reported calibration, selective-prediction, and Linux kernel validation settings.

    Source: §6 · p. 19

    ObliVul depends on LLM semantic-recovery capability and local CPG quality; future work will add symbolic execution, dynamic testing, and runtime information for complex-path and environmental analysis.

    Source: §6 · p. 19

Abstract

from arXiv · show

In real-world software development, the primary challenge in vulnerability detection is often not finding suspicious code, but identifying which alerts among the large number of candidate alerts produced by static analysis truly warrant attention. Existing learning-based methods mainly identify suspicious patterns at the function or line level, making it difficult to extract complete program evidence centered on an individual alert. Although large language models can infer risk sources, dangerous operations, protection conditions, and state preconditions from local program facts, such semantic information cannot be reliably aligned with specific program nodes, dependency relations, and propagation paths, and is therefore insufficient to verify whether the corresponding safety obligations truly affect the current alert. To address this problem, we propose ObliVul, an alert-conditioned safety obligation modeling and bidirectional counterfactual validation framework for vulnerability detection. For each candidate alert, ObliVul first extracts a Local Evidence Pack (LEP) from the Code Property Graph (CPG) and uses a large language model to recover candidate safety obligations. It then aligns the safety obligations with program nodes, dependency edges, and path scopes to construct a Local Safety Obligation Graph (LSOG). Finally, VAFI aggregates complementary verified alert evidence while suppressing redundant or weaker evidence to produce a function-level vulnerability prediction. Experimental results show that ObliVul effectively distinguishes vulnerable versions from fixed versions and reduces persistent false positives on fixed code. Ablation studies further confirm the necessity of each component for safety obligation recovery, risk response validation, and function-level vulnerability inference.

1 Introduction

Vulnerability detection must distinguish actionable alerts from the many potential risks raised by static analysis. ObliVul models safety obligations around individual alerts, validates their risk responses, and aggregates verified evidence for function-level predictions.

1 Introduction

  • Static analysis generates many unverified candidate alerts, so the central challenge is determining which alerts truly warrant attention.Alert validation requires relevant program evidence, safety obligations that affect validity, and verification that those obligations act on the current risk.Source: §1 · p. 1
  • Function- and line-level methods can miss whether an individual alert has complete dependency and path support, especially when vulnerable and fixed versions share most code.LLM-generated semantics are also difficult to align with concrete program nodes, dependency edges, and propagation paths for further checking.Source: §1 · p. 2
  • ObliVul extracts alert-relevant evidence from a function-level CPG, recovers safety obligations with an LLM, and anchors them to program structures in an LSOG.It then adds missing protection and removes existing protection to examine risk changes and extract minimal supporting evidence.Source: §1 · p. 2
  • VAFI aggregates complementary alert evidence while suppressing duplicated or weak evidence; Linux kernel review identified four real vulnerabilities, with three patches merged.VAFI coordinates alert-level, function-level, and vulnerable/fixed-pair constraints.Source: §1 · p. 3

2 Related Work

Prior work addresses alert prioritization, code-level vulnerability patterns, and vulnerable/fixed comparisons, but leaves limitations in representing and checking obligations for a particular alert. ObliVul structures alert-local evidence and uses bidirectional interventions to test risk responses.

2.1 Alert-Oriented Static-Analysis-Based Vulnerability Detection

  • Alert-oriented methods use slicing and path-sensitive analysis, but relevant dependencies and protection conditions remain scattered rather than unified as safety obligations.ObliVul organizes local CPG evidence around an individual alert to align semantic recovery and validation with the same program locations and paths.Source: §2.1 · p. 3

2.2 LLM-Enhanced Vulnerability Detection

  • LLM-enhanced methods can incorporate code, security knowledge, and program graphs, but graph context alone does not constrain model output to concrete program structures.ObliVul maps recovered safety semantics to CPG nodes, dependencies, and paths to form structured obligations with anchors and relational constraints.Source: §2.2 · pp. 3–4

2.3 Vulnerable/Fixed Pair Analysis and Bidirectional Counterfactual Validation

  • Vulnerable/fixed prediction differences cannot identify which patch edit reverses risk or whether added protection covers the alert’s dangerous path.ObliVul adds missing obligations in vulnerable contexts and removes existing protection in fixed contexts, using path consistency to extract minimal evidence; this is not a strict causal proof of complete program behavior.Source: §2.3 · p. 4

3.1 Framework Overview

ObliVul processes each alert through local evidence extraction, safety-obligation graph construction, bidirectional counterfactual validation, and function-level inference. The workflow is summarized in Figure 1.

3.1 Framework Overview

  • The four-module workflow begins by extracting alert-local program evidence and ends with validation-aware function-level vulnerability inference.Figure 1 presents the overall alert-driven bidirectional counterfactual validation framework.Source: §3.1 · pp. 4–5, Figure 1 · p. 5
  • The evidence-extraction module parses a function into a CPG and collects dependencies, control constraints, calls, and protection conditions relevant to the alert.This restricts subsequent analysis to entities and dependencies affecting the current alert.Source: §3.1 · p. 5
  • The graph-construction module uses an LLM to recover candidate obligations and checks their correspondence to CPG nodes, dependency edges, and propagation paths.Counterfactual validation then adds or strengthens missing protection and removes or weakens existing protection to test for risk decrease or reappearance.Source: §3.1 · p. 5
  • VAFI encodes local CPG, LSOG, and counterfactual results as alert-evidence units, models complementarity and redundancy, and outputs a function-level risk score and decision.It aggregates evidence from different alerts within a function.Source: §3.1 · p. 5

3.2 Alert-Driven Local Program-Evidence Extraction

Alert-driven local evidence extraction starts from static-analysis alerts and projects the CPG around each alert. The resulting Local Evidence Pack supplies explicit program facts while limiting unrelated context.

3.2 Alert-Driven Local Program-Evidence Extraction

  • A candidate alert is generated when a security check finds a potentially risky dangerous operation, and it represents a potential risk pattern rather than a confirmed vulnerability.The alert includes a rule or category, report location, dangerous operation, candidate path, and related program context.Source: §3.2 · p. 6
  • The alert-conditioned CPG projection follows relevant data sources, propagation variables, calls, control conditions, and candidate protection facts around the report location and dangerous operation.Interprocedural analysis retains directly relevant call sites, parameter passing, return propagation, and restricted callee context rather than expanding complete call subgraphs.Source: §3.2 · p. 6
  • Projection stops at candidate data sources, an expansion depth, or an interprocedural boundary, and limits the number and length of retained paths.The resulting LEP organizes normalized views of data propagation, control constraints, interprocedural transfer, and candidate protection coverage; these are parseable CPG facts, not independent evidence.Source: §3.2 · p. 6

3.3 Local Safety-Obligation Graph Construction

ObliVul derives alert-specific safety obligations from Local Evidence Packs, checks them against local CPG structures, and organizes supported obligations and their relations in an LSOG.

3.3 Local Safety-Obligation Graph Construction

  • The system induces alert-related obligations from the Local Evidence Pack rather than directly asking the LLM to decide alert validity.The LLM recovers security entities, program roles, and risk conditions associated with the alert.Source: §3.3 · pp. 6–7

3.3.1 Alert-Conditioned Safety-Obligation Induction.

  • Obligations cover risk sources, dangerous operations, required or ineffective protection, state preconditions, and critical dependencies, making risk conditions checkable.They describe how risk forms, why protection fails, and which alert path the conditions affect.Source: §3.3.1 · p. 7
  • Program-correspondence checking maps obligation entities and conditions to CPG nodes or paths, verifies dependencies, and checks whether scope covers the alert's dangerous path.Candidate obligations are classified as accepted, partially supported, conflicting, or rejected; missing protection can be anchored to its expected location.Source: §3.3.1 · p. 7

3.3.2 Local Safety-Obligation Graph Construction.

  • The LSOG connects filtered obligations through data propagation, control dependencies, and protection coverage while retaining program anchors and missing, partial, or conflicting states.Its nodes represent alert-related risk entities, obligations, and satisfaction states; its edges represent relations among them.Source: §3.3.2 · p. 8

3.4 Bidirectional Counterfactual Graph Validation

ObliVul compiles LSOG obligations into local graph interventions and tests whether the resulting risk changes align with the current alert path.

3.4.1 LSOG-Based Safety-Obligation Compilation and Bidirectional Counterfactual Validation.

  • LSOG obligations become forward or reverse graph interventions according to their types, states, anchors, and scopes.Missing or ineffective protection is added or strengthened; present protection is removed or weakened, while obligations without clear anchors or path support are not directly intervened on.Source: §3.4.1 · pp. 8–9
  • Both intervention directions reanalyze risk against a shared baseline derived from the alert's local path state.Forward validation adds or strengthens missing protection; reverse validation removes or weakens protection already on the path.Source: §3.4.1 · p. 9
  • The alert-level validation score combines bidirectional risk changes, path consistency, and obligation conflict, then passes minimal supporting evidence to VAFI.Evidence selection retains intervention items that preserve the risk response and records obligations, operation locations, and related paths.Source: §3.4.1 · p. 9

3.4.2 Alert Evidence Flow in a PrimeVul Paired Case.

  • In the CVE-2021-37681 paired case, adding the missing guard lowers vulnerable-path risk, while removing it from the fixed version raises risk.The two responses support a minimal witness containing the required guard, state precondition, and critical path.Source: §3.4.2 · p. 10, Figure 2 · p. 10

3.5 Verified Alert-Driven Function-Level Inference

VAFI encodes validated alert evidence and aggregates it to make function-level vulnerability decisions, accounting for evidence strength and semantic reliability.

3.5 Verified Alert-Driven Function-Level Inference

  • VAFI encodes each alert's local CPG features, LSOG obligations, alert risk, counterfactual results, and minimal evidence as an alert-evidence unit for function-level inference.The method aggregates multiple alerts because validating an individual alert does not by itself determine whether its function is vulnerable.Source: §3.5 · p. 10
  • VAFI separately estimates alert-risk evidence strength and security-semantic reliability, then uses validation-aware attention to weight alert evidence.The attention weight is based on the alert representation, the two estimates, and a temperature parameter.Source: §3.5 · p. 11
  • Function-level aggregation combines the strongest alert, the alert mean, and function-level anchoring information, limiting amplification from duplicate alerts.Valid alerts from different paths can jointly support the prediction.Source: §3.5 · p. 11
  • Joint training constrains function classification, semantic and alert consistency, risk ranking, vulnerable/fixed pairs, and consistency with aggregated semantics and program anchors.Vulnerable/fixed pairs are used only during training; at test time, each function's alerts are aggregated independently.Source: §3.5 · p. 11

4 Experimental Evaluation

The evaluation uses vulnerable/fixed function pairs to assess function-level classification and paired-state diagnosis. Results show ObliVul balances vulnerability discovery with recognition of repairs, including on LinuxVul despite dataset shift.

4.1.1 Dataset and Experimental Protocol.

  • PrimeVul pair-function preserves correspondence between vulnerable functions before a fix and fixed functions after the same commit, enabling paired risk-state analysis.The dataset originates from real-world C/C++ vulnerability reports and corresponding fixing commits.Source: §4.1.1 · pp. 11–12

4.1.2 Evaluation Metrics.

  • Evaluation covers function-level Accuracy, Precision, Recall, FPR, and F1-score, plus paired-state metrics P-C, P-V, P-B, P-R, and VPS.P-C means both versions are correctly distinguished; VPS summarizes effective vulnerable-versus-fixed discrimination, with higher values indicating better identification and fewer persistent fixed-version alerts.Source: §4.1.2 · p. 12

4.2 Comparison Experiments

  • The PrimeVul comparison is not a strict same-protocol comparison because methods may differ in preprocessing and effective test-set size.ObliVul was evaluated on 422 retained test pairs using deterministic greedy decoding; valid-pair denominators also differ across methods.Source: §4.2 · pp. 12–13
  • On PrimeVul pair-function, ObliVul balances vulnerability discovery and repair recognition, unlike methods that retain fixed-code alerts or miss vulnerabilities.The comparison is a performance reference, not a strict same-protocol comparison, because preprocessing and effective test-set sizes differ across methods.Source: §4.2 · pp. 12–13, Table 2 · p. 13
  • 89.69% accuracy, 88.89% precision, 90.72% recall, and 89.80% F1-score on LinuxVul test data indicate performance despite dataset shift.The LinuxVul test set is used only for final evaluation in a CVE-disjoint partition.Source: §4.1.1 · p. 12, §4.2 · p. 13
  • Paired-state results show many methods label both versions vulnerable; ObliVul models safety obligations, tests risk responses, and aggregates verified alert evidence for function-level judgments.This combination is reported to support both vulnerability and repair recognition, avoiding persistent alerts on fixed code or code constrained by effective obligations.Source: §4.2 · pp. 13–14, Table 3 · p. 13

4.3 Ablation Experiments

The ablations examine structured safety semantics, bidirectional risk validation, and alert-evidence aggregation, finding that each component addresses a distinct part of function-level vulnerability inference.

4.3.1 RQ1: Effect of LSOG-Based Structured Semantics.

  • Complete LSOG organizes risk entities, protections, scopes, and dependencies; core-entity labels alone cannot establish whether a dangerous operation is a vulnerability in context.The comparison distinguishes structured obligations from simply adding source, sink, guard, and sanitizer labels.Source: §4.3.1 · p. 14, Table 4 · p. 14

4.3.2 RQ2: Effect of Bidirectional Counterfactual Validation.

  • Two-direction risk responses alone do not ensure better discrimination; full validation with minimal evidence and quality fusion produces the strongest controlled RQ2 result.The compared branches include no validation, forward-only, direct bidirectional, bidirectional without minimal evidence, and the complete configuration.Source: §4.3.2 · pp. 14–15, Table 5 · p. 15

4.3.3 RQ3: Effect of VAFI Aggregation and Consistency Learning.

  • Simple averaging or weighted aggregation misses alert complementarity, redundancy, and validation reliability, and can repeatedly amplify risk from the same dangerous path.The ablation compares pooling and VAFI variants with ranking, path-alignment, and hierarchical-consistency constraints.Source: §4.3.3 · p. 15, Table 6 · p. 15

4.3.4 Summary of the Ablation Experiments.

  • The ablation summary assigns LSOG to structuring obligations, bidirectional validation to testing their relationship with alert risk, and VAFI to aggregating verified alert evidence.Together, these stages produce a function-level vulnerability conclusion.Source: §4.3.4 · p. 15

5 Behavioral and Trustworthiness Analysis

The analyses examine score reliability, confidence-based review prioritization, and risk changes across vulnerable/fixed pairs. A preliminary Linux kernel validation reports three fixed defects and one candidate still under validation.

5.1.1 Probability Calibration Analysis.

  • Raw scores generally track vulnerability proportions, while pair adjustment does not automatically improve calibration; validation-fitted temperature scaling can correct probabilistic deviations.The calibration analysis evaluates raw and pair-adjusted outputs and applies a temperature parameter fitted on validation data.Source: §5.1.1 · p. 16, Figure 3 · p. 16

5.1.2 Selective Prediction Analysis.

  • As coverage expands, error generally rises, so confidence ranks relatively reliable predictions first and can guide where analysts focus review.The Coverage–Risk curves compare pair-adjusted scores before and after calibration.Source: §5.1.2 · pp. 16–17, Figure 3 · p. 16

5.2 Paired Counterfactual Risk-Response Analysis

  • Paired comparisons show repair operations produce directionally correct, practically meaningful risk changes in most samples, despite shared function structure and semantics.Most feature trajectories point from fixed to vulnerable versions, and risk differences are mainly positive.Source: §5.2 · pp. 17–18, Figure 4 · p. 18

5.3 Preliminary Validation on a Real Linux Kernel Project

  • In preliminary Linux kernel screening, three identified defects were fixed and merged upstream, while another candidate and patch remained under validation pending physical-machine PCI tests.The workflow used scoped alerts, top-k filtering, manual review, and upstream repair information.Source: §5.3 · p. 18

6 Conclusion and Future Work

ObliVul combines alert-centered evidence, safety-obligation modeling, bidirectional counterfactual validation, and VAFI for function-level vulnerability judgments. The paper reports improved vulnerable/fixed discrimination and review support, while noting dependencies on LLM semantic recovery and local CPG quality.

6 Conclusion and Future Work

  • ObliVul models alert-centered safety obligations, tests risk responses by adding or removing protection, and aggregates alert evidence through VAFI for function-level judgments.The framework constructs a Local Safety Obligation Graph from local program evidence.Source: §6 · pp. 18–19
  • PrimeVul experiments report reduced persistent false positives on fixed versions and more stable distinction between vulnerable and fixed versions; calibration, selective prediction, and Linux validation provide risk scores, priorities, and local review evidence.The authors associate these outputs with improved interpretability and practical engineering value.Source: §6 · p. 19
  • ObliVul depends on LLM semantic-recovery capability and local CPG quality; future work will combine symbolic execution, dynamic testing, and runtime information for complex-path and environmental analysis.Source: §6 · p. 19
Loading 2610.11801v1…