PaperlayerView exact revision on arXiv

Source-linked AI summary

SymLift: Neural Semantic Lifting for Compile-Free Vulnerability Detection in Context-Incomplete Code Fragments

Xiarun Chen, Jiaxing Xu, Bozhi Wu, Chenhui Cui, Chunrong Fang, Weiping Wen, Tao Li, Rubing Huang, Tsong Yueh Chen

arXiv:2610.34431v1cs.SEcs.CR

TL;DR

Missing project context can leave conventional vulnerability analysis without the program relations its rules need, while neural predictions alone do not establish auditable derivations. SymLift reconstructs vulnerability-relevant relations in LLVM-like IR and uses a deterministic parser and explicit Datalog rules to produce traceable findings. It reports 94.41% Target-F1 on FormAI and 91.66% micro-averaged Target-F1 on MegaVul.

  • Problem

    Context-incomplete code may lack the program relations needed for vulnerability detection, and neural predictions alone do not establish auditable derivations.

    Source: §I · p. 1

  • Method

    SymLift uses a fine-tuned LLM to reconstruct vulnerability-relevant relations in constrained LLVM-like IR, which a deterministic parser converts to evidence facts for explicit Datalog rules.

    Source: §I · p. 2

  • Results

    94.41% Target-F1 on FormAI and 91.66% micro-averaged Target-F1 on MegaVul; Juliet compilation success falls from 100% to 12.00% while SymLift analysis completion drops only 1.17 percentage points.

    Source: §I · p. 2

  • Takeaways & Limitations

    SymLift supports compile-free analysis and deterministic, auditable vulnerability reasoning for fixed evidence facts and rules.

    Source: §I · p. 2

    Evaluation covers C code, six vulnerability families, selected datasets and baselines; fact-level and similarity measures may not capture full behavioral equivalence.

    Source: §VIII · p. 12

Abstract

from arXiv · show

Missing project context can prevent build-dependent analyzers from constructing program relations required for vulnerability detection, while neural predictions alone do not establish auditable derivations. We present SymLift, a compile-free framework centered on Vulnerability-Oriented Neural Semantic Lifting for context-incomplete code fragments. A fine-tuned large language model reconstructs vulnerability-relevant control-flow, data-flow, memory, and resource relations in a constrained LLVM-like intermediate representation (IR), without reconstructing the full project context or directly predicting vulnerability labels. A deterministic parser validates the IR and maps its supported operations and relations into normalized evidence facts. Explicit Datalog rules derive findings with supporting facts and derivation traces, ensuring deterministic reasoning for fixed facts and rules. We evaluate SymLift on FormAI, Juliet, and MegaVul. Reducing complete Juliet programs to function-level fragments lowers Clang compilation success from 100% to 12.00%, whereas SymLift's analysis completion rate decreases by only 1.17 percentage points. After task-specific fine-tuning, the lifter achieves a control-flow intersection over union (IoU) of 96.85% on FormAI. SymLift-Qwen-7B achieves a Target-F1 of 94.41% on FormAI and, across six vulnerability families on MegaVul, a micro-averaged precision of 98.24% and a micro-averaged Target-F1 of 91.66%. These results support accurate and auditable detection in the evaluated context-incomplete settings, even with partial program relations.

I. INTRODUCTION

Context-incomplete fragments may lack the program relations conventional vulnerability analysis needs, while neural predictions alone do not provide auditable derivations. SymLift reconstructs vulnerability-relevant relations and applies explicit rules to produce traceable findings without requiring successful compilation.

I. INTRODUCTION

  • The compilation gap is the mismatch between available source code and recovery of the program relations needed by downstream vulnerability detection.Missing context can cause failed or incomplete relation construction, or divergence from representations obtained with more complete context.Source: §I · p. 1
  • SymLift’s fine-tuned LLM reconstructs vulnerability-relevant relations in constrained LLVM-like IR without rebuilding full project context or directly predicting vulnerability labels.A deterministic parser maps supported IR operations and relations to evidence facts, then explicit Datalog rules produce findings linked to facts and derivation traces.Source: §I · p. 2
  • The IR interface makes reconstructed relations inspectable but does not eliminate neural-lifting uncertainty; deterministic reasoning holds for fixed facts and rules.The design separates neural semantic reconstruction from vulnerability judgment.Source: §I · p. 2
  • Reducing Juliet programs to function-level fragments lowers compilation success from 100% to 12.00%, but analysis completion by only 1.17 points; SYMLIFT-Qwen-7B achieves 94.41% Target-F1 on FormAI and 91.66% micro-averaged Target-F1 on MegaVul.FormAI control-flow IoU is 96.85%; MegaVul precision is 98.24%.Source: §I · p. 2

A. Context-Incomplete Code and Program Representations

  • Context-incomplete fragments can lack information needed for semantic analysis, and successful parsing alone does not ensure that required program relations are available.The background identifies LLM-generated code and function-level datasets as examples of such fragments.Source: A. Context-Incomplete Code and Program… · p. 2

B. LLVM Intermediate Representation

  • The LLVM-like IR uses a restricted, parser-supported subset of LLVM conventions, and analysis completion does not establish semantic equivalence.Generated outputs may be malformed; inference extracts facts without requiring assembly acceptance.Source: B. LLVM Intermediate Representation · p. 2

C. Fact-Based Vulnerability Reasoning with Datalog

  • Datalog derives relations from input facts and rules, with fixed facts and rules yielding deterministic results traceable to supporting evidence.Its reasoning is limited to available input relations.Source: C. Fact-Based Vulnerability Reasoning… · p. 2

III. MOTIVATING EXAMPLE

The motivating function contains an upper-bound check but no locally visible lower-bound check, and its missing declarations prevent independent compilation. SymLift illustrates how reconstructed relations and explicit rules can make the resulting analysis traceable.

III. MOTIVATING EXAMPLE

  • Under the example’s assumptions, idx < count supplies an upper bound but not idx >= 0, so a negative index may reach the array access.The example does not establish whether that access is reachable across the complete program’s calling contexts.Source: §III · p. 3
  • The fragment cannot compile independently because Context and get_entry_count are undefined, and tolerant parsing may not recover the typed access or required program relations.Build-dependent analyzers may therefore fail to construct, or only partially construct, representations required by their vulnerability rules.Source: §III · p. 3
  • For the example, SymLift maps reconstructed IR to Base Facts and derives DefUse, ControlDependent, and ArrayAccess relations before applying the missing-bounds rule.The report traces through the reconstructed instructions, extracted facts, derived relations, and matched rule.Source: §III · p. 3

A. Overview

  • The overview’s three stages lift functions to LLVM-like IR, parse IR into normalized evidence facts, and evaluate explicit vulnerability rules with Soufflé Datalog.The IR route reconstructs instruction-level structure before the parser extracts Base Facts; typed operations, branches, pointer computations, and SSA expose dependencies for structural validation.Source: A. Overview · p. 3

B. Stage I: Vulnerability-Oriented Neural Semantic Lifting

SYMLIFT lifts target functions and selected project context into LLVM-like IR that preserves vulnerability-relevant relations. Its two-stage fine-tuning combines structural conformity with security-oriented semantic preservation.

B. Stage I: Vulnerability-Oriented Neural Semantic Lifting

1) Analysis Granularity and Preprocessing:

  • The target function is analyzed with up to 50 nearby project-local functions within three call edges, while unavailable context is omitted rather than fabricated.Without project sources, only the target is analyzed; selected functions are lifted independently and their facts analyzed jointly.Source: 1) Analysis Granularity and… · p. 3

2) Two-Stage Fine-Tuning:

  • Fine-tuning first targets structural fidelity with paired C functions and Clang-generated IR, using token-level cross-entropy and validator-based structural feedback.The validators are llvm-as and the custom parser; their non-differentiable feedback is incorporated through a policy-gradient objective.Source: 2) Two-Stage Fine-Tuning · p. 3
  • Structural feedback rewards LLVM assembly validity and downstream-parser compatibility, not semantic equivalence with the source function.The final objective combines IR reconstruction supervision with structural feedback, with α = 0.1.Source: 2) Two-Stage Fine-Tuning · pp. 3–4
  • The second fine-tuning stage uses security-oriented FormAI functions and matched non-vulnerable examples to preserve vulnerability-family relations.It applies semantics-preserving augmentation and retains 25,746 training records after rejecting 214 variants that fail parser structural checks.Source: 2) Two-Stage Fine-Tuning · p. 4

3) Relation-Oriented Model Validation:

SYMLIFT validates LLVM-like IR, maps supported content into normalized evidence facts, and applies explicit rules to those facts. Its fault-tolerant parsing preserves parseable content while recording uncertainty and coverage limitations.

3) Relation-Oriented Model Validation:

  • The held-out validation set contains 1,000 functions from zlib and libpng and is used for model selection, never gradient updates.Generated and reference IR are compared as operation types, definition-use edges, branches, and control-flow transitions.Source: 3) Relation-Oriented Model Validation · p. 4

1) Vulnerability-Oriented Fact Schema:

  • The Vulnerability-Oriented Fact Schema provides a stable interface that normalizes equivalent IR relations and permits new rules without retraining when required IR information is present.Source: 1) Vulnerability-Oriented Fact Schema · p. 4

2) Deterministic IR-to-Fact Parsing:

3) Fault Tolerance and Parallelization:

  • Missing positive premises can cause false negatives, while unavailable safety facts can satisfy negated premises and cause false positives when other conditions hold.Non-derivability from recovered facts does not prove the complete program lacks a safety condition.Source: 3) Fault Tolerance and Parallelization · p. 5

D. Stage III: Explicit Vulnerability Rules and Datalog Execution

1) Rule Specification:

SYMLIFT organizes vulnerability rules into layered decisions over structural and operational evidence, including path-sensitive checks and interprocedural relations. Reports include supporting facts and derivation traces, but trace completeness does not establish whole-program recovery or reachability.

1) Rule Specification:

  • Four rule layers derive shared structure, identify positive operational evidence, recognize safety conditions and counterexamples, and resolve findings with decision rules and priorities.A missing safety fact alone cannot produce a report because decision rules also require positive operational evidence.Source: 1) Rule Specification · p. 5

2) Path-, Dependency-, and Safety-Aware Reasoning:

  • Rule reasoning combines operand provenance with control dependence to check whether guards apply on the relevant path, and requires cleanup across applicable acquisition-to-exit paths.An upper-bound check does not establish a lower bound; a NULL check applies only when the checked value reaches the dereference and its branch dominates the path.Source: 2) Path-, Dependency-, and… · pp. 5–6

3) Inter-Procedural and External-Call Rules:

  • Interprocedural mappings propagate provenance and resource states across selected call edges, but are path-insensitive and alone do not establish caller-side guards.Recognized external callees use explicit summaries; an unknown external call alone does not create a vulnerability verdict.Source: 3) Inter-Procedural and External-Call… · p. 6

4) Rule Execution and Reporting:

  • Reports include the CWE category, target function, source location, matched rule, supporting facts, and derivation trace; fixed facts and rules yield repeatable verdicts.Trace completeness establishes neither full program recovery nor whole-program vulnerability reachability.Source: 4) Rule Execution and Reporting · p. 6

V. EMPIRICAL STUDY

The empirical study evaluates representation availability, vulnerability detection, component contributions, and generalizability across FormAI, Juliet, and MegaVul. It describes datasets, training, overlap analysis, and comparison methods; these passages do not report empirical outcomes.

A. Research Questions •

  • The study poses four questions covering compilation and relation recovery, detection effectiveness, component contributions, and generalizability and practical applicability.Source: A. Research Questions • · p. 6

1) Datasets:

  • FormAI contains 1,400 programs, Juliet provides paired full-program and target-function sets from 600 programs, and MegaVul contains 1,371 functions.FormAI includes 4,347 functions for IR analysis; MegaVul includes 1,171 vulnerable functions and 200 controls.Source: 1) Datasets · p. 6

2) Training Configuration:

  • The structural-fidelity corpus contains 30,000 source-IR records, and the vulnerability-oriented stage fine-tunes Qwen2.5-Coder-7B-Instruct using LoRA.Structural-feedback optimization jointly uses cross-entropy and policy-gradient terms with α = 0.1.Source: 1) Datasets · p. 6, 2) Training Configuration · p. 7

1) Datasets:

  • The vulnerability-oriented corpus starts with 13,000 FormAI records and retains 12,980 after deduplication and removal of label-conflicting groups.Source: 1) Datasets · p. 7

3) Data Leakage Analysis:

  • The overlap analysis compares normalized sources and nearest-neighbor TF-IDF AST cosine similarity, reporting no normalized exact duplicates across the evaluated comparisons.FormAI training and test sets share a generator, whereas Juliet is excluded from both training corpora.Source: 3) Data Leakage Analysis · p. 7

4) Compared Methods:

  • Comparisons include static analyzers, learned detectors, direct LLM classifiers, neural lifters, and a Direct-Facts variant; MegaVul also includes RepoAudit and industrial coding-agent baselines.Source: 4) Compared Methods · p. 7

5) Evaluation Protocol:

The evaluation protocol separates analysis completion from structural availability and defines benchmark-specific detection measures. RQ1 finds that SYMLIFT can complete vulnerability analysis with context-incomplete inputs, including when representations are partial.

C. RQ1: Compilation Gap Characterization

5) Evaluation Protocol:

  • Target-F1 uses benchmark-specific target matching, counts wrong-family and non-vulnerable alerts as false positives, and treats technical failures as no-alert outputs in the fixed denominator.A true positive must match the target family and benchmark criterion; unmatched targets are false negatives.Source: 5) Evaluation Protocol · p. 7
  • FormAI ACC counts a vulnerable program correct if any alert matches its target family, while MegaVul micro-averages TP, FP, and FN over 8,226 sample–family decisions.FormAI ACC does not penalize additional wrong-family alerts; MegaVul exact accuracy requires all six family decisions to match.Source: 5) Evaluation Protocol · pp. 7–8

C. RQ1: Compilation Gap Characterization

  • Table III compares source-level representation availability across datasets; Juliet AST recovery remains unchanged despite compilation loss, while MegaVul has lower definition-use than CFG availability.The paired Juliet comparison measures relation counts, not node correspondence; equal counts establish neither identical structure nor semantic equivalence, and neither representation is ground truth.Source: Table III · p. 8, C. RQ1: Compilation Gap Characterization · p. 8

5) Evaluation Protocol:

  • Complexity tiers use five equally weighted ranked features and contain 457 functions each; comparisons across the unbalanced tiers are descriptive, not causal.Features cover token count, branch complexity, nesting, calls, and pointer/array operations; ascending ranks use averaged ties.Source: 5) Evaluation Protocol · p. 8

C. RQ1: Compilation Gap Characterization

  • 1.17 percentage points versus 88 percentage points: context removal reduced SYMLIFT’s Juliet analysis completion rate far less than independent compilation success.FormAI analysis completion was not reported because its function-level experiment evaluated structural availability, not end-to-end vulnerability analysis.Source: Table IV · p. 8, C. RQ1: Compilation Gap Characterization · p. 8
  • On MegaVul, analysis completion exceeded independent compilation and complete-CFG/DFG rates, showing that usable partial representations can support downstream analysis.SYMLIFT enables vulnerability analysis without independent compilation; parsing success alone does not ensure usable semantic relations.Source: C. RQ1: Compilation Gap Characterization · p. 8

D. RQ2: Vulnerability Detection Effectiveness

On FormAI, SYMLIFT-Qwen-7B leads vulnerability detection, while RQ3 examines relation fidelity, runtime, and symbolic-rule contributions. IR-based lifting outperforms direct fact generation on overall IoU and Target-F1, and removing safety-aware rules most harms MegaVul Target-F1.

D. RQ2: Vulnerability Detection Effectiveness

  • 94.41% Target-F1 makes SYMLIFT-Qwen-7B the best FormAI configuration, exceeding the strongest direct-LLM and learned baselines by 24.08 and 25.81 percentage points.All four general-purpose models improve when used for semantic lifting followed by symbolic reasoning rather than direct classification.Source: D. RQ2: Vulnerability Detection… · p. 8, Table V · p. 9

E. RQ3: Component Contribution Analysis

  • 87.5%–95.0% of total analysis time is spent generating IR across evaluated neural lifters.SYMLIFT-GPT-5.6-Terra has the shortest end-to-end analysis time among the evaluated neural lifters.Source: E. RQ3: Component Contribution Analysis · p. 9, Table VI · p. 9
  • 87.83% overall IoU and 96.85% control-flow IoU are achieved by SYMLIFT-Qwen-7B when generated IR relations are compared with reference relations.The comparison normalizes generated IR, directly generated facts, and reference IR into shared relation sets.Source: E. RQ3: Component Contribution Analysis · p. 9, Figure 3 · p. 9
  • 38.67- and 28.70-percentage-point reductions in overall IoU and Target-F1 occur for SYMLIFT-Qwen-7B when Direct-Facts replaces IR.The benefit is not uniform across relation types: Gemini's Direct-Facts variant has higher base-feature and definition-use IoU than its IR counterpart.Source: E. RQ3: Component Contribution Analysis · p. 9, Table VII · p. 9
  • 10.86 percentage points is the largest Target-F1 decrease when the combined safety, counterexample, and priority rules are removed.TP remains 1006 while FP rises from 18 to 313; every evaluated rule-group removal reduces Target-F1.Source: E. RQ3: Component Contribution Analysis · p. 9

F. RQ4: Generalizability and Practical Applicability

Across MegaVul, Juliet context removal, and complexity tiers, SYMLIFT-Qwen-7B remains the strongest reported configuration, though general-purpose lifters do not improve uniformly. Division by Zero remains its principal family-level limitation, with missed cases linked to relations omitted during IR generation.

F. RQ4: Generalizability and Practical Applicability

  • 91.66% Target-F1 for SYMLIFT-Qwen-7B exceeds the strongest direct-LLM baseline by 61.65 percentage points and the strongest non-SYMLIFT baseline by 45.23 points on MegaVul.It also achieves 98.24% micro-averaged precision and 87.02% sample-level exact accuracy.Source: F. RQ4: Generalizability and Practical… · p. 10, Table X · p. 10
  • General-purpose lifters have 88.56%–90.05% precision but only 14.43%–17.85% recall on MegaVul, and their Target-F1 changes versus direct classification are not uniform.GPT-5.6-Terra and Claude-Opus-5 improve slightly, whereas Gemini-3.5-Flash and DeepSeek-V4-Flash decline.Source: F. RQ4: Generalizability and Practical… · p. 10
  • A 0.54-percentage-point change in Target-F1 accompanies Juliet context removal, and SYMLIFT-Qwen-7B outperforms the strongest direct LLM by 5.94 points on fragments.It achieves the highest Target-F1 in both Juliet Full and Fragment settings.Source: F. RQ4: Generalizability and Practical… · p. 10, Table XI · p. 11
  • 61.90% Target-F1 for Division by Zero marks the main family-level limitation.Missed cases commonly require divisor propagation through parameters, aliases, or relations omitted during IR generation.Source: F. RQ4: Generalizability and Practical… · p. 10, Table XII · p. 11
  • SYMLIFT-Qwen-7B has the highest Target-F1 in every complexity tier, with scores ranging from 90.08% to 93.44%.General-purpose lifters score higher in the High tier than in the Low or Medium tiers but remain below SYMLIFT-Qwen-7B throughout.Source: F. RQ4: Generalizability and Practical… · p. 10

VI. DISCUSSION

The discussion interprets empirical findings and examines SYMLIFT’s practical scope.

VI. DISCUSSION

  • The discussion addresses interpretation of empirical findings and the practical scope of SYMLIFT.Source: §VI · p. 11

A. Practical Implications

SymLift is presented as useful for auditing code without a full build, while its failure cases and stated limitations identify boundaries in semantic lifting, available context, vulnerability scope, and deployment cost.

A. Practical Implications

  • Function auditing, IDE-based code generation, and code review are settings where SymLift supports analysis without full-build reconstruction.Its stated mechanism is recovering sufficiently faithful local semantics.Source: A. Practical Implications · p. 11
  • IR-generation cost motivates quantization, distillation, caching, incremental generation, or efficient decoding, provided generated IR satisfies the parser contract.The proposed optimizations retain the rule library.Source: A. Practical Implications · p. 11

B. Failure Cases

  • An 8192-token output limit truncated a protocol-parsing routine’s IR, omitting division instructions and causing a false negative before symbolic reasoning.The parser retained the parseable prefix, but no Division fact could be derived.Source: B. Failure Cases · p. 11
  • Resource effects from out-of-context callees were unavailable without matching external-call summaries, preventing ReleasedOnPath derivation for acquired descriptors.The authors identify extending relevant summaries as the response, without reconstructing the complete project.Source: B. Failure Cases · p. 11
  • Collapsing a pointer’s checked and unchecked definitions into one SSA value removed MayBeNull evidence and led to a null-dereference false negative.The rules established NullGuard, but the unchecked producer and redefinition were lost during semantic lifting.Source: B. Failure Cases · p. 11

C. Limitations

  • Generated IR may omit instructions, recover incorrect types, corrupt definition-use relations, or be truncated; deterministic Datalog does not remove dependence on semantic fidelity.The authors identify stronger structural validation and uncertainty estimation as important directions.Source: C. Limitations · p. 11
  • Function-level context may omit unavailable implementations, unresolved indirect calls, or excluded functions, leaving aliases, callbacks, and global resource lifecycles challenging when context or summaries are insufficient..Source: C. Limitations · p. 12
  • The rules cover six representative vulnerability families, while comparisons vary in language, granularity, build requirements, and output semantics, and IR-generation cost remains a barrier to low-cost real-time deployment.The authors state that broader CWE coverage requires new facts and reasoning rules.Source: C. Limitations · p. 12

VII. RELATED WORK

Related work spans structured static analysis, learned and LLM-based detection, and hybrid reasoning systems. The paper reports validity boundaries for its evaluation and concludes that SymLift combines compile-free analysis with deterministic reasoning while retaining limits from generation, context, and cost.

VII. RELATED WORK

  • Structured static analyzers use program representations, while lightweight syntactic matchers reduce compilation dependence; missing declarations, types, or dependencies can block representation construction.The cited work reports false positives and missed defects, and tool combinations can improve coverage without eliminating these limitations.Source: §VII · p. 12
  • Learned vulnerability scores and LLM-generated explanations do not alone establish derivations from explicit program relations, motivating attention to evidence, localization, and evaluation representativeness.The passage names VulDeePecker, CodeBERT, GraphCodeBERT, UniXcoder, and VulPrompt among learning-based approaches.Source: §VII · p. 12
  • Hybrid systems combine symbolic analysis with learned or LLM components, but several approaches depend on frontend-recovered relations; SymLift assigns IR reconstruction to a neural lifter and vulnerability rules to Soufflé.The comparison with RepoAudit, LLMDFA, and NESA concerns evidence construction and vulnerability judgment, not compilation freedom or Datalog alone.Source: §VII · p. 12

VIII. THREATS TO VALIDITY

  • Evaluation on C code, six vulnerability families, selected datasets, and selected baselines may not generalize to other languages, projects, or vulnerability families.Fact-level IoU, text similarity, and manual inspection may not capture full behavioral equivalence; parser and rule defects or assumptions may also influence results.Source: §VIII · p. 12
  • Cross-method comparisons are limited by differences in supported analyses and resource requirements, even when methods receive the same permitted context within each benchmark condition.Duplicate and AST-similarity checks do not exclude all semantic similarity or foundation-model pretraining overlap, and manual inspection is not proof of behavioral equivalence.Source: §VIII · p. 12

IX. CONCLUSION

  • SymLift separates neural semantic lifting from rule-based inference, supports compile-free operation at analysis time, and achieves 94.41% Target-F1 on FormAI and 91.66% micro-averaged Target-F1 on MegaVul.The conclusion also reports a 0.54-percentage-point Target-F1 change under controlled Juliet context removal and names generation errors, incomplete context, and generation cost as limitations.Source: §IX · p. 12

Related papers

Loading 2610.34431v1…