Source-linked AI summary
The History Is the Detector: Executing CVE Patch History, End-to-End
Qiushi Wu, Kevin Eykholt, Youngja Park, Xiaokui Shu, Dhilung Kirat, Douglas Lee Schales, Ian Molloy
TL;DR
CVE records and fixing commits contain reusable vulnerability knowledge, but that knowledge is not directly usable for automated detection. BUGSTONE-E2E mines verified fixes into deployable rules and applies them through a cost-ordered funnel with runtime validation and two-sided patch testing. Across 14 programs, the resulting system produced runtime evidence for 644 findings from a knowledge base of 1,033 rules in 172 skills.
Problem
CVE fixes explain unsafe conditions but do not directly provide reusable detectors for recurring vulnerabilities across projects and languages.
Method
BUGSTONE-E2E mines verified fixing commits into CWE- and language-organized rules, then funnels candidates through deterministic filtering, LLM inspection, runtime validation, and verified remediation.
Results
Across 14 projects, BUGSTONE-E2E produced runtime evidence for 644 findings using 1,033 rules across 56 CWE families packaged into 172 skills.
Takeaways & Limitations
Historical vulnerability fixes can be transformed into executable detection knowledge and applied with increasingly strong evidence requirements.
Takeaways & Limitations
The system is limited to recurring patterns with suitable local anchors and may miss one-off, configuration, global, or stateful vulnerabilities.
Abstract
from arXiv · showhide
Public vulnerability databases collect rich information about known software flaws, including their weakness types, affected components, and related patches. Fixing commits provide the exact code changes that removed these flaws. While these records capture why the original code was unsafe, they are documented mainly for human inspection rather than automated reuse. Consequently, the same unsafe conditions may still exist elsewhere in code without a known advisory, leaving much of this detection knowledge unused. We present BUGSTONE-E2E, a framework that transforms vulnerability history into executable detection rules and validates their findings. First, BUGSTONE-E2E mines reusable rules from verified fixing commits, capturing scan anchors, fix semantics, and CVE provenance and organizing them by CWE and language. Second, detection follows a funnel-shaped pipeline: early stages process a large pool of candidates using lightweight analysis, while later stages apply increasingly capable and expensive models to a shrinking set of targets. Specifically, BUGSTONE-E2E first enumerates call sites matching rule anchors using Tree-sitter, then removes benign sites using lightweight heuristics without LLM calls. Next, LLM-based agents inspect the remaining candidates guided by the rule. Following this inspection, the system re-triages surviving candidates and builds runtime verifications, then generates scope-checked patches validated via two-sided differential tests. Using 19,325 high-severity CVEs from 2022 to 2026, BUGSTONE-E2E identifies 2,710 fixing commits and constructs 1,033 detection rules across 56 CWE families, packaged into 172 skills. When applied across 14 programs, it produced runtime evidence for 644 findings. These results demonstrate that CVE history can be turned into an executable workflow, transforming past vulnerabilities into reproducible detection and repair.
1 Introduction
BUGSTONE-E2E addresses the gap between rich CVE patch history and reusable automated detection by mining fixing commits into rules and executing them through staged analysis, runtime validation, and verified repair.
- CVE fixes explain why code was unsafe, but their lessons rarely become detectors for finding the same conditions elsewhere.
- Tree-sitter enumeration and deterministic filtering remove benign candidates before LLM agents inspect taint flows, callers, guards, and sanitizers.
- Runtime validation distinguishes confirmed exploitable findings from non-exploitable or untestable candidates using observable execution markers.
- Phase D accepts a repair only when the exploit fails after patching and succeeds again after reverting the patch.
- 19,325 high-severity CVEs yielded 2,710 verified fixing commits and 1,033 rules across 56 CWE families, packaged into 172 skills.
- BUGSTONE-E2E mines verified fixing commits into reusable rules containing vulnerability anchors, fix semantics, and provenance.
2 Background and Problem
Recurring vulnerability conditions motivate CVE-derived detection, while existing static, dynamic, and LLM-based techniques each leave gaps in rule creation, context, or validation.
- Known vulnerability patterns recur across code locations, projects, weakness classes, and programming languages.
- Thirteen CVEs support the Python command-injection pattern in which external data reaches shell APIs without sufficient quoting or sanitization.
- Static analyzers scan efficiently once rules exist, but encoding new vulnerability patterns still requires analyst effort and contextual false-positive review.
- Dynamic techniques provide stronger behavioral evidence, but they depend on reaching vulnerable behavior and having an oracle that recognizes it.
- LLM-based systems add semantic context through retrieved knowledge, static-analysis results, code property graphs, or agentic workflows.
- BUGSTONE-E2E targets recurring API-misuse patterns visible near call sites, excluding one-off design flaws, configuration errors, and whole-program symbolic cases.
3 System Overview
BUGSTONE-E2E converts vulnerability history into skills, applies them to target projects, and progressively raises the evidence requirement through runtime validation and verified remediation.
- The pipeline has three parts: historical vulnerability knowledge becomes reusable skills, skills identify plausible static candidates, and survivors undergo validation and remediation.
- Phase C re-triages Phase B survivors and records runtime evidence such as sink reachability, sanitizer violations, or crashes.
- Only findings satisfying the exploit-confirmation criterion are marked CONFIRMED_EXPLOITABLE, while failed environment construction remains distinct from non-exploitability.
- Phase D generates a minimal scope-checked patch and requires the proof of concept to fail after patching and succeed after reversion.
4 Design
The design addresses rule mining, scalable semantic analysis, and evidence-based validation by converting verified fixes into deployable skills and funneling candidates through increasingly expensive checks.
- Research Challenges: BUGSTONE-E2E must recover reusable vulnerability conditions from incomplete CVE records and patches without overfitting to individual fixes.
- CVE-to-Skill Pipeline: 19,325 high-severity CVEs are surveyed as the input to the CVE-to-skill pipeline.
- CVE-to-Skill Pipeline: 1,033 rules are organized into 172 deployable skills after CVE enrichment, fix recovery, rule synthesis, and consolidation.
- Research Challenges: It decomposes analysis so deterministic indexing and filtering reduce the candidates requiring expensive semantic reasoning.
- Research Challenges: Runtime evidence and two-sided patch testing raise the burden of proof beyond model judgments about findings or repairs.
- CVE-to-Skill Pipeline: Rule extraction identifies vulnerable operations, scan anchors, unsafe conditions, and the changes that remove them, rejecting unsupported or nongeneralizable rules.
- CVE-to-Skill Pipeline: Consolidation reduces 1,757 rules across 266 CWE categories to 1,033 rules across 56 CWE families while preserving distinct conditions.
- CVE-to-Skill Pipeline: Each skill combines rule-specific anchors and conditions with Phase B semantic checks and an inexpensive deterministic filtering script.
5 Evaluation
BUGSTONE-E2E converts CVE-fix history into consolidated, language-specific detection skills and evaluates their efficiency, coverage, and reproducibility. The evaluation shows substantial rule recovery and candidate reduction, while exposing scope boundaries and stochastic variation in semantic verification.
- CVE Knowledge Construction: 1,033 production rules across 56 CWE families were organized into 172 language-specific detection skills after consolidating 1,757 rules across 266 CWE categories.The pipeline began with 19,325 high-severity CVEs, resolved 5,902 to open-source projects, and verified 2,710 fixing commits.
- CVE Knowledge Construction: 766 of 2,710 verified fixing commits, or 28.3%, came from repository-history recovery beyond direct commit references.This recovery path expanded usable patch evidence, although 3,192 resolved records still lacked a verified fixing commit.
- Scanner Efficiency and Task Decomposition: 60.6% of candidates were removed before model-based verification, reducing the aggregate Phase A pool from approximately 745K to 293K across 15 targets.Filter synthesis additionally removed 261 candidates, or 32.1% of observed false positives, for seven of ten high-volume skills.
- Run-to-Run Reproducibility: Across ten identical Phase B runs, individual outputs contained 125–147 findings, but their union reached 243 distinct locations while only 56 appeared in every run.At the candidate level, 93.1% received the same verdict across runs, indicating a stable core alongside a stochastic tail.
- Run-to-Run Reproducibility: Three runs recovered 75.8% and five recovered 85.7% of the ten-run finding union, while an extrapolated 302 Phase B-reportable locations suggested ten runs reached about 80.4%.The estimate applies only to locations reachable by the evaluated rule set and candidate stage, not to the true number of Pillow vulnerabilities.
- Run-to-Run Reproducibility: A single gpt-5-mini run cost approximately $161 versus about $900 for GPT-5.5 and $869 for GPT-5.6-sol, while reporting comparable or larger Phase B finding counts.Across ten gpt-5-mini runs, coverage of stronger-model finding sets rose to 97–99%.
6 Real-World Bug Detection
BUGSTONE-E2E evaluates real-world detection through nested stages from filtered candidates to runtime evidence, while documenting false-positive patterns and model-volume trade-offs. Across 14 targets, runtime evidence supports 644 findings, with substantial untested backlog affecting the reported lower bound.
- Detection outcomes: 644 findings carry runtime evidence across 14 targets, while every verification tier is strictly stronger than the preceding tier.Runtime evidence means a sanitizer report, crash, or controlled-sink signal; Live and CONFIRMED_EXPLOITABLE impose stricter requirements.
- Detection outcomes: 2,933 Phase B findings were deduplicated, 2,125 reached Phase C, and 644 received runtime evidence.The remaining findings include rejected cases and findings with static reachability but no runtime signal.
- Detection outcomes: 173,724 post-filter candidates reach Phase B after deterministic filtering removes benign sites without model calls.Phase B follows callers, weighs guards on taint paths, and returns BUGGY only with code evidence.
- Detection outcomes: Runtime-evidence rates vary sharply by target and weakness class, with FreeBSD contributing 459 runtime signals from 2,189 Phase B findings.PyTorch contributes 95 runtime observations from 195 Phase C findings, while guava has none and openssh has one unverified finding.
- Error analysis: False positives mainly arise from benign wrappers, machine-generated code, and incomplete taint chains.Phase B identifies effective guards, bounded index arithmetic, or absent user-controlled flow to explain these dismissals.
- Model and verification trade-offs: Larger or stronger scan models mainly change which candidates Phase B flags, while increasing raw volume and Phase C compute demand.The resulting workload contributes to an 808-finding Phase C backlog, so model choice affects verification cost more than runtime-supported volume.
7 Discussion
The discussion bounds BUGSTONE-E2E's scope: it targets locally anchored recurring vulnerabilities, while runtime validation and remediation remain constrained by environments, cost, and project-specific review. The authors identify broader coverage, richer context, reusable environments, and human-in-the-loop evaluation as future directions.
- Detection scope and completeness: BUGSTONE-E2E does not target one-off design flaws, configuration errors, or global and stateful invariants without suitable local anchors.CVE-derived rules can also miss equivalent APIs or conditions absent from source fixes.
- Detection scope and completeness: Conservative filtering, bounded reasoning, and stochastic Phase B verdicts can produce false negatives, especially when deeper interprocedural context is required.Reruns may recover some semantic-analysis misses but cannot recover vulnerabilities outside deployed rule and candidate coverage.
- Runtime validation: Runtime confirmation depends on reproducible builds and test environments, so unexercised plausible findings should be treated as unverified rather than benign.Complex dependencies, unavailable inputs, platform-specific behavior, and partial dynamic coverage constrain Phase C.
- Remediation guarantees: Phase D's differential oracle shows that a patch blocks the demonstrated PoC and that reverting it restores the behavior, but not completeness or regression freedom.Project-specific testing and substantial human review remain necessary to assess maintainability and broader repair correctness.
- Future directions: Future work proposes broader CVE coverage, rule auditing, richer interprocedural context, reusable build environments, and more human-in-the-loop remediation evaluation.These directions directly address the stated limits in generality, detection coverage, runtime confirmation, and patch assessment.
8 Related Work
Related work spans static rule-based analysis, vulnerability signatures, learned prediction, LLM-guided detection, agentic repair, exploit generation, and public vulnerability infrastructure. BUGSTONE-E2E distinguishes itself by combining CWE/API-level rules with staged execution, executed exploit confirmation, and evidence-grounded remediation.
- Static and query-based analysis: Static analyzers scale after suitable queries exist, but query creation and maintenance remain manual bottlenecks.Recent systems synthesize or agentically generate queries from CVE metadata and validate findings through exploit synthesis.
- Vulnerability clone and patch-signature detection: Vulnerability-signature systems reuse code, component, or patch-line history, whereas BUGSTONE-E2E matches CWE- and API-level rules without requiring seed-code cloning.Its confirmation criterion additionally requires an executed exploit.
- Learning-based vulnerability prediction: Learning-based systems predict vulnerability signals from learned representations, while BUGSTONE-E2E uses mined rules and staged evidence-driven analysis.The related work also includes agents trained or distilled for evidence checking and localization.
- LLMs for vulnerability detection: LLM vulnerability systems combine models with static analysis, graphs, retrieved knowledge, or repository-scale agent pipelines.BUGSTONE-E2E belongs to this broader agentic direction while fixing its candidate set before model execution and using runtime confirmation.
- Agentic repair and validation: Agentic repair and execution-based patch acceptance provide prior art, but adversarial reports and stricter oracles expose fragile repair acceptance.BUGSTONE-E2E extends this setting from findings without predefined PoCs or validation environments by constructing and checking runtime evidence itself.
- Exploit generation and determinism: Exploit-generation studies show that agent results vary across runs and inference backends, motivating fixed candidate sets and reproducibility measurement.BUGSTONE-E2E fixes Phase A candidates before model calls and measures run-to-run reproducibility.
- Security taxonomies and public vulnerability data: BUGSTONE-E2E builds on public CVE records, NVD metadata, CWE taxonomy, and OSV advisories as infrastructure for its rule-mining workflow.These resources support transforming vulnerability history into reusable detection knowledge.
9 Conclusion
BUGSTONE-E2E turns vulnerability history into reusable detection knowledge and applies it through staged detection, runtime validation, and candidate remediation. Its 1,033 rules across 56 CWE families support 644 runtime-evidenced findings across 14 projects, grounding stronger claims in observable evidence.
- Conclusion: 1,033 rules across 56 CWE families are deployed through 172 skills and applied across 14 projects.The rules are consolidated from vulnerability history and organized for staged detection and validation.
- Conclusion: 644 findings receive runtime evidence, supporting the use of historical fixes as reusable knowledge for finding and validating future vulnerabilities.The workflow concentrates increasingly expensive analysis on progressively fewer candidates.