PaperlayerView exact revision on arXiv

Source-linked AI summary

CodeMechanic: Bug-Property-Guided Program Mitigation

Han Zheng, Rafaila Galanopoulou, Ilia Shumailov, Tianqi Fan, Aiden Hall, Dominik Maier, Mathias Payer

arXiv:2608.22275v1cs.SEcs.CR

TL;DR

Automated repair agents can generate risky patches because PoC-only validation may accept unrelated behavior changes. CodeMechanic instead reconstructs and validates spatial memory-safety properties before inserting a local fail-stop guard, producing substantially more plausible and semantically equivalent patches at lower cost.

  • Problem

    Open-ended LLM repairs validated by PoC replay can silence observed crashes through unrelated changes, while crash sites often lack the context needed to reconstruct memory-safety boundaries.

  • Method

    CodeMechanic combines two-dimensional static and dynamic context, in-prompt debugging knowledge, and property validation to generate a local guard before the dangerous access.

  • Results

    47.6% more plausible patches were produced than by the best competitor using 9% of the tokens, and manual audits found 240% and 325% more semantically equivalent patches than PatchAgent and San2Patch.

  • Takeaways & Limitations

    The temporary mitigation trades availability for security by terminating execution on a violated boundary while developers prepare a permanent repair.

  • Takeaways & Limitations

    The evaluation uses a selected benchmark and does not establish that every generated patch is safe for automatic production deployment.

Abstract

from arXiv · show

Automated testing discovers vulnerabilities faster than developers can investigate and repair them, leaving an interval in which known memory corruptions remain exploitable. End- to-end LLM repair agents can shorten this interval, but they synthesize open-ended code changes and commonly validate them only by replaying a proof of concept (PoC). This weak oracle accepts patches that silence the observed crash by changing unrelated behavior, making unintended deployment risky. We present CodeMechanic, a bug-property-guided system for generating constrained mit- igations for spatial memory corruption. Instead of asking an LLM to generate a permanent repair, CodeMechanic reconstructs the violated memory-safety property from the crash, validates the dereferenced pointer and its buffer range, and inserts a local fail-stop guard before the dangerous access. The guard terminates execution when the boundary check fails. The resulting mitigation deliberately trades availability for security: it can convert potential remote code execution into controlled termination while developers investigate the root cause and prepare a permanent repair. CodeMechanic combines a two-dimensional static and dynamic context extractor with in-prompt debugging knowledge and stepwise val- idation to limit the effect of LLM errors. On 101 real-world ARVO bugs, the first attempt of CodeMechanic produces 47.6% more plausible patches (i.e., patches that pass PoC- replay validation) than the best baseline while using 91% fewer tokens. Manual audit further shows that CodeMechanic produces 3.4x - 4.3x more patches semantically equivalent to developer-written repairs.

1 Introduction

CodeMechanic addresses the backlog and risk of automated vulnerability repair by constraining LLM-generated mitigations for spatial memory corruption. It reconstructs memory-safety properties from richer context, validates them, and produces local guards with strong reported gains over AVR baselines.

  • Automated vulnerability discovery now outpaces developers’ ability to triage and resolve bugs, including approximately 1,500 unresolved Linux kernel reports.
  • PoC-only validation can accept arbitrary patches that silence crashes by changing unrelated functionality, making unrestricted LLM repairs risky.
  • CodeMechanic constrains mitigation patches to a validated local guard derived from the violated spatial memory-safety property.
  • Its two-dimensional context combines source-level code views with live debugger data, sanitizer metadata, and variable types and values.
  • Embedded debugging knowledge guides buffer validation, macro resolution, tool use, and error handling during mitigation generation.
  • 47.6% more plausible patches were produced than by the best competitor using only 9% of the tokens, while audits found 240% and 325% more semantically equivalent patches than PatchAgent and San2Patch.

2 Background

The background motivates temporary, bounded mitigations between open-ended LLM repair and template-only constraint repair. CodeMechanic targets rapid mitigation of spatial memory corruption while preserving permanent repair as the desired endpoint.

  • Memory corruption remains a critical and actively exploited vulnerability class, including more than 70% of Chromium high-severity security issues.
  • Permanent repair remains the desired endpoint, but a targeted local guard can block a known dangerous access while developers prepare it.
  • Automated bug-finding has produced more security-impacting vulnerabilities than developers can address, leaving over 1,500 Linux kernel bugs unfixed.
  • PoC replay can mark functionality-breaking changes as plausible when tests do not exercise the modified region, so plausible patches require human audit.
  • Constraint-based repair reduces patch freedom but often assumes an explicit bug location, lacks source-level comprehension, or produces templates that may not compile.
  • CodeMechanic uses an LLM for source-level expression recovery while restricting output to a validated boundary guard, aiming for rapid mitigation with bounded side effects.

3 Motivation and Problem Definition

CodeMechanic defines a narrow mitigation problem: reconstruct the spatial memory-safety boundary from a crash and prevent the observed unsafe access with predictable termination. Its design combines code and runtime context, debugging guidance, and explicit property validation.

  • The goal is not permanent repair but a narrow guard that prevents the observed unsafe memory access with predictable termination.
  • PoC-based validation may confuse arbitrary functionality changes with repairs because it checks only whether one observed failure disappears.
  • Reconstructing a spatial boundary requires the pointer, buffer start, and buffer end; a crash-site snippet alone may omit these expressions.
  • LLMs may repeatedly invoke debugging tools or fail to resolve macros and interpret errors without embedded debugging guidance.
  • CodeMechanic combines static and dynamic context extraction with in-prompt human knowledge to recover expressions and validate a local mitigation.
  • For spatial corruption, its boundary condition flags a dereference when ptr lies before buf_begin or when ptr plus the dereferenced object size exceeds buf_end.
  • The guard is inserted before dereference and terminates execution on violation, deliberately converting potential corruption into denial of service rather than claiming permanent repair.

4 Design of CodeMechanic

CodeMechanic combines code and runtime context extraction with embedded debugging knowledge to recover validated pointer and buffer expressions. It then instantiates a boundary condition and inserts a local terminating guard through staged, constrained patch generation.

  • Architecture: CodeMechanic combines code-view and data-view context extraction with in-prompt human knowledge to recover candidate pointer and object-range expressions.The decomposition assigns source-level inference to the LLM while explicit validation supports the mitigation contract.
  • Code View Context: Macro resolution expands obscured symbols, while buffer inference searches related references when buffer boundaries are distant from the crash site.These tools help connect crash-site expressions to their underlying variables and candidate buffers.
  • Data View Context: ASan metadata and GDB-based analysis validate pointer values and buffer ranges against fault addresses and runtime memory information.For stack overflows without precise metadata, the system measures pointer spans and searches locals and arguments for matching ranges.
  • Data View Context: Pointer type analysis accounts for dereference width, because an in-bounds pointer can still overflow when its pointed-to type accesses multiple bytes.The analysis supplies type information alongside raw pointer expressions.
  • In-Prompt Human Knowledge: Embedded debugging workflows guide tool ordering and error handling, including buffer-size derivation, candidate-address validation, and macro resolution after symbol errors.Few-shot prompts reduce ineffective repeated calls and misinterpretation of tool errors.
  • Bug-Property-Guided Mitigation: Validated expressions instantiate a boundary condition, which a second agent inserts as a local fail-stop guard rather than rewriting arbitrary program logic.The first agent constructs the constraint, while the second adapts the guard to local C or C++ context and compilation requirements.
  • Stepwise Validation: Validation failures are routed by cause: compilation errors repeat guard insertion, whereas persistent crashes rerun boundary-condition construction.This separates repetition between the two agents’ distinct responsibilities.

5 Implementation

The implementation prepares reproducible debugging environments, validates intermediate outputs step by step, and separates the mitigation framework from containerized reproduction environments.

  • System Implementation: CodeMechanic accepts a reproduction environment and crashing PoC file, then outputs key-variable analysis and a candidate patch.The implementation is approximately 5,500 lines of Python and currently uses the ARVO input format.
  • Validation: Step-by-step validation resolves the pointer before inferring buffer bounds and generates the guard only after intermediate checks pass.Failed steps are retried or halted when no progress is made.
  • Debugging Environment: A custom Clang wrapper enables full debugging information with -O0 -g despite benchmark builds that strip symbols through aggressive optimization.This supplies pointer types and memory addresses needed for debugging.
  • Debugging Environment: CodeMechanic builds GDB 12.1 from source to support DWARF formats ranging from version 3 to 5 across Ubuntu and Clang environments.The selected version is the minimum supporting all formats used in the evaluation.
  • Deployment: The mitigation framework runs on the host and interfaces with reproduction environments through Docker, separating mitigation logic from bug reproduction.This avoids co-located dependency installations and improves deployment flexibility.

6 Evaluation

CodeMechanic is evaluated against PatchAgent and San2Patch on 101 ARVO vulnerabilities using plausible-patch counts, cost, failure stages, and manual correctness audits. It generally improves patch yield and cost efficiency, but context extraction and boundary inference remain important limitations.

  • Plausible-Patch Count: 37.2 plausible patches per single run gives CodeMechanic 47.6% more than San2Patch1 and nearly three times PatchAgent1.With five attempts, CodeMechanic still leads PatchAgent and San2Patch by 41.7% and 13.3%, respectively.
  • Cost Efficiency: CodeMechanic1 costs $2.75 for all 101 bugs, while San2Patch1 costs $21.3; in five-attempt mode, CodeMechanic costs $9.90 versus San2Patch’s $106.39.The five-attempt CodeMechanic configuration produces 51 plausible patches, while San2Patch produces 10% fewer; PatchAgent costs $7.92 but produces fewer patches.
  • Attempts: CodeMechanic’s first attempt outperforms PatchAgent across all five attempts, while adjustable attempt counts let users trade cost for performance.The evaluation supports increasing attempts beyond the tested five-attempt configuration.
  • Agent Complementarity: Combining CodeMechanic with an open-ended repair agent covers 41% more bugs than CodeMechanic alone for an additional $0.93.Across agents, 81 distinct bugs receive plausible patches, but only 8 receive them from every agent.
  • Vulnerability Categories: Stack-based overflows are most challenging, whereas heap-based overflows favor CodeMechanic and global overflows yield nearly identical CodeMechanic results across attempt settings.CodeMechanic produces at most two patches for eight stack cases, about 60% and 14% more heap patches than PatchAgent and San2Patch, and 8.6 versus 9 global patches in single- versus five-attempt modes.
  • Failure Analysis: Context extraction fails for 23 of 101 bugs, with about 42% of buffer analyses failing because definitions are often distant from dereferences.When all required variables are recovered, CodeMechanic generates plausible mitigations for 37 of 45 bugs, or 82%, within three attempts.
  • Correctness Audit: Among 51 CodeMechanic plausible patches, 34 are non-IP mitigations and 17 are semantically equivalent to developer repairs, showing PoC replay alone overestimates safety and equivalence.CodeMechanic’s local guard terminates on violation, trading potential exploitation for an explicit availability failure; incorrectly inferred boundaries can still terminate valid executions.

7 Related Works

CodeMechanic builds on automated bug discovery and LLM-based vulnerability repair while narrowing the repair space to constrained mitigation. Its approach differs from unrestricted repair systems by using bug reports and runtime debugging context to guide safer local guards.

  • Bug Discovery: Automated bug-finding techniques, especially fuzzing combined with sanitizers, generate reproducible reports that support developer analysis and bug-property-guided mitigation.CodeMechanic adopts bug reports and corresponding environments from OSS-Fuzz.
  • LLM-Based Automated Vulnerability Repair: PatchAgent and San2Patch use sanitizer logs for localization and root-cause analysis, but pursue unrestricted LLM-generated repairs validated commonly by PoC replay.VulDebugger additionally incorporates dynamic debugging, while CodeMechanic takes a different approach.
  • LLM-Based Automated Vulnerability Repair: CodeMechanic occupies a middle ground by using an LLM to recover source expressions while restricting output to a validated boundary guard at the crash site.Its two-dimensional context extractor combines code and data context, and in-prompt debugging knowledge guides tool use and error handling.

8 Threats to Validity

The evaluation is limited by validation, bug-category, and deployment boundaries. CodeMechanic targets spatial memory corruptions and defensive termination, while plausible-patch results do not establish production safety.

  • Scope of Validation: CodeMechanic’s selected-benchmark results do not establish that every generated patch is safe for automatic production deployment.The authors identify this as a limitation of the evaluation scope.
  • Internal Threats: Sanitizer metadata identifies the nearest valid buffer region but does not guarantee that it is the intended buffer.The authors use a conservative 2048-byte ASan redzone to reduce possible misclassifications.
  • External Threats: Runtime-specific constants can pass PoC validation yet fail on other executions, leaving the bug exploitable and requiring human review.This limitation concerns generated guards that embed absolute addresses or runtime-specific sizes.
  • External Threats: CodeMechanic is currently focused on spatial memory corruptions, while temporal corruptions are hindered by the lack of reliable object-liveness oracles.The paper motivates this scope by the security severity and prevalence of spatial memory errors.
  • Defensive Termination: The mitigation intentionally terminates execution on boundary violations, trading availability for security by converting potential remote code execution into denial of service.Intentional termination is described as a defensive policy for exploitable memory corruptions.
  • Human Debugging Knowledge: Prompts use abstract debugging instructions developed from ten bugs per category, and evaluation suggests that this knowledge transfers to a broader benchmark of 101 bugs.The prompt design aims to limit overfitting while guiding tool invocation and error handling.
  • Validation Oracle: PoC replay confirms only that the sanitizer-reported error disappears and does not prove protection against other inputs or preservation of valid behavior.The authors therefore manually audit plausible patches.

9 Conclusion

CodeMechanic presents constrained, bug-property-guided mitigation for spatial memory corruption rather than permanent repair. The framework is intended to reduce the interval during which disclosed vulnerabilities remain exploitable and to support defensive security techniques.

  • Conclusion: CodeMechanic reconstructs and validates a boundary property before inserting a local guard that temporarily mitigates spatial memory corruption.The design trades availability for security while developers prepare a permanent repair.
  • Conclusion: The framework targets the mitigation-generation problem rather than discovering new vulnerabilities.Its stated purpose is to reduce the period during which a disclosed vulnerability remains exploitable.
Loading 2608.22275v1…