ZeroLeak: Using LLMs for Scalable and Cost Effective Side-Channel Patching

1. Introduction

• Repositories are difficult to patch because vulnerable code can be interdependent and execution time also depends on platform and compiler configuration.The authors note that reported vulnerabilities in open-source crypto libraries used by millions often go unpatched because maintainers lack expertise or resources.Source: §1 · pp. 1–2
• The study examines zero-shot LLM patching with leakage and vulnerability detectors for data-dependent timing, secret-dependent branches, and Spectre v1 gadgets.The authors describe it as the first comprehensive study of GPT, PaLM 2, and LLaMA for automatically patching these microarchitectural vulnerabilities.Source: §1 · p. 2, §1.1 · p. 2

1.1. Contributions

• The toolchain tests compiled binaries with leakage and Spectre detectors, then uses LLMs to generate source-file patches.The named tools include Microwalk, Pitchfork, Spectector, and KLEESpectre.Source: §1.1 · p. 2
• CI/CD integration retains source-code review and revision while testing binaries on a target machine to account for compiler and platform effects.The authors say this approach allows software to be continuously improved as hardware systems and software stacks evolve.Source: §1.1 · p. 2
• GPT4 patched 97% of benchmark leakages for $1.34 across 33 leaks; GPT3.5 fixed 62% at 19 times lower cost, while chat-bison and LLaMa2 patched 56% and 35%.These results are from a microbenchmark of C code compiled from known vulnerabilities.Source: §1.1 · p. 2
• Up to ∼10× faster than Spectre v1 patches from existing methods, the toolchain outperformed compiler-based techniques such as clang lfence injection.The authors present this as an efficiency advantage while retaining security.Source: §1.1 · p. 2

2.1. Constant-Time Implementations

• Constant-time cryptographic code takes the same time regardless of input size or values, but contemporary CPU features make this difficult to guarantee.Implementations must avoid branches and other operations contingent on secret data.Source: §2.1 · p. 3
• Automated constant-time verification tools exist, yet developers often overlook them because of resource constraints.Microwalk generates traces for random inputs and compares them using mutual information, but requires a per-function input template and report interpretation.Source: §2.1 · p. 3

2.2. Spectre v1

• Spectre v1 tricks a processor into speculatively executing code after a branch misprediction, potentially leaking sensitive data.A speculative access can leave a cache footprint even after the processor rolls back the mispredicted execution.Source: §2.2 · p. 3
• Spectre v1 remains difficult to mitigate because it is a hardware-level issue that software measures cannot fully protect against.Source: §2.2 · p. 3

2.2.1. Testing for Spectre-v1.

• KLEESpectre [46] models cache usage with symbolic execution to detect speculative interference and was evaluated on Kocher’s variants and several software projects.Source: §2.2.1 · p. 4
• Pitchfork [9] uses underconstrained symbolic execution and dynamic taint tracking to verify constant-time execution, while Spectector uses symbolic execution to prove speculative non-interference or detect violations.Pitchfork can verify at protocol level using function hooks; Spectector detected all 15 Kocher Spectre v1 variants.Source: §2.2.1 · p. 4

• Automated program repair studies have generally focused on syntactic and build errors, with fewer addressing security vulnerabilities and none addressing microarchitectural vulnerabilities.Source: §3 · p. 4
• DeepFix, BIFI [57], and Graph2Diff [42] target programming, syntactic, or build errors rather than security vulnerabilities.Their reported limitations include dependence on error-location prediction, remaining uncorrected errors, and difficulty with complex diff prediction.Source: §3 · p. 4
• Pearce et al. explored zero-shot LLM vulnerability repair but focused largely on basic software bugs, not microarchitectural vulnerabilities or sophisticated crypto implementations.Source: §3 · p. 4
• LLM repair studies examined Verilog bugs [2], Java vulnerabilities [56], and C buffer-overflow or pointer-dereference failures [12], among other domains.Wu et al. [56] reported that out-of-the-box Codex fixed about 20% of vulnerabilities; Charalambous et al. [12] reported up to 80% repair success.Source: §3 · p. 4
• Other work used zero-shot LLMs to suggest C# performance improvements [16] or generate hardware vulnerability assertions [25].Garg et al. [16] reported suggestions equivalent to or better than a developer in 60% of cases; Kande et al. [25] reported 4.53% correct assertions.Source: §3 · p. 4
• The related-work gap is automated repair of complex security and especially microarchitectural vulnerabilities in intricate cryptographic implementations.Source: §3 · p. 4

• The scope is software-enabled leakage through secret-dependent code or data access patterns, execution time, or speculative execution.Physically visible leakage such as power or electromagnetic emanations is excluded.Source: §4 · p. 5
• The model assumes software is free of bugs and works as intended, excluding common bugs such as buffer overflows and use-after-free.It assumes an attacker can measure execution time or collect metadata through shared components such as CPU caches.Source: §4 · p. 5
• The study uses zero-shot learning and leaves custom-trained LLMs out of scope because training from scratch is costly in time and energy and has environmental impact.Source: §4 · p. 5

• A divide-and-conquer strategy generates an algorithm function by function to manage code length and preserve coherence between pieces.The iterative process first requests a function-and-constant roadmap, then implements listed functions, and finally generates a driver for testing.Source: §5 · p. 5
• The prompt template supplies context and instructions to reduce redundant explanations and guide iterative code generation.The figure marks prompt variables in bold; the process manually initializes constants such as S-box entries and checks ciphertext and decryption results.Source: §5 · p. 5, Figure 3 · p. 5
• Functional correctness is checked by verifying expected ciphertext and matching plaintext after decryption.Constants such as S-box entries are initialized manually according to the target algorithm’s standards.Source: §5 · p. 5

6.1. Evaluating Side-Channel Leakage

• Microwalk estimates leakage using mutual information and localizes sources in code or binaries, while tracking execution paths and memory accesses across secret inputs.Repeated binary executions improve its mutual-information estimate but can increase computational cost.Source: §6.1 · p. 6

6.2. Patching for Constant-timeness

• Constant-time patching is difficult because complex implementations may have multiple leakage points, making single-shot LLM fixes unreliable and localization essential.Prompts must distinguish input-independent execution from the unrelated goal of O(1) runtime complexity.Source: §6.2 · p. 6
• The patching loop analyzes vulnerable functions, prompts the LLM, and feeds syntax, correctness, and leakage failures back for another attempt.Iterations can be bounded by resource limits.Source: §6.2 · pp. 6–7
• Leakage reports are translated into natural language with the identified code lines and mechanism, while prompt instructions preserve function compatibility and avoid undefined additions.Tool- and language-specific workarounds address compatibility problems such as unsupported JavaScript features.Source: §6.2 · p. 7
• Prompts target secret-dependent memory accesses, conditional execution, or loop sizes, and use parser, compiler, or test feedback to correct invalid patches.Functional correctness is checked with assertions in the test benches.Source: §6.2 · p. 7
• The workflow uses semi-adaptive template prompts because the available options are limited; more adaptive prompt design is left to generative AI or chained LLMs.When context reaches the model's token limit, earlier exchanges are dropped while retaining the system prompt and original function.Source: §6.2 · p. 7

7.1. Finding Spectre-v1 Gadgets

• Pitchfork, Spectector, and KLEESpectre cover different detection properties; the study considers their common Spectre-v1 PHT variant.Pitchfork uses speculative constant time, Spectector uses speculative non-interference, and KLEESpectre extends symbolic execution with cache features.Source: §7.1 · p. 8

7. Mitigating Spectre-v1

• ZeroLeak applies LLMs to patch functions with known leakage points in the transient domain, with mitigations verified on the compiled binary.This targets low-overhead software fixes while accounting for compiler effects that can remove source-level mitigations.Source: §7.2 · p. 8, §7 · p. 8

7.2. Patching Spectre-v1 Gadgets

• For patching, symbolic-execution results localize leakage at assembly level, which is traced back to source code before a prompt targets the speculative conditional.Prompts replace “constant-time” with “secure” and describe speculative execution even when a branch condition is false.Source: §7.2 · pp. 8–9

8. Experiments

• The experiments used fixed model versions and specified Microwalk, compiler, and hardware configurations; the LLM comparison covered nine models.Only LLaMA2 with 70B parameters was entirely open-source among the nine evaluated models.Source: §8 · p. 9, Table 1 · p. 9

8.1. Generating Constant Time AES in C

• GPT4’s AES-128 patch made KeyExpansion, SubBytes, MixColumns, and InvSubBytes constant time, but InvMixColumns remained leaky.The patch removed multiplication-table accesses and changed the MixColumns design.Source: §8.1 · p. 9, Figure 8 · p. 10

8.2. Patching Spectre-v1 Gadgets

• ZeroLeak had the best performance among compared Spectre-v1 mitigations and was verified secure by multiple tools; nine of 16 cases had overhead of two cycles or less.Inline lfence was not possible for one case using a ternary operator as an array index.Source: §8.2 · p. 10, Figure 9 · p. 10, Table 2 · p. 11

8.3. Comparison of LLMs

• GPT4 patched 97% of benchmark leakage points for $1.34 across 33 leaks; GPT3.5 fixed 62% while costing 19 times less than GPT4.The comparison evaluated patching across the benchmark’s leakage types.Source: Table 3 · p. 11, §8.3 · p. 11
• Text-davinci performed similarly to chat-bison, which cost 38 times less; increasing bison-model temperature did not improve generated-code quality.The bison models often returned unchanged code, while code-specific models performed worse than several general chat models.Source: §8.3 · p. 12, Figure 10 · p. 12
• Binary-level dynamic analysis rejected generated code when compilation introduced conditional jumps, even when the source lacked an if statement or ternary operator.The framework tests the compiled binary rather than relying on source appearance alone.Source: §8.3 · p. 12

8.4. Case Study 1: Patching a Real World Spectre-v1 Gadget

• On an OpenSSL Spectre-v1 gadget, GPT4 generated an index-masking patch in its third iteration, directing malicious indices to element zero during speculation.The patch also applied the mask to the remaining code to preserve functional correctness.Source: §8.4 · p. 12, Figure 11 · p. 13

8.5. Case Study 2: Patching Javascript Crypto Libraries for Constant-Timeness

• GPT4 patched 117 of 127 unique leakage points across three JavaScript libraries in about 90 minutes, while patching also introduced a new branch leakage.The libraries were aes-js, base64-js, and node-forge.Source: Table 4 · p. 13, §8.5 · p. 13

9. Ethical Questions with AI Contributions

• Generated patches were not pushed to security-critical libraries; instead, the authors planned to share them with library maintainers for review and disclose their AI origin.The authors cite ethical and legal concerns.Source: §9 · p. 13

10. Conclusion

• ZeroLeak patched side-channel leakage in C and JavaScript, including three widely used libraries in under 1.5 hours and a real-world OpenSSL Spectre-v1 instance.The authors report the library patches cost only cents per patch.Source: §10 · p. 14

Appendix A. Microbenchmark of leaky functions compiled from literature

• The source comments attribute examples to Pitchfork, CacheD, CacheAudit, FlowTracker, Blazer, ctgrind, SC-Eliminator, and DATA.The comments also identify textbook 3-way cipher code and bubblesort and selection-sort examples from CacheAudit.Source: Appendix A · pp. 16–17
• The cited microbenchmark fragments show an indexed lookup returning z[x % 3] and an option-controlled conditional whose condition is option > 3, illustrating lookup and conditional code.Source: Appendix A · p. 16
• Branch-case fragments include loops and array comparisons, such as iterating while i < low and comparing neighboring array elements.Other snippets include insertion-sort-style traversal and a comparison of a[j] with a[j+1].Source: Appendix A · pp. 16–17