PaperlayerView exact revision on arXiv

Source-linked AI summary

ret2spec: Speculative Execution Using Return Stack Buffers

Giorgi Maisuradze, Christian Rossow

arXiv:1807.10364v2cs.CR

TL;DR

The paper addresses the limited study of security risks from return-address prediction in speculative execution. It analyzes return stack buffers and develops two RSB-based attack variants, showing cross-process speculative execution and sandbox-bypassing memory disclosure in JIT environments.

  • Problem

    The security implications of return-address predictors and their return stack buffers had not been publicly documented despite broader concern about speculative execution and microarchitectural attacks.

  • Method

    The authors study RSB behavior and develop attack techniques that trigger speculative execution across processes and inside JIT-compiled WebAssembly.

  • Results

    The attacks enable arbitrary speculative code execution across processes and arbitrary memory disclosure inside a sandboxed process.

  • Takeaways & Limitations

    RSBs are a powerful attack surface, and cross-process defenses depend on flushing them at context switches while sandboxed JIT environments remain exploitable in the demonstrated setting.

  • Takeaways & Limitations

    The cross-process attack is destroyed when RSBs are flushed at context switches, and required gadgets depend on the compiler and must be found for each target.

Abstract

from arXiv · show

Speculative execution is an optimization technique that has been part of CPUs for over a decade. It predicts the outcome and target of branch instructions to avoid stalling the execution pipeline. However, until recently, the security implications of speculative code execution have not been studied. In this paper, we investigate a special type of branch predictor that is responsible for predicting return addresses. To the best of our knowledge, we are the first to study return address predictors and their consequences for the security of modern software. In our work, we show how return stack buffers (RSBs), the core unit of return address predictors, can be used to trigger misspeculations. Based on this knowledge, we propose two new attack variants using RSBs that give attackers similar capabilities as the documented Spectre attacks. We show how local attackers can gain arbitrary speculative code execution across processes, e.g., to leak passwords another user enters on a shared system. Our evaluation showed that the recent Spectre countermeasures deployed in operating systems can also cover such RSB-based cross-process attacks. Yet we then demonstrate that attackers can trigger misspeculation in JIT environments in order to leak arbitrary memory content of browser processes. Reading outside the sandboxed memory region with JIT-compiled code is still possible with 80\% accuracy on average.

1 INTRODUCTION

Modern CPU optimizations improve performance but can undermine software security through observable effects of speculative and out-of-order execution. This paper studies overlooked return-address prediction and demonstrates RSB-based attacks across processes and within sandboxed browser processes.

  • Hardware security motivation: Modern CPU optimizations, including caching, out-of-order execution, and speculative execution, can violate software security guarantees through detectable microarchitectural effects.Speculatively executed instructions are rolled back architecturally, but can still influence detectable state.
  • Return-address prediction: Return stack buffers (RSBs) predict function-return addresses by remembering recent calls, avoiding potentially expensive memory accesses.The paper investigates RSBs as the core of return-address prediction.
  • Paper contribution: The authors are the first, to their knowledge, to systematically study and demonstrate the security implications of abusing RSBs.They show that attackers can provoke RSB-based speculative execution by overflowing buffers, crafting entries before context switches, or using asymmetric call/return pairs.
  • Cross-process attack: Local attackers can poison RSBs so colocated processes speculatively execute arbitrary code, potentially exposing another user’s process memory.Operating systems may mitigate this cross-process attack by flushing RSBs on context switches, but that defense is not always active across CPU generations.
  • In-process attack: Attackers can abuse RSBs in JIT-compiled WebAssembly within a browser process, bypass memory sandboxing and read arbitrary memory addresses.This in-process attack does not require a context switch, allowing it to evade the aforementioned defense.
  • Relation to Spectre: RSB manipulation provides speculative-execution capabilities similar to Spectre while using principles orthogonal to known branch-target-buffer poisoning strategies.The paper discusses existing and new countermeasures against RSB-based attacks.

2 BACKGROUND

Modern x86 CPUs improve performance through out-of-order and speculative execution, using prediction structures such as BTBs and RSBs. RSBs predict return targets by storing recent return addresses, but limited capacity and microarchitectural underflow behavior can produce mispredictions.

  • Out-of-Order Execution: x86 CPUs translate complex instructions into micro-ops, then use a reorder buffer to execute them out of order before committing architectural state.The reorder buffer preserves original order while micro-ops wait, execute, or await retirement.
  • Speculative Execution: Speculative execution predicts conditional outcomes and indirect branch destinations so the out-of-order engine can continue along a predicted path.BTBs map branch sources to likely destinations for indirect calls and jumps.
  • Return Stack Buffers: RSBs predict returns by storing the addresses following the most recent calls; their size varies by microarchitecture, commonly reaching 16 entries.Unlike generic BTB prediction, RSB prediction exploits the paired structure of calls and returns.
  • Return Stack Buffers: RSB calls add return addresses and discard the oldest entry when full, while returns consume the top entry and decrement the stack pointer.The consumed value becomes the predicted return destination.
  • RSB Underflow: When returns exceed RSB capacity, implementations may stop predicting, switch to BTBs, or cycle through RSB entries; Intel and AMD differ in these behaviors.The paper calls cyclic behavior variant (c), and the other two behaviors non-cyclic.

3 GENERAL ATTACK OVERVIEW

The attack overview shows how attackers trigger return-prediction misspeculation, divert execution to controlled code, and observe speculative architectural effects. Different control-flow patterns apply to cyclic and non-cyclic RSBs, including overflow, exceptions, context switches, and direct overwrites.

  • Attack Structure: The attack plan has three stages: trigger return misprediction, divert speculative execution to controlled code, and modify architectural state so the effect is externally detectable.The overview labels these stages A1, A2, and A3.
  • Triggering Misspeculation: Triggering misspeculation is essential for reliably redirecting speculative execution, and the available techniques depend on the RSB’s underflow behavior.The paper distinguishes abnormal control flow for non-cyclic RSBs from overflow-based techniques for cyclic RSBs.
  • Non-Cyclic RSBs: Non-cyclic RSBs can be misaligned through exception handling, setjmp/longjmp, context switches, or direct return-address overwrites.A direct overwrite can force a return misprediction with a single store followed by ret.
  • Cyclic RSBs: Cyclic RSBs can mispredict after overflow: with N = 4, returning from E predicts H instead of D after four correctly predicted returns.Recursive functions can likewise prepare cyclic RSBs for later mispredictions.
  • Cyclic RSBs: With two recursive functions, B’s first 16 returns are predicted correctly, while remaining returns mispredict and speculate B’s call site instead of A’s.The example fills a 16-entry RSB through recursive calls to B.
  • Diverting Speculative Execution: A mispredicted return can execute code from one function in another function’s context, such as using a secret in rax to access attacker-controlled memory.This provides the mechanism for leaking the secret through speculative execution.

RSB of OS

The attack poisons return stack buffers so a victim speculatively follows attacker-chosen code, using cache effects to recover cross-process data. The evaluation targets user input and reports approximately 84% precision, requiring two password inputs.

  • Cross-process attack: After the context switch, the victim may mispredict returns to entries remaining from the attacker process.This cross-process control-flow confusion occurs because only some RSB entries are evicted during switching.
  • Feedback channel: Speculative code leaks register-dependent values through secret-dependent memory accesses that alter cache state.Flush+Reload is more accurate but requires shared memory; Prime+Probe works without shared memory.
  • Limitations: The cross-process demonstration assumes unflushed RSBs, attacker-required victim gadgets, and disabled or defeated ASLR.Flushing the RSB at context switches is identified as a limitation that destroys this attack.
  • Cross-process attack: Attackers fill RSB entries with gadget addresses before forcing a context switch to the victim process.The attack assumes knowledge of the victim address space and suitable leakage gadgets.
  • Evaluation scenario: The demonstration targets terminal programs such as bash or sudo that process character-by-character user input.The victim blocks in read until a key arrives, then resumes through a chain of returns to the main loop.
  • Evaluation scenario: 84% overall precision was achieved across 1000 pangram sentences, and two password inputs sufficed to derive a complete entered password.Characters were supplied every 50 milliseconds, corresponding to 1200 characters per minute.

5 SPECULATIVE EXEC. IN BROWSERS

The paper shows that cyclic RSB misspeculation in JIT-compiled browser code enables speculative reads beyond the sandbox, including attacker-chosen memory addresses. The attack uses recursive functions to overwrite and consume RSB entries, then recovers leaked bytes through cache state.

  • Threat model: The browser attack assumes a victim visits an attacker-controlled site and supports JIT-compiled JavaScript or WebAssembly with precise timing.The attacker may use a high-precision timer or an indirect timer source.
  • Cyclic RSB construction: Recursive functions A and B create cyclic RSB behavior that causes returns from A to mispredict to code in B.B fills the RSB through recursion, consumes its entries, and causes subsequent returns from A to target B speculatively.
  • Context confusion: Context confusion lets speculative code from B use attacker-controlled register values established by A, bypassing normal JIT address sanitization.A can set rax to an arbitrary value before returning, while speculative B executes a memory access using that value.
  • Cache-based reporting: The attack reports a leaked byte by multiplying it by 4096 and accessing the corresponding WebAssembly heap page, which is later probed through cache timing.The cached page identifies the byte value after speculation.
  • Absolute memory access: An indirect-call variant lets speculation restore an attacker-controlled heap pointer, enabling absolute memory accesses rather than only accesses relative to the WebAssembly heap.Because speculative rsp points to attacker-controlled arguments, the heap pointer can be set to 0.
  • Evaluation: ≈80% of read bytes were correct, while the measured reading speed was ≈55 bps, or 146 ms per byte.Each 1024-byte reading iteration was run 10 times and took 150 seconds on average; increasing iterations or speculations improves precision at the expense of speed.

6 COUNTERMEASURES

The paper discusses hardware, compiler, timer, and isolation defenses for RSB-based speculative attacks. It emphasizes that broad hardware restrictions can be costly, while JIT defenses must account for context confusion and future WebAssembly features.

  • Hardware defenses: Disabling speculative execution would eliminate the general problem but drastically reduce performance by making branches serializing.Blocking speculative memory accesses would also incur significant slowdown because memory accesses already bottleneck modern CPUs.
  • Hardware defenses: Flushing RSBs at every context switch can counter the cross-process attack with little expected performance slowdown.The paper argues predictions after context switches usually mispredict because RSB state is rarely shared between processes.
  • Hardware defenses: Stack-based RSB implementations do not fully prevent the browser attack because exceptions or JIT bailouts can still trigger misspeculation.The paper suggests combining hardware and compiler solutions for more reliable protection.
  • Compiler defenses: Compiler defenses include fencing call sites, replacing returns with speculation-trapping constructs, and improving memory-access sanitization.The proposed return construct preserves architectural control flow while trapping speculative execution in a loop.
  • Compiler defenses: 32-bit register use alone is insufficient because speculative execution can also modify base addresses.The paper recommends reevaluating WebAssembly features such as exception handling and threading, which may introduce additional RSB speculation or timing channels.
  • Browser defenses: Browser mitigations include reducing or randomizing timers and isolating sites into separate processes.Google’s Site Isolation is described as preventing attackers from reading across origins.

7 RELATED WORK

The paper situates ret2spec among work on speculative execution, cache side channels, and microarchitectural attacks. It distinguishes its RSB-based techniques from BTB poisoning and notes closely related contemporaneous work.

  • Speculative-execution attacks: Meltdown exploits delayed exception handling in Intel’s out-of-order engine to disclose privileged kernel data.The paper contrasts this implementation-specific flaw with Spectre’s use of speculative execution itself.
  • Speculative-execution attacks: Spectre injects arbitrary branch targets into predictors shared across processes and privilege levels on the same core.The paper presents ret2spec as targeting a different prediction mechanism.
  • Cache side channels: Cache side channels infer activity or data by distinguishing cache hits from misses through access timing.The paper reviews instruction-cache attacks on execution traces and data-cache attacks on access patterns.
  • Related attacks: Prior microarchitectural attacks also exploit BTB collisions, transactional execution, and exception-timing differences to leak or derandomize protected memory.These examples include KASLR attacks and kernel-address probing.
  • RSB-related work: Spectre Returns independently described the general RSB attack idea, while ret2spec additionally targets JIT environments and attacks that remain effective despite RSB flushing.The paper highlights arbitrary browser-process memory disclosure as its distinguishing target.

8 CONCLUSION

The paper concludes that RSBs can enable speculative attacks beyond BTB poisoning. It demonstrates cross-process arbitrary speculative execution unless RSBs are flushed at context switches, and in-process arbitrary memory disclosure from a sandboxed process.

  • Conclusions: RSBs provide attackers with a simple way to trigger speculative execution and arbitrary speculative code execution across processes.The cross-process result is qualified by RSB flushing upon context switches.
  • Conclusions: In-process RSB speculation can cause arbitrary memory disclosure from a sandboxed process.This conclusion extends the demonstrated risk from cross-process attacks to sandboxed browser settings.
Loading 1807.10364v2…