PaperlayerView exact revision on arXiv

Source-linked AI summary

Not So Fast: Analyzing the Performance of WebAssembly vs. Native Code

Abhinav Jangda, Bobby Powers, Emery Berger, Arjun Guha

arXiv:1901.09056v3cs.PL

TL;DR

The paper addresses whether WebAssembly performance generalizes from small scientific kernels to substantial Unix applications, whose operating-system dependencies complicate browser execution. It builds BROWSIX-WASM and evaluates SPEC CPU benchmarks, finding that WebAssembly remains substantially slower than native code while identifying causes and optimization targets.

  • Problem

    Prior WebAssembly evaluations focused on roughly 100-line scientific kernels, while large Unix applications require browser-unavailable services such as filesystems, synchronous I/O, and processes.

  • Method

    The paper extends BROWSIX into BROWSIX-WASM, uses BROWSIX-SPEC, and compares Clang-native and WebAssembly-compiled SPEC CPU2006 and CPU2017 benchmarks in Chrome and Firefox.

  • Results

    Across SPEC benchmarks, WebAssembly is 1.55× slower than native in Chrome and 1.45× slower in Firefox on average, with peak slowdowns of 2.5× and 2.08×, respectively.

  • Takeaways & Limitations

    The identified performance gaps provide actionable guidance for future WebAssembly optimization efforts.

  • Takeaways & Limitations

    Some identified issues are tied to WebAssembly’s online-compilation constraint, which limits how much time implementations can spend generating optimized code.

Abstract

from arXiv · show

All major web browsers now support WebAssembly, a low-level bytecode intended to serve as a compilation target for code written in languages like C and C++. A key goal of WebAssembly is performance parity with native code; previous work reports near parity, with many applications compiled to WebAssembly running on average 10% slower than native code. However, this evaluation was limited to a suite of scientific kernels, each consisting of roughly 100 lines of code. Running more substantial applications was not possible because compiling code to WebAssembly is only part of the puzzle: standard Unix APIs are not available in the web browser environment. To address this challenge, we build Browsix-Wasm, a significant extension to Browsix that, for the first time, makes it possible to run unmodified WebAssembly-compiled Unix applications directly inside the browser. We then use Browsix-Wasm to conduct the first large-scale evaluation of the performance of WebAssembly vs. native. Across the SPEC CPU suite of benchmarks, we find a substantial performance gap: applications compiled to WebAssembly run slower by an average of 45% (Firefox) to 55% (Chrome), with peak slowdowns of 2.08x (Firefox) and 2.5x (Chrome). We identify the causes of this performance degradation, some of which are due to missing optimizations and code generation issues, while others are inherent to the WebAssembly platform.

1 Introduction

WebAssembly targets portable, browser-executable code, but prior evidence largely covered small scientific kernels rather than full applications. Evaluating larger Unix programs is difficult because browsers lack standard operating-system services, and existing approaches introduce scalability or performance limitations.

  • WebAssembly is a portable, low-level browser target designed for fast compilation and execution, safety, and interoperability with JavaScript.
  • Prior PolyBenchC results showed near-native performance on small scientific kernels, but those benchmarks were roughly 100 lines each.The suite measured polyhedral loop optimizations rather than full applications.
  • WebAssembly’s intended uses include full scientific, multimedia, visualization, simulation, interpreter, virtual-machine, and POSIX applications.Strong performance on PolyBenchC therefore does not establish performance across other application types.
  • Large Unix applications cannot be compiled directly to browser WebAssembly because they require filesystems, synchronous I/O, processes, shells, and other Unix facilities.Programmers must otherwise modify applications to avoid or mimic missing operating-system services.
  • Emscripten does not scale to SPEC-sized applications because its default MEMFS loads the entire filesystem image into memory before execution.The SPEC files are too large to fit into memory.
  • BROWSIX supports unmodified Unix applications but is JavaScript-only and adds substantial overhead that would confound WebAssembly benchmarking.Its capabilities include processes, files, pipes, and blocking I/O.

Contributions

The paper introduces Browsix-Wasm and Browsix-SPEC to run unmodified Unix programs in browsers and measure WebAssembly performance comprehensively. Its evaluation finds a substantial WebAssembly–native gap and identifies implementation and platform-related causes.

  • Browsix-Wasm: Browsix-Wasm compiles unmodified Unix programs to WebAssembly and runs them in browsers, with performance optimizations for CPU-intensive applications.The system extends Browsix with functional extensions and optimizations intended to impose virtually no overhead.
  • Browsix-SPEC: Browsix-SPEC automates detailed timing and hardware performance-counter collection for application-performance measurement.
  • Performance analysis: The evaluation uses SPEC CPU 2006 and 2017 to provide the first comprehensive WebAssembly performance analysis across these benchmark suites.
  • Performance analysis: 1.55× slower in Chrome and 1.45× slower in Firefox than native code, WebAssembly contradicts prior near-parity findings.The evaluation also finds WebAssembly averages 1.3× faster than JavaScript across SPEC CPU.
  • Root-cause analysis: The paper performs forensic root-cause analysis using performance counters and provides guidance for implementers to reduce the WebAssembly–native performance gap.
  • Root-cause analysis: 2.02× more loads and 2.30× more stores in Chrome, and 1.92× more loads and 2.16× more stores in Firefox, arise from register and addressing limitations.The paper attributes these differences to reduced register availability, sub-optimal register allocation, and limited exploitation of x86 addressing modes.
  • Root-cause analysis: WebAssembly generates more branches because it requires dynamic safety checks, and more instructions lead to additional L1 instruction-cache misses.

2 From BROWSIX to BROWSIX-WASM

BROWSIX-WASM extends BROWSIX to run WebAssembly programs while addressing BROWSIX’s process-memory and communication constraints. Its modified runtime uses shared auxiliary buffers and performance fixes to support browser-based execution.

  • Motivation and architecture: BROWSIX originally compiled native programs to JavaScript, while BROWSIX-WASM adds support for WebAssembly programs.The extension addresses BROWSIX’s limitation of targeting JavaScript rather than WebAssembly.
  • Limitations of BROWSIX: BROWSIX’s shared process memory prevented on-demand heap growth and limited concurrent processes under browser heap caps.In Chrome, a roughly 2.2 GB context limit allowed at most four concurrent 500 MB processes.
  • BROWSIX-WASM design: BROWSIX-WASM communicates through a 64MB auxiliary SharedArrayBuffer, copying system-call data between process memory and shared memory.Calls exceeding 64MB are divided into several calls.
  • BROWSIX-WASM design: The auxiliary buffer enables the process and kernel to use the Atomic API for communication, unlike shared WebAssembly process memory.The runtime allocates corresponding auxiliary buffers when system calls access process-memory data.
  • Performance optimization: The authors identified kernel performance issues that could threaten WebAssembly–native comparisons, including inefficient buffer reallocation during file appends.BROWSERFS originally allocated a larger buffer and copied contents on every append.

3 BROWSIX-SPEC

BROWSIX-SPEC combines BROWSIX-WASM with browser automation, benchmark serving, performance-counter collection, and output validation. The evaluation compares native and WebAssembly builds across three benchmark suites in unmodified browsers.

  • Harness: BROWSIX-SPEC manages browser instances, benchmark assets, perf processes, performance-counter recording, and output validation.It was developed to execute WebAssembly benchmarks reliably while capturing performance data.
  • Benchmark evaluation: The evaluation uses SPEC CPU2006, SPEC CPU2017, and PolyBenchC, compiling benchmarks to native code with Clang 4.0 and to WebAssembly with BROWSIX-WASM.Chrome and Firefox run with standard sandboxing and isolation enabled.
  • Execution workflow: BROWSIX-SPEC launches a browser, loads the benchmark environment over HTTP, initializes BROWSIX-WASM, and starts the benchmark process.The process executes runspec, which spawns the standard specinvoke component.

4 Evaluation

Using BROWSIX-WASM and BROWSIX-SPEC, the evaluation compares WebAssembly with native code across PolyBenchC and SPEC CPU, and with asm.js. SPEC exposes substantially greater overhead than PolyBenchC, while WebAssembly remains faster than asm.js.

  • SPEC CPU: SPEC CPU showed higher overall overhead than PolyBenchC, indicating a substantial WebAssembly–native performance gap for larger applications.The SPEC evaluation excludes four data points that failed to compile or exceeded WebAssembly’s memory limit.
  • Benchmark design: SPEC CPU and PolyBenchC benchmarks were evaluated with BROWSIX-WASM across Chrome and Firefox, alongside native execution.The study also includes SPEC comparisons with asm.js.
  • PolyBenchC: 0.2% average overhead was measured for BROWSIX-WASM on PolyBenchC, with a maximum of 1.2%.These results reproduce most results from the original WebAssembly evaluation.
  • SPEC CPU: 1.55× slower in Chrome and 1.45× slower in Firefox was the average SPEC CPU slowdown relative to native code.The maximum overhead was 2.5× in Chrome and 2.08× in Firefox; 7 of 15 benchmarks were within 1.5× of native in each browser.
  • WebAssembly versus asm.js: 1.3× faster than asm.js was WebAssembly when each technology used its best-performing browser.The comparison allowed the best browser to differ between WebAssembly and asm.js.

5 Case Study: Matrix Multiplication

The matrix-multiplication case study shows WebAssembly running substantially slower than native code because Chrome’s JIT-generated code has more instructions, greater register pressure, and extra branches.

  • Performance comparison: 2×–3.4× slower than native was WebAssembly matrix multiplication across matrix sizes in both Chrome and Firefox.The function was compiled with -O2 and automatic vectorization disabled because WebAssembly lacked vectorized instructions.
  • Generated code: Chrome’s JITed native code has more instructions, increased register pressure, and extra branches than Clang-generated native code.Chrome also uses register spills and additional jumps in the generated loop structure.
  • Generated code: 53 instructions were generated by Chrome for matmul, compared with 28 instructions generated by Clang.The paper identifies Chrome’s poor instruction selection as one reason for the larger code size.
  • Instruction selection: Chrome’s code generator uses multiple instructions where Clang uses a single x86 memory-addressing instruction.This failure to exploit available addressing modes further increases register pressure.
  • Register allocation: Native Clang code uses fewer registers and avoids spills, whereas Chrome’s generated code uses all available general-purpose registers and spills three values.Chrome reserves registers for runtime purposes and uses a temporary register when not exploiting register addressing modes.

6 Performance Analysis

Performance-counter analysis connects WebAssembly’s SPEC CPU slowdown to pervasive code-generation inefficiencies, especially increased register pressure and unnecessary memory references.

  • Measurement approach: Performance counters were collected for SPEC CPU programs compiled to WebAssembly and native code in Chrome and Firefox.The measurements were used to explain WebAssembly’s overhead relative to native execution.
  • Increased register pressure: 2.02× more loads and 2.30× more stores were retired by Chrome’s WebAssembly code than by native code.Firefox retired 1.92× more loads and 2.16× more stores than native code.
  • Increased register pressure: 1.92× more loads and 2.16× more stores were retired by Firefox’s WebAssembly code than by native code.The results indicate increased register pressure and increased memory references.
  • Sources of inefficiency: Reserved runtime registers are unavailable to WebAssembly code in both Chrome and Firefox.Chrome reserves r13, r10, and xmm13 for runtime purposes; Firefox reserves r15, r11, and xmm15.
  • Sources of inefficiency: Increased register pressure is linked to reduced register availability, suboptimal register allocation, and ineffective use of x86 addressing modes.Chrome and Firefox use linear-scan allocators, while Clang uses greedy graph coloring, which consistently generates better code.

6.2 Extra Branch Instructions

WebAssembly executes more branch instructions than native code because of unnecessary loop jumps and safety checks inserted around function calls. These overheads arise from both code-generation behavior and WebAssembly’s runtime safety mechanisms.

  • 6.2 Extra Branch Instructions: 1.75× more unconditional and 1.65× more conditional branch instructions are retired in Chrome than native code.Firefox retires 1.65× more unconditional and 1.62× more conditional branches than native code.
  • 6.2.1 Extra Jump Statements for Loops: Chrome generates unnecessary jump statements for loops, producing significantly more branch instructions than Firefox.The comparison is reported alongside matmul’s loop behavior.
  • 6.2.2 Stack Overflow Checks Per Function Call: WebAssembly tracks current stack size with a global variable that increases on every function call.Programs can define a maximum stack size to prevent stack overflow.
  • 6.2.2 Stack Overflow Checks Per Function Call: Stack-overflow checks add comparisons and conditional jumps at every function’s start in Chrome and Firefox.These instructions test whether current stack size remains below the maximum stack size.
  • 6.2.3 Function Table Indexing Checks: WebAssembly dynamically checks indirect-call targets and verifies that their runtime function types match the call-site types.Function tables store functions and their types for these checks.
  • 6.2.3 Function Table Indexing Checks: Indirect-call checks add comparisons and conditional jumps before every indirect function call.These checks support function pointers and virtual functions in C/C++.

6.3 Increased Code Size

WebAssembly produces substantially larger code than native code, increasing retired instructions and instruction-cache misses. The effect varies across benchmarks, and lower cache misses can occasionally offset the larger instruction count.

  • 6.3 Increased Code Size: The generated code is considerably larger than code produced by Clang, and the evaluation measures instructions, cycles, and L1 instruction-cache misses.These counters compare WebAssembly execution in Chrome and Firefox with native execution.
  • 6.3 Increased Code Size: 1.80× more instructions than native code are executed in Chrome and 1.75× more in Firefox on average.Poor instruction selection, register spills, and extra branches contribute to the larger generated code.
  • 6.3 Increased Code Size: 429.mcf runs faster than native despite 1.6× more Chrome and 1.5× more Firefox instructions retired.Its slowdown relative to native is 0.81× in Chrome and 0.83× in Firefox because its main-loop instructions fit in the L1 instruction cache.
  • 6.3 Increased Code Size: 458.sjeng exhibits 26.5× more L1 instruction-cache misses in Chrome and 18.6× more in Firefox than native code.The increased WebAssembly code size leads to more instruction-cache misses.
  • 6.3 Increased Code Size: Virtual functions can produce 4.6× more L1 instruction-cache misses in Chrome and Firefox than native code.The passage attributes these misses to the execution of several virtual functions and resulting indirect calls.

6.4 Discussion

The discussion separates performance problems that improved implementations could mitigate from costs imposed by WebAssembly’s design constraints. Safety checks and reserved registers remain tied to WebAssembly’s safety and browser-interoperability requirements.

  • 6.4 Discussion: Poor register allocators and code generators are not considered fundamental because improved implementations could ameliorate them.WebAssembly compilers must generate code quickly online, unlike offline compilers such as Clang.
  • 6.4 Discussion: Hot-code optimization techniques used by other JITs are identified as likely applicable to WebAssembly.The proposed direction addresses implementation quality rather than WebAssembly’s safety design.
  • 6.4 Discussion: Stack-overflow checks, indirect-call checks, and reserved registers appear to arise from WebAssembly’s design constraints.These mechanisms impose runtime costs and increase generated code size.
  • 6.4 Discussion: The runtime checks are necessary for WebAssembly’s safety guarantees.Richer types for memory and function pointers might move some checks to compile time, but could complicate WebAssembly compiler implementations.
  • 6.4 Discussion: Browser interoperability can impose additional constraints because JavaScript implementations reserve registers for their own use.Reserved JavaScript registers increase register pressure on WebAssembly.

7 Related Work

Prior browser-native execution systems differ in their sandboxing, portability, language, and validation trade-offs. WebAssembly combines browser portability with type and memory safety, while related efforts provide narrower or different guarantees.

  • Precursors to WebAssembly: Earlier attempts to execute native code in browsers did not satisfy all of WebAssembly’s design criteria.The paper frames these criteria as including performance, portability, safety, and browser execution requirements.
  • Precursors to WebAssembly: ActiveX embeds signed x86 libraries with unrestricted Windows API access, whereas WebAssembly modules are sandboxed.ActiveX is now deprecated.
  • Precursors to WebAssembly: Native Client executes platform-specific machine code near native speed but supports only subsets of x86, ARM, and MIPS instruction sets.Static validation requires code generators to follow prescribed instruction patterns, creating a portability limitation.
  • Related JavaScript Approaches: asm.js is a JavaScript subset that uses type coercions to avoid JavaScript’s dynamic type system.Adding native features such as 64-bit integers requires extending JavaScript.
  • Related JavaScript Approaches: Compared with asm.js, WebAssembly uses compact binaries, is easier to validate, and provides formal type-safety and isolation guarantees.The passage also states that WebAssembly has shown better performance than asm.js.
  • WebAssembly Execution Model: WebAssembly is a stack machine like the JVM and CLR but does not support objects or unstructured control flow.The comparison highlights structural differences from those managed-language platforms.
  • Formal Verification: The WebAssembly specification’s operational semantics and type system have been mechanized using the Isabelle theorem prover.The mechanization found and addressed specification issues; RockSalt provides a related Coq verification effort for NaCl.
  • SPEC Benchmark Analysis: Prior SPEC studies analyze benchmark similarities and workload differences across SPEC CPU2006 and SPEC CPU2017.These studies provide benchmark analysis rather than the paper’s WebAssembly performance evaluation.

8 Conclusions

The paper enables comprehensive WebAssembly performance analysis on large SPEC CPU applications in Chrome and Firefox. Across these benchmarks, WebAssembly is substantially slower than native code, with the paper identifying causes of the gap.

  • BROWSIX-WASM and BROWSIX-SPEC enable SPEC CPU2006 and CPU2017 benchmarks to run as WebAssembly in Chrome and Firefox.The tools support detailed performance analysis of these benchmarks.
  • 1.55× mean slowdown for Chrome and 1.45× for Firefox compared with native execution across SPEC benchmarks.
  • 2.5× peak slowdown in Chrome and 2.08× in Firefox.
  • The identified performance gaps provide actionable guidance for future WebAssembly optimization efforts.
Loading 1901.09056v3…