Source-linked AI summary
The ARM Scalable Vector Extension
Nigel Stephens, Stuart Biles, Matthias Boettcher, Jacob Eapen, Mbou Eyole, Giacomo Gabrielli, Matt Horsnell, Grigorios Magklis, Alejandro Martinez, Nathanael Premillieu, Alastair Reid, Alejandro Rico, Paul Walker
TL;DR
SVE addresses the need for more capable ARM vector processing while avoiding software costs as vector lengths change. It introduces a scalable, vector-length-agnostic architecture with features for compiler auto-vectorization, and the paper reports improved vectorization opportunities and performance scaling across implementations.
Problem
ARM SIMD needed gather-scatter, per-lane predication, longer vectors, and stronger support for workloads requiring more radical vector-processing capabilities.
Method
SVE combines implementation-selected vector lengths with vector-length-agnostic programming and architectural features for complex control flow, non-contiguous accesses, and data-dependent exits.
Results
SVE can achieve speedups of up to 3× at the same vector size as Advanced SIMD, and performance scales when the same executable runs on implementations with larger vectors.
Takeaways & Limitations
SVE enables software to run across different vector lengths without recompilation while expanding the loops that compilers can vectorize.
Takeaways & Limitations
SVE uses destructive predicated forms for most data-processing instructions because encoding space was insufficient for predication and constructivity throughout the instruction set.
Abstract
from arXiv · showhide
This article describes the ARM Scalable Vector Extension (SVE). Several goals guided the design of the architecture. First was the need to extend the vector processing capability associated with the ARM AArch64 execution state to better address the computational requirements in domains such as high-performance computing, data analytics, computer vision, and machine learning. Second was the desire to introduce an extension that can scale across multiple implementations, both now and into the future, allowing CPU designers to choose the vector length most suitable for their power, performance, and area targets. Finally, the architecture should avoid imposing a software development cost as the vector length changes and where possible reduce it by improving the reach of compiler auto-vectorization technologies. SVE achieves these goals. It allows implementations to choose a vector register length between 128 and 2,048 bits. It supports a vector-length agnostic programming model that lets code run and scale automatically across all vector lengths without recompilation. Finally, it introduces several innovative features that begin to overcome some of the traditional barriers to autovectorization.
1 INTRODUCTION
SVE responds to demand for more capable ARM SIMD by combining scalable vectors with features designed to improve compiler auto-vectorization. Its vector-length-agnostic model avoids software changes as implementations choose different vector widths.
- ARM SIMD extensions increasingly need gather-scatter accesses, per-lane predication, and longer vectors for new markets and workloads.
- SVE leaves vector length as an implementation choice from 128 to 2048 bits because research found no single preferred length.The programming model adjusts dynamically without recompilation or rewriting hand-coded SVE assembly or compiler intrinsics.
- Scalable vector length increases parallelism while allowing implementations to choose their design point.
- Rich addressing modes enable non-linear data accesses, including patterns supported by gather-scatter operations.
- Per-lane predication and predicate-driven loop control target complex control flow while reducing vectorization overhead relative to scalar code.
- Horizontal operations, vector partitioning, and scalarized intra-vector sub-loops address reducible dependencies and loops with data-dependent exits.
2 SVE OVERVIEW
The paper introduces SVE’s architectural state and illustrates its key features with code examples. The overview covers vector registers, predicate registers, the first-fault register, and control registers.
- Section 2 describes SVE architecture, its vector-length-agnostic programming model, and representative code examples.
- SVE architectural state includes vector registers, predicate registers, the first-fault register, and exception-level-specific control registers.
2.1 Architectural State
SVE adds scalable vector, predicate, first-fault, and control-register state to AArch64. These structures provide scalable data containers, predicate control, fault handling, and privilege-level vector-width virtualization.
- Thirty-two scalable vector registers Z0–Z31 have implementation-dependent widths and hold 64-, 32-, 16-, and 8-bit data elements.
- Sixteen scalable predicate registers P0–P15 accompany a special-purpose first-faulting register FFR.
- Control registers ZCR EL1–ZCR EL3 let each privilege level reduce the effective vector width.
2.2 Scalable Vector Length
SVE uses a scalable vector-length-agnostic design rather than a fixed vector width. Implementations can choose among widths while software scales without recompilation or porting effort.
- SVE supports any vector length that is a multiple of 128 bits between 128 and 2048 bits.The flexible width targets different performance-power-area optimization points without requiring a different instruction set for each width.
- Vector-length-agnostic software scales to different vector lengths without additional instruction encodings, recompilation, or software porting effort.Vector partitioning supports this model while also allowing conventional fixed-length sub-vectors.
2.3 Predicate-centric Approach
SVE makes predicates central to scalable vector execution, supporting loop control, mixed element sizes, and safe handling of data-dependent exits. Its predicate-driven mechanisms enable vectorization without fixed vector lengths or explicit iteration counts.
- Predicate registers: SVE uses predicate registers to control general memory and arithmetic operations, while predicate-only instructions can use the full P0–P15 register file.General operations are restricted to P0–P7; predicate-generating and predicate-only instructions can use P0–P15.
- Predicate registers: Each predicate provides per-byte-granularity control and supports mixed element sizes by selecting the least significant enable bit for each element size.This design targets vectorized code containing multiple data types.
- Loop control: SVE while instructions populate loop-control predicates from scalar counts and limits, avoiding the vector sequence and element-size alignment overhead of conventional predicate generation.They also handle loop-counter wrap-around consistently with sequential-code semantics.
- Loop control: SVE Daxpy has no instruction-count overhead versus equivalent scalar code, enabling compilers to opportunistically vectorize loops with unknown trip counts.The example compares C, scalar ARMv8-A, and SVE representations.
- Dynamic exits: First-fault loads suppress faults after the first active invalid access and record unsuccessfully loaded elements in the first-fault register, enabling speculative vectorization.A later iteration retries the failed access as the first active element, where it traps for servicing or termination.
- Dynamic exits: Vector partitioning lets SVE process safe elements before a data-dependent break while preventing side-effecting operations after the loop exit from being architecturally performed.The mechanism supports uncounted loops such as do-while and break-based loops.
2.4 Horizontal Operations
SVE addresses dependencies across loop iterations with horizontal operations, including logical, integer, floating-point, and strictly ordered floating-point reductions.
- Horizontal reductions: Horizontal operations act across elements within the same vector register to resolve dependencies spanning multiple loop iterations.They differ from normal SIMD instructions by operating across one vector's elements.
- Horizontal reductions: SVE provides logical, integer, and floating-point reduction operations for reducible loop-carried dependencies.
- Floating-point reductions: Strictly ordered floating-point reduction, such as fadda4, preserves a specified addition order while accumulating into a scalar register.
3 COMPILING FOR SVE
Compiling for SVE required extending LLVM vectorization to handle scalable lengths, predication, speculative vectorization, and floating-point ordering concerns.
- Compiler strategy: SVE's wide, vector-length-agnostic vectors, predication, and first-faulting loads require a revised compilation strategy.
- Vector-length agnosticism: Fixed-length “Unroll and Jam” cannot determine a scalable vector length at compile time, so the vectorizer directly maps scalar operations to vector operations.
- Compiler support: The compiler handles vector-length-dependent stack accesses by introducing stack regions instead of assuming every object has a constant stack-frame offset.
- Predication: Predication converts conditions into predicates, while brk partitions active lanes when loops exit conditionally.
- Floating-point reductions: SVE's fadda enables vectorization when the precise order of floating-point additions is critical to correctness.
- Speculative vectorization: LLVM extensions support the existing vectorization pass, while speculative vectorization uses a separate pass focused on broader loop coverage.
4 IMPLEMENTATION CHALLENGES
SVE's implementation addresses tight encoding budgets and memory-access demands through selective instruction forms, shared register resources, and enhanced load/store capabilities.
- Encoding trade-offs: Most data-processing instructions use destructive predicated forms because encoding both predication and constructivity would exceed the budget.
- Encoding trade-offs: movprfx supplies fully constructive predicated operations by combining with the immediately following instruction or executing as a vector copy.
- Encoding space: SVE keeps its encoding footprint below 28 bits while leaving room for future A64 instruction-set expansion.
- Implementation cost: The scalable vector register file overlays the existing SIMD and floating-point register file, minimizing area overhead, especially for smaller cores.
- Memory access: SVE couples wider vector processing with contiguous addressing, load-and-broadcast operations, and gather-scatter support for discontiguous data.
5 SVE PERFORMANCE
SVE is evaluated against Advanced SIMD using a representative modeled processor, an experimental auto-vectorizing compiler, and configurations with 128-, 256-, and 512-bit vectors. The results show higher vector utilization, speedups up to 3× at equal vector size, scaling up to 7× with larger vectors, and compiler- or algorithm-dependent limits.
- Evaluation setup: The evaluation uses a representative out-of-order microprocessor model, with latencies corresponding to RTL synthesis results and vector cross-lane penalties proportional to vector length.The model is not a real design and uses a true dual-ported cache with a 512-bit maximum access size.
- Evaluation setup: The evaluation compares Advanced SIMD with SVE at 128-bit, 256-bit, and 512-bit vector lengths using the same processor configuration.The lines show speedup relative to Advanced SIMD, while the bars show additional vectorization achieved with SVE.
- Performance results: SVE achieves higher vector utilization than Advanced SIMD, with speedups of up to 3× even when both use vectors of the same size.SVE features support vectorization of code with complex control flow and non-contiguous memory accesses; HACCmk is cited as an example involving conditional assignments.
- Performance results: Some benchmarks show much higher SVE vectorization and performance scaling up to 7× with vector length, although gather-scatter operations can limit scaling.The assumed implementation conservatively cracks gather-scatter operations, reducing their scaling with vector length.
- Performance limitations: Other benchmarks show little or no performance benefit because of code structure, unavailable vectorized library functions, non-vectorizable algorithms, or compiler code-generation decisions.Examples include Graph500, where pointer-based graph traversal is not expected to benefit unless the algorithm is refactored, and SMG2000 and MILCmk, where compiler issues constrain performance.
- Performance limitations: The authors expect many observed compiler and library issues to be resolved over time.The performance results therefore depend partly on the maturity of the experimental toolchain and supporting software.
6 CONCLUSIONS
SVE expands ARM’s opportunities for larger-scale vector processing, while its tools and software ecosystem remain early and require continued development. The paper links this work to HPC momentum and a path toward efficient Exascale computing.
- Conclusion: SVE opens a new chapter for ARM by increasing the scale and opportunity for vector processing on ARM processor cores.The authors describe this as an early stage for SVE tools and software.
- Conclusion: HPC is the current focus and catalyst for compiler work, supporting development of Linux distributions, optimized SVE libraries, and tools from ARM and third parties.The paper describes broader engagement with the open-source community and ARM ecosystem around SVE and the HPC market.
- Conclusion: The authors describe this ecosystem development as enabling a path to efficient Exascale computing.