PaperlayerView exact revision on arXiv

Source-linked AI summary

uops.info: Characterizing Latency, Throughput, and Port Usage of Instructions on Intel Microarchitectures

Andreas Abel, Jan Reineke

arXiv:1810.04610v3cs.PFcs.ARcs.PL

TL;DR

Precise instruction-performance models are hard to obtain, despite their value for predicting and optimizing software performance. The paper automatically generates microbenchmarks to characterize x86 instruction latency, throughput, and port usage, and reports more accurate and precise characterizations than prior work, including previously undocumented differences and discrepancies.

  • Problem

    Detailed latency, throughput, and port-usage information for individual instructions is difficult to obtain, although it is needed for performance prediction and optimization.

    Source: §1 · p. 1

  • Method

    The paper develops automatically generated microbenchmarks and algorithms to characterize instruction latency, throughput, and port usage on Intel Core CPUs.

    Source: §1 · p. 2

  • Results

    The characterizations are more accurate and precise than prior work, revealing undocumented latency differences across microarchitectures and discrepancies in previously published information.

    Source: §1 · p. 2, §9 · p. 12

  • Takeaways & Limitations

    The machine-readable instruction characterizations are intended to support performance-analysis tools, simulators, and optimizing compilers.

    Source: §1 · p. 1

    The tool supports only instructions usable in 64-bit mode and excludes the mostly obsolete x87 floating-point instruction set.

    Source: §8 · p. 12

Abstract

from arXiv · show

Modern microarchitectures are some of the world's most complex man-made systems. As a consequence, it is increasingly difficult to predict, explain, let alone optimize the performance of software running on such microarchitectures. As a basis for performance predictions and optimizations, we would need faithful models of their behavior, which are, unfortunately, seldom available. In this paper, we present the design and implementation of a tool to construct faithful models of the latency, throughput, and port usage of x86 instructions. To this end, we first discuss common notions of instruction throughput and port usage, and introduce a more precise definition of latency that, in contrast to previous definitions, considers dependencies between different pairs of input and output operands. We then develop novel algorithms to infer the latency, throughput, and port usage based on automatically-generated microbenchmarks that are more accurate and precise than existing work. To facilitate the rapid construction of optimizing compilers and tools for performance prediction, the output of our tool is provided in a machine-readable format. We provide experimental results for processors of all generations of Intel's Core architecture, i.e., from Nehalem to Coffee Lake, and discuss various cases where the output of our tool differs considerably from prior work.

1 Introduction

The paper addresses sparse, imprecise instruction-performance data by automatically characterizing latency, throughput, and port usage on Intel Core CPUs. Its measurements expose differences and inaccuracies in prior information, while its machine-readable output is intended for performance tools and optimizing compilers.

1 Introduction

  • Detailed instruction latency, throughput, and port data are difficult to obtain; Intel manuals cover only some commonly used instructions and omit µop decomposition and execution-port information.Such information can support performance-analysis tools, cycle-accurate simulators, optimizing compilers, and manual code tuning.Source: §1 · p. 1
  • The tool automatically generates microbenchmarks to characterize latency, throughput, and port usage for instructions on Intel Core CPUs.The algorithms account for explicit and implicit dependencies, including status flags, using instruction-set information extracted from Intel XED configuration files.Source: §1 · p. 2
  • The proposed latency definition distinguishes dependencies between different input-output operand pairs, supporting more accurate performance predictions.Source: §1 · p. 2
  • The tool was applied to Intel Core generations from Nehalem to Coffee Lake, both on hardware and through a variant running benchmarks on IACA.The paper notes that IACA is updated infrequently and its results are not always accurate.Source: §1 · p. 2
  • The measurements uncover undocumented latency differences across microarchitectures, explain discrepancies in prior information, and reveal errors in IACA and inaccuracies in manuals.The authors report these findings from comparing their measurements with previously published data.Source: §1 · p. 2

2 Related Work

The section reviews Intel-provided instruction data and measurement-based approaches, describing gaps in coverage, update frequency, and automation. It also surveys tools and datasets that measure or infer instruction performance.

2.1 Information provided by Intel

  • Intel’s Optimization Reference Manual provides latency and throughput for some commonly used instructions, but coverage is incomplete and it omits detailed port usage.IACA can analyze loop-kernel throughput and port usage, but not generally individual µop bindings; its support is infrequently updated and lacks Kaby Lake and Coffee Lake.Source: §2.1 · p. 2
  • LLVM scheduling models include architect-provided latency and port-usage information for Sandy Bridge, Haswell, Broadwell, and Skylake.Source: §2.1 · p. 2

2.2 Measurement-based Approaches

  • Measurement-based resources include Agner Fog’s manually interpreted instruction tables, CQA’s latency and throughput module, EXEgesis’s instruction-list and inference goals, and narrower datasets or tools from Granlund and AIDA64.Agner Fog’s lists are incomplete and sometimes imprecise; CQA mentions port-counter measurements without further details.Source: §2.2 · pp. 2–3

3 Background

The background describes how Intel Core CPUs move instructions through their pipeline and explains the paper’s assembler notation. It also introduces hardware performance counters used to measure cycles and port-level µop execution.

3.1 Pipeline of Intel Core CPUs

  • The pipeline decodes instructions into µops, tracks and renames them in the reorder buffer, then schedules ready µops for dispatch to execution ports and their functional units.Ports accept at most one µop per cycle; most functional units are pipelined, unlike divider units.Source: §3.1 · p. 3
  • Some microarchitectures execute special µops directly in the reorder buffer, including NOPs, zero idioms, and move-eliminated register moves.Source: §3.1 · p. 3

3.2 Assembler Instructions

  • Assembler instructions use a mnemonic followed by operands, typically with the destination first; operands may be registers, memory, immediates, or implicit state such as status flags.Memory operands use base, index, scale, and displacement components, and instructions can have multiple operand types or widths.Source: §3.2 · p. 3
  • Assembler text does not map one-to-one to machine code, and some machine instructions cannot be generated through assembler code.The assembler generally controls which encoding it selects when multiple encodings are possible.Source: §3.2 · p. 3

3.3 Hardware Performance Counters

  • Recent Intel processors provide performance counters for elapsed core cycles and the number of µops executed on each port.Source: §3.3 · p. 4

4 Definitions

Section 4 defines instruction latency, throughput, and port usage. It proposes an operand-pair latency definition and distinguishes throughput conventions and port combinations.

4 Definitions

  • The section establishes the three microarchitectural properties that the paper’s algorithms aim to infer.Source: §4 · p. 4

4.1 Latency

  • Latency is a mapping from each source–destination operand pair to its readiness time, avoiding a single value that can overstate observed latency.The mapping assumes other dependencies are not on the critical path.Source: §4.1 · p. 4

4.2 Throughput

  • Throughput has competing definitions: Intel measures cycles until issue ports can accept the same instruction again, while this paper uses cycles per instruction.Fog’s definition averages cycles per independent same-kind instruction; the paper notes that these definitions are not equivalent.Source: §4.2 · p. 4

4.3 Port Usage

  • Port usage counts µops by the combinations of ports on which they can execute, not by assigning each µop to one port.For a µop executable on ports 0 and 1, the count is one for {0, 1} and zero for {0} or {1}.Source: §4.3 · p. 4

5 Algorithms

The algorithms infer port usage by probing instructions alongside blocking instructions. They use port-combination measurements to distinguish µops that can execute on the same ports from those with narrower bindings.

5.1 Port Usage

  • Isolation averages can confuse fixed port assignments with flexible µops, producing different behavior when an instruction competes for a port.For example, 1 µop each on ports 0 and 5 can look like 2 µops eligible on either port 0 or 5.Source: §5.1 · pp. 4–5
  • Blocking instructions occupy a port combination so the algorithm can identify µops that can execute only on that combination.Source: §5.1 · p. 5

5.1.1 Finding Blocking Instructions

  • The algorithm assumes a one-µop blocker exists for each relevant port combination, then selects the highest-throughput candidate in each isolated-use group.The stated assumption excludes port combinations connected to store-data and store-address units.Source: §5.1.1 · p. 5

5.1.2 Port Usage Algorithm

  • For each port combination, the algorithm probes with repeated blockers, subtracts counts already assigned to stricter subsets, and records the remaining µops.It processes smaller combinations first and can restrict iterations to ports observed in isolation.Source: §5.1.2 · pp. 5–6

5.2 Latency

The latency algorithms construct dependency chains to isolate source-to-destination timing. They adapt the chains for operand types, implicit dependencies, memory operations, and value-dependent division latency.

5.2 Latency

  • For matching operand types, repeated dependent instructions yield latency from chain runtime; mismatched types require a chain instruction with reversed operand types.For mismatched types, the chain instruction’s known latency is subtracted from the measured chain time.Source: §5.2 · p. 6

5.2.1 Register →Register

  • Register-to-register chains use MOVSX for general-purpose registers to avoid move elimination, while different-type chains may provide only an upper bound.For different register types, the algorithm measures compositions with possible chain instructions and subtracts one from the minimum time; SIMD chains use shuffle instructions to avoid zero-latency MOVs.Source: §5.2.1 · pp. 6–7

5.2.2 Memory →Register

  • For memory-to-register latency, the method makes the next load depend on the previous result, using a double-XOR chain to link the destination register to the address register.It breaks flag dependencies introduced by XOR and uses MOVSX for 8- or 16-bit destinations to avoid partial-register stalls.Source: §5.2.2 · p. 7

5.2.3 Status Flags →Register

  • For status-flag-to-register dependencies, TEST on a general-purpose register creates a dependency from the register to the flags.Source: §5.2.3 · p. 7

5.2.4 Register →Memory

  • Store latency cannot be measured directly; store-and-load chain time may be shortened by forwarding or early load-address computation.The paper nevertheless measures chains pairing register-reading, memory-storing instructions with suitable loads for insight into their timing.Source: §5.2.4 · p. 7

5.2.5 Divisions

  • Divider latency depends on operand values, so the algorithm tests high- and low-latency values and builds chains that preserve a chosen register value.The usual automatic dependency-chain technique does not work for these cases because a high-latency division may produce a value associated with lower latency.Source: §5.2.5 · pp. 7–8

5.3 Throughput

The paper measures throughput using sequences designed to minimize dependencies and computes a port-based throughput estimate under stated assumptions. Divider instructions are handled as a special case.

5.3.1 Measuring Throughput

  • Measured throughput comes from timing repeated sequences of mostly independent instruction instances and dividing by the number executed.The algorithm tests sequence lengths of 1, 2, 4, and 8 because longer sequences can sometimes increase time per instruction.Source: §5.3.1 · p. 8
  • For instructions with implicit read/write operands, dependency-breaking instructions are also tested, though their resource use may not reduce sequence time.Divider throughput is tested with operand values associated with high and low throughput.Source: §5.3.1 · p. 8

5.3.2 Computing Throughput from Port Usage

  • Port-derived throughput assumes ports are the only limiting resource and there are no implicit dependencies; divider instructions are not fully pipelined.For other instructions, throughput is computed from port usage by minimizing the maximum port load across a schedule.Source: §5.3.2 · p. 8

6 Implementation

The implementation converts Intel’s x86 instruction description into a simpler representation for benchmark generation, then measures instruction behavior using hardware performance counters. Measurements run in kernel space, allowing system instructions to be tested and preemption and interrupts to be disabled.

6 Implementation

  • The implementation section describes how the algorithms from Section 5 are implemented.Source: §6 · p. 8

6.1 Details of the x86 Instruction Set

  • The implementation converts XED’s complex configuration files into simpler XML containing the information needed to generate assembler for instruction variants, including implicit operands.The XED descriptions are complete but complex to parse and include encoding details unnecessary for this purpose.Source: §6.1 · p. 9

6.2 Measurements on the Hardware

  • The tool uses hardware performance counters for port-µop counts and execution time, and performs measurements in kernel space to test system instructions and disable preemption and interrupts.The kernel-space setup also permits privileged counter configuration.Source: §6.2 · p. 9

6 AsmCode

The measurement harness isolates and averages instruction-sequence measurements, while a variant runs the generated sequences through IACA for comparison. Results are stored in machine-readable XML.

6 AsmCode

  • Measurements use 10 and 110 sequence copies to subtract counter-reading overhead, divide the difference by 100, and average 100 warmed-up repetitions.The copy counts balance instruction-cache fit with measurement accuracy; counter reads are serialized to prevent reordering.Source: §6 · p. 9

6.3 Analysis Using Intel IACA

  • The IACA variant analyzes generated sequences as loop bodies, and versions 2.1, 2.2, 2.3, and 3.0 can differ on the same instruction and microarchitecture.Newer versions add support for newer microarchitectures while dropping support for older ones.Source: §6.3 · p. 9

6.4 Machine-readable Output

  • Results for tested microarchitectures are stored in XML, including hardware measurements and IACA analyses.Source: §6.4 · p. 9

7 Evaluation

The evaluation compares hardware measurements with IACA and presents instruction-level cases where operand-specific latency or port usage differs from prior reports. It also identifies newly observed latency differences and dependency-breaking idioms.

7.1 Experimental Setup

  • Tool runtime ranged from 50 minutes on Coffee Lake to 110 minutes on Broadwell.Source: §7.1 · p. 10

7.2 Hardware Measurements vs. Analysis with IACA

  • 91.04%–98.24% of cases with matching µop counts also had matching port usage; IACA’s reported µop counts matched hardware for 84.65% (Westmere) to 90.06% (Broadwell).The authors also report that IACA ignores some dependencies and has incorrect port assignments in some cases; different IACA versions sometimes disagree for the same instruction.Source: §7.2 · pp. 10–11

7.3.1 AES Instructions

  • AESDEC’s Sandy Bridge latency was 8 cycles for XMM1→XMM1 but about 1.25 cycles for XMM2→XMM1; Haswell measured 7 cycles for both operand pairs.The authors observed the same Sandy Bridge and Ivy Bridge behavior for AESDECLAST, AES-ENC, and AESENCLAST, which they say had not previously been documented to their knowledge.Source: §7.3.1 · p. 11

7.3.2 SHLD

  • SHLD on Nehalem measured 3 cycles for R1→R1 and 4 cycles for R2→R1, matching different previously reported latency values.On Skylake, the measured latency was 1 cycle with the same register for both operands and 3 cycles with different registers.Source: §7.3.2 · p. 11

7.3.3 MOVQ2DQ

  • For MOVQ2DQ, our approach shows the second µop can use ports 0, 1, and 5, whereas IACA and LLVM model both µops as using only port 5.With blocking instructions for ports 1 and 5, all MOVQ2DQ µops use port 0.Source: §7.3.3 · p. 12

7.3.4 MOVDQ2Q

  • For MOVDQ2Q, hardware measurements found different port usage on Haswell and Sandy Bridge, exposing inaccuracies or imprecision in prior reports.On Haswell, the measured usage was one µop on port 5 and one eligible for ports 0, 1, and 5; on Sandy Bridge, it was 1∗p015+1∗p5.Source: §7.3.4 · p. 12

7.3.5 Instructions with Multiple Latencies

  • Latency differences between input-output operand pairs appeared in many instructions, including memory-operand forms and numerous non-memory instructions.For several listed instructions these differences were previously undocumented to the authors’ knowledge.Source: §7.3.5 · p. 12

7.3.6 Zero Idioms

  • The authors’ results identify (V)PCMPGT(B/D/Q/W) instructions as dependency-breaking idioms, though they are absent from the Optimization Manual’s list.Source: §7.3.6 · p. 12

8 Limitations

The tool’s coverage is limited to instructions usable in 64-bit mode and excludes x87, some system instructions, and several sources of performance variation.

8 Limitations

  • The tool excludes x87 and some system instructions, and does not test performance differences from register or immediate values (except division) or memory addressing modes beyond the base register.It supports only instructions usable in 64-bit mode.Source: §8 · p. 12

9 Conclusions and Future Work

The authors characterize instruction latency, throughput, and port usage across all Intel Core microarchitectures, with results more accurate and precise than prior work. They provide machine-readable characterizations, implement a performance-prediction tool, and identify further extensions for future work.

9 Conclusions and Future Work

  • The experimental evaluation finds the instruction characterizations more accurate and precise than prior work across Intel Core microarchitectures.The algorithms and their implementations characterize latency, throughput, and port usage.Source: §9 · p. 12
  • The characterizations are available in machine-readable form, and the authors implemented an IACA-like performance-prediction tool for all Intel Core microarchitectures.They also planned to release the characterization tool's source code as open source.Source: §9 · p. 12
  • Future work includes adapting the algorithms to AMD x86 CPUs and characterizing additional pipeline properties, including fusion and decoder usage.The paper names micro and macro-fusion, simple and complex decoders, and the Microcode-ROM as examples.Source: §9 · p. 12
Loading 1810.04610v3…