PaperlayerView exact revision on arXiv

Source-linked AI summary

LIKWID: A lightweight performance-oriented tool suite for x86 multicore environments

Jan Treibig, Georg Hager, Gerhard Wellein

arXiv:1004.4431v3cs.DCcs.PF

TL;DR

Complex multicore systems make it difficult to connect hardware topology and thread placement with performance. LIKWID provides command-line tools for topology, affinity, and counter measurement, and the case studies show that pinning outcomes depend on topology. The paper reports consistently high performance in the pinned STREAM case and a twofold reduction when the topology-aware stencil code is pinned incorrectly.

Abstract

from arXiv · show

Exploiting the performance of today's processors requires intimate knowledge of the microarchitecture as well as an awareness of the ever-growing complexity in thread and cache topology. LIKWID is a set of command-line utilities that addresses four key problems: Probing the thread and cache topology of a shared-memory node, enforcing thread-core affinity on a program, measuring performance counter metrics, and toggling hardware prefetchers. An API for using the performance counting features from user code is also included. We clearly state the differences to the widely used PAPI interface. To demonstrate the capabilities of the tool set we show the influence of thread pinning on performance using the well-known OpenMP STREAM triad benchmark, and use the affinity and hardware counter tools to study the performance of a stencil code specifically optimized to utilize shared caches on multicore chips.

I. INTRODUCTION

LIKWID is a lightweight command-line tool suite intended to make performance optimization on complex multicore x86 systems more accessible. Its tools expose topology, support thread affinity and performance-counter measurement, and address hardware prefetching.

I. INTRODUCTION

  • Complex multicore, multicache, multisocket systems make resource mapping and thread binding difficult for users.Binding practices vary across compilers and MPI libraries, while hardware thread IDs may be hard to relate to cores, caches, sockets, and NUMA domains.Source: §I · p. 1
  • LIKWID is designed for performance-oriented programming on Linux, supports Intel and AMD processors, and requires no kernel patching.The suite also supports multithreaded and hybrid shared/distributed-memory code.Source: §I · p. 1
  • The suite combines topology probing, thread-core affinity, performance-counter measurement, and hardware-prefetcher control.The tools are presented as addressing typical application-programming problems on multicore and multisocket systems.Source: §I · p. 1

II. TOOLS

  • LIKWID supports x86-based processors; the authors cite x86 prevalence in HPC as a reason not to consider this a severe limitation.The passage gives an example of 90% of systems in the latest Top 500 list being x86-based.Source: §II · pp. 1–2

A. likwid-perfCtr

likwid-perfCtr measures hardware performance-counter events through a command-line wrapper or a marker API. It offers event groups and multiplexing, while requiring care with core-based counts and short measurements.

A. likwid-perfCtr

  • Hardware counters count processor events during execution; likwid-perfCtr uses runtime event collection because it is sufficient in many cases and practically overhead-free.The alternative described is interrupt-based sampling for a finer-grained view, with inherent statistical errors.Source: A. likwid-perfCtr · p. 2
  • Counter counts are core-based, not process-based, so events generated by anything running on a measured core are included.The user is responsible for affinity to obtain sensible results.Source: A. likwid-perfCtr · p. 2
  • The command-line wrapper measures multiple cores without source changes, and socket locks assign shared uncore-event counts to one thread per socket.Measurements can cover an application's runtime; the marker API instead delimits named code regions and accumulates counts across calls.Source: A. likwid-perfCtr · p. 2
  • Preconfigured event groups derive standard metrics such as memory bandwidth and Flops per second, reducing the need to specify vendor event names.The authors aim to provide the same groups across supported architectures when native events allow it.Source: SIMD_COMP_INST_RETIRED_SCALAR_DOUBLE:PM… · p. 3
  • Multiplexing assigns counters to event sets in round-robin order when events outnumber available counters, but short-running measurements then carry large statistical errors.Without multiplexing, the application must be run more than once when the requested event count exceeds available counters.Source: SIMD_COMP_INST_RETIRED_SCALAR_DOUBLE:PM… · p. 3

SIMD_COMP_INST_RETIRED_SCALAR_DOUBLE:PMC1 \

B. likwid-topology

likwid-topology extracts processor and cache topology so users can relate hardware threads to physical resources. It reports thread, cache-sharing, and extended data-cache information.

B. likwid-topology

  • likwid-topology uses processor-specific CPUID information to extract thread and cache topology, with methods covering multiple Intel and AMD processor generations.The thread topology is determined from the APIC ID; the core functionality is also available as a C library module.Source: B. likwid-topology · p. 4
  • Its output identifies which hardware threads map to physical resources and which threads share each cache level.It also reports extended parameters for data caches.Source: B. likwid-topology · p. 4
  • The tool can present an ASCII-art overview of a node's cache and socket topology.The described listing shows one socket, corresponding to the first L3 cache group.Source: B. likwid-topology · p. 4

C. likwid-pin

likwid-pin enforces thread affinity by pinning newly created threads to a supplied core list, without requiring application code changes. Its portable wrapper supports multiple threading implementations, with thread-creation details conveyed through configuration such as a skip mask.

C. likwid-pin

  • The wrapper overloads pthread_create and pins threads as they are created, using core IDs and parameters specified through environment variables.The application is started with the library preloaded.Source: C. likwid-pin · p. 5
  • Intel OpenMP creates OMP_NUM_THREADS+1 threads and uses the first as a management thread, so likwid-pin must be told not to pin it.For Intel-compiled binaries using Intel MPI, the stated skip mask is 0x3, which avoids binding the first two newly created threads.Source: C. likwid-pin · p. 5
  • The skip mask can also leave MPI shepherd threads unpinned, enabling likwid-pin to handle hybrid applications.POSIX threads, Intel OpenMP, and GNU OpenMP are supported; other implementations can use a skip mask.Source: C. likwid-pin · p. 5
  • Unlike taskset, likwid-pin can pin threads individually but does not create a Linux cpuset for the application..Source: C. likwid-pin · p. 5
  • likwid-pin offers a portable way to pin threads across applications, compilers, MPI implementations, and processor types.The application must be dynamically linked, but no code changes are required.Source: C. likwid-pin · p. 5

D. likwid-features

likwid-features displays and changes switchable processor-feature states, including hardware prefetchers. In the described version, it works only on Intel Core 2 processors.

D. likwid-features

  • Hardware prefetching can hide data-access latency, but turning it off can sometimes increase performance, motivating inspection of its state.On Intel Core 2, prefetcher settings can be changed through bits in the IA32_MISC_ENABLE MSR register.Source: D. likwid-features · p. 5
  • likwid-features reports switchable feature states and can disable adjacent cache line prefetching with the -u CL_PREFETCHER option.The example reports CL_PREFETCHER as disabled after the command.Source: D. likwid-features · pp. 5–6
  • The current likwid-features version works only on Intel Core 2 processors; support for other architectures was planned.Source: D. likwid-features · p. 6

III. COMPARISON WITH PAPI

The paper compares LIKWID with PAPI, emphasizing their different primary uses: LIKWID as command-line tools for end users and PAPI as a library for other tools. The authors caution that the comparison is incomplete and that some issues are difficult to quantify.

III. COMPARISON WITH PAPI

  • PAPI relies on other software for architecture-specific implementation and focuses on a portable interface to performance metrics across platforms and architectures.It also includes a small collection of command-line utilities.Source: §III · p. 6
  • LIKWID primarily provides command-line tools for end users, whereas PAPI [4] primarily serves as a library used by other tools.The comparison is presented as an impression of differences, not a complete assessment.Source: §III · p. 6

A. Case study 1: Influence of thread topology on STREAM triad performance

In STREAM triad tests, distributing threads across sockets and pinning them produced consistently high performance, while unpinned runs varied. Intel OpenMP scatter matched likwid-pin, and performance distributions differed by compiler and processor.

A. Case study 1: Influence of thread topology on STREAM triad performance

B. Case Study 2: Influence of thread topology on a topology-aware stencil code

The stencil’s topology-aware optimization requires a specific thread-to-core mapping, not merely an even distribution across the machine. Incorrect pinning reverses the optimization’s benefit by preventing effective use of the shared cache.

B. Case Study 2: Influence of thread topology on a topology-aware stencil code

  • Unlike the earlier case study, this stencil requires exact thread-to-core placement because its optimization is designed to exploit shared caches.Source: B. Case Study 2: Influence of thread… · p. 8
  • The 3D Jacobi smoother uses a 7-point stencil, POSIX threads, assembly-language kernels, and implicit temporal blocking via pipeline parallelism [8].Source: B. Case Study 2: Influence of thread… · p. 8
  • Incorrect pinning cuts the optimized stencil’s performance by a factor of two, making it slower than a naive threaded baseline without temporal blocking.The optimization relies on shared-cache use to increase computational intensity, so the machine requires a specific thread-core mapping.Source: B. Case Study 2: Influence of thread… · p. 8

C. Case Study 3: Examining the effect of temporal blocking

Performance counters quantify how temporal blocking changes memory traffic in a 7-point Jacobi stencil. Although the optimized version cuts traffic substantially, the reduction does not produce a proportional performance increase on Nehalem EP.

C. Case Study 3: Examining the effect of temporal blocking

  • Temporal blocking reduces memory traffic 4.5-fold, but this does not yield a proportional performance boost on Nehalem EP.A single stream cannot fully use its memory bandwidth, while the performance gap between saturated main memory and L3 bandwidth is small for Jacobi on this processor.Source: C. Case Study 3: Examining the effect… · p. 9
  • Nontemporal stores reduce data transfer volume by about one-third versus temporal stores by eliminating write allocation on store misses.Source: C. Case Study 3: Examining the effect… · p. 9
  • The comparison measures main-memory transfer volume for three 7-point stencil versions executing the same number of updates on four physical cores of one socket.Source: C. Case Study 3: Examining the effect… · pp. 8–9

V. CONCLUSION AND FUTURE PLANS

LIKWID offers lightweight command-line tools for performance-oriented programming, with an emphasis on simplicity and low overhead. The authors report demonstrations on STREAM and a topology-aware stencil, while noting the software’s alpha-stage status and planned extensions.

V. CONCLUSION AND FUTURE PLANS

  • LIKWID aims to make important performance-tool functionality accessible with few obstacles, simplicity, and low overhead.Source: §V · p. 9
  • Demonstrations cover thread-topology effects on STREAM and pinning and performance-counter analysis of an optimized topology-aware stencil.Source: §V · p. 9
  • LIKWID remains in alpha; planned work includes broader processor support, NUMA information in likwid-topology, cpuset support in likwid-pin, and MPI counter-data collection.Source: §V · p. 9
  • Future plans include profiling and low-level bandwidth mapping, a Windows port, and XML output.Source: §V · p. 9
Loading 1004.4431v3…