Source-linked AI summary
LIKWID: Lightweight Performance Tools
Jan Treibig, Georg Hager, Gerhard Wellein
TL;DR
Modern multicore systems make topology, affinity, and performance analysis difficult for users. LIKWID addresses these issues with a lightweight command-line tool set for topology probing, affinity enforcement, hardware-counter measurement, and related optimization tasks. Its demonstrations show performance variation from thread placement and expose ccNUMA bandwidth problems, including 9.83 GB/s with sequential initialization versus 23.15 GB/s with correct first-touch placement.
Problem
Modern multicore, multicache, and multisocket systems complicate resource mapping, thread placement, and performance analysis, while conventional tools can be difficult to use and require extra system components.
Method
LIKWID combines topology probing, prefetch control, hardware-counter measurement, source-independent thread pinning, and MPI-compatible affinity support in Linux command-line tools.
Results
LIKWID demonstrations show that pinned thread placement produces consistently high STREAM performance, while correct first-touch placement raises effective bandwidth from 9.83 GB/s to 23.15 GB/s.
Takeaways & Limitations
LIKWID makes topology and affinity effects accessible with simple, low-overhead tools that require no code changes for core pinning workflows.
Takeaways & Limitations
LIKWID supports only x86-based processors.
Abstract
from arXiv · showhide
Exploiting the performance of today's microprocessors requires intimate knowledge of the microarchitecture as well as an awareness of the ever-growing complexity in thread and cache topology. LIKWID is a set of command line utilities that addresses four key problems: Probing the thread and cache topology of a shared-memory node, enforcing thread-core affinity on a program, measuring performance counter metrics, and microbenchmarking for reliable upper performance bounds. Moreover, it includes a mpirun wrapper allowing for portable thread-core affinity in MPI and hybrid MPI/threaded applications. To demonstrate the capabilities of the tool set we show the influence of thread affinity on performance using the well-known OpenMP STREAM triad benchmark, use hardware counter tools to study the performance of a stencil code, and finally show how to detect bandwidth problems on ccNUMA-based compute nodes.
1 Introduction
LIKWID addresses the difficulty of optimizing modern multicore systems by combining topology awareness, affinity control, performance measurement, and related tools in an accessible command-line suite.
- Motivation: Modern multicore, multicache, and multisocket systems make hardware-resource relationships and thread placement difficult to understand.Users may not know how hardware thread IDs map to cores, caches, sockets, and NUMA domains, while binding details vary across compilers and MPI libraries.
- Contribution: LIKWID is an easy-to-use Linux command-line tool set for performance-oriented programming without kernel patching.It supports Intel and AMD processors and both multithreaded and hybrid shared/distributed-memory parallel code.
- Contribution: Its tools probe topology, control prefetching, measure hardware counters, enforce thread affinity, and support MPI-oriented pinning.Preconfigured event groups also simplify obtaining standard metrics such as memory bandwidth and FLOP counts.
- Contribution: The tools are presented as a unified set because they address typical problems when porting and running applications on complex multicore and multisocket systems.The paper describes selected tools, demonstrates three case studies, and concludes with a summary and outlook.
2 Tools
LIKWID provides topology-aware resource specification, low-overhead performance-counter measurement, and source-independent thread pinning for complex x86 systems.
- Scope: LIKWID is limited to x86-based processors, although the authors regard this as non-severe given x86 prevalence in HPC systems.The paper states that 90% of systems in the latest Top 500 list were x86 at the time of writing.
- Resource specification: Thread domains and logical resource IDs let users specify cores, caches, sockets, and NUMA resources independently of Linux numbering schemes.Ranges and multiple ID lists support flexible selections such as M0:0,1@M2:0,1.
- likwid-perfctr: likwid-perfctr measures hardware events over complete applications or marked code regions, with predefined groups and derived metrics simplifying standard analyses.It targets quick, flexible, multi-core measurements with minimal system requirements and no required user-code changes for basic functionality.
- likwid-perfctr: 0.693493 CPI is reported for core 0 in the Init region of a four-core Intel Core 2 measurement.The CPI metric is derived from the always-counted INSTR_RETIRED_ANY and CPU_CLK_UNHALTED_CORE events.
- likwid-pin: likwid-pin enforces affinity externally by preloading a wrapper that intercepts pthread creation and assigns threads to a configured core list.The application must be dynamically linked, and implementation-specific thread behavior such as Intel OpenMP's management thread must be accounted for.
- likwid-pin: The portable pinning approach is intended to work across most applications, compilers, MPI implementations, and processor types.It also supports hybrid MPI-threaded programs when process startup establishes a Linux cpuset for each process.
3 Case studies
The case studies show how thread affinity, time-resolved counter monitoring, and NUMA-aware placement expose and address performance variability and bandwidth problems on multicore systems.
- 3.1 Case study 1: Equal distribution across two sockets gives the best STREAM triad performance on the tested ccNUMA system.Without pinning, performance varies substantially; pinning maintains consistently high performance.
- 3.2 Case study 2: Daemon-mode likwid-perfctr samples FLOPS_DP and MEM performance groups every 800 ms while keeping measurement overhead low.It reads counters and prints differences between successive measurements.
- 3.3 Case study 3: LIKWID’s MEM and NUMA performance groups help developers detect inefficient use of ccNUMA memory organization.The NUMA group is separately needed on AMD processors, while newer Intel systems may measure all events with MEM.
- 3.3 Case study 3: 6.9273 MBytes/s versus 6998.71 MBytes/s memory bandwidth distinguishes the measured core behaviors in the NUMA memory-copy example.The table also reports CPI, remote-read bandwidth, remote-write bandwidth, and total remote bandwidth for each core.
- 3.3 Case study 3: 23.15 GB/s with correct first-touch placement exceeds 9.83 GB/s after sequential initialization on one socket.Interleaving is a viable compromise when applications cannot readily use parallel first-touch placement, although it is less effective than correct first touch.
4 Conclusion and future plans
LIKWID provides simple, low-overhead command-line tools for exploiting multicore processor resources without kernel patches or code changes. Future work targets profiling, usability, Windows support, and newer processor architectures.
- Conclusion: LIKWID prioritizes simplicity and low overhead rather than a complex tooling environment.Its topology and pinning tools account for thread and cache topology and support physical-resource binding without code changes.
- Conclusion: LIKWID is open source, released under GPL2, and provides command-line support for performance-oriented developers targeting multicore processors.The conclusion highlights topology, pinning, and performance-counter use across the demonstrated examples.
- Future plans: Future plans include assembly-level profiling, improved usability, Windows support, and ports to architectures such as Intel Sandy Bridge.These items are presented as planned or ongoing development directions.