Source-linked AI summary
Characterizing the I/O Behavior of HPC Applications through Modeling and Simulation
Njoud O. Almaaitah, David E. Singh, Taylan Özden, Jesus Carretero, Raffaele Montella
TL;DR
HPC applications’ intensive and contended I/O makes their behavior important to understand for efficient system use. The paper creates ElastiSim applications from real HPCIO traces, reproduces I/O access patterns, and reports reproduction accuracy of up to 98.8% and 98.1%. The framework also supports fine-grained I/O performance and congestion analysis.
Problem
Parallel HPC applications exhibit complex I/O access patterns and contention risks, motivating analysis of their I/O behavior for performance optimization.
Method
The framework analyzes HPCIO traces, samples application profiles, and generates ElastiSim applications that reproduce real-world I/O behavior.
Results
98.8% and 98.1% accuracy: the framework with ElastiSim reproduces the performance behavior of both evaluated applications.
Takeaways & Limitations
The framework provides feasible, accurate simulation of native applications and fine-grained insights into platform performance from the I/O perspective.
Abstract
from arXiv · showhide
Parallel applications process large amounts of data, leading to intensive parallel I/O operations. These operations can exhibit different levels of complexity, including, among others, multiple I/O access patterns, data staging, and contention risks. Therefore, in order to exploit high-performance computing (HPC) systems efficiently and optimize the I/O performance, it is crucial to consider the I/O behaviour of the HPC applications. In this work, we have developed a framework that reproduces the I/O access pattern of real applications in a simulated environment provided by ElastiSim, a batch-system simulator for rigid, malleable, and evolving workloads. The simulated applications are generated based on I/O traces captured from real applications provided by the HPC Input/Output (HPCIO) analysis repository. The HPCIO analysis database includes traces combined with information about real applications' performance across different parallel I/O libraries and layers of the I/O stack. We have conducted detailed case studies of real-world applications' traces to demonstrate how the proposed modeling framework can provide insights into the performance characteristics of I/O applications, including the I/O congestion analysis based on the application's I/O access pattern.
I. INTRODUCTION
The paper addresses HPC I/O performance challenges caused by application behavior and concurrent-access contention. It proposes reproducing real application I/O traces in ElastiSim to support simulation-based analysis and future I/O scheduling techniques.
- Parallel applications can underperform because of resource contention, network congestion, and inefficient I/O, including contention from simultaneous accesses.
- Modeling I/O workloads supports storage evaluation, bottleneck identification, and improved scheduler decisions, but accurate models are difficult because hardware and input parameters affect performance.
- The framework uses HPCIO application profiles and creates ElastiSim applications that reproduce repository traces under different resource utilization levels.
- The work evaluates real use cases for I/O congestion and validates ElastiSim with Jacobi and EpiGraph on real and simulated HPC platforms.
II. RELATED WORK
Prior work models and predicts HPC I/O performance through statistical analysis, predictive analytics, replay, and workload generation. This paper instead ports real HPCIO traces into ElastiSim to reproduce access patterns and analyze I/O congestion.
- Existing HPC I/O performance techniques include statistical analysis, predictive analytics, replay-based methods, and workload generation.
- Predictive approaches analyze historical data to identify performance issues in advance, while autotuning systems reduce the complexity exposed by the I/O stack.
- Replay-based approaches analyze historical traces to generate workloads that reproduce the original application's I/O behavior.
- This work uses real HPCIO traces to replicate application I/O access patterns in ElastiSim and analyze congestion based on those patterns.
III. SIMULATION TOOL
The paper uses ElastiSim to simulate HPC applications containing computation and I/O tasks. It applies this environment to several scientific use cases and validation applications.
- ElastiSim supports performance models for computation and I/O tasks and focuses on rigid, malleable, and evolving workloads.
- ElastiSim is a discrete-event simulator whose application models specify the load introduced to the simulated platform.
- The use cases include Quantum Espresso distributions for electronic-structure calculations and materials modeling.
- Nek5000 is an open-source solver, while WaComM++ supports simulations and predictions of pollutant transport and dispersion in coastal environments.
- Validation applications include Jacobi for iterative linear-system solutions and EpiGraph for stochastic simulations of COVID-19 propagation.
V. MODELING FRAMEWORK
The framework imports native I/O traces, profiles computation and I/O phases, translates them to a reference platform, samples trace intervals, and calibrates an ElastiSim application model.
- Framework overview: The framework collects native traces and execution-environment information before constructing simulated application profiles.Native traces record read/write volumes over equal-duration intervals, while native platforms are examined by I/O bandwidth and computation power.
- Framework overview: The application profile algorithm estimates computation and I/O durations for every native trace interval.It assumes CPU and I/O times exceed communication times, so communication overhead is omitted in the current framework version.
- Framework overview: The converter translates profiles from multiple native systems to a single reference platform using CPU-power and I/O-bandwidth differences.This enables comparison of trace behaviors in one simulated execution environment.
- Framework overview: The sampling phase converts simulated profiles into sampled I/O traces containing interval-specific data amounts and computation durations.A sampling window may contain one or more intervals and is defined by its start and end times.
- Trace importing and application profiles: QePh illustrates the workflow by comparing native JUWELS execution with simulated reference-platform traces and profiles.Platform-performance differences can make simulated interval times differ from original times; faster platforms reduce CPU and I/O durations.
B. Application profile sampling and model calibration
The sampling and calibration procedures preserve interval data distribution while iteratively tuning computation so the ElastiSim model matches the sampled I/O trace timing.
- Application profile sampling: The sampling algorithm processes profile intervals within a specified window to generate the simulated I/O trace.It uses the sampled data amount, window start and end times, and interval execution times to determine sample sizes.
- Model calibration: The calibrated application model alternates CPU and I/O phases, with each I/O phase processing the data amount assigned to its corresponding trace interval.The initial model contains as many phases as the I/O trace has intervals.
- Model calibration: Algorithm 1 takes task times, an application model, and a threshold as input and outputs an updated application model.Its loop continues while the error exceeds the threshold and updates the model after comparing I/O times and phase durations.
- Model calibration: Calibration iteratively adjusts phase Flops upward or downward until simulated phase durations match sampled-trace intervals within a threshold.The procedure reduces Flops when observed duration is too long and increases them when it is too short.
VI. EVALUATION
The evaluation examines modeled application behavior under interference and validates the simulation by comparing benchmark execution times between real and simulated HPC platforms.
- Evaluation: The study analyzes modeled applications under interference scenarios and separately validates simulation against real benchmark executions.Jacobi and EpiGraph are the benchmark applications used for comparing simulated and real execution times.
A. System platform characterization
The reference platform models a large HPC system with compute nodes and a shared parallel file system for studying application performance and I/O behavior.
- System platform characterization: The reference platform contains 4300 nodes with 48 cores each and a modeled peak node performance of 3.2 Tflops.Each processor is specified as an Intel Xeon Platinum 8160 24C at 2.1 GHz.
- System platform characterization: Its parallel file system is modeled as one shared storage tier accessible to all compute nodes.The storage tier uses the MareNostrum 4 GPFS 4.2.2.0 type.
B. Application modelling
The four use cases are simulated with uniform data distribution across varying node counts, while normalizing traces to the same number of intervals for comparison.
- The modeled use cases are Quantum Expresso-PHonon, Quantum Expresso Car-Parrinello, Nek5000, and WaComM++.
- Figure 4 encodes I/O data volume per interval against node count and interval number.The z-axis is I/O data volume, the y-axis is nodes, and the x-axis is trace intervals.
- Each trace retains the same number of intervals across node counts, normalizing traces even though execution time decreases with more nodes.
C. Framework and ElastiSim validation
The framework and ElastiSim are validated by transferring application models between HPC platforms and comparing simulated with real I/O execution behavior. Both applications show closely matching behavior, including EpiGraph’s complex non-periodic pattern, with reported phase accuracy up to 98.8% and 98.1%.
- Application models generated on the HPC4AI cluster are tested for reproducing Jacobi and EpiGraph behavior on UC3M’s C3 cluster.
- Jacobi and EpiGraph traces are captured with Darshan, converted into ElastiSim-compatible traces, and used to generate application performance models.
- Figure 5 compares simulated and real I/O phases using timestamps and execution-time accuracy for Jacobi and EpiGraph.Each bar represents an application I/O phase.
- EpiGraph’s checkpoint spacing changes across simulated COVID-19 waves, yet the HPC4AI-derived model reproduces this complex pattern on C3.
- 98.8% and 98.1% accuracy are reported for reproducing the performance behavior of Jacobi and EpiGraph with ElastiSim.
D. Performance evaluation
The evaluation examines I/O interference as applications overlap, scale in job size, and vary their data-distribution patterns. Results show that contention and degradation depend on both workload composition and I/O access behavior.
- Experimental setup: The experiments assume same-type jobs with equal node allocations, different submit times, no resource oversubscription, and sufficient simulated CPU cores.
- Concurrent applications: Increasing concurrent QePh jobs reduces one application's I/O throughput because they compete for parallel-file-system bandwidth.QePh reaches the 250 GB/s file-system bandwidth with two simultaneous jobs; contention is lower near the final 20% of execution.
- Concurrent applications: WaComM++ I/O throughput is affected only when a much larger number of jobs runs simultaneously.
- Increasing job size: Increasing QePh job size degrades one application's throughput under both uniform and distributed data patterns.With uniform data, replicated data increases I/O pressure; with distributed data, aggregated throughput exceeds parallel-file-system throughput.
- Increasing job size: The distributed pattern reduces contention during the final 20% of execution because data is distributed across nodes, lowering filesystem pressure.
- Mixed workloads: When QePh and Nek constitute 10% of a distributed workload, degradation begins at 32 concurrent jobs and remains below 20%.At 25% and 50% I/O-intensive workload profiles, degradation begins with fewer jobs and becomes larger.
VII. CONCLUSION
The paper presents a framework that reproduces real-world HPCIO I/O traces in ElastiSim and evaluates it with real-world application use cases. The framework provides feasible, accurate trace reproduction and fine-grained I/O-oriented platform-performance insights.
- The framework creates ElastiSim applications that reproduce real-world I/O traces from the HPCIO repository.
- Evaluation across real-world application use cases indicates feasible and accurate reproduction of native applications in simulation.
- The framework enables fine-grained insights into platform performance from the I/O perspective.
- Future work includes simulating malleable jobs, developing I/O-aware malleable scheduling, adding burst buffers, and refining performance models.