Source-linked AI summary
Performance Evaluation of Container-based Virtualization for High Performance Computing Environments
Carlos Arango, Rémy Dernat, John Sanabria
TL;DR
HPC applications can suffer substantial performance penalties in virtualized environments, while container implications for scientific computing remain unclear. The paper compares LXC, Docker, and Singularity with bare metal across compute, memory, network, I/O, and GPU workloads. Its preliminary conclusions identify Singularity as generally more suitable for HPC, with workload- and infrastructure-specific trade-offs.
Problem
The paper addresses uncertainty about container-based virtualization's implications for scientific computing and HPC, where virtualized contexts can perform slowly compared with bare metal.
Method
The study benchmarks LXC, Docker, and Singularity against bare metal across compute, memory, network, I/O, and GPU performance dimensions.
Results
Singularity containers are usually more suitable for HPC than Docker or LXC, while LXC is very efficient from a network-performance perspective.
Takeaways & Limitations
Singularity is presented as a good alternative for HPC, while container images can be advantageous for I/O-intensive workloads and Docker-based AUFS solutions should be avoided because of overhead issues.
Abstract
from arXiv · showhide
Virtualization technologies have evolved along with the development of computational environments since virtualization offered needed features at that time such as isolation, accountability, resource allocation, resource fair sharing and so on. Novel processor technologies bring to commodity computers the possibility to emulate diverse environments where a wide range of computational scenarios can be run. Along with processors evolution, system developers have created different virtualization mechanisms where each new development enhanced the performance of previous virtualized environments. Recently, operating system-based virtualization technologies captured the attention of communities abroad (from industry to academy and research) because their important improvements on performance area. In this paper, the features of three container-based operating systems virtualization tools (LXC, Docker and Singularity) are presented. LXC, Docker, Singularity and bare metal are put under test through a customized single node HPL-Benchmark and a MPI-based application for the multi node testbed. Also the disk I/O performance, Memory (RAM) performance, Network bandwidth and GPU performance are tested for the COS technologies vs bare metal. Preliminary results and conclusions around them are presented and discussed.
I. INTRODUCTION
The paper motivates container-based virtualization for HPC by contrasting its lower-overhead path to hardware with hypervisor limitations and unresolved scientific-computing implications. It introduces benchmarks comparing LXC, Docker, and Singularity with bare metal across computational and system-performance dimensions.
- HPC applications have generally received slower performance in virtualized contexts than on bare metal, despite advances in virtualization and tailored cloud tools.
- Hypervisors can penalize applications that rely on native optimized processor instructions by presenting virtual machines with a simplified hardware view.
- Containers remove the hypervisor layer and use namespaces and cgroups to provide isolation and resource accounting.
- Container technologies accelerate application development and deployment, but their implications for scientific computing, including HPC, remain uncertain.
- The study compares Docker, LXC, and Singularity with bare metal using benchmarks for compute, memory bandwidth and latency, network bandwidth, and I/O bandwidth.
II. CONTAINER-BASED OPERATING SYSTEM VIRTUALIZATION TECHNOLOGIES
The paper describes container-based operating-system virtualization as a lightweight alternative built around kernel isolation and resource controls, then distinguishes LXC, Docker, and Singularity. Their differing features and design priorities motivate HPC benchmarking, where early results suggest that fewer features can improve performance.
- Container-based operating systems isolate resources such as processes, users, filesystems, and networks while remaining closer to operating-system services than traditional virtualization.
- LXC: LXC uses Linux namespaces and cgroups for resource isolation, accounting, and limits, while supporting route- and bridge-based networking and container scheduling.
- Docker: Docker extends LXC with APIs, network-service virtualization, cgroup-based resource control, and the layered AuFS filesystem.
- Singularity: Singularity emphasizes application portability, compute mobility, reproducibility, and isolation rather than broad host virtualization features.
- Comparison: Preliminary HPC benchmark results indicate that the absence of some container features can positively affect performance.
III. METHODOLOGY AND BENCHMARKS
The study evaluates container-based operating-system virtualization against bare metal using computational, memory, I/O, network, and GPU benchmarks across single-node and multi-node environments.
- Methodology: Experiments compared current COS implementations with bare metal across computational performance, memory, network, I/O, and GPU overhead.The experiments were conducted at two cluster facilities.
- Basic operation: Six basic-operation variants compared native execution with Singularity, Docker, and LXC commands, including container startup and shutdown.The measurements used /usr/bin/time on the host; shutdown operations were the slowest.
- CPU performance: 5.42% better than native performance was reported for Singularity on HPL, while LXC and Docker incurred 7.76% and 2.89% overheads, respectively.The authors associate Docker’s overhead with default CPU-use restrictions and Singularity’s result with mount-namespace-based execution.
- Disk I/O performance: 37.28% write and 65.25% read performance losses were observed with Docker, whereas LXC and Singularity had similar write results and Singularity slightly reached native read performance.The authors relate Docker’s larger I/O overhead to AUFS copy-on-write and layered-file searches.
- Memory and network performance: Docker showed approximately 36% average memory-throughput overhead, while LXC had the best network scores and Docker’s bandwidth was 16.96% below native.The STREAM benchmark measured sustainable memory bandwidth; network results were evaluated with MPI bandwidth and latency tests.
- GPU performance: For the NAMD benchmark, Docker and Singularity achieved better performance than native, while LXC did not reach native performance.The benchmark reports days/ns, with lower values indicating better performance.
IV. RELATED WORK
Prior work examined container overhead mainly against classic virtual machines, while broader scientific-computing implications and GPU virtualization remained insufficiently studied. This paper contributes a preliminary standardized assessment of three container-based virtualization technologies for HPC.
- Prior studies: Earlier studies mainly compared container-based virtualization overhead with classic virtual-machine technologies such as KVM and LinuxVServer.Those studies also reported poor isolation and security from current LXC and Docker resource-management implementations.
- Scientific computing: Containers support scientific portability and reproducibility, but their broader implications for scientific computing remained incompletely characterized.The related work identifies existing scientific use cases while indicating that important implications were still under study.
- GPU virtualization: GPU virtualization had not been broadly studied because virtualizing GPU technologies presents substantial challenges.The motivation reflects growing use of GPUs in research and cloud services.
- This work: The paper claims no similar publication assessing three container-based operating-system virtualization technologies with standardized benchmarks.The approach is presented as a preliminary characterization of container-based and virtualization technologies generally.
- HPC evaluation gap: Research on container solutions for heavy HPC applications was limited, and prior HPL evaluations differed in benchmark version, compilation workflow, and CPU architecture.The authors compiled the benchmark binary inside each container to reproduce a normal HPC-cluster workflow.
V. CONCLUSIONS
The paper compares Docker, LXC, and Singularity with bare metal across HPC-relevant workloads and system dimensions. Singularity is generally the most suitable for HPC, while deployment choices remain workload- and infrastructure-dependent.
- Overall comparison: Singularity containers are usually more suitable for HPC implementation than Docker or LXC.The comparison covers container-based virtualization tools against bare-metal execution.
- Network and CPU: LXC is very efficient from a network perspective, but Singularity may perform better when its libraries are more efficient than the host's.The authors attribute overhead in LXC and Docker to cgroups resource control for CPU-intensive processes.
- I/O workloads: For I/O-intensive workloads, container images can outperform execution against shared storage, while standard Docker AUFS is discouraged because of overhead issues.The paper relates the advantage to keeping data close to the process.
- Small and memory jobs: Docker is discouraged for small tasks and memory jobs unless the container is already running on the host.Docker image distribution and startup can impose additional HPC-system costs, whereas Singularity uses one shareable file.
- Security and access: Singularity blocks privilege escalation, whereas Docker's isolation can preclude access to high-performance networks and optimized storage platforms.The comparison highlights security and infrastructure-access trade-offs between the container technologies.
- Scope and boundary: COS, particularly Singularity, is presented as an alternative for reducing virtualization overhead in mixed HPC and HPSS environments.HPC deployment must still address container overhead, privilege escalation, network and filesystem access, and workflow compatibility.