Source-linked AI summary
Performance Evaluation of Microservices Architectures using Containers
Marcelo Amaral, Jordà Polo, David Carrera, Iqbal Mohomed, Merve Unuvar, Malgorzata Steinder
TL;DR
The paper addresses how to implement microservices-oriented related-process ensembles with containers while balancing deployment flexibility and performance. It compares master-slave and nested-container models using CPU and network benchmarks. Nested containers preserve CPU performance but introduce network trade-offs relative to bare-metal and regular containers.
Problem
The paper asks how master-slave and nested-container models differ in CPU and network performance when implementing related processes per container for microservices.
Method
The study benchmarks CPU and network behavior across master-slave and nested-container implementations, including single-host networking configurations.
Results
Nested containers have no significant CPU-performance impact, but incur network-performance trade-offs compared with bare-metal and regular containers.
Takeaways & Limitations
Nested containers are a suitable model when improved resource sharing, IPC, fate sharing, and infrastructure-management flexibility are valuable.
Abstract
from arXiv · showhide
Microservices architecture has started a new trend for application development for a number of reasons: (1) to reduce complexity by using tiny services; (2) to scale, remove and deploy parts of the system easily; (3) to improve flexibility to use different frameworks and tools; (4) to increase the overall scalability; and (5) to improve the resilience of the system. Containers have empowered the usage of microservices architectures by being lightweight, providing fast start-up times, and having a low overhead. Containers can be used to develop applications based on monolithic architectures where the whole system runs inside a single container or inside a microservices architecture where one or few processes run inside the containers. Two models can be used to implement a microservices architecture using containers: master-slave, or nested-container. The goal of this work is to compare the performance of CPU and network running benchmarks in the two aforementioned models of microservices architecture hence provide a benchmark analysis guidance for system designers.
I. INTRODUCTION
Containers provide lightweight, fast-starting operating-system-level virtualization, while RPPC groups related processes per container for deployment and microservices. The paper compares master-slave and nested-container implementations of RPPC to understand their performance differences.
- Containers: Containers isolate and control process resources through operating-system-level virtualization, avoiding the overhead of emulating physical hardware.Their core mechanisms rely on Linux namespaces and cGroups.
- Related Processes Per Container: RPPC places one process or a few related processes in each container instead of treating a container as a lightweight server.The paper presents RPPC as a useful building block for microservices architectures.
- Related Processes Per Container: Grouping related processes into containers allows subsets of an application ensemble to be redeployed when functionality changes.This avoids repackaging the entire application unit.
- Microservices: Microservices organize systems as independently developed, managed, and scaled services connected through a routing fabric.Kubernetes uses pods as deployable groups of containers whose members share the same fate.
- Compared Models: The study compares master-slave, where a parent manages peer child containers, with nested-container, where child containers run inside a privileged parent namespace.The comparison aims to characterize performance differences for future system design.
II. MICROSERVICES ARCHITECTURE USING CONTAINERS
Containers offer lightweight virtualization, but privileged nested containers introduce security considerations and additional virtual-networking layers. The paper therefore examines how networking configurations affect communication performance on a single host.
- A. Containers: Containers provide operating-system-level virtualization that isolates and controls processes with less overhead than virtual machines.The approach uses Linux container mechanisms rather than physical-hardware emulation.
- A. Containers: A privileged Docker container can access host devices and run another Docker daemon to create child containers.This capability enables the nested-container approach.
- A. Containers: Sharing system resources with other users can create security challenges for privileged nested containers.The paper identifies this as a caution associated with the approach.
- B. Key virtual networking differences: Single-host communication may traverse native, virtual-switch, nested-switch, or virtual-machine networking layers.The default Docker setup connects containers through Linux Bridge or OpenvSwitch, while nested containers add another virtual-switching level.
C. Microservices
Microservices decompose applications into independent services, and containers provide a lightweight deployment vehicle for these services. The paper considers master-slave and nested-container organization models, which trade management simplicity and shared resources against possible overhead.
- C. Microservices: Microservices replace monolithic applications with many independent services that work together.The architecture supports simpler individual codebases and isolated service updates and scaling.
- C. Microservices: Docker containers suit microservices because they are lightweight, start quickly, and encapsulate implementation dependencies.Container images can include preconfigured middleware, databases, and applications.
- D. Achieving Microservices with Containers: Deploying services as container ensembles leads to two organizational models: master-slave and nested-containers.The paper illustrates both models in Figure 1.
- D. Achieving Microservices with Containers: Master-slave uses a coordinating master and peer slave containers, whereas nested-containers hierarchically place child containers inside a parent.Nested containers may simplify management, IPC, fate sharing, and resource sharing, but can add overhead from two Docker-daemon layers.
III. RELATED WORK
Prior work established containers, Docker, and microservices management, but this paper addresses a specific gap: performance analysis of nested and master-slave container implementations.
- This paper identifies its work as the first analysis of microservices performance using nested and master-slave container implementations.
- Containers reduce virtualization overhead relative to virtual machines, while providing somewhat less isolation.
- Nested-containers draw inspiration from Google’s pod concept for managing Docker containers, whose performance over virtual machines or bare metal had not been studied.
- Kubernetes and Mesos represent related approaches to microservices or cluster management, although the cited Mesos work did not consider container technology.
IV. EVALUATION
The evaluation studies nested-containers as a microservices architecture approach and compares them with bare metal, regular containers, and virtual machines across CPU- and network-oriented experiments.
- Five experiments examine CPU and network performance, including CPU-intensive execution and virtual-container creation overhead.
- The study compares bare metal, regular containers representing master-slave deployment, nested-containers, and virtual machines in every experiment.
B. Experiment 1: CPU Performance Evaluation
The CPU experiment compares Sysbench execution across bare metal, regular containers, nested-containers, and virtual machines as concurrency increases. All environments show similar CPU behavior relative to bare metal, with slowdown accelerating beyond the machine’s core capacity.
- Sysbench CPU mode calculates prime numbers up to 40,000 using 64-bit integers.
- The experiment measures mean execution time for 1–64 concurrent instances, with containers and virtual machines each running one Sysbench instance.
- Figure 3 reports slowdown relative to one Sysbench instance running on bare metal and averages 10 executions per environment and instance count.
- All environments show similar behavior, indicating no significant CPU performance impact for containers or virtual machines compared with bare metal.
- Slowdown increases slowly through 8–16 concurrent instances and faster thereafter because the 12-core machine must share cores among additional instances.
C. Experiment 2: Comparing Overhead of Virtual Container Creation
The second experiment evaluates the scalability and overhead of creating containers, nested-containers, and virtual machines. Regular containers are fastest, nested-containers are intermediate, and their overhead reflects parent initialization and becomes less favorable at higher concurrency.
- The experiment measures creation time for 1–64 concurrent hosting entities that launch a negligible dummy application.
- Regular containers are always fastest, followed by nested-containers and virtual machines.
- Almost 8 times more overhead is required for one nested-container than one regular container, while nested-containers remain more than twice as fast as virtual machines.
- Nested-container overhead comes from initializing Docker in the privileged parent, loading its local image, and creating the child container.
- 6.2s average image loading contributes to overhead; sharing a preloaded read-only volume reduced creation time from 8s to 1.7s.
- Above 8 nested-containers, creation time appears more than linear and approaches virtual-machine performance, although nested-containers remain twice as fast in most scenarios.
D. Experiment 3: Overhead of Nested-Container Creation
Experiment 3 measures nested-container creation time across different parent-to-child ratios. Creation is dominated by parent initialization, while distributing children across a small number of parents can reduce elapsed time.
- Measurement method: Parent initialization includes starting Docker, loading the image, starting child containers, and exiting them.
- Creation-time results: 111 seconds is required for one parent with 256 children, compared with 2312 seconds for 256 parents with one child.The difference is attributed to the initialization bottleneck of parent containers.
- Creation-time results: With fewer than 16 parents, concurrent child creation across parents benefits the creation of 256 children.
- Creation-time results: More than 139 seconds is required when the host is overloaded with more parents than available cores.
E. Experiment 4: Network Performance - Local traffic in one host
Experiment 4 evaluates local network performance for bare-metal, containers, virtual machines, and nested-containers using TCP throughput and request/response tests. Containers generally outperform virtual machines, while Host-Network can match bare-metal performance but introduces security concerns for parent containers.
- Method: TCP_STREAM measures throughput, while TCP_RR exchanges synchronous requests and responses to assess communication timing.
- Experimental setting: Local throughput can exceed 1Gbps because packets use the loopback interface rather than the host’s single 1-Gigabit NIC.CPU throughput is the bottleneck in this same-host experiment.
- Results: Containers generally provide higher network performance than virtual machines and can be as fast as bare-metal under some configurations.
- Results: Host-Network containers display approximately bare-metal throughput and latency, whereas Linux Bridge and OpenvSwitch configurations perform significantly worse.
- Nested-container results: Nested-container configurations with Host-Network are approximately twice as fast as Linux Bridge or OpenvSwitch in both throughput and latency.Host-Network in a parent container poses security concerns because privileged parents may access host packets.
F. Experiment 5: Network Performance - Remote traffic across two hosts
Experiment 5 measures network performance across two hosts connected by a physical 1-Gigabit switch. The physical network becomes the primary bottleneck, reducing differences among container configurations, while virtual machines show higher latency than containers.
- Method: The experiment connects two hosts through a physical 1-Gigabit switch and runs the client across bare-metal, container, and virtual-machine environments.
- Method: All configurations are evaluated with TCP_STREAM and TCP_RR, using five repetitions and reporting averages with standard deviations.
- Results: Remote throughput is lower and latency higher than in the local experiment, changing from Gbps and µs to Mbps and ms.
- Results: The physical network becomes the bottleneck, so performance varies less across virtual-container and network-virtualization configurations.
- Results: Virtual-machine latency is approximately 12% higher because the guest operating system simulates the entire network stack.
- Nested-container results: Nested-container configuration differences have no significant performance impact once the physical network is the bottleneck.Host-Network remains associated with security concerns when used in the parent container.
V. CONCLUSIONS
The paper evaluates nested-containers as an RPPC implementation and finds them suitable for microservices because they support resource sharing and IPC without significant CPU impact. Network performance, security, and simplicity remain trade-offs when selecting an implementation model.
- Contribution: RPPC groups related processes into container ensembles intended to improve infrastructure management and application deployment.
- Compared models: The paper compares master-slave and nested-container models for implementing RPPCs.Master-slave uses peer child containers managed by a parent, while nested-container places child containers inside a privileged parent’s namespace.
- Conclusions: Nested-containers support resource sharing, IPC, and fate sharing among containers in the same nested-container.
- Conclusions: Nested-containers do not significantly affect CPU performance but involve network-performance trade-offs relative to bare-metal and regular containers.
- Trade-offs: Selecting a microservices implementation model requires considering network performance, security, and simplicity.