Source-linked AI summary
About Microservices, Containers and their Underestimated Impact on Network Performance
Nane Kratzke
TL;DR
The performance of containers on hypervisor-based VMs with overlay networking and encryption had been little analyzed. This study benchmarks those layers in a cloud-based HTTP/REST ping-pong system. It finds measurable container losses, larger SDVN impacts, and comparatively minor encryption impacts, and derives recommendations for minimizing losses.
Problem
Performance impacts of containers on hypervisor-based VMs connected by overlay networks, sometimes encrypted, had received little analysis.
Source: §I · p. 1Method
The study benchmarks message-transfer rates in an HTTP/REST ping-pong system across configurations adding containers, SDVN, and encryption.
Source: §III · pp. 2–3Results
Weave SDVN retained 60% of reference performance for small messages and about 25% for messages greater than 100 kB, while encryption reduced transfer performance to about 90% of non-encrypted rates.
Source: §IV · p. 4Takeaways & Limitations
The paper recommends minimizing message sizes, running SDVN routers directly on hosts, and deploying them on multi-core virtual machines to reduce performance impacts.
Source: §V · p. 5
Abstract
from arXiv · showhide
Microservices are used to build complex applications composed of small, independent and highly decoupled processes. Recently, microservices are often mentioned in one breath with container technologies like Docker. That is why operating system virtualization experiences a renaissance in cloud computing. These approaches shall provide horizontally scalable, easily deployable systems and a high-performance alternative to hypervisors. Nevertheless, performance impacts of containers on top of hypervisors are hardly investigated. Furthermore, microservice frameworks often come along with software defined networks. This contribution presents benchmark results to quantify the impacts of container, software defined networking and encryption on network performance. Even containers, although postulated to be lightweight, show a noteworthy impact to network performance. These impacts can be minimized on several system layers. Some design recommendations for cloud deployed systems following the microservice architecture pattern are derived.
I. INTRODUCTION
Microservices compose large, horizontally scalable applications from independent processes, and containers are promoted as a lightweight alternative to hypervisors. The study addresses the limited analysis of containers layered on hypervisor-based VMs with overlay networking and encryption, and outlines its experiments and design recommendations.
I. INTRODUCTION
- Microservices combine small, independent, highly decoupled processes through language-agnostic APIs to build complex, horizontally scalable applications.The paper notes Docker-based containers are often presented as a high-performance hypervisor alternative, while prior studies reported little performance impact.Source: §I · p. 1
- Performance impacts of containers on hypervisor-based VMs connected by encrypted overlay networks had received little analysis, despite this combination appearing in IaaS microservice frameworks.Prior studies typically compared containers with virtual machines, rather than examining containers on top of virtual machines.Source: §I · p. 1
- The study benchmarks container, SDVN, and encryption impacts, then derives recommendations for application, overlay-network, and IaaS infrastructure layers.The benchmark tooling and collected performance data are provided online.Source: §I · p. 1
II. RELATED WORK
Prior work found Docker containers close to bare-metal performance, but did not study containers running on hypervisors. This study selects Weave as its SDVN candidate and targets HTTP-based, REST-like microservice traffic.
II. RELATED WORK
- Felter et al. found Docker benchmarks near bare-metal performance across processing, memory, networking, storage, and database workloads, but did not test containers on hypervisors.Their study compared traditional virtual machines with Linux containers.Source: §II · pp. 1–2
- Weave was the only identified open-source Docker SDVN able to encrypt transfers, so flannel and docknet were excluded.Open vSwitch was also excluded because it was not designed for operating-system virtualization.Source: §II · p. 2
- Earlier cloud-network performance studies,,,,, did not explicitly target horizontally scalable HTTP-based, REST-like systems.The paper proposes its experiment design to address that use case.Source: §II · p. 2
III. EXPERIMENT DESIGN
The study uses a REST-like HTTP ping-pong benchmark to measure message transfer rates across five experiments. The experiments add containers, SDVN, encryption, or cross-region deployment to a reference configuration.
III. EXPERIMENT DESIGN
- Five experiments measured transfer rates for m-byte messages in an HTTP-based REST-like ping-pong system using Apachebench on AWS m3.medium instances in Ireland.The ping and pong applications were written in Dart.Source: §III · p. 2
- The reference experiment ran ping and pong on separate virtual machines without containers, SDVN, or encryption.Siege ran Apachebench with increasing requested message sizes; each request was relayed from ping to pong and answered with a response.Source: §III · p. 2
- The Docker experiment added containers for the ping and pong services to isolate the impact of an additional container layer.Its measured transfer rates are compared with the reference experiment.Source: §III · p. 3
- The SDVN experiment connected the ping and pong containers through an SDVN, so transfers passed through the network solution.The measured rates were used to calculate SDVN impact.Source: §III · p. 3
- The encryption experiment added data encryption to the SDVN layer and measured the resulting transfer rates.This configuration encrypted the SDVN network.Source: §III · p. 3
- The cross-region experiment placed ping in Ireland and pong in Tokyo, using the reference setup otherwise.Its data were used only for qualitative comparison with container, SDVN, and encrypted SDVN impacts.Source: §III · p. 3
IV. DISCUSSION OF RESULTS
The benchmarks found measurable network-performance losses from containers and larger losses associated with SDVN, while encryption had a comparatively minor impact. The paper recommends host-based SDVN deployment and multi-core systems to reduce these impacts.
Experiment reliability
- More than 12 hours of benchmarking transferred over 316 GB across more than 6 million HTTP requests, and the authors judged the collected data reliable.Reference and Docker experiments had minor standard deviations; deviations increased with SDVN and were attributed to its technical impacts.Source: §IV · p. 3
Container impact
- An additional container layer on a bare virtual machine reduced data-transfer performance to 80% for message sizes smaller than 100 kBytes and 90% for sizes greater than 100 kBytes; cross-zone deployment showed almost the same performance as one-zone deployment.Source: §IV · pp. 3–4
SDVN impact
- SDVN performance fell from 60% of reference for small messages to about 25% for messages over 100 kB; the SDVN experiment also included containerization.Figure 3 compares relative performance indicators; the reported SDVN measurements combine containerization and SDVN effects.Source: §IV · p. 4, Figure 3 · p. 4
Design recommendations
- To reduce performance losses, the paper recommends running SDVN directly on the host and on multi-core systems to avoid container losses and CPU contention.The tested m3.medium virtual machine was single-core; under saturated network load, the Weave router contended with application processes.Source: §IV · p. 4
Encryption impact
- Encryption reduced transfer performance to about 90% of non-encrypted rates, with smaller effects for smaller messages; most losses were attributed to SDVN.For larger messages, encryption’s impact was comparable to containerization.Source: §IV · p. 4
V. CONCLUSION
For HTTP-based, REST-like services on IaaS, containers and overlay networks can impose substantial network-performance costs, while encryption loss is minor especially for small messages. The paper recommends reducing those costs at the application, overlay-network, and infrastructure layers without dismissing the flexibility and manageability of container and SDVN technologies.
V. CONCLUSION
- Container performance impact is about 10% to 20%, SDVN impact is about 30% to 70%—comparable to cross-regional Ireland–Japan deployment—and encryption loss is minor, especially for small messages.The study concludes that container impact on network performance is not negligible.Source: §V · pp. 4–5
- Minimize inter-component message sizes; host SDVN routers to reduce performance impact by 10% to 20%; and use multi-core VMs to avoid router contention with application processes.The reported 10% to 20% optimization is attributed to general container losses.Source: §V · p. 5
- Container and SDVN technologies should not be avoided generally, but used with their performance implications in mind alongside their flexibility and manageability benefits.The recommendations concern cloud-deployed HTTP-based and REST-like services on IaaS; the authors say conclusions should be cross-checked with other SDVN solutions.Source: §V · pp. 4–5