The Architectural Implications of Microservices in the Cloud

1 INTRODUCTION

• Microservices offer modularity, faster deployment, language and framework choice, and component-level debugging, unlike monoliths’ broader update and troubleshooting burden.They also fit container-based datacenters, with each microservice accommodated in a container.Source: §1 · p. 1
• Microservices shift the computation-to-communication ratio, prompting renewed study of server size, instruction-cache pressure, and end-to-end performance unpredictability.Representative end-to-end applications are needed to investigate these questions.Source: §1 · p. 1
• The authors present a movie renting, streaming, and reviewing application built from tens of microservices.The service combines languages and frameworks including node.js, Python, C/C++, Java/Javascript, Scala, Go, nginx, memcached, MongoDB, Mahout, and Xapian; its services communicate over Apache Thrift RPCs.Source: §1 · p. 1
• The study characterizes end-to-end service scalability on a local cluster of 2-socket, 40-core servers, identifies latency-heavy microservices, and quantifies kernel/user time, communication/computation, and instruction-cache pressure.It also relates the shift to microservices to the big-versus-little server debate, putting more pressure on single-thread performance. For server implications, it uses RAPL on high-end servers and low-power SoCs like Cavium’s ThunderX.Source: §1 · p. 1

2 RELATED WORK

• Cloudsuite, TailBench, and Sirius cover batch, interactive, and intelligent-assistant workloads, including web, database, speech, translation, and voice-to-text services,,,,.These suites have been used to study cloud benchmark architecture, interactive performance, and machine-learning acceleration.Source: §2 · p. 1
• Current benchmark suites focus exclusively on single-tier workloads, sometimes modeling traditionally multi-tier applications as independent leaf nodes.This differs from production deployments and prevents study of problems arising from dependencies between tiers.Source: §2 · p. 1

3.1 Description

• The service combines popular open-source microservices through Apache Thrift RPCs to support movie display, reviews, renting, streaming, advertisements, and recommendations.Table 1 reports newly developed and RPC-interface lines of code, plus a per-language breakdown; Docker containers simplify setup and deployment.Source: §3.1 · p. 2, Table 1 · p. 2
• Movie display routes requests through a load balancer and nginx/php-fpm to ComposePage, which aggregates eight Thrift services for movie information.Memcached and MongoDB hold cached and persistent data; video files reside in NFS.Source: §3.1 · p. 2, Figure 1 · p. 2
• Review submission assigns a unique ID and movie ID, then ComposeReview propagates the review to movie and user databases.MovieReview also updates movie statistics and ratings through UpdateMovie.Source: §3.1 · p. 2, Figure 2 · p. 2
• The section characterizes throughput-tail-latency curves for different query types of the end-to-end service.Source: Figure 3 · p. 2
• The service also includes movie and advertisement recommenders, search, Spark analytics, and video playback plugins beyond those pictured in the figures.Source: §3.1 · p. 2

3.2 Methodological Challenges of Microservices

• Distributed tracing records RPC-level latency by timestamping requests at microservice arrival and departure, then storing accumulated data in Cassandra.The system also tracks network-request processing separately from application computation; reported overhead is below 0.1% on 99th-percentile latency and 0.2% on throughput.Source: §3.2 · p. 2

4.2 Implications in Server Design

• Rent/stream requests saturate at lower loads from video bandwidth, while high-load latency shifts toward backend databases; at low load nginx dominates and MongoDB takes 10.3%.The per-microservice latency breakdown compares low and high load.Source: Figure 4 · p. 3, §4.2 · p. 3, §4.1 · p. 3

4.1 Scalability and Query Diversity

• Movie browsing, review, and rent/stream requests all saturate with load, but browsing latency varies with the amount of data stored per movie.The dataset contains 1,000 top-rated IMDB movie records.Source: §4.1 · p. 3

4.2 Implications in Server Design

• Only 22% of cycles go toward committing instructions on average; microservices have more front-end stalls than the same service as a monolith.The monolith has fewer front-end stalls and improved IPC, while misspeculation increases in the monolith.Source: §4.2 · p. 3, Figure 5 · p. 3
• A large fraction of execution occurs in kernel mode and an almost equal fraction in libraries, reflecting network handling and reliance on existing libraries.Kernel work includes interrupt and TCP-packet processing; libraries include libc, libgcc, libstdc, and libpthread.Source: Figure 6 · p. 3, §4.2 · p. 3
• ThunderX meets end-to-end QoS at low load but saturates earlier than high-end servers; Xeon at 1.8GHz still considerably outperforms ThunderX.Low-power machines may suit microservices outside the critical path or insensitive to frequency scaling.Source: Figure 8 · p. 4, §4.2 · p. 4
• Microservices are more sensitive to poor single-thread performance than traditional cloud applications, as each service faces stricter tail-latency constraints.Among the monoliths, MongoDB tolerates near-minimum frequency at maximum load; Xapian is the most sensitive to frequency drops.Source: Figure 7 · p. 4, §4.2 · p. 4
• TCP consumes 5–70% of execution time at low load, while network request queuing and processing dominate at high load.The Java/JS monolith spends 18% of time on client network requests at low load and 41% at high load.Source: §4.2 · p. 3, Figure 9 · p. 4
• Instruction-cache pressure remains high for nginx, memcached, and MongoDB but is lower for other microservices; LLC and D-TLB misses are also below traditional cloud applications.The authors attribute microservices’ better instruction-cache locality to their simplicity and report most instruction-cache misses in the kernel, attributed to Thrift.Source: Figure 10 · p. 4, §4.2 · p. 4

• The study quantifies microservices’ instruction-cache pressure, the trade-off between big and small servers, and their shift in the computation-to-communication ratio.Source: §5 · p. 4
• The analysis uses a new end-to-end movie reviewing and streaming service built with tens of microservices.Source: §5 · p. 4
• As cloud and IoT applications adopt microservices, the paper argues that assumptions used to build and manage cloud systems should be revisited.Source: §5 · p. 4