PaperlayerView exact revision on arXiv

Source-linked AI summary

Container-based Cluster Orchestration Systems: A Taxonomy and Future Directions

Maria A. Rodriguez, Rajkumar Buyya

arXiv:1807.06193v2cs.DC

TL;DR

Container orchestration systems must manage diverse workloads across large clusters while balancing utilization, placement, scalability, and availability. This paper proposes a taxonomy and applies it to state-of-the-art systems. The classification surfaces system trade-offs and open research directions, including cloud-cluster elasticity and QoS management.

  • Problem

    Orchestration systems must meet diverse application requirements while scaling across large clusters and balancing throughput, deployment delay, utilization, fault tolerance, availability, and fairness.

    Source: §1 · p. 3

  • Method

    The paper introduces a reference architecture and classifies container orchestration systems by application, scheduling, and resource-management characteristics.

    Source: §7 · p. 27

  • Results

    Two-level schedulers offer framework-level flexibility, while shared-state schedulers support simultaneous pluggable schedulers; commercial systems include more advanced features than open-source systems.

    Source: §5 · pp. 20–21

  • Takeaways & Limitations

    The identified research directions include elastic cloud-based clusters, cost-aware scheduling, rescheduling for dynamic clusters, and stronger application QoS management.

    Source: §7 · p. 27

    Scaling shared-state systems to many schedulers while remaining efficient is an open research question.

    Source: §5 · p. 21

Abstract

from arXiv · show

Containers, enabling lightweight environment and performance isolation, fast and flexible deployment, and fine-grained resource sharing, have gained popularity in better application management and deployment in addition to hardware virtualization. They are being widely used by organizations to deploy their increasingly diverse workloads derived from modern-day applications such as web services, big data, and IoT in either proprietary clusters or private and public cloud data centers. This has led to the emergence of container orchestration platforms, which are designed to manage the deployment of containerized applications in large-scale clusters. These systems are capable of running hundreds of thousands of jobs across thousands of machines. To do so efficiently, they must address several important challenges including scalability, fault-tolerance and availability, efficient resource utilization, and request throughput maximization among others. This paper studies these management systems and proposes a taxonomy that identifies different mechanisms that can be used to meet the aforementioned challenges. The proposed classification is then applied to various state-of-the-art systems leading to the identification of open research challenges and gaps in the literature intended as future directions for researchers.

1 | INTRODUCTION

Containers offer OS-level isolation and flexible resource sharing, and their rapid deployment and broad workload support have driven adoption of cluster orchestration platforms. These systems must manage large, diverse workloads while balancing placement speed, utilization, scalability, availability, and fairness.

1 | INTRODUCTION

  • Containers package software and dependencies with OS-level process isolation, sharing a host kernel rather than virtualizing hardware like virtual machines.They use resources as needed and can be built from lightweight filesystem layers.Source: §1 · p. 2
  • Orchestration platforms manage containerized workloads at large scale, with systems capable of running hundreds of thousands of jobs across thousands of machines.Their adoption followed container benefits including flexible deployment and resource utilization.Source: §1 · p. 2
  • Container orchestration must balance fast placement and high resource utilization with scalability, fault tolerance, availability, and fair allocation.Workloads may include highly available services, deadline-constrained batch jobs, or latency-sensitive applications.Source: §1 · pp. 2–3
  • The paper studies how orchestration systems meet these requirements and compares their capabilities, building on surveys of VM-based resource managers.The paper presents its reference architecture, taxonomy, system classifications, and future directions in later sections.Source: §1 · p. 3

2 | REFERENCE ARCHITECTURE FOR CONTAINER ORCHESTRATION SYSTEMS

The reference architecture describes how jobs move through a cluster manager to worker nodes and the infrastructure hosting them. Master and worker components monitor resources and tasks, make placement decisions, and manage cluster capacity.

2 | REFERENCE ARCHITECTURE FOR CONTAINER ORCHESTRATION SYSTEMS

  • The architecture has four layers: jobs, a cluster-manager master, a compute cluster, and the physical infrastructure.Users submit jobs composed of tasks; the master assigns tasks to worker nodes for execution.Source: §2 · pp. 3–4
  • Jobs may contain heterogeneous tasks and express resource needs and quality-of-service requirements such as fault tolerance, deadlines, priorities, or QoS classes.Jobs can belong to different users and range from long-running latency-sensitive services to short-lived batch workloads.Source: §2 · p. 3
  • The accounting module records infrastructure metrics such as utilization, energy, and cloud cost, as well as user job and resource-consumption metrics.These measurements may support quotas or billing estimates.Source: §2 · p. 4
  • Master-side modules track resource usage, enforce admission decisions, schedule and relocate tasks, and launch and monitor running containers.Admission control may reject jobs, preempt lower-priority jobs, or prompt adding nodes when resources are insufficient.Source: §2 · pp. 3–5
  • Worker agents report local resource metrics, manage task execution and resources, and monitor containerized tasks on each worker node.A resource provisioner may add nodes manually or dynamically add and remove virtual nodes as demand changes.Source: §2 · p. 5

3 | TAXONOMY

The taxonomy organizes container-based cluster orchestration systems by their application, scheduling, and resource models. It describes characteristics and elements in these perspectives and relates them to system examples.

3 | TAXONOMY

  • The section identifies multiple co-located tasks as a taxonomy topic.Source: §3 · p. 6
  • The taxonomy examines container orchestration systems from application, scheduling, and resource-model perspectives.Its classifications concern characteristics and elements of these platforms.Source: §3 · p. 6
  • The taxonomy explains each classification and points to system examples and references presented in the system survey.The classifications are intended to characterize container-based cluster orchestration systems.Source: §3 · p. 6

3.1 | Application Model

The application-model taxonomy describes workload types and ways users can express jobs. Mixed workloads can improve resource utilization, while their differing QoS needs complicate scheduling.

3.1 | Application Model

  • The application-model taxonomy covers ways users describe jobs and the characteristics those applications may have.It is linked to the job components in the reference architecture.Source: §3.1 · p. 6
  • Shared clusters commonly combine high-availability, latency-sensitive services with finite-lifetime batch jobs and periodically submitted cron jobs.Cron-job timing is known in advance and can inform scheduling decisions.Source: §3.1 · p. 6
  • Mixed workloads bring different QoS requirements but can improve utilization; Google reported needing 20-30% more machines when long-running and batch jobs ran in separate clusters.Less performance-sensitive jobs can use resources claimed but not currently used by jobs with stricter requirements.Source: §3.1 · p. 6
  • Jobs may be single tasks, sets of independent identical tasks, co-located dependent tasks, or task graphs with communication patterns and dependencies.Borg uses independent-task jobs, Kubernetes supports co-located dependent tasks, and Apollo lets users define task graphs.Source: §3.1 · p. 7

3.2 | Scheduling

The scheduling taxonomy distinguishes scheduler architecture from job-placement policies. These choices balance scalability, flexibility, scheduling speed, and placement quality.

3.2.1 | Scheduler Architecture

  • Scheduler architectures range from centralized global-view schedulers, which can optimize placement but face single-point-of-failure and scalability risks, to decentralized replicas and two-level designs.Decentralized replicas may be monolithic or specialize by workload; two-level systems delegate placement to application frameworks.Source: §3.2.1 · pp. 7–8
  • Decentralized schedulers improve scalability through replicas, while shared cluster state with optimistic concurrency increases parallelism but can waste effort when conflicts occur.Two-level schedulers separate resource management from application-framework placement decisions, offering flexibility and reducing central-scheduler load; offer-based locking is best suited to frameworks making fast decisions on small, short-lived tasks.Source: §3.2.1 · pp. 7–8

3.2.2 | Job Scheduling

  • Job-scheduling policies classify placement constraints, node-pool selection, preemption, and rescheduling as key choices.Constraints include value-based, label-based, query-based, limit-based, and affinity-based forms; affinity and anti-affinity rules place jobs relative to other jobs and can affect scheduling performance.Source: §3.2.2 · pp. 8–10
  • Schedulers may evaluate every node or sample a subset, reducing placement time and increasing request throughput at the cost of considering fewer nodes.Borg samples nodes randomly; another option partitions the cluster among scheduler replicas.Source: §3.2.2 · p. 8
  • Preemption lets higher-priority tasks take resources from lower-priority tasks, but can waste work; rescheduling also handles failures and resource-based evictions.Constraints can tailor placement to application requirements, while two-level schedulers leave placement decisions to application frameworks.Source: §3.2.2 · p. 9

3.3 | Cluster Infrastructure and Management

The section classifies cluster infrastructure, resource-management mechanisms, system objectives, and multi-tenancy features. It connects these design choices to resource use, availability, and tenant isolation.

3.3.1 | Cluster Infrastructure

  • Infrastructure may be static or elastic, with resources added or removed manually or by an autoscaler that reacts to system state.Autoscaling adds complexity but may improve utilization, performance, and energy consumption.Source: §3.3.1 · pp. 10–11

3.3.2 | Resource Management

  • Resource management combines user task requests and limits with allocation granularity, oversubscription, consumption estimation, and reclamation.Requests guide placement against available resources; limits may trigger throttling for compressible CPU use or eviction for non-compressible RAM. Fine-grained requests can improve utilization, while oversubscription reuses reserved resources based on consumption estimates and may require reclaiming them.Source: §3.3.2 · pp. 11–12

3.3.3 | System Objectives

  • System objectives include scalability, availability, utilization, throughput, and application QoS; scalability mechanisms include scheduler replicas, optimistic concurrency, quick decisions, caching, equivalence classes, and node sampling.High throughput can reduce placement delay but may require compromising placement optimality; the surveyed systems do not address heterogeneous QoS parameters.Source: §3.3.3 · pp. 12–13
  • The taxonomy counts an objective only when its enabling functionality is built into core system components, not when users can configure it through constraints, replication, or QoS classes.This scope distinguishes built-in system behavior from user-provided mechanisms.Source: §3.3.3 · p. 13

3.3.4 | Multi-tenancy Features

  • Multi-tenancy across organizations calls for stronger isolation than sharing a cluster among trusted teams within one organization.Cross-organization deployments may need separate control-plane objects, enforced quotas, strong network and security isolation, or sole-tenant nodes; within an organization, shared control planes and OS-container isolation are commonly sufficient.Source: §3.3.4 · p. 14
  • Multi-tenancy features cover security and container-centric isolation, network isolation, and performance isolation through quotas and scheduling policies.Network isolation can restrict communication between tenant domains; performance isolation aims to prevent one tenant’s resource use from negatively affecting others.Source: §3.3.4 · pp. 14–15

4 | SURVEY

The survey applies the proposed taxonomy to state-of-the-art orchestration systems. Borg is presented as a large-scale system whose reported utilization and task throughput accompany a broad set of resource-management features.

4.1 | Borg

  • Borg serves hundreds of thousands of heterogeneous jobs across clusters with tens of thousands of machines, including production services and lower-priority batch jobs.Jobs comprise homogeneous tasks, and Borg uses a proprietary container stack based on Linux cgroups.Source: §4.1 · p. 15
  • Borg schedules pending tasks by priority and round robin, then finds feasible machines and scores them for package availability, failure-domain spreading, and workload packing.It dynamically estimates resource use, reclaims excess for lower-quality work, throttles CPU, and generally kills tasks that exceed RAM reservations.Source: §4.1 · pp. 15–16
  • Borg reports 60–70% cluster utilization and approximately 10 thousand tasks per minute across tens of thousands of nodes.The survey characterizes Borg as advanced, citing oversubscription with consumption estimation, mixed workloads, fine-grained requests, and preemption.Source: §4.1 · p. 16

4.2 | Kubernetes

Kubernetes combines pod-based workload placement and resource controls with pluggable scheduling options. Swarm offers lightweight Docker scheduling, but does not automatically recover failed nodes or services.

4.2 | Kubernetes

  • Pod resource requests inform placement, and Kubernetes prevents the sum of requests on a node from exceeding its capacity.Limits can trigger different outcomes: memory-limit exceedance may terminate a container, while storage-limit exceedance leads to eviction.Source: §4.2 · p. 16
  • Kubernetes is mature and supports pluggable schedulers, although its performance and scalability may not reach Borg’s levels.As of version 1.10, it supports clusters of up to 5000 hundred nodes.Source: §4.2 · p. 17

4.3 | Swarm

  • Swarm offers spread, binpack, and random placement strategies, with value-, label-, and affinity-based placement constraints.Constraints can target nodes by operating system, stored image, deployed service, or cluster rack.Source: §4.3 · p. 17
  • Swarm’s simple scheduling policies support fast placement; its performance was evaluated in an environment running 30,000 containers.It specializes in Docker containers and is described as lightweight and flexible.Source: §4.3 · p. 17
  • Unlike most surveyed systems, Swarm does not automatically handle node failures by redeploying failed replicas.The framework also does not automatically handle service failures.Source: §4.3 · p. 17

4.4 | Mesos

Mesos distributes cluster resources to application frameworks through offers, leaving frameworks to schedule their own tasks. Aurora and Marathon build on Mesos with additional application models and service-management features.

4.4 | Mesos

  • Mesos offers cluster resources to frameworks, which can accept or reject them before scheduling tasks with their own schedulers.Offers follow policies that preserve priorities and fairness, and Mesos launches tasks on the corresponding hosts.Source: §4.4 · p. 18
  • Time-bounded offers incentivize frameworks with long scheduling cycles to decide quickly, while resource reservations can be configured for short tasks.Reserved resources are associated with a maximum task duration, after which tasks running on them are killed.Source: §4.4 · p. 18
  • Mesos combines framework flexibility and fine-grained resource sharing, which the paper associates with high utilization, adaptability to workload changes, and robustness.The paper states that Mesos scales to 50,000 nodes.Source: §4.4 · p. 18

4.4.1 | Aurora

  • Aurora runs long-running services, cron jobs, and ad-hoc jobs on Mesos, and intelligently reschedules jobs onto healthy machines after machine failures.Aurora supports jobs composed of multiple near-identical tasks, each managed by an executor process.Source: §4.4.1 · p. 18

4.4.2 | Marathon

  • Marathon focuses on long-running services and aims to maintain applications despite node failures, with health checks, monitoring, placement constraints, and autoscaling.Version 1.4 supports co-located pods whose applications can share storage, networking, and other resources.Source: §4.4.2 · p. 19
  • Marathon handles only service-like jobs, while Aurora supports service, cron, and ad-hoc jobs; Marathon is described as simpler to set up and use.Marathon includes a user interface through which users can schedule tasks directly.Source: §4.4.1 · p. 18, §4.4.2 · p. 19

4.5 | Apache Hadoop YARN

The surveyed systems use different architectures to allocate resources and schedule work. YARN coordinates framework-specific execution through resource and application masters, while Omega, Apollo, and Fuxi use distinct scheduling strategies.

4.5 | Apache Hadoop YARN

  • YARN assigns each framework an application master that requests containers from a per-cluster resource manager and manages the job lifecycle.Frameworks coordinate their own execution flows and optimizations; requests can include locality preferences and container properties.Source: §4.5 · p. 19

4.6 | Omega

  • Omega uses parallel schedulers over shared cluster state, with optimistic concurrency control allowing conflicts that may require rescheduling tasks.The paper reports that the overhead was acceptable and that eliminating blocking offered better scalability.Source: §4.6 · p. 19

4.7 | Apollo

  • Apollo coordinates distributed schedulers to balance scalability and scheduling quality, using runtime estimates and opportunistic scheduling to drive utilization while maintaining low job latencies.Regular tasks are scheduled for low latency; opportunistic tasks fill slack left by regular tasks.Source: §4.7 · p. 20

4.8 | Fuxi

  • Fuxi’s application master requests resources from FuxiMaster, then sends work plans to agents that enforce constraints with Linux cgroups.Incremental or locality-based scheduling makes decisions in microseconds, considering applications queued on the machine when resources are free.Source: §4.8 · p. 20

5 | SYSTEM CLASSIFICATION AND DISCUSSION

The surveyed systems are classified across application, scheduling, and cluster infrastructure and management, with discussion highlighting differences in capabilities and open design trade-offs. The comparison identifies flexibility, resource utilization, and system-specific constraints as recurring concerns.

Classification framework

  • The surveyed classification is organized by originating organization and runtime, application model, scheduling, and cluster infrastructure and management; N/A marks inapplicable properties and N/S marks unspecified details.The classification is presented across Tables 1–8.Source: §5 · p. 20, Table 1 · p. 21, Table 2 · p. 21

System comparison

  • Commercial systems include more advanced features than open-source systems, but organization-specific features such as preemption and rescheduling may not suit general-purpose open-source systems.The paper notes that allowing users or developers to choose system goals could help, but calls this complex.Source: §5 · p. 20

Scheduling

  • Two-level schedulers such as Mesos and YARN allow application frameworks to define scheduling policies, while shared-state systems such as Kubernetes and Omega support simultaneous pluggable schedulers.Framework schedulers in two-level systems have access only to cluster-state information supplied by the resource manager; efficiently scaling to many schedulers remains an open question.Source: §5 · pp. 20–21

Resource management

  • Resource oversubscription is common as a way to pursue high cluster utilization, but user-provided resource estimates may leave little room for it or contribute to resource exhaustion and task eviction.Borg estimates task usage from fine-grained measurements over time rather than relying solely on user requests.Source: §5 · pp. 21–22
  • All surveyed systems require users to specify at least CPU and memory needs, and overestimation can leave resources idle.The paper identifies automatic resource-requirement determination as a potential way to facilitate deployment, improve QoS, and better utilize cluster resources.Source: §5 · p. 22

Scheduling

  • Preemption is common in proprietary systems, while Aurora is the only surveyed open-source system supporting it.The passage begins to describe how preemption enables mixed job workloads, but does not specify the workload types.Source: §5 · p. 22

Cluster infrastructure

  • Kubernetes was the only surveyed system with a cluster autoscaler, an add-on that provisions nodes for unschedulable pods but is restricted to nodes resembling those already in the cluster or node group.The cost and resource-utilization consequences of this provisioning are not considered.Source: §5 · pp. 23–24

Application model

  • Most surveyed systems, except Swarm, were designed for intra-organizational multi-tenancy, but proprietary-system isolation details were elusive and Omega's presumed Borg-like features could not be confirmed.The classification of Omega, Apollo, and Fuxi was partial for this reason.Source: §5 · p. 24

6 | FUTURE DIRECTIONS

The section identifies gaps in cloud-aware cluster management, including static-resource assumptions and the need for rescheduling, heterogeneous-resource placement, stronger isolation, and QoS-aware orchestration.

6 | FUTURE DIRECTIONS

  • Existing frameworks often assume static clusters, overlooking cloud elasticity, resource costs, and pricing and service heterogeneity.For cloud deployments, the paper says this can mean unnecessary costs, potentially reduced application performance, and substantial tuning effort.Source: §6 · p. 24
  • Rescheduling long-running tasks can counter placement degradation through consolidation, freeing nodes or making use of newly added nodes.The discussion focuses on applications that tolerate a component being shut down and restarted.Source: §6 · pp. 24–25
  • Cloud-aware placement could match applications to resources by price, location, type, and size, using filtering and affinity ranking to select resources.The paper gives reserved instances for customer-facing services and rebated resources for batch jobs as examples.Source: §6 · p. 26
  • Stronger isolation remains needed for inter-organizational tenancy; tenant failures can cascade across shared nodes, and denial-of-service mitigation remains open.The discussion notes that non-replicated critical applications may suffer grave consequences from a shared-node failure.Source: §6 · p. 26
  • Application QoS support remains limited: batch-job deadline guarantees are absent from surveyed open-source systems, and heterogeneous QoS constraints remain unexplored.The passage also describes CPU-threshold autoscaling as a baseline approach for services.Source: §6 · p. 26

7 | SUMMARY AND CONCLUSIONS

The paper surveys container orchestration systems through a reference architecture and taxonomy, then identifies research gaps and directions. Its conclusion calls for continued optimization as these systems gain use across organizations and as-a-service models.

7 | SUMMARY AND CONCLUSIONS

  • Container orchestration systems manage diverse workloads at large scale, with designs addressing scalability, availability, resource utilization, and request throughput.The paper describes systems capable of running hundreds of thousands of jobs across thousands of machines.Source: §7 · pp. 26–27
  • The paper introduces a four-layer reference architecture and taxonomy covering workloads, multi-tenancy, scheduling, elasticity, node management, and system objectives.It also surveys state-of-the-art systems and applies the taxonomy to them.Source: §7 · p. 27
  • Identified research directions include elastic cloud clusters, cost-aware scheduling, rescheduling, application QoS management, and stronger multi-tenancy isolation.The paper notes that surveyed open-source systems lack batch-job execution-time guarantees against specified deadlines.Source: §7 · p. 27
  • The paper concludes that continued development and optimization are important as orchestration systems are adopted within organizations and in as-a-service models.It cites ease of use, flexibility, resource use, performance offerings, and advances in container technologies among the reasons for growing adoption.Source: §7 · p. 27
Loading 1807.06193v2…