PaperlayerView exact revision on arXiv

Source-linked AI summary

Auto-scaling Web Applications in Clouds: A Taxonomy and Survey

Chenhao Qu, Rodrigo N. Calheiros, Rajkumar Buyya

arXiv:1609.09224v6cs.DC

TL;DR

Cloud auto-scaling must provision resources for dynamic workloads while balancing cost and QoS. This paper surveys the field through a MAPE-based challenge analysis and taxonomy, maps existing techniques to it, and identifies weaknesses and future research directions.

  • Problem

    Auto-scaling must provision resources without human intervention while avoiding both resource waste and QoS or SLA violations.

  • Method

    The paper models auto-scaling as a MAPE loop, surveys developments, and classifies existing auto-scalers by challenges and key properties.

  • Results

    The taxonomy comprehensively covers identified challenges, organizes existing techniques by their solutions, and supports analysis of their strengths and weaknesses.

  • Takeaways & Limitations

    The analysis identifies gaps between current solutions and an ideal auto-scaler and proposes directions for improving the state of the art.

  • Takeaways & Limitations

    Workload prediction is limited to algorithms based on workload history, excluding external signals such as social media.

Abstract

from arXiv · show

Web application providers have been migrating their applications to cloud data centers, attracted by the emerging cloud computing paradigm. One of the appealing features of the cloud is elasticity. It allows cloud users to acquire or release computing resources on-demand, which enables web application providers to automatically scale the resources provisioned to their applications without human intervention under a dynamic workload to minimize resource cost while satisfying Quality of Service (QoS) requirements. In this paper, we comprehensively analyze the challenges that remain in auto-scaling web applications in clouds and review the developments in this field. We present a taxonomy of auto-scalers according to the identified challenges and key properties. We analyze the surveyed works and map them to the taxonomy to identify the weaknesses in this field. Moreover, based on the analysis, we propose new future directions that can be explored in this area.

1. INTRODUCTION

Cloud elasticity lets web application providers dynamically acquire and release resources, but effective auto-scaling must balance cost against QoS under changing workloads. The paper surveys the field, organizes auto-scalers by challenges and properties, and identifies weaknesses and future directions.

  • Elasticity enables cloud users to acquire and release computing resources dynamically according to their needs.
  • Auto-scaling automatically provisions and deprovisions resources to minimize cost while satisfying QoS requirements.Overprovisioning wastes resources and money, whereas under-provisioning degrades performance and can violate SLAs.
  • General-purpose auto-scalers are difficult to design because workloads, application resource requirements, and cloud pricing models vary.
  • The paper presents a taxonomy, compares existing infrastructure- and application-level approaches, and maps them to strengths and weaknesses.
  • The survey proposes future research directions intended to improve current auto-scaling approaches.
  • The work broadens an earlier survey focused mainly on resource estimation by discussing primary challenges and newer developments.

2. PROBLEM DEFINITION AND CHALLENGES

Auto-scaling is an autonomous MAPE-loop control problem that provisions resources for fluctuating workloads while minimizing cost and satisfying SLAs or SLOs. Its challenges span monitoring, analysis, planning, execution, and multi-cloud coordination.

  • Auto-scaling provisions or deprovisions resources dynamically to handle fluctuating workloads while minimizing cost and satisfying SLAs or SLOs.
  • The MAPE loop repeats monitoring, analysis, planning, and execution to tune resource types and quantities toward SLA-based performance goals.
  • Monitoring: Monitoring requires selecting suitable performance indicators and intervals while balancing sensitivity, monitoring cost, and oscillation risk.Very short intervals increase computing and financial costs and may cause oscillations.
  • Analysis: Analysis must decide when to scale, predict workload changes, adapt to changing conditions, and mitigate oscillations.Oscillation involves frequent opposite scaling actions and can waste resources and increase SLA violations.
  • Planning: Planning estimates required resources and chooses resource combinations, scaling directions, VM types, and pricing models within a large optimization space.
  • Execution: Execution carries out scaling plans through cloud-provider APIs, but supporting APIs from different providers is an engineering challenge.
  • Multi-cloud: Multi-cloud auto-scaling combines data-center selection, geographical load balancing, and resource provisioning while respecting SLAs.

3. TAXONOMY

The proposed taxonomy classifies cloud auto-scalers by challenges in each MAPE phase and by their target environment. It organizes approaches across application, adaptivity, indicators, estimation, oscillation, timing, scaling methods, and environment characteristics.

  • The taxonomy classifies existing auto-scalers according to MAPE-phase challenges and targeted environments.
  • Its dimensions include application architecture, session stickiness, adaptivity, scaling indicators, resource estimation, and oscillation mitigation.
  • The taxonomy also covers scaling timing, workload prediction, scaling methods, resource combinations, and single- versus multi-cloud environments.
  • The surveyed characteristics can be correlated, and individual auto-scalers may span multiple taxonomy subcategories.
  • The taxonomy is feature-based rather than a performance ranking, because surveyed approaches target diverse workloads, architectures, and pricing models.
  • The paper compares existing auto-scalers according to the taxonomy in Sections 4 through 12.

4. APPLICATION ARCHITECTURES

Web applications use single-tier, multi-tier, and service-oriented architectures, which determine the granularity and difficulty of resource management. Existing auto-scalers often manage tiers or services separately, though holistic application-level provisioning can better preserve aggregate QoS.

  • The literature identifies single-tier, multi-tier, and service-oriented web application architectures.
  • A tier is the minimum separately deployable component, and single-tier architecture consists of only one tier.The paper notes that few real web applications contain only one tier.
  • Most auto-scalers separately manage individual tiers or services because this approach is simple and general.
  • Separate management can produce globally suboptimal provisioning because whole-application SLA requirements must be divided into tier- or service-level requirements.
  • Multi-tier architectures: Multi-tier applications connect sequential tiers, commonly including frontend, application-logic, and database tiers.The database tier is often treated as dynamically unscalable and ignored by auto-scalers.
  • Application-level management: Holistic provisioning can satisfy whole-application SLAs but requires more effort to model and estimate resource consumption.
  • Service-oriented architectures: Service-based applications consist of standalone services interacting through APIs and are commonly represented as directed graphs rather than sequential tiers.

5. SESSION STICKINESS

Sessions preserve client state across interactions, but session stickiness can constrain elastic scaling. Common remedies move session data outside web servers, while specialized auto-scalers manage stateful instances directly.

  • A session is a sequence of client–application interactions whose intermediate state must be preserved for subsequent operations.Without that state, clients may lose completed operations and need to repeat them.
  • Sticky sessions bind a client to the same server when session data remains server-side.This stateful access pattern limits terminating under-utilized instances while their sessions remain unfinished.
  • Moving session data to the user side or a shared Memcached cluster is the most adopted way to support stateless scaling.The paper notes that most auto-scalers require the scaling cluster to be stateless.
  • Stateful-instance auto-scalers can wait for active sessions to clear or consolidate sessions across fewer instances before termination.These approaches address state retention without requiring every instance to be stateless.

6. ADAPTIVITY

Adaptivity matters because production workloads and applications can change, making fixed control models insufficient. The survey distinguishes non-adaptive, self-adaptive, and self-adaptive-switching approaches by how they respond to those changes.

  • 6. ADAPTIVITY: Auto-scalers need adaptivity because workload characteristics and applications may change at any time in dynamic production environments.The paper frames auto-scalers as control systems that tune resources toward target performance.
  • 6.1. Non-adaptive Approaches: Non-adaptive approaches use a predefined control model and make decisions from current input, with users specifying scaling conditions and actions.They do not automatically adjust settings during production and often require substantial offline testing.
  • 6.2. Self-adaptive Approaches: Self-adaptive auto-scalers tune a fixed core control model using real-time control-action quality and can retrain after substantial changes.This reduces offline preparation and maintenance, but early training can cause poor performance before convergence.
  • 6.3. Self-adaptive Switching: Self-adaptive switching concurrently maintains multiple controllers and transfers control to the one performing best on the application.Included self-adaptive controllers continue tuning in parallel, while only the selected controller provisions resources at a given moment.

7. SCALING INDICATORS

Auto-scalers use performance indicators from physical infrastructure through the application layer, sometimes combining them with external data. Each indicator type offers different measurement and control trade-offs.

  • 7. SCALING INDICATORS: Scaling decisions rely on indicators monitored from low-level physical or hypervisor metrics to high-level application metrics.The monitoring level determines what aspect of application performance the auto-scaler can observe.
  • 7.1. Low-Level Metrics: Low-level metrics include CPU, memory, and network utilization, swap, and cache misses, but they do not directly reveal application performance accurately.This makes faithful SLA assurance difficult when scaling solely from infrastructure measurements.
  • 7.2. High-Level Metrics: High-level metrics such as request rate, response time, throughput, and session creation rate can directly trigger scaling operations.They are visible to service providers but not cloud providers because they are observed at the application layer.
  • 7.2. High-Level Metrics: Service time and request mix support efficient resource planning but cannot directly trigger scaling and are difficult to measure.Service time may require profiling, application support, logs, filters, regression, or runtime feedback; request mix requires application understanding.
  • 7.3. Hybrid Metrics: Hybrid indicators combine request rate, response time, and resource utilization to build resource models or support workload prediction.Some approaches use machine learning to improve estimation accuracy without complex analytical models.
  • 7.4. Other Indicators: External signals such as weather and political events can also inform workload-intensity prediction.The survey cites a fuzzy-based approach using these factors alongside platform and application measurements.

8. RESOURCE ESTIMATION

Resource estimation seeks the minimum resources needed for a workload, because estimation quality affects provisioning efficiency and convergence. The survey covers rule-based, fuzzy, profiling, analytical, machine-learning, and hybrid approaches, along with their trade-offs.

  • 8. RESOURCE ESTIMATION: Resource estimation identifies the minimum resources needed for a workload and determines whether and how scaling should occur.Accurate estimates can accelerate convergence, whereas errors can cause insufficient provisioning or inefficient resource use.
  • 8. RESOURCE ESTIMATION: The survey groups resource-estimation methods into rule-based, fuzzy inference, application profiling, analytical modeling, machine learning, and hybrid approaches.These categories span basic rules through more sophisticated models.
  • 8.1. Rule-based Approaches: Rule-based auto-scalers use predefined metric thresholds and actions, such as adding or removing instances when CPU utilization crosses specified bounds.Their simplicity is offset by dependence on application knowledge, expert-selected parameters, and limited adaptation to change.
  • 8.1. Rule-based Approaches: Adaptive step sizes adjust scaling increments to workload conditions and performed best for bursty workloads but offered limited improvement for other workload types.One extension also dynamically tunes aggressiveness according to QoS requirements.
  • 8.1. Rule-based Approaches: Dynamic thresholds mitigate inefficient utilization caused by fixed thresholds, especially as cluster size changes.Fixed bounds can prevent removing instances even when large clusters have substantial spare capacity.
  • 8.3. Application Profiling: Application profiling tests resource saturation under synthetic or recorded workloads to learn how many resources handle a workload intensity.Offline profiling is more precise but must be manually repeated after application updates; online profiling is less fine-grained under urgent demand.
  • 8.4. Analytical Modeling: Analytical modeling predominantly uses queuing theory, with single queues for simpler components and queuing networks for multi-tier or multi-service applications.Holistically optimizing a complex queuing network is difficult and computationally heavy.

9. OSCILLATION MITIGATION

Oscillation is repeated opposite scaling caused by overly frequent operations or poor configuration. The survey covers fixed delays, refined triggers, adaptive parameters, and formal identification of risky settings.

  • Oscillation repeatedly provisions and deprovisions resources, often because monitoring and scaling operations occur too frequently or thresholds are poorly configured.Rule-based auto-scalers are especially susceptible because their resource estimates are empirical and coarse-grained.
  • Cooling time mitigates frequent scaling by enforcing a minimum delay, ideally no shorter than acquiring, booting, and configuring a VM.Long cooling times can increase SLA violations because scale-out is delayed.
  • Refining scaling conditions can prolong the interval between operations by requiring threshold exceedances for several consecutive monitoring intervals.With one-minute monitoring, consecutive-trigger rules avoid reacting to every isolated exceedance.
  • Adaptive approaches tune scaling parameters, including scale-down thresholds, hysteresis, stability factors, or prediction horizons, to reduce oscillation.These methods adjust controller aggressiveness or prediction behavior as resource allocation, control settings, or prediction error changes.
  • Formal models can identify settings that may cause oscillations, enabling restrictions intended to eliminate the associated risk rather than merely mitigate it.The cited approaches analyze potential oscillation conditions in rule-based auto-scalers.

10. SCALING TIMING

Scaling timing depends on workload behavior and application preferences for cost and QoS: reactive scaling suits smooth changes, while proactive scaling uses predictions for abrupt or strict-SLA workloads. Prediction methods draw on workload history, external information, trends, regular patterns, or direct resource-usage forecasts, but bursty workloads remain difficult.

  • Reactive scaling is generally preferred for gradual, smooth workloads because it can save resources without significant SLA violations.Proactive scaling is more appropriate when workload changes are drastic or SLA requirements are strict.
  • Proactive scaling relies on predictions because provisioning takes time, but workload prediction cannot effectively handle random request bursts.The survey identifies news-feed and social-network bursts as examples where current approaches fall back to best-effort reactive handling.
  • Workload Prediction: Prediction algorithms commonly use workload history to forecast average or maximum load over a future prediction horizon.The horizon specifies how far ahead the auto-scaler predicts, while control strategies may use only the next horizon or all intervals within it through MPC.
  • Workload Prediction Data Source: External information such as weather, political events, social media, and other channels can reveal application-specific workload events beyond historical workload data.The survey notes that integrating many external parameters into a general-purpose predictor is difficult, while timely event detection remains viable.
  • Prediction Algorithms: Workload prediction methods use either recent trends or regular patterns, with the latter requiring an extended workload archive.Representative techniques include regression, autoregressive models, neural networks, wavelets, decision-tree selection, and pattern matching.
  • Resource-Usage Prediction: Some auto-scalers directly predict resource usage from historical usage data, especially for vertical scaling, while horizontal-scaling proposals use it for combined prediction and resource estimation.Examples include signal processing with dynamic time warping, Markov chains, linear regression, neural networks, and pattern matching.

11. SCALING METHODS

Cloud auto-scalers choose vertical, horizontal, or hybrid scaling while accounting for provisioning speed, capacity limits, VM combinations, pricing, and transition costs. The survey covers homogeneous and heterogeneous instances, spot markets, billing periods, and hybrid plans.

  • Vertical Scaling: Vertical scaling adds or removes CPU, memory, I/O, or network resources from existing VMs, but major providers may require shutting instances down first.This platform restriction can make runtime vertical scaling unavailable in commercial clouds.
  • Vertical Scaling: Vertical scaling can act nearly instantaneously but is unsuitable for highly scalable applications because VM and host capacity impose limits.Multiple VMs sharing a physical host further restrict the available scaling capability.
  • Horizontal Scaling: Horizontal scaling is central to cloud elasticity, but VM sizes, customizable configurations, and multiple pricing models complicate provisioning decisions.For a single tier, homogeneous VMs simplify management, while the preferred VM type depends on resource profile and workload characteristics.
  • Heterogeneity: VM cost-efficiency depends on application and workload changes, so some auto-scalers detect changes online and reselect the VM type.One cited approach combines Hierarchical Temporal Memory with a dynamically trained artificial neural network.
  • Heterogeneity: Heterogeneous VMs can provide limited cost savings under conventional linear billing, although searching combinations is often computationally intensive.The survey also describes heterogeneous spot instances used to improve cluster reliability and save cost.
  • Hybrid Scaling: Hybrid scaling uses vertical scaling for rapid adaptation and horizontal scaling when vertical capacity is exhausted, and optimization methods can search for lower-cost plans.The two methods can also be assigned separately to components such as database servers that are difficult to scale horizontally.

12. ENVIRONMENT

The survey organizes auto-scalers by environment and design properties, then maps prior work to its taxonomy. Multi-cloud deployment adds latency, availability, cost, and lock-in motivations, but coordinated scaling must also select locations and route requests under changing workloads and prices.

  • Surveyed Designs: Existing auto-scalers include rule-based, regression, analytical-model, reinforcement-learning, profiling, fuzzy-inference, and hybrid approaches across diverse application and scaling settings.The reviewed table entries cover both single-tier and multi-tier applications and multiple adaptivity categories.
  • Taxonomy Dimensions: The taxonomy distinguishes surveyed systems using properties such as session adaptivity, scaling indicators, resource estimation, oscillation mitigation, proactive scaling, and scaling methods.The table contains entries spanning single-tier and multi-tier systems, adaptive and non-adaptive designs, and vertical, horizontal, hybrid, homogeneous, and heterogeneous methods.
  • Single-Cloud Environment: Single-cloud auto-scaling is abstracted as an MAPE loop, with design challenges identified for each phase and surveyed works mapped to taxonomy categories.Table I summarizes the general design properties of key single-cloud auto-scalers.
  • Multi-Cloud Environment: Multi-cloud deployment can reduce response latency, improve availability, exploit vendor price differences, and prevent vendor lock-in.These benefits motivate deploying applications across multiple cloud data centers.
  • Multi-Cloud Environment: Standalone local auto-scalers are easy to manage but are not optimal when workload and resource prices change dynamically across data centers.A coordinated multi-cloud auto-scaler must decide resource provisioning together with location selection and request routing.

13. DISCUSSION AND FUTURE DIRECTIONS

The analysis identifies gaps between current auto-scaling solutions and an ideal system, then proposes directions spanning estimation, prediction, pricing, scaling mechanisms, reliability, sustainability, preferences, and containers.

  • Current solutions leave gaps across multiple aspects of auto-scaling, motivating improvements identified through the taxonomy and analysis.
  • 13.3. Resource Estimation Models: Resource estimation models still require improvements in accuracy, generality, computing requirements, and ease of use.The paper highlights hybrid analytical and machine-learning models, queuing networks, and online profiling as promising directions.
  • Future auto-scalers should incorporate broader pricing models, vertical scaling, and reliability constraints across multi-cloud deployments.The discussion covers non-Amazon rebated pricing, live vertical scaling limitations, and data-center outage impacts on availability.
  • 13.6. Event-based Workload Prediction: Existing workload prediction mainly uses past history, while external real-time information could improve burst prediction for application-specific workloads.Examples include news applications affected by physical events and outdoor applications affected by weather.
  • Auto-scaling research should consider energy and carbon footprint alongside financial cost and QoS, while respecting application locality and user preferences.Provider-level allocation may favor renewable-energy availability, but geographic, regulatory, temporal, and resource-type preferences constrain placement.
  • Dedicated auto-scaling solutions for container-based microservices and service pods remain to be explored, although VM-based techniques may provide inspiration.

14. SUMMARY AND CONCLUSIONS

Auto-scaling dynamically adjusts application resources to changing workloads without human intervention, seeking lower resource bills while meeting customer QoS expectations. The paper surveys this field through a MAPE-based taxonomy, analyzes existing techniques, and proposes future research directions.

  • Auto-scaling automatically adjusts application resources to real-time workloads without human intervention while targeting lower resource bills and customer QoS expectations.
  • The paper abstracts auto-scaling as a MAPE loop and identifies key challenges in its Monitoring, Analysis, Planning, and Execution phases.
  • The taxonomy categorizes auto-scalers by key properties and by how existing works address the loop’s identified challenges.
  • The survey analyzes existing techniques to discuss their strengths and weaknesses, then proposes promising directions for future research.
Loading 1609.09224v6…