PaperlayerView exact revision on arXiv

Source-linked AI summary

Artificial Intelligence (AI)-Centric Management of Resources in Modern Distributed Computing Systems

Shashikant Ilager, Rajeev Muralidhar, Rajkumar Buyya

arXiv:2006.05075v2cs.DCcs.LG

TL;DR

Modern distributed systems must manage complex, heterogeneous infrastructure and varying workloads, while existing heuristic RMS methods are inadequate and data preparation remains challenging. This paper surveys AI-centric resource management, identifies challenges and directions, proposes a conceptual model, and demonstrates two use cases. The use cases report energy savings in GPU scheduling and data-centre management.

Abstract

from arXiv · show

Contemporary Distributed Computing Systems (DCS) such as Cloud Data Centres are large scale, complex, heterogeneous, and distributed across multiple networks and geographical boundaries. On the other hand, the Internet of Things (IoT)-driven applications are producing a huge amount of data that requires real-time processing and fast response. Managing these resources efficiently to provide reliable services to end-users or applications is a challenging task. The existing Resource Management Systems (RMS) rely on either static or heuristic solutions inadequate for such composite and dynamic systems. The advent of Artificial Intelligence (AI) due to data availability and processing capabilities manifested into possibilities of exploring data-driven solutions in RMS tasks that are adaptive, accurate, and efficient. In this regard, this paper aims to draw the motivations and necessities for data-driven solutions in resource management. It identifies the challenges associated with it and outlines the potential future research directions detailing where and how to apply the data-driven techniques in the different RMS tasks. Finally, it provides a conceptual data-driven RMS model for DCS and presents the two real-time use cases (GPU frequency scaling and data centre resource management from Google Cloud and Microsoft Azure) demonstrating AI-centric approaches' feasibility.

I. INTRODUCTION

IoT, mobile, and other data-intensive applications are increasing demand for computing resources and diverse QoS, while distributed infrastructures make efficient, reliable resource management difficult. The paper motivates AI-centric resource management and presents its challenges, research directions, conceptual model, and two real-time use cases.

I. INTRODUCTION

  • IoT, mobile, and other applications generate large amounts of data, increasing demand for computing resources to process it and derive insights.Cloud and Edge offer on-demand service access, while microservices and serverless models are becoming mainstream.Source: §I · p. 1
  • DCS resource management must meet diverse application QoS requirements across large-scale, heterogeneous, multi-tenant infrastructure.The paper notes that the number of possible configurations can make manual tuning impractical.Source: §I · p. 1
  • Static, heuristic RMS methods are often reactive and cannot adapt or improve at runtime in dynamic Cloud and Edge environments.The paper contrasts these methods with data-driven models that can predict workload demand and scale resources beforehand.Source: §I · p. 2
  • AI-centric RMS is more feasible because AI techniques have matured, DCS platforms generate extensive operational data, and infrastructure complexity demands automated decisions.Data-centre sensors monitor workload, energy, temperature, and weather parameters.Source: §I · p. 2
  • The paper surveys DCS evolution and RMS, identifies data-driven challenges and research directions, proposes a conceptual model, and presents two real-time use cases.The use cases concern energy-efficient GPU clock configurations and data-centre resource management.Source: §I · p. 2

II. DCS EVOLUTION AND THE STATE-OF-THE-ART

DCS have evolved from scientific clusters toward Cloud data centres and IoT-driven systems spanning Edge to Cloud, increasing the need for autonomous resource management. Although data-driven methods have been applied to RMS tasks, their adoption remains at an early stage and requires scalable frameworks and standards.

II. DCS EVOLUTION AND THE STATE-OF-THE-ART

  • DCS evolved from scientific parallel-task clusters through service-oriented and virtualized Cloud data centres toward IoT systems spanning Edge to remote Cloud.Figure 1 presents an approximate, representative timeline of this evolution.Source: §II · p. 2, Figure 1 · p. 3
  • IoT applications require autonomous task composition, deployment, and execution, with RMS provisioning resources, scheduling tasks, and managing QoS and latency.The paper describes this as a sense-connect-act model involving more machine-to-machine interaction.Source: §II · p. 2
  • Microservices and serverless execution models add to the need for intelligent resource-management decisions; discussion focuses mainly on Cloud and Edge..Source: §II · p. 2
  • Data-driven RMS research includes a reported 40% efficiency improvement in Google cooling management and work on placement, scheduling, scaling, and front-end stalls,,,.The paper characterizes AI solutions for RMS as still superficial in adoption and calls for scalable, manageable frameworks and standards.Source: §II · pp. 2–3

III. CHALLENGES

  • The paper identifies adoption challenges for AI solutions in distributed resource management.This section introduces the discussion of those challenges.Source: §III · p. 3

A. Availability of Data

Data-driven RMS faces challenges in obtaining and preparing suitable data, deploying models without disrupting resource-constrained applications, and meeting service requirements despite stochastic outputs.

A. Availability of Data

  • DCS data comes from different abstraction levels and formats, making integration and ML-ready preprocessing difficult; public datasets also rarely include infrastructure configurations and energy footprints.The paper calls for data pipelines, standardized formats, and collaborative data access.Source: A. Availability of Data · pp. 3–4

B. Managing the Deployment of Models

  • Model training and inference compete with latency-sensitive applications for scarce edge resources, while cloud deployment can delay inference and feedback.The paper highlights lifecycle decisions about training amount and placement, and mentions transfer or distributed learning as ways to reduce computational demands.Source: B. Managing the Deployment of Models · p. 4

C. Non-Deterministic Outputs

  • ML stochasticity can produce different outputs for identical inputs, complicating DCS services governed by SLAs and consistent-pricing expectations.The paper says stochasticity remains inherent and requires careful output monitoring and control.Source: C. Non-Deterministic Outputs · p. 4

D. Black Box Decision Making

AI-based RMS faces challenges in explaining decisions, representing heterogeneous entities efficiently, and orchestrating resources across evolving network architectures. The paper highlights the need for explanations, lightweight semantics, and standardized data models.

D. Black Box Decision Making

  • Black-box ML decisions make it difficult to justify resource preemption fairly and provide users with reasons, a trust concern in DCS.Explainable AI has been explored, but the paper says the problem remains challenging.Source: D. Black Box Decision Making · p. 4

E. Lightweight and Meaningful Semantics

  • Heterogeneous, multi-tier DCS need lightweight semantic models to represent resources, entities, applications, and services without added overhead.Existing semantic models are described as either heavy or inadequate for these environments.Source: E. Lightweight and Meaningful Semantics · p. 4

F. Complex Network Architectures, Overlays, Upcoming Features

  • Evolving cloud–telecom architectures and network slicing require complex orchestration, while RMS must meet end-to-end bandwidth, throughput, and latency requirements.The paper calls for standardized data formats and models across the different network abstraction levels.Source: F. Complex Network Architectures,… · pp. 4–5

G. Performance, Efficiency, and Domain Expertise

AI can improve resource management in dynamic distributed systems, but cold starts, safety and reward constraints, real-time inference, energy use, and domain adaptation complicate deployment. The paper identifies provisioning, scheduling, and related tasks as avenues for data-driven approaches.

G. Performance, Efficiency, and Domain Expertise

  • Cold starts, safety and reward constraints, real-time inference, energy use, and the need to adapt AI models to RMS tasks complicate practical adoption.The paper calls for approaches balancing accuracy, energy, and performance overhead, alongside domain-specific algorithms and frameworks.Source: G. Performance, Efficiency, and Domain… · p. 5

A. Data-driven Resource Provisioning and Scheduling

  • AI techniques can support resource provisioning and scheduling, including resource estimation, workload co-location, spot-instance planning, and adaptive placement across edge tiers.Spot-instance planning may estimate runtime to guide resource quantity and checkpointing; historical data and adaptive RL can support scheduling and provisioning.Source: A. Data-driven Resource Provisioning… · p. 5

B. Managing Elasticity using Predictive Analytics

Predictive analytics can guide proactive elasticity, while AI techniques are proposed for energy and carbon management and for security and privacy tasks.

B. Managing Elasticity using Predictive Analytics

  • Forecasting future load from past usage can support proactive scaling before demand bursts, improving SLAs and saving costs compared with reactive scaling.The paper identifies time-series methods such as ARIMA and LSTM for predicting future load.Source: B. Managing Elasticity using… · p. 5

C. Energy Efficiency and Carbon footprint Management

  • Predictive and reinforcement-learning models can address energy tasks spanning frequency management, workload scheduling, cooling, power budgets, and carbon-aware workload shifting.The paper also proposes forecasting renewable energy to guide workload shifts across clouds.Source: C. Energy Efficiency and Carbon… · pp. 5–6

D. Security and Privacy Management

  • AI techniques are proposed for privacy-aware placement and security tasks such as intrusion and anomaly detection and DDoS traffic filtering.A cited scheduler classifies medical records by sensitivity and places sensitive records on a private cloud and nonsensitive records on AWS EC2.Source: D. Security and Privacy Management · p. 6

E. Managing Cloud Economics

The paper describes data-driven opportunities in cloud economics, training-data generation, system architecture, and other resource-management tasks.

E. Managing Cloud Economics

  • Demand forecasts can support more dynamic cloud pricing and pooling excess resources in open markets, addressing inflexible approaches that may overprice services or reduce provider revenue.The paper frames this as potentially benefiting both consumers and providers.Source: E. Managing Cloud Economics · p. 6

F. Generating the Large-scale Data Sets

  • GANs can generate large-scale synthetic DCS time-series data from small datasets or help complete incomplete datasets for predictive-model training and RL-agent bootstrapping.The paper notes that limited access to large-scale infrastructure data can stem from privacy and data-generation constraints.Source: F. Generating the Large-scale Data Sets · p. 6

G. Future System Architectures

  • Microservice monitoring data can inform future data-centre server design, while deep learning is proposed for diagnosing unpredictable cloud-system performance.The cited monitoring tracks stalls and cache misses across Google warehouse-scale computers.Source: G. Future System Architectures · p. 6

H. Other Avenues

  • Other proposed applications include optimizing heuristics, network routing and TCP settings, storage management, and learning-based optimization across the computing stack.The listed stack spans hardware design, compilers, operating-system policies, and distributed platforms.Source: H. Other Avenues · p. 6

V. CONCEPTUAL MODEL FOR AI-CENTRIC RMS

The paper proposes a conceptual data-driven RMS architecture for distributed systems and describes two demonstration use cases. It emphasizes integration challenges across middleware, models, and heterogeneous infrastructure.

V. CONCEPTUAL MODEL FOR AI-CENTRIC RMS

  • The conceptual model organizes data-driven RMS around users or applications, an AI-centric middleware, and heterogeneous DCS infrastructure.The model is identified as Figure 2.Source: §V · p. 7, Figure 2 · p. 7
  • The middleware supports provisioning, scheduling, monitoring, migration, and consolidation using predictive models trained offline or adaptive RL models improved through runtime feedback.The two model types can interact with RMS operations through RESTful APIs; offline models use historical DCS data and preprocessing.Source: §V · p. 7
  • Adopting the model requires changes to RMS design, monitoring agents, interfaces, and deployment policies for integration with heterogeneous infrastructure.The paper also describes scalable, generic integration of models into DCS platforms as challenging and still at a conception stage.Source: §V · pp. 6–7

VI. DEMONSTRATION CASE STUDIES

  • The demonstration cases cover data-driven GPU frequency configuration for energy-efficient workload scheduling and ML-based data-centre resource management.The system model is shown in Figure 3.Source: §VI · p. 7, Figure 3 · p. 7

A. Data-Driven GPU Clock Configuration and Deadline-aware Scheduling

The section presents data-driven GPU frequency prediction and deadline-aware scheduling to reduce energy while meeting application deadlines. On NVIDIA Tesla P100, CatBoost predicts energy and execution time with low average RMSE, and scheduling uses less energy than default and maximum-clock policies.

A. Data-Driven GPU Clock Configuration and Deadline-aware Scheduling

  • The approach addresses the nonlinear, application-specific relationship among GPU runtime performance, energy, and execution time that simple analytical solutions and standard heuristics fail to capture.The motivating application settings include video processing, autonomous cars, and machine-learning training.Source: A. Data-Driven GPU Clock Configuration… · p. 7
  • The scheduler uses predictions across frequency settings, clustering correlated applications from minimal profiling data to select energy-efficient frequencies under deadline requirements.Training uses profiling, energy-time measurements, and frequency configurations; evaluation covers 12 Rodinia and Polybench applications and about 120 features.Source: A. Data-Driven GPU Clock Configuration… · p. 8
  • 15.07% less energy than default and max-clock policies while meeting application deadlines; frequency scaling uses energy-efficient settings that meet performance requirements.The reported scheduling result is shown alongside average total GPU energy and frequency-scaling policy figures.Source: Figure 6 · p. 8, Figure 7 · p. 8, A. Data-Driven GPU Clock Configuration… · p. 9
  • CatBoost achieves average RMSE values of 0.38 for energy prediction and 0.05 for execution-time prediction.Figure 4 compares models for both prediction tasks, with lower RMSE preferred.Source: Figure 4 · p. 8, A. Data-Driven GPU Clock Configuration… · p. 9

B. Industrial (Google Cloud and Microsoft Azure) Data Centre Management

The section describes AI-based resource-management examples at Microsoft Azure and Google data centres. Azure's prediction-serving system informs runtime resource decisions, while Google's cooling-system models are reported to achieve 40% energy savings.

B. Industrial (Google Cloud and Microsoft Azure) Data Centre Management

  • Data-centre resource management must tune hundreds of subsystem parameters across a complex facility, where static or heuristic solutions may not yield better results.The passage notes data centres can dissipate up to 20 kW per rack and that a 1% efficiency improvement can save millions of dollars annually.Source: B. Industrial (Google Cloud and… · p. 9
  • Azure Resource Control serves gradient-boosting predictions about VM utilization, deployment size, lifetime, and blackout time to the resource manager for runtime decisions.For example, the scheduler queries predicted VM lifetime to inform decisions intended to increase infrastructure efficiency.Source: B. Industrial (Google Cloud and… · p. 9
  • Google's neural-network models predict cooling-system settings from workload, network, and environmental inputs; the reported outcome is 40% energy savings.Inputs include outside temperature, wind speed, and active-chiller count; cooling subsystems are configured according to predictions.Source: B. Industrial (Google Cloud and… · p. 9

VII. CONCLUSIONS

The conclusion argues that modern distributed platforms require resource-management approaches suited to scale, heterogeneity, and changing workloads. It summarizes the paper's challenges, future directions, conceptual model, and two AI-centric use cases.

VII. CONCLUSIONS

  • The paper identifies adoption challenges and future research directions, presents a conceptual AI-centric RMS model, and demonstrates two distributed-systems resource-management use cases.The use cases concern GPU frequency scaling and data-centre management.Source: §VII · p. 9
  • The paper concludes that rule-based and heuristic resource-management solutions are inadequate for modern platforms' scale, heterogeneity, and varying workload requirements.It argues that AI techniques can be used across tasks including monitoring, provisioning, and scheduling.Source: §VII · p. 9

Related papers

Loading 2006.05075v2…