Source-linked AI summary
An Overview of the Commercial Cloud Monitoring Tools: Research Dimensions, Design Issues, and State-of-the-Art
Khalid Alhamazani, Rajiv Ranjan, Karan Mitra, Fethi Rabhi, Samee Ullah Khan, Adnene Guabtni, Vasudha Bhatnagar
TL;DR
Cloud monitoring must manage heterogeneous, dynamic resources and large data volumes while providing visibility across cloud layers and application components. The paper surveys research dimensions, design issues, and academic and commercial tools, finding that cloud-dependent systems are mostly commercial while cloud-agnostic systems are typically open source. It concludes that QoS dimensions, aggregation, visibility, scalability, and real-time monitoring remain important areas for improvement.
Problem
Cloud monitoring lacks comprehensive treatment of layer-specific, predictive, machine-learning-supported, interoperable, and cross-layer monitoring for complex cloud environments.
Method
The paper surveys cloud provisioning and monitoring concepts, identifies research dimensions and design issues, and maps them to existing academic and commercial monitoring tools.
Results
Cloud-dependent monitoring systems are mostly commercial, whereas cloud-agnostic monitoring systems are typically open source; QoS quality is identified as the most important monitoring dimension.
Takeaways & Limitations
Effective cloud monitoring requires scalable, fast, real-time, and customizable support for diverse and dynamic environments, with continued work on QoS aggregation and layer visibility.
Takeaways & Limitations
Cloud-dependent tools may not monitor application components hosted by other providers, while many commercial tools remain focused on IaaS resources rather than QoS across multiple layers.
Abstract
from arXiv · showhide
Cloud monitoring activity involves dynamically tracking the Quality of Service (QoS) parameters related to virtualized resources (e.g., VM, storage, network, appliances, etc.), the physical resources they share, the applications running on them and data hosted on them. Applications and resources configuration in cloud computing environment is quite challenging considering a large number of heterogeneous cloud resources. Further, considering the fact that at each point of time, there will be a different and specific cloud service which may be massively required. Hence, cloud monitoring tools can assist a cloud providers or application developers in: (i) keeping their resources and applications operating at peak efficiency; (ii) detecting variations in resource and application performance; (iii) accounting the Service Level Agreement (SLA) violations of certain QoS parameters; and (iv) tracking the leave and join operations of cloud resources due to failures and other dynamic configuration changes. In this paper, we identify and discuss the major research dimensions and design issues related to engineering cloud monitoring tools. We further discuss how aforementioned research dimensions and design issues are handled by current academic research as well as by commercial monitoring tools.
1. Introduction
The paper frames cloud monitoring as increasingly difficult because cloud environments are heterogeneous, dynamic, and generate large, complex data volumes. It identifies gaps in layer-specific, predictive, and machine-learning-supported monitoring and outlines the paper’s contributions and organization.
- Motivation: Cloud providers offer dynamically provisioned resources across large-scale, heterogeneous infrastructures, making application deployment and management challenging.Amazon EC2 is described as operating around half a million physical hosts, each supporting multiple virtual machines.
- Research gap: Existing surveys do not comprehensively address monitoring across IaaS, PaaS, and SaaS layers, predictive monitoring, or machine-learning use with monitored data.The paper presents these omissions as gaps in prior cloud-monitoring surveys.
- Research gap: Cloud monitoring tools must cope with heterogeneous, multimodal, dynamic data streams and very large data volumes generated by cloud infrastructures and connected devices.The paper links these conditions to the need for tools capable of managing big-data floods.
- Research questions: The paper investigates layer-specific monitoring, consumer-defined monitoring information, and predictive status for applications’ future behavior.These are stated as three central cloud-monitoring challenges.
- Contributions: Its contributions are understanding provisioning and monitoring concepts, identifying research dimensions and design issues, and proposing future directions for cloud-monitoring techniques.The paper also maps these issues to academic and commercial monitoring tools in its stated organization.
2. Cloud Resource Provisioning
Cloud resource provisioning covers deployment and management across virtual machines, physical hosts, and applications. The paper presents provisioning steps and connects runtime monitoring with QoS, SLA compliance, and dynamic configuration.
- Provisioning overview: Cloud resource provisioning is the complex process of deploying and managing applications on cloud infrastructure.The paper notes that current providers do not completely automate software-resource deployment and configuration.
- Provisioning stages: Virtual machine provisioning instantiates suitable VMs and configurations, with consumers sometimes selecting appliances during or after VM provisioning.Bitnami provisions a VM-and-appliance stack, whereas Amazon EC2 can separate VM and appliance selection.
- Provisioning stages: Resource provisioning maps and schedules instantiated VMs onto physical cloud servers through cloud-based hypervisors.The described EC2 sequence includes selecting a VM type, configuring it, deploying it, selecting applications, and deploying the combined resources.
- Provisioning stages: Application provisioning deploys applications on cloud-hosted VMs either together with VM hosting or as a separate subsequent step.Tomcat deployment on an Amazon EC2 VM illustrates application provisioning.
- Runtime monitoring: After provisioning, workflows may span multiple services and providers, making quality monitoring more complex during runtime.Runtime QoS monitoring is needed to help avoid and handle violations.
- Runtime monitoring: Improved monitoring is presented as supporting SLA fulfillment and dynamic configuration, including autoscaling and other runtime changes.The paper characterizes monitoring, SLAs, and dynamic configuration as interrelated.
3. Cloud Monitoring
Cloud monitoring continuously observes resources and applications across cloud layers to support QoS, SLA compliance, and corrective control. The paper emphasizes cross-layer visibility because application components and QoS requirements vary by architecture and application type.
- Application Life Cycle: Cloud application architectures contain layered software and hardware resources whose selection, deployment, monitoring, and control form an application life cycle.The layers may include clients, load balancers, web servers, streaming servers, application servers, and databases.
- Application Life Cycle: Monitoring continuously gathers resource information, detects abnormal behavior, and can notify an orchestrator for policy-based corrective action.Resource control handles runtime uncertainties such as upgrading or downgrading resource types or functionality.
- Cloud Monitoring: Monitoring supports resource management, performance, security, billing, troubleshooting, data-center operations, and SLA management for providers and consumers.The paper describes monitoring as essential to cloud-system health and elastic operation.
- Cloud Monitoring: High-level monitoring concerns virtual-platform status, whereas low-level monitoring concerns the physical infrastructure’s status.The monitoring system is described as self-adjusting and typically multi-threaded.
- Cloud Monitoring: Monitoring tests may be computation-based or network-based and collect measurements such as CPU speed, CPU utilization, and disk information.These measurements assess cloud platforms and applications in terms of performance, reliability, power usage, SLA fulfillment, and security.
- QoS and SLA: Unpredictable availability, load, and throughput make current status information necessary for handling runtime uncertainty and meeting QoS targets.The paper presents QoS fulfillment as nontrivial because resource conditions vary over time.
- QoS and SLA: Because application components span PaaS and IaaS, QoS monitoring should cover all relevant layers of the cloud stack.The paper identifies web, streaming, and indexing servers alongside compute, storage, and network services.
- QoS and SLA: QoS targets differ by application type, so SLAs must match application-specific requirements and monitoring must track the relevant parameters.The paper states that missed QoS tracking can lead to SLA violations and support more realistic dynamic SLAs.
4. Evaluation Dimensions
Cloud monitoring tools can be evaluated through architectural dimensions such as centralized versus decentralized organization and the operational trade-offs each design entails. Centralized monitoring simplifies management but introduces scalability, failure, communication, and capacity concerns, while decentralized designs distribute responsibility and reduce dependence on any single component.
- Evaluation Dimensions: Cloud monitoring tools can be evaluated using dimensions that include network architecture and the organization of monitoring components.The section introduces evaluation dimensions and discusses centralized and decentralized monitoring architectures.
- Centralized: Centralized monitoring has PaaS and IaaS resources send QoS status queries to a central monitoring server through periodic probing.The central server continuously pulls information from cloud components.
- Centralized: Centralized monitoring is prone to a single point of failure, limited scalability, network bottlenecks, and insufficient computational capacity for many requests.These design issues affect reliability, communication cost, and the ability to serve large monitoring workloads.
- Decentralized: Decentralized monitoring distributes system components so that no component is more important than the others.A component failure does not influence the operation of other components in the system.
- Decentralized: Structured, unstructured, and hybrid peer-to-peer overlays represent different ways to distribute monitoring authority and search functions.Structured overlays eliminate a central point of failure, unstructured overlays decentralize the search directory, and hybrid overlays combine both arrangements.
4.2. Interoperability
Interoperability evaluates whether monitoring frameworks can observe applications and components deployed across multiple cloud providers. Cloud-dependent tools are tightly integrated with one provider, whereas cloud-agnostic monitoring must address the lack of a unified API for runtime QoS statistics.
- Interoperability: Interoperability concerns a monitoring framework’s ability to monitor application components deployed across multiple cloud providers.Designing a generic multi-cloud framework remains more challenging than implementing a cloud-specific framework.
- Cloud Dependent: Cloud-dependent tools monitor services within a provider’s infrastructure but generally cannot monitor components hosted by other providers.CloudWatch is presented as an example that monitors AWS EC2 applications but not components on GoGrid or Azure.
- Interoperability: Table 2 distinguishes monitoring tools that are specific to a cloud provider from tools that are cloud agnostic.The classification organizes commercial monitoring tools according to their interoperability scope.
- Cloud Agnostic: Cloud-agnostic monitoring is challenging because cloud services lack a common unified API for runtime QoS statistics.Cloud-agnostic tools are intended to retrieve QoS data from services and applications distributed across multiple clouds.
4.3. Quality of Service Matrix
The QoS matrix dimension addresses how monitoring tools represent targets across cloud layers, from individual atomic parameters to composite groups. This is necessary because applications may span multiple PaaS and IaaS components whose collective behavior determines performance quality.
- Quality of Service Matrix: Application developers must determine which QoS parameters and targets to specify across each layer of the cloud stack.The relevant parameters may be represented individually or as a group.
- Single Parameter: A single parameter represents one specific system QoS target, such as CPU utilization, that must be tracked continuously.A violation of an atomic SLA value can lead to serious system failure.
- Composite Parameter: A composite parameter combines multiple parameters to evaluate the performance of one or more cloud software services.The resulting QoS assessment reflects the collective behavior of the services involved.
- Cross-Layer QoS: Multimedia streaming components distributed across PaaS and IaaS layers require cross-layer QoS monitoring to guarantee application-level targets.The components include streaming, web, indexing, compute, storage, and network services.
4.4. Visibility
Visibility concerns the cloud layers and resources that a monitoring tool can observe. Layer-specific tools focus on one layer, while layer-agnostic monitoring provides access across layers for applications whose components span PaaS and IaaS.
- Visibility: Applications can distribute components such as streaming, web, indexing, compute, storage, and network services across multiple cloud layers.Monitoring QoS across these layers is critical when targets apply to the application as a whole.
- Layer Specific: Layer-specific monitoring tools monitor services in one of SaaS, PaaS, or IaaS, and many commercial tools focus on IaaS resources.CloudWatch is described as unable to monitor how CPU-service behavior affects QoS delivered by hosted PaaS services.
- Layer Agnostic: Layer-agnostic monitoring lets consumers retrieve application data from multiple layers simultaneously.This approach is especially effective for consumers seeking complete awareness of running cloud applications.
- Visibility: Table 3 classifies monitoring tools according to the visibility of the cloud layers they support.The table is identified as covering monitoring tools and layers’ visibility.
4.5. Programming Interfaces
Programming interfaces provide unified ways to monitor components across cloud layers, using APIs, widgets, and command-line interaction. Commercial tools use communication protocols to transfer monitoring data.
- Programming interfaces combine APIs, widgets, and command-line interaction to monitor multiple components across cloud layers.
- An API enables software programs to communicate, and commercial tools such as Rackspace, Nimsoft, RevealCloud, and LogicMonitor provide such interfaces.
- Widgets let consumers run and display applets through graphical interfaces, with Monitis and RevealCloud offering multiple customizable monitoring widgets.
- Communication protocols transfer data into and out of commercial monitoring tools, including HTTPS and FTP for Monitis and Rackspace and encrypted SNMP for LogicMonitor.
5. Commercial Monitoring Tools
Commercial monitoring tools differ in cloud-layer coverage, infrastructure scope, supported resources, and monitoring architecture. Several provide unified or multi-layer views, while CloudWatch remains tied to Amazon EC2 and limited across layers.
- Monitis provides a unified dashboard with multiple widgets and remotely monitors websites, servers, and networked devices through installed agents.
- RevealCloud monitors SaaS, PaaS, and IaaS across popular cloud providers and supports maintaining historical monitoring data.
- LogicMonitor supports SaaS, PaaS, and IaaS monitoring across multi-cloud resources, using SSL connections and SNMP for data retrieval.
- Nimsoft supports multi-layer monitoring of virtual and physical resources across multiple cloud infrastructures and both private and public clouds.
- Nagios supports multi-layer monitoring across cloud and in-house infrastructures, extending network monitoring to virtual instances and storage through plugins.
- SPAE emphasizes security and vulnerability monitoring through multiple network protocols and SNMP but does not support monitoring across IaaS, PaaS, and SaaS layers.
- CloudWatch monitors resources residing on Amazon EC2, does not support multi-cloud infrastructure monitoring, and offers limited cross-layer monitoring.
6. Conclusion, Discussion and Future Research Directions
The paper synthesizes cloud-monitoring research, design dimensions, and commercial-tool capabilities while identifying the need for more scalable, real-time, and mature monitoring systems. Its discussion highlights QoS dimensions, decentralization, interoperability, and cloud-environment diversity as continuing research concerns.
- The paper presents state-of-the-art cloud-monitoring research, design issues, commercial tools, tool shortcomings, and future research directions.
- Increasing cloud complexity demands monitoring systems that are more scalable, effective, fast, and capable of real-time performance reporting.
- Not all identified monitoring evaluation dimensions are adopted by open-source or commercial systems, and further effort is needed to reach maturity.
- Cloud-dependent monitoring systems are mostly commercial, whereas cloud-agnostic systems are typically open source; decentralized approaches are gaining trust.
- Quality-of-service measurement is identified as the most important monitoring dimension, including which parameters are monitored, detected, reported, and aggregated.
- Monitoring platforms are compared against evaluation dimensions including decentralized network architecture.