Source-linked AI summary
Cloudbus Toolkit for Market-Oriented Cloud Computing
Rajkumar Buyya, Suraj Pandey, Christian Vecchiola
TL;DR
The paper addresses how computing can become an on-demand utility while handling the architectural, market, scalability, security, and reliability challenges of Cloud deployment. It defines a market-oriented Cloud architecture and presents Cloudbus technologies for brokering, application development, federation, simulation, and real-world use. The paper also identifies interoperability and software licensing as boundaries for broader Cloud adoption.
Problem
Cloud computing requires architectures and management mechanisms that can deliver utility-style services while addressing security, regulatory, scalability, reliability, and interoperability challenges.
Method
The paper defines a market-oriented Cloud architecture and presents the Cloudbus toolkit, including brokering, PaaS, federation, simulation, and deployment technologies.
Results
Cloudbus technologies have been deployed in private Clouds, hybrid scientific workflows, commercial demonstrations, and industrial Cloud computing research.
Takeaways & Limitations
Cloudbus provides a concrete technology base for deploying and studying Cloud applications and for integrating heterogeneous Cloud resources through brokering and federation.
Takeaways & Limitations
Interoperability between Cloud providers and licensing proprietary software for public virtual appliances remain major hurdles to enterprise-public Cloud integration.
Abstract
from arXiv · showhide
This keynote paper: (1) presents the 21st century vision of computing and identifies various IT paradigms promising to deliver computing as a utility; (2) defines the architecture for creating market-oriented Clouds and computing atmosphere by leveraging technologies such as virtual machines; (3) provides thoughts on market-based resource management strategies that encompass both customer-driven service management and computational risk management to sustain SLA-oriented resource allocation; (4) presents the work carried out as part of our new Cloud Computing initiative, called Cloudbus: (i) Aneka, a Platform as a Service software system containing SDK (Software Development Kit) for construction of Cloud applications and deployment on private or public Clouds, in addition to supporting market-oriented resource management; (ii) internetworking of Clouds for dynamic creation of federated computing environments for scaling of elastic applications; (iii) creation of 3rd party Cloud brokering services for building content delivery networks and e-Science applications and their deployment on capabilities of IaaS providers such as Amazon along with Grid mashups; (iv) CloudSim supporting modelling and simulation of Clouds for performance studies; (v) Energy Efficient Resource Allocation Mechanisms and Techniques for creation and management of Green Clouds; and (vi) pathways for future research.
1 Introduction - Technology Trends
Cloud computing extends the long-standing vision of computing as an on-demand utility, delivering services through shared infrastructure and virtualized resources. Its expected economic impact and broad industry adoption motivate the Cloudbus initiative.
- Utility Computing: Computing utilities provide on-demand services that users pay for when accessed, without building or maintaining complex IT infrastructure.Users access services according to their requirements, regardless of where they are hosted.
- Cloud Service Models: Cloud computing delivers infrastructure, platforms, and applications as subscription-based, pay-as-you-go services.These offerings are commonly classified as IaaS, PaaS, and SaaS.
- Technology Trends: Virtualized data centers expose hardware, databases, interfaces, and application logic as services accessible on demand worldwide.This frees IT companies from low-level infrastructure tasks so they can focus on innovation and business value.
- Market Potential: Cloud services were projected to grow from $16 billion worldwide in 2008 to $42 billion in 2012.Cloud applications also act as catalysts or market makers that connect buyers and sellers.
- Adoption Outlook: Cloud computing was identified as an emerging technology with adoption expected to progress from peak expectations toward mainstream computing.The paper describes Gartner’s Hype Cycle as representing technology emergence, adoption, maturity, and impact.
- Paper Scope: The paper proceeds from Cloud computing definitions and challenges to Cloudbus architecture, toolkit technologies, integrations, and future trends.This organization frames Cloudbus as a response to the practical development of Cloud computing.
2 Cloud Computing
Cloud computing delivers the entire computing stack as pay-per-use services through virtualized, distributed infrastructure. The paper presents a market-oriented architecture and identifies security, regulatory, pricing, scalability, reliability, and energy challenges addressed by Cloudbus.
- Cloud Definition: Cloud computing delivers hardware infrastructure and software applications as services consumed on a pay-per-use basis.The approach spans applications and data-center infrastructure delivered through the Internet.
- Market-Oriented Architecture: Cloud brokers, coordinators, and the Cloud Exchange support application scheduling, resource allocation, workload migration, and SLA-based service trading.The Cloud Exchange aggregates demand, evaluates available supply, and supports secure SLA-related financial transactions.
- Open Challenges: Cloud adoption must address security, privacy, trust, legal and regulatory constraints, flexible pricing, SLA violations, and energy consumption.Data-center electricity use is described as equivalent to the power consumption of the Czech Republic worldwide.
- Open Challenges: Cloud infrastructures must scale reliably across geographically distributed and potentially multiple-vendor data centers while supporting application development.InterCloud protocols and mechanisms are needed for cross-provider scaling, and virtualization requires effective VM management under changing QoS demands.
- Cloudbus Contribution: The Cloudbus Toolkit is presented as a step toward making the Cloud vision concrete by addressing these challenges through models, software frameworks, and applications.Its scope includes Cloud deployment, management, and application support across the broader reference model.
- Cloud Computing Reference Model: The reference model spans system-level infrastructure delivery through user-level applications accessed from anywhere.Cloud services are commonly organized into SaaS, PaaS, and IaaS/HaaS categories.
3 Cloudbus Vision and Architecture
Cloudbus envisions a Cloud marketplace where users obtain computing, storage, and content-delivery services through brokers that match QoS, budget, and resource needs across heterogeneous providers.
- A Cloud marketplace would expose computing, storage, and content-delivery Clouds to end-users and enterprises.
- A meta-broker would select providers offering the best service under users’ budgets, constraints, and SLA-specified QoS requirements.Applications or separate broker clients submit resource needs to the market maker through provider-native or standardized interfaces.
- Cloud peering would federate heterogeneous providers, enabling service offloading and standardized interconnection policies.PaaS systems such as Aneka could balance private resources with virtual resources provisioned from public Clouds.
- MetaCDN would unify access to storage Clouds by placing content according to users’ QoS and budget preferences.The approach extends beyond compute-intensive services to storage and content delivery.
- Repeatable, controllable simulation environments are needed to evaluate provisioning policies, resource-allocation algorithms, and energy-efficient physical-resource use.Such tools help researchers reproduce problem settings and obtain reliable results.
- The Cloudbus Toolkit assembles brokering infrastructure and middleware that connect applications, market makers, providers, and Cloud marketplace resources.Its layered architecture supports third-party brokering services and multiple middleware implementations, including Cloudbus technologies.
4 Cloudbus / CLOUDS Lab Technologies
The Cloudbus Toolkit is a layered collection of technologies for market-oriented Cloud computing, spanning application deployment, brokering, federation, content delivery, simulation, and energy efficiency.
- The toolkit provides technologies for science, engineering, business, creative-media, and consumer applications on Clouds.
- Aneka: Aneka is a PaaS for developing and deploying Cloud applications through configurable containers, distributed programming models, prototyping tools, and market-oriented services.
- Aneka: Aneka’s flexible, customizable design targets education, engineering, scientific computing, and financial applications.Its container can be deployed on physical or virtual Internet-connected resources, supporting transparent integration with public and private Clouds.
- Aneka: Standardized interfaces allow Aneka to integrate with client applications and third-party brokers that negotiate QoS and submit applications to Aneka Clouds.
- Federation and services: InterGrid links Grid islands through peering arrangements for inter-Grid resource sharing, while MetaCDN places content on storage resources using QoS and budget preferences.
- Simulation and energy efficiency: CloudSim models compute, storage, network, and other Cloud data-center components, alongside research on scalability and energy efficiency.
4.2 Broker
The Grid Service Broker mediates distributed resource access by discovering, selecting, mapping, deploying, monitoring, and reporting on application execution.
- The broker discovers data sources, selects computational resources, maps jobs, deploys and monitors execution, accesses data, and presents results.
- It supports parameter-sweep, workflow, parallel, and bag-of-tasks application models.
- Plug-ins connect the broker with Globus, Aneka, Unicore, Condor, Unix platforms, PBS, and SGE.
- The broker can provision compute and storage services in Cloud resources via SSH and describe QoS parameters for applications requiring mixed services.
4.3 Workflow Engine
The Workflow Management System lets users compose and execute Cloud applications through a higher-level workflow abstraction, with editing, monitoring, discovery, and scheduling support.
- The WMS provides a workflow editor, XML-based workflow language, and portal components for discovery, monitoring, and scheduling.It can manage applications running on distributed resources with Aneka and/or the Broker.
- The WMS has supported fMRI brain imaging, evolutionary multi-objective optimization, and intrusion-detection applications.
4.4 Market Maker/Meta-broker
The Market Maker/Meta-broker mediates between Cloud users and providers by discovering suitable services and mapping jobs to them within a global marketplace.
- The Market Maker/Meta-broker works for both Cloud users and service providers.It mediates access to distributed resources and seeks suitable matches between applications and published services.
4.5 From InterGrid to InterCloud
The InterCloud vision connects otherwise disparate Clouds into a scalable, shared infrastructure. InterGrid and MetaCDN illustrate execution across sites and unified use of multiple storage providers.
- 4.5 From InterGrid to InterCloud: The InterCloud model links Cloud islands through peering to enable inter-Cloud resource sharing.It also aims to provide scalable interconnection and a global Cyberinfrastructure for e-Science and e-Business applications.
- 4.5 From InterGrid to InterCloud: InterGrid uses virtual machines to construct execution environments spanning multiple computing sites.Those sites may combine physical Grid machines with virtual machines running on infrastructures such as Amazon EC2.
- 4.6 MetaCDN: MetaCDN integrates Storage Cloud resources from multiple IaaS vendors into an overlay network for content delivery.It matches and places content according to quality of service, coverage, and budget preferences through a unified namespace.
4.7 Energy Efficient Computing
Cloud data centers consume substantial electricity and contribute to carbon emissions, motivating research on scalable and energy-efficient Cloud infrastructure. The work explores power-aware scheduling as one response.
- 4.7 Energy Efficient Computing: Data centers supporting elastic applications are expensive to operate because they consume significant electric power.The paper notes that Google’s data-center energy consumption is equivalent to the power consumption of cities such as San Francisco.
- 4.7 Energy Efficient Computing: The research investigates application scalability and energy efficiency to support service-oriented Green ICT technologies.Its power-aware scheduling work selects appropriate supply voltages for processing elements to reduce energy consumption in large data centers.
4.8 CloudSim
CloudSim is a customizable toolkit for modeling, simulating, and executing applications on extensible Cloud environments. Its virtualization and policy abstractions support research on resource allocation and scheduling.
- 4.8 CloudSim: CloudSim models and simulates extensible Clouds while allowing applications to execute on top of them.Its customizable design supports research without requiring provisioning and configuring real physical resources.
- 4.8 CloudSim: CloudSim supports large-scale infrastructure simulation and self-contained modeling of data centers, brokers, scheduling, and allocation policies.It can model large-scale Cloud infrastructure, including data centers, on a single physical computing node.
- 4.8 CloudSim: Its virtualization engine manages independent co-hosted virtualized services and switches between space-shared and time-shared core allocation.These capabilities are intended to speed development of Cloud resource-allocation policies and scheduling algorithms.
- 4.8 CloudSim: CloudSim evolved from GridSim, which provided heterogeneous resource modeling, resource brokering, scheduling, and extensions for failures and reservations.This lineage places CloudSim within prior simulation work for parallel and distributed computing.
5 Related Technologies, Integration, and Deployment
Cloudbus integrates its technologies with one another and with third-party solutions, supporting deployment across private, public, hybrid, and federated Cloud environments. Real-world uses include enterprise productivity, scientific workflows, and industrial Cloud research.
- Integration: Cloudbus technologies are integrated with each other and with third-party technologies, supporting ubiquitous access to computation and data through external infrastructures.Integration is presented as fundamental to enterprises and end-users offloading computation to third-party infrastructures.
- Related technologies: The reference model combines virtual server containers, infrastructure services, and higher-level brokers that provision compute resources with or without SLAs.Examples of virtualisation technologies include VMWare, Xen, and KVM; brokers select suitable services for end-users.
- Deployment: A private Aneka Cloud at GoFront increased product-design productivity and return on investment, while the Workflow Engine executed fMRI workflows on hybrid Clouds.The hybrid environment combined EC2 virtual resources with clusters worldwide.
- Deployment: Aneka supports hybrid Clouds by dynamically leasing public resources when private capacity is exceeded and federating additional private Clouds managed by Aneka or other vendors.Supported external technologies include XenServer and VMWare.
- Deployment: Cloudbus technologies have been demonstrated at international computing events through fMRI brain-imaging workflows and other applications, and CloudSim is used by HP Labs for industrial research.The demonstrations included the IEEE e-Science conference and the IEEE International Scalable Computing Challenge.
6 Future Trends
The paper anticipates service-oriented distributed computing becoming influential while identifying licensing, security, cost, energy, reliability, and interoperability challenges. It positions Cloud computing and Cloudbus as evolving toward utility computing, broader scalability, and more robust support for users.
- Future trends: Service-oriented distributed computing is expected to shape industry and service delivery, including health, financial, and government services.The paper connects this trend with increased dependence on ICT technologies and the need to support elastic applications serving millions of users.
- Challenges: Proprietary software licensing is identified as a major hurdle when software must be delivered to millions of users through public virtual appliances.The paper links customized software with potential enterprise integration with public Clouds for scalability and customer reach.
- Challenges: Cloud adoption raises security and privacy concerns for corporate data, motivating proposals for a globally accredited body to certify providers, standardize formats, and enforce SLAs.The proposed body would also handle trust certificates.
- Challenges: Dynamic negotiation and SLA management are expected to influence revenue as providers compete over pricing, usage costs, and delivered services.The paper presents this as an issue alongside broader technological challenges.
- Green Clouds: Energy consumption is a significant component of industrial-scale Cloud operating costs, encouraging energy-efficient allocation policies and renewable energy use.These measures are discussed as ways to conserve energy, reduce cooling costs, and lower carbon footprints.
- Challenges: Service continuity and interoperability remain essential because disruptions can destabilize markets and paralyze dependent IT services, while vendor lock-in remains a concern.The paper calls for adequate interoperability between Cloud service providers to prevent these effects.
- Future direction: Cloud computing is presented as a route toward treating distributed compute, storage, and application software as commodities, while Cloudbus continues evolving for robustness and scalability.The toolkit is being consolidated to support increasing numbers of users.