Queue Based Q-Learning for Efficient Resource Provisioning in Cloud Data Centers

Queue Based Q-Learning for Efficient Resource Provisioning in Cloud Data Centers

A. Meera (Department of Information Science and Technology, Anna University, Chennai, India) and S. Swamynathan (Department of Information Science and Technology, Anna University, Chennai, India)
Copyright: © 2015 |Pages: 18
DOI: 10.4018/IJIIT.2015100103
OnDemand PDF Download:
No Current Special Offers


Cloud Computing is a novel paradigm that offers virtual resources on demand through internet. Due to rapid demand to cloud resources, it is difficult to estimate the user's demand. As a result, the complexity of resource provisioning increases, which leads to the requirement of an adaptive resource provisioning. In this paper, the authors address the problem of efficient resource provisioning through Queue based Q-learning algorithm using reinforcement learning agent. Reinforcement learning has been proved in various domains for automatic control and resource provisioning. In the absence of complete environment model, reinforcement learning can be used to define optimal allocation policies. The proposed Queue based Q-learning agent analyses the CPU utilization of all active Virtual Machines (VMs) and detects the least loaded virtual machine for resource provisioning. It detects the least loaded virtual machines through Inter Quartile Range. Using the queue size of virtual machines it looks ahead by one time step to find the optimal virtual machine for provisioning.
Article Preview

1. Introduction

Cloud Computing is defined as a model for enabling ubiquitous, convenient, on demand network access to a shared pool of configurable computing resources that could be rapidly provisioned and released with minimal management effort or service provider interaction (NIST, 2010, p.2). The basic service models of cloud computing are Infrastructure as a Service (IaaS) such as IBM Blue house, Amazon EC2, Microsoft Azure Platform, Platform as a Service such as AWS, IBM Virtual images, Google App Engine and Software as a Service such as Google Calender, IBM Lotus Live.

Cloud infrastructure services deliver computer infrastructures like computing, memory storages and networking. Cloud platform services deliver a computing platform or solution stack as a service, often consuming cloud infrastructure and sustaining cloud applications. It facilitates deployment of applications without the cost and complexity of buying and managing the underlying hardware and software layers. Cloud application services deliver software as a service over the Internet, eliminating the need to install and run the application on the customer's own computers and simplifying maintenance and support.

Due to vast on-demand services, anything could be outsourced as a service which was commonly referred to as anything as a Service (Rittinghouse & Ransome, 2009). Apart from the basic services, there were other services that were offered from the cloud. They were Communication as a Service (CaaS) that outsourced enterprise communication solutions to customers. The CaaS Vendors were responsible for management of hardware and software voice over IP, instant messaging services, video conferencing, soft phones and multimedia conferencing. Database as a Service (DaaS) provided remotely hosted database service to users, need sharing of database and make to function as if the database were local. Integration as a Service allowed to access integration software functionalities on a pay per use model. Governance as a Service allowed the ability to manage the different cloud services.

The cloud services are classified into 3 categories such as public cloud, private cloud and hybrid cloud. A public cloud is available to anyone on the Internet. Any user can be signed in to use a public cloud. (E.g. Microsoft Windows Azure, Amazon Cloud). A private cloud is a proprietary cloud environment that only provides cloud services to a limited number of users or clients. They are normally built within their own data center. A hybrid cloud is a combination of private and public cloud. It provides services that run on a public cloud infrastructure using a virtual private network.

Cloud Computing is a recently developed cost and energy efficient computing paradigm that provides computing resources on a “pay as you go model” with a predefined quality of service. The quality of service is determined by scalability, availability, throughput, efficiency and other parameters. The performance of the cloud services are directly influenced by these parameters. A cloud data center is a repository to house various computational resources such as computing machines, storage and network and its associated components. Thousands of computing machines, blade servers and rack servers are hosted in a data center. Appropriate virtual machines must be created when a customer request is accepted by a cloud provider. This process is called as resource provisioning.

With the evolution of virtualization technologies, cloud environment has been deployed with resource consolidation and management. The physical resources can be consolidated to meet the demands of cloud users which results in efficient utilization of computing resources, power consumption and carbon footprint. To meet the variable demands, computing resources are virtualized to form virtual machines. The number of active virtual machines may vary according to the time as well as the type of application, thus require an intellect dynamic resource management framework. So, cloud datacenters and services that are based on virtualization require artificial intelligence for optimized resource allocation. Efficient usage of power, CPU, storage and network resources is resulted from these frameworks. The scope of this work depends on the Infrastructure as a Service (IaaS).

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 18: 4 Issues (2022): Forthcoming, Available for Pre-Order
Volume 17: 4 Issues (2021): 3 Released, 1 Forthcoming
Volume 16: 4 Issues (2020)
Volume 15: 4 Issues (2019)
Volume 14: 4 Issues (2018)
Volume 13: 4 Issues (2017)
Volume 12: 4 Issues (2016)
Volume 11: 4 Issues (2015)
Volume 10: 4 Issues (2014)
Volume 9: 4 Issues (2013)
Volume 8: 4 Issues (2012)
Volume 7: 4 Issues (2011)
Volume 6: 4 Issues (2010)
Volume 5: 4 Issues (2009)
Volume 4: 4 Issues (2008)
Volume 3: 4 Issues (2007)
Volume 2: 4 Issues (2006)
Volume 1: 4 Issues (2005)
View Complete Journal Contents Listing