Article Preview
Top1. Introduction
Cloud computing is regarded as a new software delivery paradigm and a 5th utility service after water, electricity, gas and telephony (Buyya et al., 2008). Businesses are shifting their technologies to the Cloud in order to save on the costs of infrastructure, maintenance and personnel. However there are various risks associated with using the Cloud and new research is turning to building trust and security standards in order for the customers to use the Cloud with greater confidence (Djemame et al., 1983; Khan et al., 2012; Kiran et al., 2011). Testing becomes an essential part of these standards.
Software testing is challenging and expensive requiring time and resources to scrutinize the application’s reliability, functionality and performance. Testing for the Cloud should, in essence, be simple and cost effective; and go beyond the traditional kind of functional and non-functional testing practiced by developers. This is because there is still a need to build trust within the community of Cloud users, and the need for repeated revalidation of the same software, as it is extended, customized or migrated from one platform to another. The specific testing approaches used differ depending on the various kinds of Cloud ecosystem, which vary according to infrastructure, platform and software architecture and the different stakeholder roles involved. Testing should cover the functional and non-functional aspects of the services. In this paper, various testing issues are discussed in the context of future Cloud ecosystems, which will require a spread of testing methods to validate services at the different interaction points between the stakeholders. The paper also surveys the current state-of-the-art in Cloud testing, identifying the gaps between this and the needs that any future complete Cloud testing methodology should satisfy.
Figures 1 and 2 describe various testing dimensions across Cloud platforms. Depending on the different kinds of platforms, depicting the Cloud environment being used, testing issues vary across these. Examples of these issues include functional testing methods, penetration testing or multi-tenancy testing. These depend on the scenario to show which ones would be relevant to the Cloud ecosystems being used. For instance security testing may have a greater influence in multi-cloud environments rather than private Cloud environments. Besides the environments, Clouds exist in three forms depending on the functionality being offered,
Figure 1. Examples of testing issues for various cloud aspects
Figure 2. Cloud testing issues lie across the platforms being used
- •
SaaS (Software as a Service): Uses the Web to deliver third-party applications to Clients. For example:Gmail.
- •
PaaS (Platform as a Service): provides framework to build applications on top as well. It provides the computer infrastructures, hardware and highly scalable. Example: GoogleAppEngine, Heroku.
- •
IaaS (Infrastructure as a Service) third party allows you to install a virtual server on their IT infrastructure.
Testing SOA applications, where most literature exists, focuses on the SaaS functionality of Clouds. Further work for the other two functionalities may need to include a collection of testing issues as shown in Figure 3. These are discussed below in this section with an extensive discussion on related issues of testing with Cloud-related aspects, highlighting, functional, non-functional, service oriented architectures and specific Cloud issues.
Figure 3. Categories of publications when searching for “testing cloud computing” papers (2004-1014)