Testing Software Services in Cloud Ecosystems

Testing Software Services in Cloud Ecosystems

Mariam Kiran (School of Electrical Engineering and Computer Science, University of Bradford, Bradford, UK) and Anthony Simons (Department of Computer Science, University of Sheffield, Sheffield, UK)
Copyright: © 2016 |Pages: 17
DOI: 10.4018/IJCAC.2016010103
OnDemand PDF Download:
No Current Special Offers


Testing in the Cloud is far more challenging than testing individual software services. A multitude of factors affect testing, including variations across platforms and infrastructure. Architectural issues include differences between private, public Clouds, multi-Clouds and Cloud-bursting. Platform issues include cross-vendor incompatibility, and diverse locales of service deployment and consumption. Software issues include integration with third-party services, the desire to validate competing service offerings to similar standards and need to re-validate services at different stages of service lifecycle. A complete approach to testing whole Cloud ecosystems should involve all relevant stakeholders, such as service provider, consumer and broker. When testing Clouds, the methodologies used should not hinder the advantages Cloud usage brings to the users or programmers and more importantly be simple and cost effective. However, these testing methodologies differ according to the various kinds of Cloud ecosystems and the different user perspectives of the actors involved such as the end-user, the infrastructures, or the different software (i.e. web services). This paper also studies the state-of-the-art in Cloud testing where most research focuses predominantly on web services, functional testing and quality-of-service, usually being considered separately. The authors suggest a framework, Quality-as-a-Service (QaaS) which integrates quality issues such as functional behaviour and performance monitoring with lifecycle governance and security of the service. This paper maps out the themes in the contemporary research literature and links them with the service lifecycle process for validating future Cloud services. Along the way, the authors identify important research questions that the future Cloud service testing agenda should seek to address.
Article Preview

1. Introduction

Cloud computing is regarded as a new software delivery paradigm and a 5th utility service after water, electricity, gas and telephony (Buyya et al., 2008). Businesses are shifting their technologies to the Cloud in order to save on the costs of infrastructure, maintenance and personnel. However there are various risks associated with using the Cloud and new research is turning to building trust and security standards in order for the customers to use the Cloud with greater confidence (Djemame et al., 1983; Khan et al., 2012; Kiran et al., 2011). Testing becomes an essential part of these standards.

Software testing is challenging and expensive requiring time and resources to scrutinize the application’s reliability, functionality and performance. Testing for the Cloud should, in essence, be simple and cost effective; and go beyond the traditional kind of functional and non-functional testing practiced by developers. This is because there is still a need to build trust within the community of Cloud users, and the need for repeated revalidation of the same software, as it is extended, customized or migrated from one platform to another. The specific testing approaches used differ depending on the various kinds of Cloud ecosystem, which vary according to infrastructure, platform and software architecture and the different stakeholder roles involved. Testing should cover the functional and non-functional aspects of the services. In this paper, various testing issues are discussed in the context of future Cloud ecosystems, which will require a spread of testing methods to validate services at the different interaction points between the stakeholders. The paper also surveys the current state-of-the-art in Cloud testing, identifying the gaps between this and the needs that any future complete Cloud testing methodology should satisfy.

Figures 1 and 2 describe various testing dimensions across Cloud platforms. Depending on the different kinds of platforms, depicting the Cloud environment being used, testing issues vary across these. Examples of these issues include functional testing methods, penetration testing or multi-tenancy testing. These depend on the scenario to show which ones would be relevant to the Cloud ecosystems being used. For instance security testing may have a greater influence in multi-cloud environments rather than private Cloud environments. Besides the environments, Clouds exist in three forms depending on the functionality being offered,

Figure 1.

Examples of testing issues for various cloud aspects

Figure 2.

Cloud testing issues lie across the platforms being used

  • SaaS (Software as a Service): Uses the Web to deliver third-party applications to Clients. For example:Gmail.

  • PaaS (Platform as a Service): provides framework to build applications on top as well. It provides the computer infrastructures, hardware and highly scalable. Example: GoogleAppEngine, Heroku.

  • IaaS (Infrastructure as a Service) third party allows you to install a virtual server on their IT infrastructure.

Testing SOA applications, where most literature exists, focuses on the SaaS functionality of Clouds. Further work for the other two functionalities may need to include a collection of testing issues as shown in Figure 3. These are discussed below in this section with an extensive discussion on related issues of testing with Cloud-related aspects, highlighting, functional, non-functional, service oriented architectures and specific Cloud issues.

Figure 3.

Categories of publications when searching for “testing cloud computing” papers (2004-1014)


Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 12: 4 Issues (2022): 2 Released, 2 Forthcoming
Volume 11: 4 Issues (2021)
Volume 10: 4 Issues (2020)
Volume 9: 4 Issues (2019)
Volume 8: 4 Issues (2018)
Volume 7: 4 Issues (2017)
Volume 6: 4 Issues (2016)
Volume 5: 4 Issues (2015)
Volume 4: 4 Issues (2014)
Volume 3: 4 Issues (2013)
Volume 2: 4 Issues (2012)
Volume 1: 4 Issues (2011)
View Complete Journal Contents Listing