Abstract
The proliferation of public cloud providers and services offered necessitate that end users have benchmarking-related information that help compare the properties of the cloud computing environment being provided. System-level benchmarks are used to measure the performance of overall system or subsystem. This chapter surveys the system-level benchmarks that are used for traditional computing environments that can also be used to compare cloud computing environments. Amazon's EC2 Service is one of the leading public cloud service providers and offers many different levels of service. The research in this chapter focuses on system-level benchmarks and looks into evaluating the memory, CPU, and I/O performance of two different tiers of hardware offered through Amazon's EC2. Using three distinct types of system benchmarks, the performance of the micro spot instance and the M1 small instance are measured and compared. In order to examine the performance and scalability of the hardware, the virtual machines are set up in a cluster formation ranging from two to eight nodes.
TopCurrent cloud computing offerings come from a number of service providers ranging from small local companies running 3rd party cloud OS software to offer up their hardware resources to other local businesses to some of the world’s largest companies running massive services running their own proprietary software to serve clients across the entire globe. The smaller providers are typically more focused on local business clients as their hardware infrastructure isn’t large enough to scale for the needs of large national or international companies. For this reason we will focus on a subset of services offered by the three largest providers; Amazon Web Services (AWS) Elastic Compute Cloud (EC2) which was used in testing for this paper, as well as Microsoft Windows Azure (Azure) VM Roles, and Google Cloud Platform (GCP) App Engine (AE).
The Elastic Compute Cloud (EC2) by Amazon is an Infrastructure as a Service (IaaS) public cloud. EC2 is available, for an hourly fee, to the public. As an IaaS product, EC2 offers users a virtual machine and options of OS to be installed on it. This is to contrast against the Platform as a Service (PaaS) product, where an OS with middleware is rented to the user, or Software as a Service (SaaS), which provides an end-product or application for use. This makes EC2 flexible for use during peak hours or as a full time enhancement or replacement of the existing server infrastructure of a business.
The VM Roles from Windows Azure are very similar to the EC2 instances and are also IaaS services but are a less mature product and as such offer far fewer sizing options. AppEngine (AE) from Google in contrast is a PaaS and offers no options of operating system and even restricts the programming languages to Java (and others that compile to java byte code), Python, and currently experimentally, the Google Go language.
Key Terms in this Chapter
Infrastructure as a Service: The hosting of various hardware and hardware configurations by a third party in a remote location accessible over the Internet. Oftentimes highly scalable in terms of resources which can be adjusted on-demand for dynamic utilization.
Cloud Computing: The practice of hosting services on a remote server(s) which are accessible through the Internet. Connected users are able to access, store, manage, and process data utilizing the resources of the remote server instead of their local computer.
Instance Type: Refers to the specifications of a virtual machine. The computational resources allocated to a virtual machine will determine its instance type. Large instance types will have more dedicated resources for the virtual machine to utilize.
Cluster: A group of computers networked together so that they are able to share resources and function as a single, distributed entity.
Computing Services: Availability of resources to perform computational work via a third party. Typically in the form of data processing or application hosting through Cloud providers.
Virtualization: Simulating the environment of a computer hardware platform, operating system, or application. To the end-user, the virtualized environment should be indistinguishable from a similar real environment.
Benchmarking: The process of measuring the performance of an application, platform, or communication medium.