While it is common to use storage in a cloud-based manner, the question of true interoperability is rarely fully addressed. This question becomes even more relevant since the steadily growing amount of data that needs to be stored will supersede the capacity of a single system in terms of resources, availability, and network throughput quite soon. The logical conclusion is that a network of systems needs to be created that is able to cope with the requirements of big data applications and data deluge scenarios. This chapter shows how federation and interoperability will fit into a cloud storage scenario. The authors take a look at the challenges that federation imposes on autonomous, heterogeneous, and distributed cloud systems, and present approaches that help deal with the special requirements introduced by the VISION Cloud use cases from healthcare, media, telecommunications, and enterprise domains. Finally, the authors give an overview on how VISION Cloud addresses these requirements in its research scenarios and architecture.
TopIntroduction
There is always a point in time, when one system is not enough anymore. History showed that applications always managed to require more resources than a single instance of a system can provide at a time. There are several reasons why one would go beyond a single cloud storage system besides the simple need of more storage space. In real life, there are enterprise borders and there is data that is not allowed to leave the enterprise for legal or compliance issues. There are geographical constraints and it is sometimes simply not efficient enough to access data at the other side of the globe, just because a company’s cloud storage is located there. Furthermore, sometimes other storage offers may simply be cheaper. We notice that there is more than enough motivation to take a closer look on federation and interoperability of cloud storage systems.
The term federation describes the interworking of two or more otherwise autonomous and possibly heterogeneous systems. Creating a federation of existing cloud systems supports the user with a unified and combined view of storage and data services across several providers and systems. A storage federation allows users to seamlessly migrate, backup, synchronize, and monitor data and resources across connected cloud storage systems and supports various use cases such as collaboration, storage outsourcing, hybrid storage management and geographical data distribution to different providers or company branches. Federating resources to multiple cloud providers to scale beyond the storage limits of a single provider, guarantees reliability in case of failures and thus leads to higher data security. How does that relate to the term interoperability of cloud systems? Even though this turns out to be a religious discussion, we define interoperability as the technical base on which we can build up a federated system. As an example, we use the interoperability characteristics of HTTP, REST, and CDMI to create a combined, federated view of user’s resources.
Federation is an important consideration for cloud storage providers and for companies consuming storage services. Storage providers have an interest in keeping their costs low which leads to an economical best practice of operating clusters at full capacity whenever possible to be cost efficient. If peak loads are to be expected, the provider needs to make the same decision as a potential cloud user: How much CAPEX (own infrastructure) should be turned to OPEX (variable cost of renting infrastructure)? Given the technical possibility of a cloud federation, a cloud provider can dimension his cluster for average load scenarios and use federated resources on a “Pay per Use” basis if needed. The same is true for supplying customers with compute resources in countries where no dedicated infrastructure is available, to grant a certain service level. Users of cloud storage solutions will also benefit from federation: using more than one cloud allows to mitigate the risk of storage failures and prevents data lock-in.
Recently, it also became obvious that there is an emerging need for hybrid clouds, that use federation and interoperability technologies to combine access to public and private clouds in parallel (Managing Private and Hybrid Clouds for Data Storage, 2010). Hybrid clouds essentially federate access to multiple pools of resources in order to meet business, performance and compliance requirements. For example, an enterprise may choose to store sensitive data in the private storage cloud, while utilizing the low cost and the elasticity of the public storage cloud for other less sensitive data. In addition, hybrid clouds are useful for creating storage tiers, with lower cost tiers being exported to public clouds, while storing only the higher level data that needs to meet the low latency requirements in private clouds. Similarly, replicating the data to external public clouds, may lower the overall cost of data backup and archiving. Finally, hybrid storage clouds are useful when elastic demand for storage exceeds the available internal resources (Managing Private and Hybrid Clouds for Data Storage, 2010).
Another not to be underestimated driver for federation, especially for new cloud service providers on the market, is the possibility to allow customers an easy migration or on-boarding between different storage clouds, while keeping data accessible for the users.