Migration of Data between Cloud and Non-Cloud Datastores

Cloud Datastore

Various cloud datastores and their data storage schemes are discussed. An application running on the cloud requires dealing with a huge amount of data. Applications must be as scalable as the database of application. RDBMS is not suitable for such a requirement, e.g. handling a large amount of unstructured data, providing elastic scalability, etc. Therefore, new document oriented distributed datastores are emerging to cater to these requirements. Moreover, different cloud datastores are following different schemes to store the data. We discuss the data storage scheme of such cloud datastores to elaborate the point.

Apache CouchDB

CouchDB is an open source document-oriented schema free database-management system, accessible using a RESTful JavaScript Object Notation (JSON) API. Couch stands for Cluster of Unreliable Commodity Hardware. It stores data in JASON format. It can include all native datatypes in programming language. It stores data in the form of a document.

CouchDB is schema free, i.e. one document can have fields that another document doesn't have. Documents are the actual representation of data objects. CouchDB stores uniquely named documents with document ID and revision number. Each document can be made of a number of fields not bound by size and with unique names. Documents can have attachments that can be both text as well as digital. When changes are made, a new version of document called revision is created. It does not have a locking mechanism. Two users can load and edit the same document at same time. It maintains consistency by ensuring updates either work or fail (Bhat, 2010).

It achieves scalability and availability by periodically copying documents between servers. It supports file attachments, which can be in the form of music, images, etc. This feature is not seen in traditional databases. It assigns a universally unique identifier to each and every document. It does not support join. It allows creating arbitrary relations between documents.

Amazon SimpleDB

Items are stored as resources in simpleDB. These resources can have multivalued attributes. These resources can be related to each other, and the relationship between these resources can be visualized as a hierarchical tree structure as shown in Figure 1.

Figure 1.

Amazon simpleDB data storage model

As it is a schema-less storage, an item can have a different set of attributes from other items in the domain. It does not store raw data. Rather, it expands input data and creates indices over multiple dimensions to quickly query that data. It can be used as a flat file store. Individual item name, attribute names, and attribute values can be up to 1024 bytes in length. Amazon simpleDB allows 10GB of storage for each domain with 100 domains per customer account, which provides 1TB of total storage.