Chapter Preview
TopMain Focus Of The Chapter
This section will give an introduction of in memory data processing systems category by category, including major products in the market and prototypes from academia. Main memory database systems (or in memory database system, IMDS) are the vanguards to leverage the big memory for higher data processing performance (Li & Patel, 2013; Balkesen, Teubner, Alonso, & Özsu, 2013).
By leveraging the large memory capacity, compression techniques, as well as various optimization techniques, main memory database systems usually could obtain a performance boost of more than one order of magnitude over traditional database systems. SAP has conducted a benchmarking (SAP HANA - b, 2012) to show the advantages of its HANA in memory database. The 100 TB (terabytes) sales and distribution data set is compressed by 20 times into 3.78 TB and distributed onto a cluster of 16 nodes, each of which is equipped with 4 CPUs, 512GB (gigabytes) of RAM, and 3.3 TB of disk storage. In the experiments, they achieved an average response time of less than 2 seconds on most queries, and remained lengthy queries could be finished in around 5 seconds.
Key Terms in this Chapter
NewSQL: Enhanced version of traditional relational database system with higher scalability yet with fully ACID support.
OLAP: Online analytic processing.
ACID: Properties of a database transaction, including atomicity, consistency, isolation, and durability. Database systems should provide the ACID guarantee for easy programming of transactions.
NoSQL: A database that uses looser consistency models than traditional relational databases in order to achieve horizontal scaling and higher availability.
YCSB: Yahoo! Cloud Serving Benchmark, a benchmark for testing cloud based data service systems.
MMDB: Main memory database systems, also call in memory database systems (IMDS). In a MMDB the whole data set is resident in main memory for fast access. Main memory database systems borrow many implementation ideas from disk resident database systems, however they are different from disk resident database systems in terms of data layout, indexing, concurrency control, and recovery etc.