Efficient Placement and Processing in Shared-Nothing Data Warehouses

Abstract

Some businesses generate giga or even terabytes of historical data that can be organized and analyzed for better decision making. This poses issues concerning systems and software for efficient processing over such data. While the traditional solution to this problem involves costly hardware and software, we focus on strategies for running large data warehouses over low-cost, non-dedicated nodes in a local-area network (LAN) and non-proprietary software. Once such a technology is in place, every data warehouse will be able to run in a small cost environment, but the system must be able to choose its placement and processing for maximum efficiency. We discuss the basic system architecture and the design of the data placement and processing strategy. We compare the shortcomings of a basic horizontal partitioning for the environment, with a simple design that produces efficient placements. Our discussion and results provide important insight into how low-cost efficient data warehouse systems can be obtained.