Placement and Scheduling over Grid Warehouses

Placement and Scheduling over Grid Warehouses

Rogério Luís de Carvalho Costal (University of Coimbra, Portugal) and Pedro Furtado (University of Coimbra, Portugal)
DOI: 10.4018/978-1-60566-364-7.ch005
OnDemand PDF Download:


The computational grid offers services for efficiently scheduling jobs on the grid, but for grid-enabled applications where data handling is a most relevant part, the data grid kicks in. It typically builds on the concept of files, sites and file transfers between sites. These use a data transfer service, plus a replica manager to keep track of where replicas are located. The authors consider a multi-site, grid-aware data warehouse, which is a large distributed repository sharing a schema and data concerning scientific or business domains. Differently from typical grid scenarios, the data warehouse is not simply a set of files and accesses to individual files. It is a single distributed schema and both localized and distributed computations must be managed over that schema. Given this difference, it is important to study approaches for placement and computation over the grid data warehouse and this is our contribution in this book chapter.
Chapter Preview


The Grid is an infrastructure that coordinates and provides transparent access to distributed and heterogeneous resources for geographically distributed users. One of its main usages is to enable the implementation of Virtual Organizations (Foster, 2001).

Virtual Organizations (VO) are composed by (members of) distinct real organizations or institutions, which cooperatively participate in the Grid, sharing resources and using the shared resources in order to solve common problems or to achieve common objectives. In such multi-institutional grid it is somewhat frequent that individual domain administrators impose some kind of usage constraints on local domains’ resources to remote users (Foster, 2001; Schopf & Nitzberg, 2002). Business historical information, molecular simulations and measurements of sensor data are examples of some current and future applications that run on VO and which can produce large amounts of distributed data.

In Data Grids (Chervenak et al., 2001; Krauter et al., 2002), the Grid software is used to manage data storage, transparent data access or the execution of data intensive jobs. Flat files are frequently used to store data. Grid-based services are used to execute efficient data movement (e.g. GridFTP (Allcock et al., 2005)) and to locate database replicas (e.g. Globus’ Replica Location Service – RLS (Chervenak et al., 2004)).

On the other hand, Data Warehouses (DWs) are huge mostly read-only databases used primarily for decision support purposes. They are target of periodic data loads as part of ETL (Extraction Transform and Load) processes: data from several OLTP (Online Transaction Processing) systems’ is transformed and loaded into the DW, which traditionally resides at a single site (Chaudhuri & Dayal, 1997) (parallel hardware is commonly used to improve performance).

Therefore, the Grid-based Data Warehouse can be used as a tool to support several processes in global Virtual Organizations, including decision making and knowledge discovery. It enables users from different geographical locations (and possibly from different real organizations), which are participating in the VO, to share and query shared data transparently and efficiently. Hence, given a data warehouse schema, grid-enabled DWs have queries, data mining analysis and computations of all kinds happening over large quantities of data. For instance, groups of scientific organizations may provide public multi-site data repositories and computing platforms for simulations and analysis, as it is already beginning to happen with biological data such as protein conformation studies.

Implementing efficient data placement and query scheduling strategies is one of the challenges in such environment. For instance, placing all the data at a single site may not be the most interesting choice. In Grids, distributed resources from multiple sites are available to be cooperatively used to perform tasks. Using such distributed resources to implement a Grid-based DW may lead to a high performance environment at a reduced monetary cost, when compared to placing all the DW at a single grid site (especially due to reduced requirements in hardware and network links at individual sites). Hence, data and query distribution across several sites should be considered in order to obtain a high performance grid-based DW.

Besides that, it is important to notice that warehouse users may query the database according to some kind of access pattern (Deshpande et al., 1998). For instance, users from distinct sites may be more interested in doing drill-down and drill-up operations in distinct geographically-related data (each user accessing data that is somewhat related to the region where the user is placed). As doing data movement across different grid sites may be time consuming, the existent users’ interests must also be considered when doing placement across distinct sites. Furthermore, as workloads and resource characteristics may change over time, the system should also have some kind of dynamic data placement (or replication) strategy, which is also useful when users’ access patterns are not known in advance (or change over time).

But the specialized data placement strategy and the abovementioned aspects lead to the need of a specialized query scheduling strategy, which may consider the existence of partitioned and replicated tables when scheduling query execution among distinct sites.

Complete Chapter List

Search this Book:
Editorial Advisory Board
Table of Contents
Mark Baker
Nik Bessis
Nik Bessis
Chapter 1
Enjie Liu, Xia Zhao, Gordon J. Clapworthy
At the heart of the Grid technology is the concept of resource sharing, which includes computers, storage and networks. Grid currently appears to be... Sample PDF
Building Service-Oriented Grid Applications
Chapter 2
Giuseppe Andronico, Roberto Barbera, Marco Fargetta, Emidio Giorgio, Salvatore Marco, Diego Scardaci
Grid computing allows for the creation of e-infrastructures providing computational power and information storage capabilities needed both by... Sample PDF
Sustainable and Interoperable e-Infrastructures for Research and Business
Chapter 3
Vassilikil Andronikou, Dimosthenis Kyriazis, Magdalini Kardara, Dimitrios Halkos, Theodora Varvarigou
The Grid has the potential to make a significant advance beyond the Internet, by turning it from a passive information medium into an active tool... Sample PDF
Scenarios of Next Generation Grid Applications in Collaborative EnvironmentsA Business-Technical Analysis
Chapter 4
Gayathri Nadarajan, Areti Manataki, Yun-Heh Chen-Burger
The infrastructure of Grid is approaching maturity and can be used to enable the utilisation and sharing of large scale, remote data storages... Sample PDF
Semantics-Based Process Support for Grid Applications
Chapter 5
Rogério Luís de Carvalho Costal, Pedro Furtado
The computational grid offers services for efficiently scheduling jobs on the grid, but for grid-enabled applications where data handling is a most... Sample PDF
Placement and Scheduling over Grid Warehouses
Chapter 6
Navonil Mustafee, Simon J.E. Taylor
The computational grid offers services for efficiently scheduling jobs on the grid, but for grid-enabled applications where data handling is a most... Sample PDF
Leveraging Simulation Practice in Industry through use of Desktop Grid Middleware
Chapter 7
Genoveffa Jeni Giambona, Nicholas L.J. Silburn, David W. Birchall
This chapter focuses on the collaborative use of computing resources to support decision making in industry. Through the use of middleware for... Sample PDF
Trust, Virtual Teams, and Grid Technology
Chapter 8
Rob Smith, Rob Wilson
Flexible and remote working is becoming more and more widespread. In particular, virtual team working is growing rapidly. Although virtual teams... Sample PDF
The Socio-Technical Virtual Organisation
Chapter 9
Marina Burakova-Lorgnier
A Virtual Organisation (VO) or Virtual Enterprise is a loosely-coupled group of collaborating organisations, acting to some extent as though they... Sample PDF
Modelling Trust–Control Dynamics for Grid-based Communities: A Shared Psychological Ownership Perspective
Chapter 10
Lu Liu, Nick Antonopoulos
The aim of this chapter is to appreciate the need for and propose some thoughts on modelling trust–control dynamics for communities that use grid... Sample PDF
Small World Architecture for Building Effective Virtual Organisations
Chapter 11
James Dooley, Andrea Zisman, George Spanoudakis
A Virtual Organisation in large-scale distributed systems is a set of individuals and/or institutions with some common purposes or interests that... Sample PDF
Runtime Service Discovery for Grid Applications
Chapter 12
Nik Bessis
This chapter describes a framework to support runtime service discovery for Grid applications based on service discovery queries in both push and... Sample PDF
Model Architecture for a User Tailored Data Push Service in Data Grids
Chapter 13
Eleana Asimakopoulou, Chimay J. Anumba, Bouchlaghem, Bouchlaghem
Much work is under way within the Grid technology community on issues associated with the development of services to foster collaboration via the... Sample PDF
Using Grid Technology for Maximizing Collaborative Emergency Response Decision Making
Chapter 14
Ahmet Sayar, Geoffrey C. Fox, Marlon E. Pierce
Geographic information is critical for building disaster planning, crisis management, and early-warning systems. Decision making in geographic... Sample PDF
Unified Data Access/Query over Integrated Data-views for Decision Making in Geographic Information Systems
About the Contributors