Assisting Efficient Job Planning and Scheduling in the Grid

Assisting Efficient Job Planning and Scheduling in the Grid

Enis Afgan (University of Alabama at Birmingham, USA) and Purushotham Bangalore (University of Alabama at Birmingham, USA)
DOI: 10.4018/978-1-60566-184-1.ch003
OnDemand PDF Download:
$37.50

Abstract

Grid computing has emerged as the next generation computing platform. Because of the resource heterogeneity that exists in the grid environment, user jobs experience variable performance. Grid job scheduling, or selection of appropriate mappings between resources and the application, with the goal of leveraging available capacity and imposed requirements is thus at the heart of successful grid utilization. Grid job scheduling can be viewed as a function of resource heterogeneity, resource and application availability, and application options. This chapter presents work that incorporates all of these factors to provision and present individual users with alternative job options in terms of cost and time tradeoffs. Inherently, this leads to more effective scheduling policies. To support these aims, a framework is introduced with a novel scheduling methodology that introduces new user-scheduler interaction levels and a new layer of scheduling that includes application parameter selection and parameter value optimization.
Chapter Preview
Top

Introduction

Over the past few years, grid computing (Foster and Kesselman, 1999) has gained popularity as the emerging architecture for next-generation high performance distributed computing. Goal of providing ubiquitous access to distributed, High Performance Computing (HPC) resources that are shared between multiple organizations through “virtualization” and “aggregation”, is only as efficient as its overall perception by the end users though. In order to realize this goal, grid middleware provides a standard set of services for authentication, authorization, resource allocation and management, job submission and monitoring, as well as data transfer and management (Berman et al., 2003), thus providing necessary abstractions from individual resources. Because grid provides access to heterogeneous pool of resources, grid job scheduling, or selection of appropriate mappings between resources and application, is at the heart of grid success. However, this scheduling process is perplexed with complexities due to resource availability, application and resource dependencies (Berman, 1998), as well as user goals and requirements. In order for grid to gain wide spread appeal and make efficient use of available resources, grid scheduling needs to be raised to the level where it can accommodate for such aspirations. User experience has to be tailored to support the individual user and provide them with the options they need and desire.

Access to grid resources is typically handled through a job submission interface (e.g., a web-based portal (Gannon et al., 2003), command line interface) where the user is requested to supply job options and parameters. Options requested depend on the scheduling engine employed behind the submission interface and can range from as few as application name and job input files to as many as an individual application supports (e.g., requirements for number of processors employed, amount of memory needed, speed of data transfer). If the latter grid job submission interface is perceived from a perspective of a typical user, such as an applied scientist, all the available resources and choices might appear equivalent and inherent differences would not be recognized, while the selection of which resource to run the job on might be random or based on previous experiences. In addition, once a routine has been established, even though the resource availability, input data, algorithms, or even the applications change, the user may always use the same resource and/or options. The resulting observation is that grid experience does not meet user’s expectations while leading to underutilized and inefficient use of resources (in terms of both, cost and time).

Grid job scheduling can be viewed as a function of resource heterogeneity, resource and application availability, and application options. In this paper, a grid scheduling framework is presented that advances grid job scheduling to include all of above stated parameters. Through incorporation of these factors, the goal of this work is to provision and present users with concrete and detailed options regarding their jobs, as well as provide more effective and efficient scheduling policies. To support these aims, the framework introduces a novel scheduling methodology that introduces new user-scheduler interaction levels and a new layer of scheduling that includes application parameter selection and parameter value optimization. To accommodate for the overall framework goals, individual components within the proposed framework will allow specification of application requirements, enabling initial application registration on the grid, an application profiling system developed and customized for data analysis tools, data analysis tools geared toward job parameter optimization, and resource-job cost normalization environment enabling comparisons between resource cost provider formats. These tools act as support mechanisms for the framework as a whole to assist users with planning and scheduling their job execution on the grid.

Aim of this work is to realize cost-to-application-runtime tradeoffs from user’s perspective. This is accomplished by providing job scheduling alternatives, thus providing support for situation-oriented user requirements and considerations. The act of resource selection and scheduling is the central point in all aspects of the grid and, as the grid evolves, will be a key to account management and cost tracking. Providing users with easily understood and applicable, yet targeted job alternatives is thus a key requirement for the grid to see wide overall acceptance.

Key Terms in this Chapter

Scheduling Optimization: It refers to grid job scheduling with a mathematically targeted goal of obtaining an optimal solution under current constraints, such as an optimum in combined execution time and cost).

Grid Job Scheduling: It is the process of goal-driven resource selection in grid computing for a given application at a given moment in time that is constrained by resource availability and application options.

Job Parameterization: Refers to targeted selection of application options and values for those options, each of which can alter job execution variables (e.g., execution time, cost).

Grid Application: An application that is capable of executing in a grid environment with a goal of taking advantage offered by grid environment, such as heterogeneous and dynamic resource availability.

Grid Computing: It is a form of distributed computing where, through networking and middleware, possibly geographically distributed and heterogeneous resources are aggregated and virtualized into a seamless resource pool available to its users.

Virtualization: Refers to abstraction of low level details otherwise associated with resource access in grid computing to simplify cross computer communication.

Aggregation: It is collection of dispersed resources into a readily available pool of resource that can be simultaneously be used to complete a task

Complete Chapter List

Search this Book:
Reset
Editorial Advisory Board
Table of Contents
Foreword
Ruth E. Shaw
Preface
Emmanuel Udoh, Frank Zhigang Wang
Acknowledgment
Emmanuel Udoh
Chapter 1
Emmanuel Udoh, Frank Zhigang Wang, Vineet R. Khare
This chapter presents a historical record of the advent of Grid with a recourse to some basic definitions commonly accepted by most researchers. It... Sample PDF
Overview of Grid Computing
$37.50
Chapter 2
Eric Aubanel
The problem of load balancing parallel applications is particularly challenging on computational grids, since the characteristics of both the... Sample PDF
Resource-Aware Load Balancing of Parallel Applications
$37.50
Chapter 3
Enis Afgan, Purushotham Bangalore
Grid computing has emerged as the next generation computing platform. Because of the resource heterogeneity that exists in the grid environment... Sample PDF
Assisting Efficient Job Planning and Scheduling in the Grid
$37.50
Chapter 4
Kuo-Chan Huang, Po-Chi Shih, Yeh-Ching Chung
Most current grid environments are established through collaboration among a group of participating sites which volunteer to provide free computing... Sample PDF
Effective Resource Allocation and Job Scheduling Mechanisms for Load Sharing in a Computational Grid
$37.50
Chapter 5
Tevfik Kosar
As the data requirements of scientific distributed applications increase, the access to remote data becomes the main performance bottleneck for... Sample PDF
Data-Aware Distributed Batch Scheduling
$37.50
Chapter 6
Gianni Pucciani, Flavia Donno, Andrea Domenici, Heinz Stockinger
Data replication is a well-known technique used in distributed systems in order to improve fault tolerance and make data access faster. Several... Sample PDF
Consistency of Replicated Datasets in Grid Computing
$37.50
Chapter 7
Ming Wu, Xian-He Sun
Rapid advancement of communication technology has changed the landscape of computing. New models of computing, such as business-on-demand, Web... Sample PDF
Quality of Service of Grid Computing
$37.50
Chapter 8
QoS in Grid Computing  (pages 75-83)
Zhihui Du, Zhili Cheng, Xiaoying Wang, Chuang Lin
This chapter first summarizes popular terms of QoS related concepts and technologies in grid computing, including SLA, End-to-End QoS Provision and... Sample PDF
QoS in Grid Computing
$37.50
Chapter 9
Kris Bubendorfer, Ben Palmer, Ian Welch
A Grid resource broker is the arbiter for access to a Grid’s computational resources and therefore its performance and functionality has a... Sample PDF
Trust and Privacy in Grid Resource Auctions
$37.50
Chapter 10
Sandro Fiore, Alessandro Negro, Salvatore Vadacca, Massimo Cafaro, Giovanni Aloisio, Roberto Barbera
Grid computing is an emerging and enabling technology allowing organizations to easily share, integrate and manage resources in a distributed... Sample PDF
An Architectural Overview of the GRelC Data Access Service
$37.50
Chapter 11
Man Wang, Zhihui Du, Zhili Cheng
Resource Management System (RMS), which manages the Grid resources and matches the applications’ requests to the proper resources, is one of the... Sample PDF
Adaptive Resource Management in Grid Environment
$37.50
Chapter 12
Vineet R. Khare, Frank Zhigang Wang
The need for a dynamic and scalable expansion of the grid infrastructure and resources and other scalability issues in terms of execution efficiency... Sample PDF
Bio-Inspired Grid Resource Management
$37.50
Chapter 13
Yuhui Deng, Frank Zhigang Wang, Na Helian
Storage Grid is a new model for deploying and managing the heterogeneous, dynamic, large-scale, and geographically distributed storage resources.... Sample PDF
Service Oriented Storage System Grid
$37.50
Chapter 14
Dominic Cherry, Maozhen Li, Man Qi
This chapter presents MediaGrid, a distributed storage system for archiving broadcast media contents. MediaGrid utilizes storage resources donated... Sample PDF
A Distributed Storage System for Archiving Broadcast Media Content
$37.50
Chapter 15
Maozhen Li, Man Qi, Bin Yu
The computational grid is rapidly evolving into a service-oriented computing infrastructure that facilitates resource sharing and large-scale... Sample PDF
Service Discovery with Rough Sets
$37.50
Chapter 16
Irfan Habib, Ashiq Anjum, Richard McClatchey
Due to some barriers to adoption we have not seen a proliferation of Grid Computing technologies throughout e-Science or other domains. This chapter... Sample PDF
On the Pervasive Adoption of Grid Technologies: A Grid Operating System
$37.50
Chapter 17
Kurt Vanmechelen, Jan Broeckhove, Wim Depoorter, Khalid Abdelkader
As grid computing technology moves further up the adoption curve, the issues of dealing with conflicting user requirements formulated by different... Sample PDF
Pricing Computational Resources in Grid Economies
$37.50
Chapter 18
Rosario M. Piro
Large, geographically distributed and heterogeneous computing infrastructures, such as the Grid, often span multiple organizations and... Sample PDF
Resource Usage Accounting in Grid Computing
$37.50
Chapter 19
Frans Arickx, Jan Broeckhove, Peter Hellinckx, David Dewolfs, Kurt Vanmechelen
Quantum structure or scattering calculations often belong to a class of computational problems involving the aggregation of a set of matrices... Sample PDF
Grid-Based Nuclear Physics Applications
$37.50
Chapter 20
Gabriel Aparicio, Fernando Blanco, Ignacio Blanquer, César Bonavides, Juan Luis Chaves, Miguel Embid, Álvaro Hernández
In the last years an increasing demand for Grid Infrastructures has resulted in several international collaborations. This is the case of the EELA... Sample PDF
Developing Biomedical Applications in the Framework of EELA
$37.50
Chapter 21
Gerald Schaefer, Roger Tait
Efficient approaches to computationally intensive image processing tasks are currently highly sought after. In this chapter, the authors show how a... Sample PDF
Distributed Image Processing on a Blackboard System
$37.50
Chapter 22
Daniele Andreotti, Armando Fella, Eleonora Luppi
The BaBar experiment uses data since 1999 in examining the violation of charge and parity (CP) symmetry in the field of high energy physics. This... Sample PDF
Simulated Events Production on the Grid for the BaBar Experiment
$37.50
Chapter 23
Diego Liberati
A framework is proposed that creates, uses, and communicates information, whose organizational dynamics allows performing a distributed cooperative... Sample PDF
A Framework for Semantic Grid in E-Science
$37.50
Chapter 24
Roberto Barbera, Valeria Ardizzone, Leandro Ciuffo
The Grid INFN virtual Laboratory for Dissemination Activities (GILDA) is a fully working Grid test-bed devoted to training and dissemination... Sample PDF
Grid INFN Virtual Laboratory for Dissemination Activities (GILDA)
$37.50
Chapter 25
Dirk Gorissen, Tom Dhaene, Piet Demeester, Jan Broeckhove
The simulation and optimization of complex systems is a very time consuming and computationally intensive task. Therefore, global surrogate modeling... Sample PDF
Grid Enabled Surrogate Modeling
$37.50
Chapter 26
Patrik Skogster
Grid computing is becoming as essential part of different business analysis. In traditional business computing infrastructures data transfer occurs... Sample PDF
GIS Grids and the Business Use of GIS Data
$37.50
Chapter 27
Gokop Goteng, Ashutosh Tiwari, Rajkumar Roy
The emerging grid technology provides a secured platform for multidisciplinary experts in the security intelligence profession to collaborate and... Sample PDF
Grid Computing: Combating Global Terrorism with the World Wide Grid
$37.50
Chapter 28
Salvatore Scifo
This chapter focuses on the efforts to design and develop a standard pure Java API to access the metadata service of the EGEE Grid middleware, and... Sample PDF
Accessing Grid Metadata through a Web Interface
$37.50
Chapter 29
Jyotsna Sharma
Efforts in Grid Computing, both in academia and industry, continue to grow rapidly worldwide for research, scientific and commercial purposes.... Sample PDF
Grid Computing Initiatives in India
$37.50
Chapter 30
Hai Jin, Li Qi, Jie Dai, Yaqin Luo
A grid system is usually composed of thousands of nodes which are broadly distributed in different virtual organizations. Owing to geographical... Sample PDF
Dynamic Maintenance in ChinaGrid Support Platform
$37.50
About the Contributors