A Simulation Model for Large Scale Distributed Systems

Introduction

Large-scale grids and other federations of distributed resources that aggregate and share resources over wide-area networks present major new challenges for scientists. In this chapter we focus on the challenge to enable scalable, high-level, online simulation of applications, middleware, resources and networks to support scientific and systematic study of Large Scale Distributed Systems (LSDSs). The field of modelling and simulation was long-time seen as a viable alternative to develop new algorithms and technologies. It enables the development of LSDS when analytical validations are prohibited by the nature of the encountered problems.

In this chapter we describe the approach used to design and implement a generic simulator for distributed systems. We present the design characteristics of a model for the simulation of such systems that integrates components and mechanisms that enable realistic simulation experiments for LSDS. The model incorporates the necessary components and characteristics that allow the complete and accurate design of realistic simulation experiments of complex Grid architectures, consisting of many resources and various technologies, ranging from data transferring to scheduling and data replication, with resources working together to provide a common set of characteristics. The proposed simulation model includes the components to describe various actual distributed system technologies, and provides the mechanisms to describe concurrent network traffic, evaluate different strategies in data replication, and analyze job scheduling procedures.

The model is part of a multithreaded, process oriented simulator for LSDS called MONARC 2. The first version of such a generic simulator for LSDS was developed in the late ’90s (Dobre & Stratan, 2003). It started with the LHC physics experiments that needed data processing and storage capacities beyond what was available at that time. The size of the LHC experiments and the unprecedented scale of data resulted in the need to look at resources outside of European Organization for Nuclear Research (CERN). This context led to the proposal of the hierarchical distribution model (the Tier architecture), according to which facilities from all around the world are putting together resources in order to provide the necessary computing power and data storage space needed for the experiments (Legrand, et al., 2003). In many ways this architecture was the predecessor of modern LSDS. Still, because the architecture represented a novel approach, and to eliminate further suspicions that the model might not in fact deliver the envisioned processing and storing capacities, a mix team of researchers from CERN and California Institute of Technology designed and developed the first version of a generic simulator, called MOdels of Networked Analysis at Regional Centers (MONARC). It was actually used for the first validation experiments of the model that was later used in the processing of the data actual generated in the LHC experiments (Dobre & Cristea, 2007).