A Failure Detection System for Large Scale Distributed Systems

A Failure Detection System for Large Scale Distributed Systems

Andrei Lavinia (University Politehnica of Bucharest, Romania), Ciprian Dobre (University Politehnica of Bucharest, Romania), Florin Pop (University Politehnica of Bucharest, Romania) and Valentin Cristea (University Politehnica of Bucharest, Romania)
Copyright: © 2011 |Pages: 24
DOI: 10.4018/jdst.2011070105

Abstract

Failure detection is a fundamental building block for ensuring fault tolerance in large scale distributed systems. It is also a difficult problem. Resources under heavy loads can be mistaken as being failed. The failure of a network link can be detected by the lack of a response, but this also occurs when a computational resource fails. Although progress has been made, no existing approach provides a system that covers all essential aspects related to a distributed environment. This paper presents a failure detection system based on adaptive, decentralized failure detectors. The system is developed as an independent substrate, working asynchronously and independent of the application flow. It uses a hierarchical protocol, creating a clustering mechanism that ensures a dynamic configuration and traffic optimization. It also uses a gossip strategy for failure detection at local levels to minimize detection time and remove wrong suspicions. Results show that the system scales with the number of monitored resources, while still considering the QoS requirements of both applications and resources.
Article Preview
Top

Fault tolerance in LSDS is based on one form of a failure detection system. Such a failure detector is generally capable of running a detection algorithm and it can communicate with other services which it monitors. This model was first proposed, in the form of an “oracle” detection service, by Chandra and Toueg (1996).

The failure detection module is independent to the main application flow and is being responsible with the monitoring of a subset of the processes within the monitored system and maintaining a list of those it currently suspects to have crashed. A process can query its local failure detector module at any time to check its status. The list of suspected processes is permanently updated such that, at any time, new processes can be added and old ones removed. This failure detector is considered unreliable because it is allowed to make mistakes up to a certain degree. Therefore a module might erroneously suspect some correct process (wrong suspicion) or fail to detect processes that have already crashed. At any given time two failure detector modules may have different lists of processes.

Complete Article List

Search this Journal:
Reset
Open Access Articles
Volume 12: 4 Issues (2021): Forthcoming, Available for Pre-Order
Volume 11: 4 Issues (2020): 2 Released, 2 Forthcoming
Volume 10: 4 Issues (2019)
Volume 9: 4 Issues (2018)
Volume 8: 4 Issues (2017)
Volume 7: 4 Issues (2016)
Volume 6: 4 Issues (2015)
Volume 5: 4 Issues (2014)
Volume 4: 4 Issues (2013)
Volume 3: 4 Issues (2012)
Volume 2: 4 Issues (2011)
Volume 1: 4 Issues (2010)
View Complete Journal Contents Listing