Abstract
Communication in large scale distributed systems has a major impact on the overall performance and widely acceptance of such systems. In this chapter we analyze existing work in enabling high-performance communications in large scale distributed systems, presenting specific problems and existing solutions, as well as several future trends. Because applications running in Grids, P2Ps and other types of large scale distributed systems have specific communication requirements, we present different the problem of delivering efficient communication in case of P2P and Grid systems. We present existing work in enabling high-speed networks to support research worldwide, together with problems related to traffic engineering, QoS assurance, protocols designed to overcome current limitation with the TCP protocol in the context of high bandwidth traffic. We next analyze several group communication models, based on hybrid multicast delivery frameworks, path diversity, multicast trees, and distributed communication. Finally, we analyze data communication solutions specifically designed for P2P and Grid systems.
TopThe current Internet architecture was designed as a public communication environment for everybody. The Internet by itself is a resource whose fair share is controlled by the used protocols. Because of this, most of the currently available data transfer tools are unaware of the network. They rely on the underlying network protocols to tune their data flow in an acceptable manner for everybody using the network in that moment. Because of this, the Internet architecture is limited in its ability to support, for example, Grid computing applications. Packet switching is a proven efficient technology for transporting burst transmission of short data packets (e.g., for remote login, consumer oriented email and web applications). Making forwarding decisions every 1500 bytes is sufficient for emails or 10-100 kB web pages. However, this is not an optimal mechanism if we are to cope with data sizes of six to nine orders larger in magnitude. For example, copying with 1.5 Terabytes of data using packet switching requires making the same forwarding decision about 1 billion times, over many routers along the path. Setting circuit or burst switching over optical links is a more effective multiplexing technique.
Over the last few years, optical component technology has rapidly evolved to support multiplexing and amplification of increasing digital modulation rates. Currently, 10 Gigabit Ethernet (GbE) network interfaces are in widespread use, and the costs of these interfaces, as well as 10 GbE ports on switches have dropped. However, because the cost for replacing the current equipment is not justified, given the state of the telecommunications market, for long connections (typically transoceanic connections) SONET optical data transmission technology will remain for several years (SONET, 2006). The cost of the existing 10Gbps network infrastructures has not yet been fully amortized, which makes the Telecom Operators to head towards the so called “next generation SONET/synchronous digital hierarchy (SDH)”. This relies on (SONET, 2007):