Isochronous Distributed Multimedia Synchronization

Introduction

A multimedia system is characterized by the integrated computer-controlled generation, manipulation, presentation, storage, and communication of independent discrete and continuous media data. The presentation of any data and the synchronization between various kinds of media data are the key issues for this integration (Georganas, Steinmetz, & Nakagawa, 1996). Clearly, multimedia systems have to precisely coordinate the relationships among all media that include temporal and spatial relationships. Temporal relationships are the presentation schedule of media, and spatial relationships are the location arrangements of media. Multimedia synchronization is a process of maintaining these relationships by employing appropriate synchronization mechanisms and algorithms. Multimedia synchronization is traditionally challenging, especially in distributed environments.

Three types of multimedia synchronization can be distinguished: intrastream synchronization, interstream synchronization, and intermedia synchronization (Crowcroft, Handley, & Wakeman, 1999). The approaches used for interstream synchronization can also be used for intermedia synchronization.

The word synchronization refers to time. The easiest way of synchronizing between streams at different sites is to use a single time reference. There are several ways to provide this time reference.

• •

  The network will have a clock serve as a single reference. This approach is used in H.261/ISDN- (integrated services digital network) based systems. A single clock time is propagated around a set of codecs and multipoint control units (MCSs).

• •

  The network deploys a clock-synchronization protocol, such as NTP (the network time protocol; Mills, 1993). The time stamps of media packets will be derived from the globally synchronized clocks. The isochronous synchronization approach as described in this article heavily relies on this time reference.

An Isochronous Synchronization Approach

The isochronous synchronization approach employs a clock-driven protocol for achieving multimedia synchronization (any one of three types of synchronization; Yang, Gay, Sun, Siew, & Sattar, 2002). This approach is particularly suitable for distributed collaborative multimedia environments where many-to-many multimedia communication is the basic interaction pattern. In this approach, multimedia synchronization is based on the use of synchronized physical clock time instead of any form of logical clock or sequence numbers, and thus clock synchronization across the distributed system is assumed. A real-time (synchronized) clock is incorporated in the system as a mechanism used for initiating significant events (actions) as a function of real time.

With globally synchronized clocks that satisfy the granularity condition, we can construct an action lattice (or event lattice; Kopetz, 1992). One dimension of this lattice represents the progression of time, the other dimension is the processes in the system (Figure 1). Processes in the system are designed to execute a simple clock-driven protocol, which requires that the events of sending and receiving messages are restricted to only occur at the lattice point of the globally synchronized space-time lattice (Figure 1). Thus, whenever an action has to be taken, it has to be delayed until the next lattice point of the event lattice.

Figure 1.

Lattice structure

This lattice structure greatly simplifies multimedia synchronization and readily maintains the temporal and causal relationship among the media.

The idea behind the clock-driven, isochronous synchronization is very simple and intuitive in that the easiest way to synchronize processes is to get them all to do the same thing at the same time. Using the simple mechanism based on the synchronized clock without requiring complex algorithms, the approach can equally well be applied to various multimedia applications in distributed environments, including live multimedia applications (live teleconferencing and CSCW) and stored media applications.