High-Performance Solutions for Adaptive and Customizable Streaming of Interactive Content to Mobile Devices

Introduction

During the first decade of this millennium, mobile devices have pervaded and changed the way of life of many people. While many types of devices, from cellular phones to laptops, may be considered mobile, we are concerned specifically with portable handheld devices, particularly focusing on the latest generation, which includes smart phones, Personal Digital Assistants (PDAs), Ultra-Mobile PCs, and e-Readers.

One of the key factors behind the success of latest generation of mobile networks is the availability of useful, attractive, and low cost services for the end-user. Hardware improvements have led to design of increasingly more powerful handheld devices and to the integration of digital imaging and wireless networking technologies, therefore enhancing the range of applications that they can support. At the same time, technological advances in wireless communications now make ubiquitous access to the network possible, allowing more users to complete their tasks while on the move. Moreover, increasingly capacious and fast wireless networks allow researchers to design and implement a wide range of multimedia services specifically tailored for the mobility scenario. For these reasons, mobile devices are the key factors for the growth of pervasive computing. Indeed, the number of handheld devices sold (mainly smart phones) is continuously increasing (Elkin, 2010), at a rate much faster than that of desktop systems, as people use handheld devices primarily for data communication and internet connections.

On the other hand, although today’s mobile devices are equipped with hardware resources that were unthinkable until a few years ago in terms of their computational power, storage, and network connectivity; they still present some limitations that prevent people from using them for intensive processing tasks, such as computer graphics applications. To overcome this issue, several applications are based on the remote visualization paradigm, where streaming based techniques are used to visualize remote multimedia content on user devices, thus moving the need for performance from client (mobile) devices to specialized, remote hardware machines.

Streaming applications are widely adopted today; they are subjected to demanding requirements and are expected to perform as so called “killer applications” on future wireless networks because of their high bandwidth usage. In particular, video streaming applications have been successful thanks to the number of higher bandwidths available after the improvement of wireless networks.

The delivery of video content may be realized in two ways: progressive download and real-time streaming. The progressive download method is based on a direct download onto the storage unit of a client from a server; it can be used whenever the applications do not require real-time visualization, and the length of the content is short. On the other hand, real-time streaming is used whenever the content must be consumed by the client device as soon as it has been received. It is pertinent to note that “real-time” in this context does not necessarily mean that the application is mission critical, and the total correctness of an operation depends on the time in which it is performed. Rather, traditional streaming systems can tolerate some delays and may result in decreased Quality of Service (QoS) while maintaining an active service. For example, during a video streaming session, some frames can be omitted while displaying a video because of network congestion (i.e., the packets that encapsulate an image frame might follow a different path and arrive after some packets from a frame sent later) or limited computational capabilities on the client device (i.e., image frames might be delivered faster than the device is capable of processing). For this reason, streaming systems belong to the so-called “soft real-time systems”.