3D Thumbnails for 3D Videos with Depth

Introduction

Today, the popularity of 3D media usage in computerized environment and the research on 3D content generation is increasing. 3D contents are frequently used in various applications such as computer games, movies and even in home environmental systems and this reputation leads visualization of 3D contents such as 3D videos and 3D images becoming more significant. For visualizing the 3D contents, thumbnail representation is used in order to provide a quick overview of multimedia files in order to allow a quick scanning over a large number of data. By using the traditional methods, the thumbnail generally shows the first frame of the video and for images, visual representation is generated by using shrinking, manual cropping or uniform scaling. However, these approaches do not preserve the important parts of the multimedia files and resulting thumbnails do not give the general idea of the content. Furthermore, in spite of the existence of 3D content databases, there is no standardization on the thumbnail representation for 3D contents while their usage area is widespread. Thus, the thumbnail representation is very crucial to get a quick overview of the content rather than downloading from the database and processing it.

Therefore, we propose a thumbnail generation system that creates meaningful, illustrative visual representations of 3D video with depth contents without losing perceivable elements in the selected video frame by using saliency-depth and layer based methodologies. Moreover, in order to represent the 3D contents realistically and enhance depth perception, the resulting thumbnail should be in 3D. Thus, the framework constructs geometries of important objects as polygon meshes and adds 3D effects such as shadow and parallax mapping. Figure 1 illustrates a layout that holds resultant 3D thumbnails for 3D videos with depth.

Figure 1.

3D thumbnails on a 3D grid layout

While creating 3D thumbnails, it is required to select suitable 3D video formats since compression and coding algorithms of 3D videos show diversity according to the varieties of 3D displays: classical two-view stereo video (CSV), video plus depth (V+D), layered depth video (LDV) and multi-view video plus depth (MDV). Some of these formats and coding algorithms are standardized by MPEG, since standard formats and efficient compression are crucial for the success of 3D video applications (F.Institute, 2008). For our framework, V+D and LDV formats are eligible because of simplicity and the depth information they provide. V+D format provides a color video and an associated depth map that stands for geometry-enhanced information of the 3D scene. The color video is original video itself and the depth map is a monochromatic, luminance-only video. Besides, LDV is an extension of V+D format. It contains all information that V+D satisfies with an extra layer called background layer which includes foreground objects and the associated depth map of the background layer. By using the properties of V+D and LDV videos, we develop two different thumbnail generation methods based on the information they present. These proposed methodologies create meaningful thumbnails without losing perceivable visual elements in the selected original video frame.

In this chapter, the previous work on 3D video formats and thumbnail generation methods, the proposed framework that generates 3D thumbnails from video plus depth (V+D) and layered depth video (LDV), two 3D thumbnail generation methodologies based on 3D meshes and parallax mapping, and several experiments showing effectiveness and recognizability of 3D thumbnails, are presented.

Background

We discuss 3D video formats and thumbnail generation approaches under two different subsections since our approach combines them.