As 3D becomes more ubiquitous with the advent of 3D scanning and display technology, methods of compressing and transmitting 3D data need to be explored. One method of doing such is depth mapping, in which 3D depth data is compressed into a 2D image, and then 2D image processing techniques may be leveraged. This chapter presents a technique of depth mapping 3D scenes into 2D images, entitled Holoimage. In this technique, digital fringe projection, a special kind of structured light technique from optical metrology, is used to encode and decode 3D scenes pixel-by-pixel. Due to the pixel-by-pixel 3D data processing nature, this technique can be used on parallel hardware to realize real-time speed for high definition 3D video encoding and decoding.
TopIntroduction
Advancements in 3D imaging and computational technology have made acquisition and display of 3D data simple. Techniques such as structured light, stereovision, and light detection and ranging (LIDAR) have led the path in 3D data acquisition (Gorthi & Rastogi, 2010). Stereoscopic displays have made the display of 3D data a reality. However, as these fields and techniques evolve, a growing problem is being confronted; how can 3D data be efficiently stored and transmitted?
Storage and transmission of 3D data has become a large problem due to the file sizes associated with 3D geometry. Standard 3D file storage techniques do not lend themselves nicely to high detailed, high frame rate scenes. Instead, traditional 3D file storage techniques aim to store models and then animated models based on constraints of a few points, typically skeletal points (Forstmann et al., 2007). This does not hold true for 3D scenes captured from 3D scanners, as they consist of a large array of 3D coordinates, all of which are animated; this animation is inherently unstructured and unconstrained making typical 3D file storage difficult. Introducing 3D models into 3D scenes only further exacerbates the problem, as both modalities need to be accounted for. How then can such scenes with both types of 3D data be encoded in a unifying way, which provides not only an efficient storage and transmission medium, but also a quick encoding and decoding of high definition (HD) data?
One solution to this problem is to use depth mapping to encode 3D scenes consisting of unstructured scanned data and structured models into 2D images, and then rely on 2D image compression and transmission techniques. The benefit of doing this is that decades of research and development in 2D image processing can be leveraged, utilizing existing compression and transmission techniques along with existing infrastructure. Existing video services such as Youtube and Vimeo can be used with slight modifications; only the video renderer needs to be modified to decode and display 3D scenes rather than 2D images.
Holoimage (Gu et al, 2006) is a technique developed to accomplish the task of depth mapping an entire HD 3D scene. Utilizing techniques developed in optical metrology, Holoimage creates a virtual fringe projection (a special kind of structured light) system which can depth map an entire 3D scene point-by-point into 2D images. The benefits of such a technique include:
- 1.
Using existing research in the field of optical metrology;
- 2.
Leveraging existing research in the field of image processing; and
- 3.
Achieving point-by-point computation though the whole process.
Employing parallel hardware such as that of a graphics processing unit (GPU), HD 3D scenes can be encoded and decoded in real-time. Thus Holoimage meets the requirements of encoding and decoding a 3D scene with little speed hindrance, lending itself nicely to 3D video and other high-speed, high-resolution 3D applications (Zhang & Huang, 2006a; Zhang & Yau, 2006; Zhang & Yau, 2007). This chapter will delve into the details of the Holoimage technique, will show compression results, and will discuss the advantages and shortcomings.