Spatio-Temporal Denoising for Depth Map Sequences

Spatio-Temporal Denoising for Depth Map Sequences

Thomas Hach (Arnold & Richter Cinetechnik (ARRI), Munich, Germany) and Tamara Seybold (Arnold & Richter Cinetechnik (ARRI), Munich, Germany)
DOI: 10.4018/IJMDEM.2016040102


This paper proposes a novel strategy for depth video denoising in RGBD camera systems. Depth map sequences obtained by state-of-the-art Time-of-Flight sensors suffer from high temporal noise. Hence, all high-level RGB video renderings based on the accompanied depth maps' 3D geometry like augmented reality applications will have severe temporal flickering artifacts. The authors approached this limitation by decoupling depth map upscaling from the temporal denoising step. Thereby, denoising is processed on raw pixels including uncorrelated pixel-wise noise distributions. The authors' denoising methodology utilizes joint sparse 3D transform-domain collaborative filtering. Therein, they extract RGB texture information to yield a more stable and accurate highly sparse 3D depth block representation for the consecutive shrinkage operation. They show the effectiveness of our method on real RGBD camera data and on a publicly available synthetic data set. The evaluation reveals that the authors' method is superior to state-of-the-art methods. Their method delivers flicker-free depth video streams for future applications.
Article Preview

2. Prior Art

Depth map denoising is actively investigated due to the common noise issues of today’s depth sensors. There are numerous methods based on spatial and most often edge-aware denoising or combined spatial upscaling and denoising (Chan, Buisman, Theobalt, & Thrun, 2008; Diebel & Thrun, 2005; Ferstl, Reinbacher, & Ranftl, 2013; He, Sun, & Tang, 2015; Kopf et al., 2007; Y. Li, Xue, Sun, & Liu, 2012; Liu, Tuzel, & Taguchi, 2013; Ma, He, Wei, Sun, & Wu, 2013; Park, Kim, Tai, Brown, & Kweon, 2011; Soh, Sim, Kim, & Lee, 2012; J. Yang, Ye, Li, Hou, & Wang, 2015; Yang et al., 2013; Yang, Yang, Davis, & Nister, 2007).

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 10: 4 Issues (2019): Forthcoming, Available for Pre-Order
Volume 9: 4 Issues (2018): 3 Released, 1 Forthcoming
Volume 8: 4 Issues (2017)
Volume 7: 4 Issues (2016)
Volume 6: 4 Issues (2015)
Volume 5: 4 Issues (2014)
Volume 4: 4 Issues (2013)
Volume 3: 4 Issues (2012)
Volume 2: 4 Issues (2011)
Volume 1: 4 Issues (2010)
View Complete Journal Contents Listing