Improved SfM-Based Indoor Localization with Occlusion Removal

Improved SfM-Based Indoor Localization with Occlusion Removal

Yushi Li (The Hongkong Polytechnic University, Hong Kong), George Baciu (The Hong Kong Polytechnic University, Hong Kong), Yu Han (The Hong Kong Polytechnic University, Hong Kong) and Chenhui Li (The Hong Kong Polytechnic University, Hong Kong)
DOI: 10.4018/IJSSCI.2018070102

Abstract

This article describes a novel 3D image-based indoor localization system integrated with an improved SfM (structure from motion) approach and an obstacle removal component. In contrast with existing state-of-the-art localization techniques focusing on static outdoor or indoor environments, the adverse effects, generated by moving obstacles in busy indoor spaces, are considered in this work. In particular, the problem of occlusion removal is converted into a separation problem of moving foreground and static background. A low-rank and sparse matrix decomposition approach is used to solve this problem efficiently. Moreover, a SfM with RT (re-triangulation) is adopted in order to handle the drifting problem of incremental SfM method in indoor scene reconstruction. To evaluate the performance of the system, three data sets and the corresponding query sets are established to simulate different states of the indoor environment. Quantitative experimental results demonstrate that both query registration rate and localization accuracy increase significantly after integrating the authors' improvements.
Article Preview
Top

Introduction

Computing accurate camera pose including position and orientation with respect to the reconstructed structure of a scene has a broad range of applications in both outdoor and indoor environments such as autonomous navigation, objects tracking, AR (augmented reality), cognitive path finding, multi-agent coordination, critical event management (accidents, fire, traffic congestions, etc.). Most of current localization systems running on mobile phones rely on radio signals like Wi-Fi and GPS. However, the availability of these signals depends upon scene scale and site conditions. To be specific, GPS signal is only feasible in outdoor environments, while the accuracy of Wi-Fi-based localization is vulnerable in complicated indoor scenes. Different from the signal-based localization systems, image-based techniques can satisfy the demands of more complex and diverse scenarios. In addition, the tremendous progress in Structure-from-Motion (SfM) techniques enables the camera pose to be estimated accurately with respect to the reconstructed 3D scene.

Most recent 3D-model-based approaches focus on determining position in large-scale outdoor environment. They can provide the camera poses with a desired accuracy. In spite of that, the effectiveness of these systems is affected by two main problems: (1) occlusions in enclosed environments; (2) drift and deformation of the reconstructed model. The prerequisite of 3D camera pose estimation is associating 2D background features detected on an image with 3D points reconstructed from the structure of the background correctly. Any object that moves in an indoor setting can cause occlusion which may result in wrong feature associations. Meanwhile, some essential features belonging to the static background might be lost because they are sheltered by various obstructions. Most importantly, even if all the 2D-to-3D correspondences are correct, the drifting structure of a 3D model would bring about large localization errors.

In order to solve these two problems we propose an obstruction removal method for 3D localization as a problem of separating the foreground from the background. Furthermore, we propose a new synthetic framework combining the state-of-the-art localization technique with refined SfM and an occlusion removing component. The basic structure of our system is similar to the recent work on 3D structure-based localization: establish 2D-to-3D correspondences by relating 2D features detected from a query image to 3D points previously created by SfM, and compute pose of query from the geometric constraints constructed by these 2D-to-3D matches.

Unlike other state-of-the-art localization techniques, our proposed framework seeks to promote the system effectiveness and localization accuracy in a crowded indoor environment. There are two essential assumptions in this paper. (1) By recovering important features occluded by obstacles with deformation it is possible to reduce the sparse “noise” of ambient occlusion. (2) Re-triangulating the failed matches of SIFT features in incremental SfM can decrease the loss of correct feature matches which would result in drifting and deformation of 3D structures. Go Decomposition (GoDec) (Zhou & Tao, 2011), which is a classic low- rank and sparse matrix decomposition algorithm, is adopted in our work to filter out the negative inferences of moving obstructions in multiple frames with static structure. Moreover, a SfM approach with RT (Wu, 2013) is integrated in our system to guarantee the geometric accuracy of reconstructed indoor scene. More importantly, the experimental results illustrate that the localization system proposed by us remains robust in complex indoor environments, especially buildings with moving people and other large moving objects.

Well-studied approaches, such as optical flow estimation (Mémin & Pérez, 1998) and Markov Random Fields (MRF) (Huang et al, 2007), can be applied to accomplish the segmentation of moving pixels as well. However, they cannot tackle the moving object occlusions with large-scale displacement or obscured contour. Furthermore, our own data sets are created to evaluate the performance of our approach in indoor environments with crowds, on account that currently existing data sets built up for location awareness only concentrate on outdoor or static indoor scenes.

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 12: 4 Issues (2020): Forthcoming, Available for Pre-Order
Volume 11: 4 Issues (2019)
Volume 10: 4 Issues (2018)
Volume 9: 4 Issues (2017)
Volume 8: 4 Issues (2016)
Volume 7: 4 Issues (2015)
Volume 6: 4 Issues (2014)
Volume 5: 4 Issues (2013)
Volume 4: 4 Issues (2012)
Volume 3: 4 Issues (2011)
Volume 2: 4 Issues (2010)
Volume 1: 4 Issues (2009)
View Complete Journal Contents Listing