Calculating Absolute Scale and Scale Uncertainty for SfM Using Distance Sensor Measurements: A Lightweight and Flexible Approach

Introduction

Structure from Motion (SfM) techniques have matured throughout the years to become viable commercial solutions for 3D reconstruction. This is due to the techniques’ scalability, relative ease of use and the fact that they do not rely on specialized equipment. This positions SfM as a useful substitute for other reconstruction approaches that require both specialized hardware and software, like structured light (Sarbolandi, 2015), stereo (Sarker, 2017) or time-of-flight cameras (Corti, 2016), when real-time performance is not necessary.

The algorithm pipeline for SfM is extensively documented by (Özyeşil, 2017) and the accuracy of different solutions for varying use cases are discussed by (Nikolov I. A., 2016), (Knapitsch, 2017) . There are several approaches to performing SfM reconstruction, but a typical algorithm takes 2D images looking at the reconstructed object or surface, from different positions and directions. Another important feature of SfM is the possibility to use it both with images from precisely calibrated capturing setups (Martell, 2018), as well as with in the wild image datasets (Makantasis, 2016), requiring more post-processing in filtering the image data and clustering it, but saving on long capturing times.

In the SfM processing pipeline, a number of feature points are extracted from each image and matched with features from the input images. These feature matches are filtered and together with the intrinsic parameters of the cameras are used in a bundle adjustment algorithm to triangulate the camera positions in 3D space, as well as a sparse point cloud. A depth map and dense point cloud are then computed. Finally, if needed, the dense point cloud is meshed and a texture is calculated from the images. One drawback of using only uncalibrated 2D images as input is that the scale of the reconstruction is ambiguous. To calculate the absolute scale, additional information is needed. This information can be captured manually, by using objects of known sizes in the images or by using additional sensors.

This chapter focuses on using additional sensors for calculating the absolute scale of the 3D reconstruction. It demonstrates a step-by-step solution which uses external distance sensors to provide the necessary information. In addition, the authors take into consideration that real-world sensors’ readings contain level of uncertainty, which in turn is transferred to the calculated scale. The discussed solutions take this into consideration and demonstrate that these uncertainties can be quantified.

The chapter’s contributions to the field of SfM can be summarized as:

• •

  A lightweight and easy to implement method for finding the absolute scale of a SfM reconstruction using distance sensors;

• •

  The method is easy to integrate into existing commercial SfM solutions, as it requires only simple outputs, such as a 3D mesh and camera positions and orientations;

• •

  The method is flexible and can be used both with expensive LiDAR solutions, as well as cheap distance measurement sensors, as it does not require capturing the object surface;

• •

  The uncertainty of the computed absolute scale can be calculated, when high precision is required.