Calculating Absolute Scale and Scale Uncertainty for SfM Using Distance Sensor Measurements: A Lightweight and Flexible Approach

Calculating Absolute Scale and Scale Uncertainty for SfM Using Distance Sensor Measurements: A Lightweight and Flexible Approach

Ivan Nikolov (Aalborg University, Denmark) and Claus B. Madsen (Aalborg University, Denmark)
Copyright: © 2020 |Pages: 25
DOI: 10.4018/978-1-5225-5294-9.ch008

Abstract

Capturing details of objects and surfaces using structure from motion (SfM) 3D reconstruction has become an important part of data gathering in geomapping, medicine, cultural heritage, and the energy and production industries. One inherent problem with SfM, due to its reliance on 2D images, is the ambiguity of the reconstruction's scale. Absolute scale can be calculated by using the data from additional sensors. This chapter demonstrates how distance sensors can be used to calculate the scale of a reconstructed object. In addition, the authors demonstrate that the uncertainty of the calculated scale can be computed and how it depends on the precision of the used sensors. The provided methods are straightforward and easy to integrate into the workflow of commercial SfM solutions.
Chapter Preview
Top

Introduction

Structure from Motion (SfM) techniques have matured throughout the years to become viable commercial solutions for 3D reconstruction. This is due to the techniques’ scalability, relative ease of use and the fact that they do not rely on specialized equipment. This positions SfM as a useful substitute for other reconstruction approaches that require both specialized hardware and software, like structured light (Sarbolandi, 2015), stereo (Sarker, 2017) or time-of-flight cameras (Corti, 2016), when real-time performance is not necessary.

The algorithm pipeline for SfM is extensively documented by (Özyeşil, 2017) and the accuracy of different solutions for varying use cases are discussed by (Nikolov I. A., 2016), (Knapitsch, 2017) . There are several approaches to performing SfM reconstruction, but a typical algorithm takes 2D images looking at the reconstructed object or surface, from different positions and directions. Another important feature of SfM is the possibility to use it both with images from precisely calibrated capturing setups (Martell, 2018), as well as with in the wild image datasets (Makantasis, 2016), requiring more post-processing in filtering the image data and clustering it, but saving on long capturing times.

In the SfM processing pipeline, a number of feature points are extracted from each image and matched with features from the input images. These feature matches are filtered and together with the intrinsic parameters of the cameras are used in a bundle adjustment algorithm to triangulate the camera positions in 3D space, as well as a sparse point cloud. A depth map and dense point cloud are then computed. Finally, if needed, the dense point cloud is meshed and a texture is calculated from the images. One drawback of using only uncalibrated 2D images as input is that the scale of the reconstruction is ambiguous. To calculate the absolute scale, additional information is needed. This information can be captured manually, by using objects of known sizes in the images or by using additional sensors.

This chapter focuses on using additional sensors for calculating the absolute scale of the 3D reconstruction. It demonstrates a step-by-step solution which uses external distance sensors to provide the necessary information. In addition, the authors take into consideration that real-world sensors’ readings contain level of uncertainty, which in turn is transferred to the calculated scale. The discussed solutions take this into consideration and demonstrate that these uncertainties can be quantified.

The chapter’s contributions to the field of SfM can be summarized as:

  • A lightweight and easy to implement method for finding the absolute scale of a SfM reconstruction using distance sensors;

  • The method is easy to integrate into existing commercial SfM solutions, as it requires only simple outputs, such as a 3D mesh and camera positions and orientations;

  • The method is flexible and can be used both with expensive LiDAR solutions, as well as cheap distance measurement sensors, as it does not require capturing the object surface;

  • The uncertainty of the computed absolute scale can be calculated, when high precision is required.

Key Terms in this Chapter

Value Uncertainty: The variance in the measured or calculated values, which can be cause by noise or random fluctuations.

LiDAR: Light detection and sensing. A type of distance sensor, which uses laser light pulses to measure distances in one or many directions.

Absolute Scale: Scale which is equal to real life measurements in a known measurement system like mm, cm, m, inches, etc.

Distance Measurement Sensor: Sensors used to measure real-life distances. They can use different hardware to do it – lasers, ultrasound, etc. The number of distance measurements can also vary from one to many.

Color Images: Images containing three color channels, represented in a certain way, for example RGB—red, green, blue—or HSI—hue, saturation, intensity.

Structure From Motion: A computer vision based 3D reconstruction technique, using a number of unordered color images, to capture both the 3D shape, as well as the color of an object or surface.

3D Reconstruction: The process of capturing the 3D shape of an object or parts of an object, in contract to images, which capture a 2D representation of an object from a certain direction.

Complete Chapter List

Search this Book:
Reset