3D Reconstruction Challenges Using Structure-From-Motion

Introduction

Over the last decades the number of available 3D digital replicas is growing and constitute an integral part of many research domains including cultural heritage, Archaeology, Architecture, Medicine, etc. The key role for this fact is the progress achieved in 3D digitization systems coupled with the ongoing development of real-time 3D graphics. 3D digital replicas can be created through the usage of various methodologies ranging from multi-image photogrammetry (Structure-from-Motion -SfM/Multiple-View-Stereovision – MVS), Laser Triangulation to Structured Light. For the multi-image photogrammetry case, the existence of low-cost software solutions or open source libraries that implement the SfM/MVS pipeline, in combination with the minimum user intervention needs, enable their use as a black box solution. Moreover, its broad implementation relies also on the cost effectiveness in terms of hardware equipment, man-hours spent during the data collection phase and its relatively low background knowledge requirements. Additionally, as far as the generated 3D digital replica is concerned, its quality and accuracy in terms of geometry and texturing satisfies a vast range of 3D digitization projects (Koutsoudis et al., 2014).

Within the cultural heritage domain 3D reconstruction is considered as a common practice (Koutsoudis et al., 2012), an application domain that is very demanding in terms of geometrical and colour accuracy. 3D digital replicas can be used for dissemination and documentation purposes and constitute a valuable tool as well for archaeologists and conservators to conduct their studies. Europeana, the European commission’s digital platform for Cultural Heritage (EUROPEANA), recognising the value of 3D digital replicas hosts a significant number of 3D items, ranging from small objects (e.g. a vase) to large scale objects (e.g. an architectural). Numerous European and national research and development projects exploiting the SfM approach (CARRARE; 3D-ICONS; 3D-COFORM), as well as independent initiatives have been conducted to enrich the digital libraries with high geometric accuracy 3D content.

Figure 1.

Structure-from-Motion overview

The main scope of this chapter is to indicate challenges that the SfM/MVS approach is facing and subsequently to present significant published scientific works that confront and offer ways to handle the different limitations and drawbacks of the traditional SfM/MVS approach. Additionally, this work attempts to provide a framework to conduct SfM/MVS even in cases that normally are either unable to produce any 3D digital replica or the outcome fails to meet the requirements in terms of geometric accuracy.

Structure-From-Motion/Multi-View Stereo Overview

The SfM-MVS pipeline involves distinct phases such as image sequence collection, feature points extraction, matching of corresponding feature points among different images in overlapping areas and calculation of the extrinsic and intrinsic parameters of the camera (Cipolla, 2008). To improve the camera trajectory and thus minimize the projection error while preventing the error-built up of the camera tracking, bundle adjustment is implemented (Engels et al. 2006). SfM/MVS utilizes a set of unordered images, depicting a static scene or an object captured from arbitrary viewpoints that have a strong overlap with each other, to recover the camera parameters, to produce a sparse point cloud of the 3D scene followed by a dense point cloud which will ultimately be converted into a polygonal mesh (Figure 1 and 2).

Figure 2.

SfM pipeline overview. Left: data collection, Middle: Image sequences and Right: 3D object