In this chapter, an online method is developed for estimating 3D structure (with proper scale) of moving objects seen by a moving camera. In contrast to traditionally developed batch solutions for this problem, a nonlinear unknown input observer strategy is used where the object’s velocity is considered as an unknown input to the perspective dynamical system. The estimator is exponentially stable, and hence, provides robustness against modeling uncertainties and measurement noise from the camera. The developed method provides first causal, observer based structure estimation algorithm for a moving camera viewing a moving object with unknown time-varying object velocities.
TopBackground
Recovering the structure of a stationary object viewed by a moving camera is called structure from motion (SfM). A number of solutions to the SfM problem exist in the form of batch or offline methods (Kahl & Hartley, 2008; Oliensis, 2000; Strum & Triggs, 1996) and causal or online methods (Dahl, Nyberg, & Heyden, 2007; Dani, Fischer, Dixon, 2012; Dani, Kan, Fischer, Dixon, 2012; Dani & Dixon, 2010; Dani, Rifai, & Dixon, 2010; Dixon, Fang, Dawson, & Flynn, 2003; Jankovic, Ghosh, 1995). The fundamental concept behind SfM algorithms is triangulation. Since the object being observed is stationary, two rays projected onto consecutive images and the camera baseline form a triangle. If the object is not stationary, then the projections of the object will be from different locations in the fixed inertial frame; hence, triangulation is not feasible and standard SfM techniques cannot be used to recover the structure of a moving object using a moving camera (Avidan & Shashua, 2000).
This chapter presents a result for the case when the viewed object is in motion. We term this problem (SaMfM) estimation. In the pioneering work by Avidan and Shashua (2000) developed an offline method, termed trajectory triangulation, to recover the structure of a moving object using a moving camera. In Avidan and Shashua (2000), a batch algorithm is applied for points moving in straight lines or conic trajectories given five or nine views, respectively. In Kaminski and Teicher (2004), a batch algorithm is presented for object motions represented by more general curves. In Han and Kanade (2004), a factorization-based batch algorithm is proposed where objects are assumed to be moving with constant speed in a straight line, observed by a weak perspective camera. An algebraic geometry approach is presented in Vidal, Ma, Soatto, and Sastry (2006) to estimate the motion of objects up to a scale given a minimum number of point correspondences. Yuan and Medioni (2006) developed an approximate batch algorithm to estimate the structure and motion of objects by assuming that one of the feature points of the moving object lies on the static background. In Park, Shiratori, Matthews, and Sheikh (2010), a batch algorithm is designed which requires approximation of the trajectories of a moving object using a linear combination of discrete cosine transform (DCT) basis vectors.