The objective of this chapter is to discuss the advantages of introducing computer vision based image processing to positioning for improved accuracy, availability, and reliability. Various methods needed to accomplish this type of visual positioning are introduced. Although visual methods in positioning have been researched for years, mainly related to automatic navigation of robots and unmanned vehicles, there are very few existing solutions targeted for pedestrian positioning, especially with smartphones as the application platform. The solutions invented so far are introduced in the chapter as well as the challenges of integrating visual positioning into a smartphone. Visual information is, however, a valuable source of information to replace or assist satellite, wireless, cellular network, and inertial positioning, especially in challenging signal environments, because images are independent from other positioning observations and the camera is typically affected by different error sources than other sensors.
What is Visual Positioning?
Visual positioning means the usage of information obtained from images to resolve the user position or ease the resolving of the user position integrating the visual information with position measurements obtained with other methods. In the latter case, the usage of visual information is called visual-aiding. The visual information may also be used to make the position solution more accurate or more widely available. The visual data may be used to obtain either an absolute position of the user by utilizing reference images in a database or a relative position, location change, by examining consecutive images. Though the viewpoint of these two approaches is quite different, some methods are the same. Both cases share the processing steps of finding correspondences from the images, in the former case between the reference image and the one taken by the user (also called a query image) and in the latter case between consecutive images. The required tasks are preparation of the camera and the positioning frame, extracting features from the image to be used in the matching, and the actual matching. The methods and techniques needed for performing these functions, together with some representative test results, are introduced in this chapter. The user positions are solved differently in the two cases and the methods and techniques needed for obtaining the user position are here discussed with some illustrative examples, with an emphasis on smartphone solutions.
The limited battery resources, processing power and memory of the smartphone set challenges for performing the calculations needed for positioning. The problems arising are presented and possible solutions given. The chapter is concluded with discussion of future research directions and a general conclusion section.
The processes of feature extracting and matching are needed for calculating the user position from images, and they set the requirements for the photographed scene. Too few features in the scene complicate the extracting. While calculating the camera motion from consecutive images, moving features distort the calculation of the user position. This is because the calculations of the camera motion are based on the assumption that the matched features in images are from objects that are static in the scene. If the objects move, the obtained motion is the sum of the motions of the objects and the camera. In general case, the degree of the motion of the objects can’t be resolved and thus can’t be separated from the motion of the camera. In the case of visual indoor positioning, the lighting conditions set also challenges for the methods used. Visual navigation is mainly needed in areas where GNSS signals are degraded, in densely constructed city centres or indoors. These areas have plenty of features, including stationary ones like buildings in city centres and inner structures of buildings indoors. These features contain many straight lines, like edges of the buildings, windows, and doors outdoors, as well as the lines where floors and walls (or walls and ceilings) meet indoors. Lines are good features to be tracked especially indoors, because they are reasonably invariant to changes in lighting.
There is no one universal method from computer vision to be used in all tasks of visual positioning. The challenge is to monitor the surroundings and flexibly adapt the means of retrieving the information from images to the one most suitable for the situation. For example the vanishing point method presented in this chapter is effective for resolving the orientation of the motion while walking indoors along a corridor, but fails completely while making a turn. Similarly, the optical flow presented later in the chapter resolves the motion accurately outdoors, but is unsatisfactory indoors with varying light conditions.