Automatic Recognition of Traffic Signs with 3D Distance Estimation for Intelligent Vehicles

1. Introduction

Automatic recognition of traffic signs is a challenging task which has attracted many researchers in the field of advanced driver assistance systems. The main challenges in this field are the complexity of the outdoor environment and the requirement of real-time execution. In fact, traffic signs images are captured in all weather conditions (sunny, shady, rainy, cloudy, and windy) and all places (freeways, expressways, highways, boulevards, streets, and country roads) which include significantly varied illumination and different complex background (Figure 1).

This has motivated many researchers to improve the performance of traffic signs recognition in complex conditions and so is the objective of our herein presented method. Hence, we introduced a new method for traffic signs recognition and distance estimation of traffic signs from a moving vehicle in complex conditions. In the recognition step, we proceeded with three stages of traffic signs detection, classification and tracking. In the detection stage, we applied a color based segmentation method to extract the candidate regions of traffic signs. For the classification step, HoG features are applied to encode the detected traffic signs and compute the feature vector. This vector is used as an input to a SVM classifier to identify the traffic sign class. Then, we track the recognized traffic signs using an optical flow-based method to keep a constant capture of the identified traffic sign. Concerning the second step of distance estimation, we used a dense disparity map between the left and right images of the recognized traffic sign region to compute its distance to the vehicle carrying the stereovision. Actually, an accurate estimation of the relative position between the vehicle and the traffic sign has an important impact on further stages such as geometrical discarding, tracking and measuring the remaining distance to a curved road portion. In our context of study, we are interested to recognize danger and prohibitory traffic signs since they constitute the important cause of accident-prone situations

Figure 1.

Traffic signs detection challenges

The remainder of this paper is organized as follows: Section 2 details the main steps of typical traffic sign recognition process and overviews the existing methods. Section 3 presents our proposed method. Section 4 discusses a set of experimental evaluations and compares the performance of our method with existing methods. Section 5 summarizes the main contributions of the present work and highlights its perspectives.

2. State Of The Art

Traffic sign recognition and estimation of their relative position can be based on monocular or stereovision approaches. Monocular based methods are characterized by their fast processing to detect and recognize the traffic sign. However, they did not afford TS depth information. They need to apply some constraints such as flat road assumption and previous knowledge about the height of the TS, which induce strong errors in the relative position estimation. In contrast to this approach, the stereovision based methods are more robust to the aforementioned problem. Nevertheless, they are computationally expensive. Moreover, they may suffer from false detections in the case of occlusions, aperture and luminance due to false matching.

Accordingly, we proposed hybrid method which combines monocular process to recognize the traffic sign, and stereovision process to compute its distance to the host vehicle.

The recognition of TSs is mainly performed using two steps: detection and classification. The detection step seeks to reduce the search space and indicate only potential regions which could be recognized as possible TSs. In the classification step, each of the already detected candidate regions is filtered to decide whether it is in the group of road signs or not. Next, we performed tracking step to keep identifying the recognized TS over frames.

In this section, we detail the existing methods in the literature for TS detection, classification and tracking (Table 1).