Abstract
The detection of person fatigue is one of the important tasks to detect drowsiness in the domain of image processing. Though lots of work has been carried out in this regard, there is a void of work shows the exact correctness. In this chapter, the main objective is to present an efficient approach that is a combination of both eye state detection and yawn in unconstrained environments. In the first proposed method, the face region and then eyes and mouth are detected. Histograms of Oriented Gradients (HOG) features are extracted from detected eyes. These features are fed to Support Vector Machine (SVM) classifier that classifies the eye state as closed or not closed. Distance between intensity changes in the mouth map is used to detect yawn. In second proposed method, off-the-shelf face detectors and facial landmark detectors are used to detect the features, and a novel eye and mouth metric is proposed. The eye results obtained are checked for consistency with yawn detection results in both the proposed methods. If any one of the results is indicating fatigue, the result is considered as fatigue. Second proposed method outperforms first method on two standard data sets.
TopBackground
The steps involved in fatigue detection are face detection, feature detection and then classifying whether the person is fatigue or not. In literature, the features mostly considered are eye state and yawn. Some researchers considered head tilt detection for determination of fatigue. Xie Y. et al (2018) built a model using deep neural network for yawn detection. The network learns from yawning video clips and also images using transfer and sequential learning. They are able to distinguish between yawning, talking and laughing. Also, yawn is detected even when the face is turned 70 degrees away from camera. Huang R et al. (2018) has built fatigue detection convolutional network (FDCN) based on common convolutional neural network (CNN). projection cores were incorporated into FDCN to make the features learnt invariant to scale. Achieving an accuracy of 94.9% to detect fatigue using eye state.
F. Zhang et al (2017) used AdaBoost and Local Binary Features to detect features and CNN for classifying and achieved an accuracy of 95%. B. N. Manu et al. (2016) used binary support vector machines to detect closed eye detection and yawn. They used linear kernel and achieved an accuracy of 95%.
Automatic fatigue detection of drivers through yawning analysis is done by Azim, Tayyaba, et al. (2009). After locating face in a video frame, the mouth region is extracted, where lips are searched using spatial fuzzy c-means (s-FCM) clustering. If the yawning state of the driver is detected for several consequent frames, the driver is said to be drowsy. Fuzzy C-means uses spectral and optimal information to segment lips region. Danisman et al. (2010) proposed variations in eyes location based on the symmetry feature along horizontal direction of the eyes. If the symmetry doesn’t occur, it corresponds to a closed eye state. Omidyeganeh et al. (2011) detected drowsiness by analyzing eye and mouth states. The person’s face is captured from a camera and then converted to color spaces of YCbCr and HSV. To extract eyes, Structural Similarity Measure (SSIM) was used which uses properties which are statistical such as mean and variance. The SSIM values vary between -1 and 1. The maximum value is gained when two images are the same. It is used along with a template to find out the best match for the eyes.
Key Terms in this Chapter
Support Vector Machines: A support vector machine (SVM) is a classifier that separates classifiers by outputting a hyperplane. In supervised learning where labeled training data is given, SVM generates a hyperplane such that data belonging to same class will be on one side of hyperplane.
Gradient Boosting: Gradient boosting is a machine learning technique for regression and classification problems, which produces a strong classifier in the form of an ensemble of weak classifiers. Gradient boosting combines weak classifiers in iteratively. Gradient boosting generalizes by minimizing loss function and loss function must be differentiable. Gradient boosting involves weak classifiers, a loss function that has to be minimized and an additive model to add weak classifiers to minimize loss function.
Mouth Map: Color map is matrix of values. Color map can be of any length and depends on the number of colors that make the color space. Each row in the matrix defines one color. For example, in RGB image, each pixel is combination of intensities of red, green and blue colors. Each row of color map matrix contains 3 columns for storing the intensities of red, green and blue colors. Image with only red component can be used for image processing. Red component dominates around mouth area than blue component in humans. Mouth can be prominently identified by considering only chromium component in YCbCr color space. Mouth Map is generally image of the detected mouth with only one component used.