Real-Time Classification of Water Spray and Leaks for Robotic Firefighting

Real-Time Classification of Water Spray and Leaks for Robotic Firefighting

Joshua G. McNeil (Department of Mechanical Engineering, Virginia Polytechnic Institute and State University, Blacksburg, VA, USA) and Brian Y. Lattimer (Jensen Hughes, Arbutus, MD, USA)
Copyright: © 2015 |Pages: 26
DOI: 10.4018/IJCVIP.2015010101


Robotic firefighting is an area of increased focus as a way of limiting the exposure of firefighters to hazardous environments. A suppression system must incorporate multiple functionalities to allow for closed-loop firefighting control. One area of development is classifying water spray as a way of correcting errors between suppressant placement and fire location. An IR vision system is presented which is capable of identifying water. Image segmentation is performed, followed by a process that classifies regions of interest as water or non-water objects. A probabilistic classification method, using Naïve Bayes classifier, was applied on a varied dataset of differing water temperatures and sprays. Objects were segmented using frame differencing with image intensity and difference thresholds. Segments were manually labeled to create a training dataset. Precision, recall, F-measure, and G-measure results of the classifier on a separate test dataset ranged from 86.1-97.4% for classifying water objects using the test dataset with water classification alone having 94.2-97.4% accuracy.
Article Preview


To limit the exposure and risk of firefighters and to improve effectiveness of indoor firefighting operations, robots have been proposed as a support tool. One aspect of robotic firefighting is a fire suppression system that autonomously detects and extinguishes fires effectively and efficiently. Many versions of firefighting robots (Kim, Y.D., Kim, Y.G., Lee, Kang, & An, 2009; Penders et al., 2011; Liljeback, Stavdahl, & Beitnes, 2006; Pack, Avanzato, Ahlgren, & Verner, 2004; Dearie, Fisher, Rajala, & Wasson, 2001; Miyazawa, 2002) have been developed; however, most use an operator to remotely control them or have not been tested in realistic fire scenarios. In the shipboard autonomous firefighting robot (SAFFiR) program, a humanoid robot was developed to autonomously locate and suppress fires located onboard naval ships in indoor structures. Advancements including artificial intelligent algorithms and perception systems (Kim, Starr, & Lattimer, 2015; Kim, Keller, & Lattimer, 2013; Starr & Lattimer, 2012, 2013, 2014) along with autonomous fire suppression approaches (McNeil, Starr, & Lattimer 2013) were developed through the program to further advance firefighting robots. In addition, a probabilistic classification technique was proposed to distinguish fire and smoke from reflections or hot objects (Kim & Lattimer, 2015) for use in determining a heading direction for the robot to locate a fire. Due to the smoke-filled environments that are encountered in firefighting, IR cameras were selected over visual RGB cameras. Handheld thermal infrared cameras (TIC) are typically used to aid in firefighting tasks within smoke-filled environments (Amon, Benetis, Kim, & Hamins, 2004; Amon, Bryner, & Hamins, 2004; Amon & Ducharme, 2009; Maxwell, 1971). For TICs and IR cameras that absorb IR radiation in the long wavelength IR (LWIR, 7-14 micron range), they are able to image surfaces even in dense smoke and zero visibility environments (Starr & Lattimer, 2014; Chacon-Murguia & Perez-Vargas, 2011). Classification of objects with thermal imagers is needed to provide the robotic system with the information required to efficiently perform firefighting tasks even in clear and low visibility environments.

Water is an unstructured, moving object that makes its segmentation and classification complex. There are many different approaches to segmenting and classifying moving objects in video sequences but active water segmentation with classification techniques in IR have not been widely studied. Some spray segmentation approaches include edge detection (Pastor, Arregle, & Palomares, 2001; Jeong, Lee, & Ikeda, 2007), which relies on high contrast between warm temperature diesel spray and a static blank background. Water spray or leaks in IR images contain a high level of noise and can be very low contrast making traditional edge-based detection algorithms unreliable (Nanda & Davis, 2002). Water does not have a distinct structure so classification techniques relying on shape cannot be used in low contrast scenarios. Image segmentation based on Otsu’s method, a cluster-based image thresholding technique, has been used when the object or spray has high contrast with the background (Shao & Yan, 2008; Zhao-dong & Zeng-ping, 2010; Kang & Chen, 2009) and can be separated as foreground. In the suppression system, the water object is often low intensity (low temperature) and merges with parts of the background. Background subtraction techniques such as Gaussian Mixture Models (GMM) (Stauffer & Grimson, 2000) and adaptive background subtraction are unreliable in IR video due to the low signal-to-noise ratio (Latecki, Miezianko, & Pokrajac, 2005). A reliable background model cannot be used in dynamic firefighting scenes involving fire, water, smoke, and reflections due to the large differences between an image and a background model that are produced as the scene changes or the system moves due to robot motion or from scanning the area.

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 10: 4 Issues (2020): Forthcoming, Available for Pre-Order
Volume 9: 4 Issues (2019): 3 Released, 1 Forthcoming
Volume 8: 4 Issues (2018)
Volume 7: 4 Issues (2017)
Volume 6: 2 Issues (2016)
Volume 5: 2 Issues (2015)
Volume 4: 2 Issues (2014)
Volume 3: 4 Issues (2013)
Volume 2: 4 Issues (2012)
Volume 1: 4 Issues (2011)
View Complete Journal Contents Listing