Anomaly Detection in Streaming Sensor Data

Introduction

The Wireless Phone-Based Emergency Response System (WIPER) is a laboratory proof-of-concept, Dynamic Data Driven Application System (DDDAS) prototype that uses cell phone network data to identify potential emergency situations and monitor aggregated population movement and calling activity. The system is designed to complement existing emergency response management tools by providing a high level view of human activity during a crisis situation using real-time data from the cell phone network in conjunction with geographical information systems (GIS). Using cell phones as sensors has the advantages of automatic deployment and sensor maintenance; however, the data available from the network is limited. Currently only service usage data and coarse location data, approximated by a Voronoi lattice defined by the cell towers, are available, although cell-tower triangulation and GPS could greatly improve the location data (Madey, Szabó, & Barabási, 2006, Madey et al., 2007, Pawling, Schoenharl, Yan, & Madey, 2008, Schoenharl, Bravo, & Madey, 2006, Schoenharl, Madey, Szabó, & Barabási, 2006, Schoenharl & Madey, 2008).

The viability of using cell phones as a sensor network has been established through the use of phone location data for traffic management (Associated Press, 2005). WIPER applies this finding to fill a need in emergency response management for a high level view of an emergency situation that is updated in near real-time. Tatomir and Rothkrantz (2005) and Thomas, Andoh-Baidoo, and George (2005) describe systems for gathering on-site information about emergency situations directly from response worker on the ground via ad-hoc networks of PDAs. While these systems can provide detailed information about some aspects of the situation, such as the location of victims and environmental conditions, the information is limited to what can be observed and reported by the responders. This provides valuable but local information, though there may be observations from different, geographically dispersed locations. In contrast, WIPER provides less detail, but instead gives an overall view of population movements that may be valuable in refining the response plans or directing response workers to gather more detailed information at a particular location.

Dynamic data driven applications systems (DDDAS) provide a framework in which running simulations incorporate data from a sensor network to improve accuracy. To achieve this, the simulations dynamically steer the measurement process to obtain the most useful data. The development of DDDAS applications is motivated by the limited ability to predict phenomena such as weather and wildfire via simulation. Such phenomena are quite complex, and the correct simulation parameterization is extremely difficult. The goal of DDDAS is to provide robustness to such simulations by allowing them to combine sub-optimal initial parameterizations with newly available, real-world data to improve performance without the expense of rerunning the simulations from the beginning (Douglas & Deshmukh, 2000).

In this chapter, we focus on one component of WIPER: the detection and alert system. This module monitors streaming data from the cell phone network for anomalous activity. Detected anomalies are used to initiate an ensemble of predictive simulations with the goal of aiding emergency response managers in taking effective steps to mitigate crisis events. We discuss methods for anomaly detection on two aspects of the call data: the call activity (the number of calls made in a fixed time interval) and the spatial distribution of network usage.

The remainder of the chapter is organized as follows: we discuss background literature related to mining data from a cell phone network. We start with a discussion of methods for detecting outliers in our data, with a focus on using data clustering to model normality in data. Those clusters of outliers in the streaming data could be indicators of a problem, disaster or emergency in a geographical area (e.g., an industrial explosion, a civil disturbance, progress of a mandated evacuation prior to a hurricane, a terrorist bombing). We then give an overview of the data set and the WIPER system, followed by descriptions of algorithms used in the detection and alert system. Finally, we discuss some of the privacy issues related to this work and our plans for future work in the spatial, graph and temporal analysis of the underlying social network.