Cognitive Based Distributed Sensing, Processing, and Communication

Introduction

The greatest challenge to modern sensor processing is computational complexity and limited communication bandwidth. Consider a multi sensor distributed image communication system for real time situation monitoring. An example of such system would be security video surveillance consisting of multiple cameras covering a certain area. Image sequences from the cameras provide multiple views of the same area from different vantage points. The timing of the images is not synchronized. Human operators are usually overwhelmed by the amount of visual information that has to be processed in real time. There is a strong need for automated detection of events of interest and bringing them to operator’s attention.

The challenges for such automation are (1) information fusion, (2) object detection, classification, and tracking, and (3) situation classification. Information fusion associates the data from different cameras that originated from the same object (Hall & Llinas, 2001). This task is difficult due to multiple objects in the view, asynchronous image sequences, and the presence of a large number of irrelevant objects. A brute-force algorithm for testing all possible combinations of data points from each camera results in exponential computational time. Object detection and classification is often difficult due to high noise, occlusions by other objects, variations in size, shape, color, and other characteristics of objects falling within the same class (Bishop, 2006). Object tracking, similarly to information fusion, is challenging due to the data association problem (Bar-Shalom & Fortmann, 1988). Finding correspondences between image points that belong to consecutive images may require exponential computational time. Finally, situation classification requires consideration of multiple objects, their spatial relationships, and time evolution. Due to the large number of irrelevant objects in the images this task could not have been automated. Each of the aforementioned challenges is an area of active research. The importance of systems meeting all of the above challenges can hardly be overstated.

Cognitive modeling shows promise in overcoming these challenges based on the evidence from neuroscience and psychology. The brain is known to perform information fusion from various senses, object tracking and recognition, and situation recognition in efficient manner. In fact, the term situational awareness is used almost exclusively to describe human ability to understand the meaning of current perceptions and predict the future events (Endsley, 1995). It seems that building an artificial agent capable of situational awareness will involve going beyond modeling the perceptual abilities of human beings, such as vision, touch and smell. In what follows we argue that successful cognitive modeling has to include such attributes of the mind as instincts, emotions, and language. All of these components of the mind find their place within the Neural Modeling Fields theory (NMF) of the mind. The objectives of the chapter are to provide an overview of NMF, examples of usage of NMF, and outline the directions of future research within this exciting area of cognitive modeling.