QuEST for Information Fusion in Multimedia Reports

QuEST for Information Fusion in Multimedia Reports

Erik P. Blasch (Air Force Research Laboratory, Rome, NY, USA), Steven K. Rogers (Air Force Research Laboratory, Rome, NY, USA), Hillary Holloway (Systems &Technology Research, Woburn, MA, USA), Jorge Tierno (Barnstorm Research Corporation, Malden, MA, USA), Eric K. Jones (Systems &Technology Research, Woburn, MA, USA) and Riad I. Hammoud (BAE Systems, Burlington, MA, USA)
DOI: 10.4018/IJMSTR.2014070101


Qualia-based Exploitation of Sensing Technology (QuEST) is an approach to create a cognitive exoskeleton to improve human-machine decision quality. In this paper, the authors present QuEST-motivated man-machine information fusion with an example for multimedia narratives. User-based situation awareness includes both elements of external sensory perception and internal cognitive explanation. The authors outline QuEST elements and tenets towards a reasoning approach that achieves human intelligence amplification (IA) in relation to data aggregation from machine artificial intelligence (AI). In a use case example for multimedia exploitation, they showcase the need for enhanced understanding of the man (mind-body cognition) and the machine (sensor-based reasoning) for establishing a cohesive narrative of situational activities. QuEST tenets of structurally coherent, situated conceptualization, and simulated experience are utilized in organizing multimedia reports of Video Event Segmentation by Text (VEST).
Article Preview

1. Introduction

For many activities, a user-machine workflow is required for data analysis and mission effectiveness (Blasch, Bosse, Lambert, 2012). For example, integrated global Intelligence, Surveillance, and Reconnaissance (ISR) operations include a five-phase process: Planning and direction; Collection; Processing and Exploitation; Analysis and production; and Dissemination (PCPAD) (Air Force Doctrine, 2012). The PCPAD process is not linear or cyclical, but rather represents an multi-machine, multi-user enterprise of interrelated, simultaneous operations that can, at any given time, feed and be fed by other operations as illustrated in Figure 1. Key PCPAD process elements are machine tools to aid automation and decision making. For processing, it could be signals alignment, filtering and collection. For exploitation, it includes data correlation and association. For dissemination, visualization, interpretation and data reporting transfer knowledge. Exploitation and dissemination necessitate user refinement for situation analysis as well as data selection for reporting, picturing, and narration.

Figure 1.

Information fusion in the enterprise


Qualia-based Exploitation of Sensing Technology (QuEST) is focused on the blending of traditional experience based interpretation with an artificially ‘conscious’ interpretation of the signals, data, and information. A driving motivation for QuEST is the theory that conscious has a role in robust decision making such as supporting coherent narrative from sparse measurements.

In designing systems to augment user needs, it is desirable to provide intelligence amplification (IA) (Rogers, et al., 2003). Qualia provide a vocabulary for subjective analysis of stimuli. Qualia are the internal conscious perceptions of the basis set used to represent the stimuli and is a subjective aspect of the human’s conscious perception of the stimuli. Qualia allow an agent to understand/evaluate relevant data in decision making. The more that a sensor/user understands and evaluates their Qualia (Rogers, et al., 2008), the more self-confident they would be in decision making. Qualia encompass an important component to uncertainty reasoning associated with subjective beliefs, trust, and narratives in decision making. This includes the conscious representation of the subconscious processing and thus represents a sense of intuition. The goal of the user-machine processing is to select relevant data in forming a cohesive narrative that explains the situation.

Situation analysis includes three domains: (1) human factors for situation awareness (Endsley, 1995A), (2) information fusion for situation assessment (Blasch, et al., 2006), and (3) psychology for situation representation (Patterson, et al., 2012). While all three domains have focused on situation understanding, the QuEST paradigm seeks to coordinate user cognitive processes with that of the sensed world as augmented by machines for situation narratives.

The PCPAD process highlights human-machine exploitation, shown in Figure 2 (Rogers, et al., 2014). Sensing-to-processing includes command and control and collection which incorporate cognitive determination for machine control. Processing-to-dissemination includes exploitation (or analysis) that focuses on cognitive interpretation of the data available, directly or indirectly, to reason about the situation. QuEST seeks to enhance processing with human-based sensing.

Figure 2.

Planning to dissemination


Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 5: 4 Issues (2017)
Volume 4: 4 Issues (2016)
Volume 3: 4 Issues (2015)
Volume 2: 4 Issues (2014)
Volume 1: 4 Issues (2013)
View Complete Journal Contents Listing