Object-Based Video Analysis and Interpretation

Abstract

In this chapter, we present a novel scheme for object-based video analysis and interpretation based on automatic video object extraction, video object abstraction, and semantic event modeling. In this scheme, video objects (VOs) are first automatically extracted, followed by a video object abstraction algorithm for identifying key frames to reduce data redundancy and provide reliable feature data for the next stage of the algorithm. Semantic feature modeling is based on a temporal variation of low-level features of video objects. Dynamic Bayesian networks (DBNs) are then used to characterize the spatio-temporal nature of the video objects. The system states in the proposed DBNs directly correspond to the physical concepts. Thus, the decoding of the DBN system states from observable variables is a natural interpretation of the behavior of the video objects. Since the video objects are generally considered as the dominant semantic features of video clips, the proposed scheme provides a powerful methodology for content description, which is critical for large scale MPEG-7 applications.