This paper introduces a complete framework for temporal video segmentation. First, a computationally efficient shot extraction method is introduced, which adopts the normalized graph partition approach, enriched with a non-linear, multiresolution filtering of the similarity vectors involved. The shot boundary detection technique proposed yields high precision (90%) and recall (95%) rates, for all types of transitions, both abrupt and gradual. Next, for each detected shot, the authors construct a static storyboard by introducing a leap keyframe extraction method. The video abstraction algorithm is 23% faster than existing techniques for similar performances. Finally, the authors propose a shot grouping strategy that iteratively clusters visually similar shots under a set of temporal constraints. Two different types of visual features are exploited: HSV color histograms and interest points. In both cases, the precision and recall rates present average performances of 86%.
TopIntroduction
Recent advances in the field of image/video acquisition and storing devices have determined a spectacular increase of the amount of audio-visual content transmitted, exchanged and shared over the Internet. In the past years, the only method of searching information in multimedia databases was based on textual annotation, which consists of associating a set of keywords to each individual item. Such a procedure requires a huge amount of human interaction and is intractable in the case of large multimedia databases. Today, existing video repositories (e.g., Youtube, Google Videos, DailyMotion, etc.) include millions of items. Thus, attempting to manually annotate such huge databases is a daunting job, not only in terms of money and time, but also with respect to the quality of annotation.
When specifically considering the issue of video indexing and retrieval applications, because of the large amount of information typically included in a video document, a first phase that needs to be performed is to structure the video into its constitutive elements: chapters, scenes, shots and keyframes This paper specifically tackles the issue of video structuring and proposes a complete and automatic segmentation methodology.
Figure 1 presents the proposed analysis framework. The main contributions proposed in this paper concern: an enhanced shot boundary detection method, a fast static storyboard technique and a new scene/chapter detection approach.
Figure 1. The proposed framework for high level video segmentation
The rest of this paper is organized as follows. After a brief recall of some basic theoretical aspects regarding the graph partition model exploited, we introduce the proposed shot detection algorithm. Then, we describe the keyframe selection procedure. The following section introduces a novel scene/chapter extraction algorithm based on temporal distances and merging strategies. The experimental results obtained are then presented and discussed in details. Finally, we conclude the paper and open some perspectives of future work.