Content Coverage and Redundancy Removal in Video Summarization

1. Introduction

With the ever decreasing cost of digital storage devices and advancement of technology in recent years, video recorders have gained immense popularity. A large number of videos are produced and uploaded at an ever increasing rate. Digital libraries and video repositories like YouTube, DailyMotion, MyVideo etc. enable users to upload, retrieve, view and share videos over the internet. This has also been facilitated by a many fold increase in communication bandwidth provided by Internet Service Providers (ISPs). Due to its inherent structure, a video encompasses the other three media types as well, i.e. text, image and audio, combining them into a single data stream. As a consequence, the analysis and retrieval of videos has attracted much attention from researchers and application developers. The research issues include feature extraction, similarity/dissimilarity measures, segmentation (temporal and semantic), key-frame extraction, indexing, annotation, classification and retrieval of videos. Video summarization is an application which lies on the crossroads of these research issues. The objective of video summarization is to reproduce and represent a video to the user in a concise manner such that its overall semantic meaning is preserved. The time and bandwidth constraints besetting the user are balanced to a great extent by such summarization task. Real-time video summarization helps in immediate indexing of videos for facilitating retrieval from the repository. Also it facilitates the user in assessing whether watching the entire video would be useful or not. Video summarization and representation through key frames have been frequently adopted as an efficient method to preserve the overall contents of the video with a minimum amount of time. The types of video summarization include static (also known as storyboard representation) and dynamic (also called video skimming). Storyboard or static representation is produced by means of selecting key-frames from a set of video frames, obtained after decomposing a video. Audio clips have temporal characteristics and are therefore not an integral part of static summaries. However, some keywords in the form of meta-data may be tagged with the static video frames to assist in keyword based indexing and retrieval. The set of key-frames serve as features of the video and these key-frames help in indexing task. During the process of retrieval, the key-frames corresponding to a video may be matched with the key-frames extracted from the query video. As such, the task of video mining is facilitated through this process.