This chapter identifies challenges and concepts inherent to the visualization and exploration of video spaces and presents an approach through ColorsInMotion, an interactive environment to process, visualize and explore a video space with a semantic focus on cultural aspects like music and dance, and stressing features such as their color dominance, rhythm and movement, at the level of the video space and the individual videos. It provides means to capture, experience, and express videos’ properties and relations, allowing to gain insights into our culture and to influence the expression of its intrinsic aesthetics in creative ways.
TopIntroduction
We have been witnessing a rapid explosion of video data in richer and broader widely accessed media spaces. It is becoming very easy to access video in a digital manner in private and public scenarios, due to the propagation of digital cameras, high bandwidth Internet connections and video sharing websites. Furthermore, video is a very rich medium combining image and sound, thus providing huge amounts of information and an excellent platform for creativity to be expressed and explored. The pioneers of the Video Art, for instance, have been exploring this creativity, using installation and performance art in conjunction with video in order to create an immersive experience for the spectator. However, all the richness that makes video based information spaces so interesting, inside each video and outside in the information realm where in many ways they relate to each other, comes with a challenging complexity to handle.
One of the problems is the fact that video is not structured data-wise, and so, accessing all the data that a video can provide is often not an easy task. Semantic descriptors can be used to tag some information of the video: by the users, a common approach in video repositories like YouTube; or through information segmentation and understanding, a more complex task. Low level data, like color and shot information, duration of the video, and scene information, provide additional information that can be retrieved by calculations and time consuming tasks. But once this information is collected, we can try to use it to our advantage, for a better organization of the individual and collective video spaces, to search, to assist with the process of editing videos and even to provide new forms of visualization and interaction.
Visualization techniques (Card et al, 1999; Tufte, 2006) can actually help to handle the complexity and express the richness in these information spaces. Video visualization can be an intuitive and effective way to convey meaningful information in video. However, there is still a lack of effective techniques to convey complex information intuitively through automatic video processing. An alternative approach is to provide an overview by extracting interesting information and present it in a meaningful way (Daniel & Chen, 2003). Summarization for e.g. allows to visualize video in concise ways, based on properties like movement, rhythm or scene change. Also in the Video Art movement, some of the works make use of visualization - as a tool to convey some kind of meaning to the viewer.
Advances in data visualization have emerged from research rooted primarily on visual perception and cognition. Trends are to evolve towards scenarios where tools become invisible while we become immersed in the exploration of data (Few, 2007). In our approach, we explore visualization techniques and interaction modalities in the direction of making the user immersed in the video spaces, while stressing significant properties.
This chapter identifies challenges and concepts inherent to the visualization and explorations of video spaces and presents an approach through ColorsInMotion, an interactive environment to support the interactive and creative visualization and exploration of videos with a strong emphasis on color and motion properties, at the crossroads of information access, culture and digital art. It provides video analysis support along with the means to help with the visualization and browsing of the collective video space and the individual videos, stressing features such as their color dominance, rhythm and movement. Interaction can be done through a traditional setting based on a screen, keyboard and mouse, or a touch screen, but we are also developing more natural and deviceless approaches, especially adequate for installation settings and ambient interaction contexts. This work is based on our earlier work: “ColorsInMotion: Interactive Visualization and Exploration of Video Spaces”, in (Martinho & Chambel, 2009) © ACM, 2009.
Next section makes a review of most relevant background. Then, ColorsInMotion is presented based on the description of its two modules, the VideoAnalyzer and the Viewer. The chapter ends with a discussion of future research directions and some concluding remarks.