Article Preview
Top1. Introduction
Multimedia content, which is provided by PowerPoint presentations or Flash documents, is widely adopted and can be found in any domain. Despite the growing interest in multimedia web search, most major web search engines currently offer only limited multimedia search functionality. Unlike existing image or video search engines, the content a multimedia search engine has to deal with is the coherent combination of different types of media assets such as images, videos, audios, and text. Those media assets can be animated or rearranged by interactions (Candan & Sapino, 2010). The semantic within multimedia documents is given through relations dealing with time and space of media assets (Boll, Sandhaus, Scherp, & Westermann, 2007), e.g., duration of a video, width, height, and position of an image, but these are just partially taken into account by previous systems. Research on search engines dealing with a single media type like images and videos is well established and there exist studies that investigate how users apply media search engines (Maniu, O'Hare, Aiello, Chiarandini, & Jaimes, 2013; Kofler & Lux, 2009). However, regarding search and retrieval of true multimedia there is a gap in the research. Thus, we have developed an early prototype of a search engine for multimedia content on the web (Tingvold, Stohr, Schneider, & Amundsen, 2013)1 (cf. Figure 1). The prototype offers users to search and explore for multimedia content. The goal of this early search engine was to provide users an initial idea of what a retrieval system for true multimedia could look like and to use this system to bootstrap requirement elicitation and detailed understanding of users’ needs for semantic multimedia search. On the basis of this prototype, we conducted a user study evaluating the features for a future multimedia search engine. The current prototype is developed for keyword-based queries only. There are filters for audio, video, animation and interaction. These filters are applied by logical OR operators. The preview of the multimedia documents in the result list indicates if the document contains media assets fulfilling the filter. The thumbnails, which show the first frame or the first slide of the multimedia document, are of different sizes, starting with larger ones in the upper position to smaller ones in the lower position of the results page. Hovering over a thumbnail shows an animated preview of the document. Clicking on a document opens a detail view in the lower part of the search engine’s window.
Figure 1. Fulgeo search interface and result view pagination is used and limits the result list to 20 documents per page. Overall, the prototype has about 4000 multimedia documents in the database.
In the subsequent sections, we present the related work in multimedia search and studies on media search engines. In Sections 3 and 4, we describe the conducted user study and its results. Based on these results, we derive requirements to a future multimedia search engine in Section 5. We designed mockups combining good features of the existing prototype, with new requirements collected from the survey, and further ideas derived by the interview section (Section 6). Finally, we discuss the overall results and the limitations of our study (Sections 7 and 8).