Article Preview
TopIntroduction
Image browsing is known as the interactive process of skimming through an image collection in order to explore the collection or to find a specific image in mind. For that purpose, image browsing tools typically provide a grid-like arrangement of thumbnails – also known as Storyboard (Arman et al., 1994) – which can be used to scroll through the list of images. Popular examples using that approach are file managers in operating systems, online photo sharing sites (e.g., Flickr, http://picasa.google.com/). These browsing tools provide a list of thumbnails that is usually sorted by some kind of metadata, such as the filename or creation date. However, users that need to find a particular image in mind rather search by visual attributes of the desired image instead of metadata. This is particularly true if metadata is invalid or incomplete.
In order to support visual search for images, it is meaningful to visualize the images in a similarity-based arrangement, as shown by existing work (Rodden et al., 1999a, 1999b, 2001; Nguyen & Worring, 2006; Schaefer, 2010). Unfortunately, these proposed approaches are bound to specific visualization layouts. For example, Rodden et al. use a more or less clustered visualization produced by the multi-dimensional scaling approach (Basalaj, 1999) and Schaefer proposes a 3D globe layout. Both approaches share the problem of overlapping of images, an issue that is solved by hierarchical refinement, which is also problematic as it destroys the browsing context (Nguyen & Worring, 2006). To the best of our knowledge, no one has proposed a color sorting method for grid-like arrangements and evaluated its performance in a user study.
In this work we first describe a few simple color sorting algorithms. In order to find out which of these color sorting algorithms is considered as most intuitive by humans, we evaluate the intuitiveness through a user survey with 14 users. Secondly, we use the most intuitive color sorting algorithm for a follow-up user experiment with 12 users that evaluate the achievable visual search performance in a color-sorted storyboard in direct comparison to a randomly arranged storyboard (this random arrangement should simulate a list of images sorted by some sort of metadata).
Our results of the first evaluation show that a color-sorting algorithm, based on grouping of images according to their dominant Hue values (in the HSV color space) and group-wise sorting of images by an HSV histogram, is considered as the most intuitive approach. This color sorting approach has the following advantageous characteristics:
- •
It is a general color sorting algorithm that can be used for a storyboard as well as other layouts.
- •
It does not result in overlapping of images and does not require hierarchical browsing.
- •
It produces an intuitively sorted list of images, where users can easily estimate the position of a desired image (as our user experiment has shown).
- •
It is simple and fast and can be used on mobile devices too, as shown in Schoeffmann (2011).
In the follow-up user experiment we applied this color sorting algorithm to a randomly arranged storyboard with 150 images and compared the achievable visual search time for known-item-search tasks with 1440 trials. The statistical evaluation of our results show that users are significantly faster (by 20%) with the color-sorted storyboard and also find it more preferable.