Cocktail: Exploiting Bartenders’ Gestures for Mobile Interaction

Cocktail: Exploiting Bartenders’ Gestures for Mobile Interaction

Jong-Woon Yoo (KAIST, South Korea), Woomin Hwang (KAIST, South Korea), Hyunchul Seok (KAIST, South Korea), Sung Kyu Park (KAIST, South Korea), Chulmin Kim (KAIST, South Korea), Woong Ho Choi (KAIST, South Korea) and Kyu Park (KAIST, South Korea)
DOI: 10.4018/978-1-4666-0194-9.ch011
OnDemand PDF Download:


Today’s mobile devices are capable of creating and storing a large amount of multimedia data, but sharing such data with others remains challenging. Even when a user wishes to send a picture to a friend located next to the user, current wireless data transfer techniques usually demand several steps requiring inconvenient user involvement, such as manipulation of tiny buttons on a small touch screen. This paper presents ‘Cocktail’, a new gesture-based mobile interaction system that exploits gestures employed by bartenders for easy data sharing between co-located users. In our system, users can pour (transfer) multimedia data in their mobile devices to other friends’ devices in a manner akin to a bartender pouring a drink into a glass. Cocktail also provides an intuitive way of creating new content by mixing existing data with a shaking gesture. For example, users can make music videos with their favorite music and pictures in the mobile phone by selecting them and shaking the mobile phone. The authors have implemented a prototype of Cocktail with commercial smart phones and evaluated its usability via user studies.
Chapter Preview

1. Introduction

Most advanced mobile devices are capable of creating new multimedia data. For example, mobile phones are equipped with high resolution cameras, allowing users to take pictures or make movies anytime, anywhere. However, sharing those data with others remains challenging even when users are co-located. Transferring the taken pictures from one mobile phone to another nearby one usually involves several steps of inconvenient user intervention, including manipulating buttons to execute the file exchange program, enabling radio interfaces (e.g., Bluetooth) followed by wait time for identifying the target’s network address (e.g., Bluetooth MAC address), selecting pictures to be sent, and pressing the ‘transmit’ button. Therefore, it is essential to devise novel solutions for exchanging multimedia data in more user-friendly ways to enhance usability of mobile devices.

This paper describes a gesture-based mobile interaction system, Cocktail, which is designed for intuitive data exchange between co-located users and content creation. Motivated by the analogy of bartenders mixing drinks to make cocktails; our system uses related gestures for mobile interaction: a user can pour (transfer) data in the mobile phone to another device, like a bartender pouring drink mix into a shaker. The user can mix his/her pictures and music files into a music video by shaking the mobile phone, as a bartender mixes drinks to create a new cocktail. In addition, our system includes a touch-screen-based table-like computer system called SmartTable, which supports ‘pushing’ interaction with stationary devices, such as a TV or printer. In the same manner that a bartender pushes glasses or bills to customers along the bar, users can push icons on SmartTable towards the stationary devices to transfer the data. Figure 1 compares the gestures used in a cocktail bar and our system.

Figure 1.

Comparison of gestures used in a bar and our system


2. Cocktail System

2.1. Overview

Figure 2 shows the overall Cocktail system. It consists of mobile devices (e.g., mobile phones), a touch-screen called SmartTable, and several types of stationary devices (e.g., networked storage systems, displays, or printers). Cocktail provides gesture-based intuitive interaction among them.

Figure 2.

An example of a Cocktail system

The devices shown in Figure 2 are mapped to objects in a real cocktail bar: a mobile device is mapped to a bottle, multimedia data in the mobile device represents the drink in the bottle, and SmartTable is mapped to a table in the real bar. In this environment, mobile users become digital bartenders handling multimedia data in their mobile phones.

Our system provides three types of interactions: (1) data transfer from a mobile device to another device (including both mobile and stationary devices); (2) creation of new content; and (3) data transfer from SmartTable to stationary devices. All three interactions are based on gestures used by bartenders.

First, in order to transfer data in a mobile device to another device, we use a sprinkling gesture (similar to a pouring gesture). Similar to how a bartender pours a drink from a bottle to a glass, a mobile user can transfer data, such as pictures, in his/her mobile phone to another device, such as his/her friend’s phone, by manipulating the device with a sprinkling motion above the target device. Our system then automatically finds the target, establishes a connection, and begins the data transfer. Section 2.2 discusses the sprinkling interaction in detail.

Second, akin to a bartender making a new cocktail by mixing various drinks, a mobile user can create new content by shaking his/her mobile phone. The content (music or pictures) in his/her mobile phone is then mixed into new content (e.g., a music video). This interaction is described in Section 2.3.

Finally, we have designed interaction via SmartTable to support intuitive data transfer to stationary devices. SmartTable is a touch-screen-based computer system. A user can transfer data from the mobile phone to SmartTable with the sprinkling gesture, i.e., by sprinkling the mobile phone above the touch-screen. An icon that represents the received data then appears on the touch-screen. In addition, SmartTable allows the user to transfer data from SmartTable to nearby stationary devices, such as a TV, printer, or computer, by dragging the icon in the direction of the target device. Details about the SmartTable interaction are given in Section 2.4.

Complete Chapter List

Search this Book: