Article Preview
TopIntroduction
Through our attempts at using a multi-display environment for interactive contents, we have particularly addressed realization of a flexible screen layout. Our objective in undertaking these works is to pursue a multi-display usage that can enrich contents’ representation. Herein, we designed and implemented a reconfigurable multi-display system with commodity mobile devices, and used a pinching gesture for the intuitive interface for connection of the screens.
A multi-display environment is generally used for offering extremely large and high-resolution virtual screens (Ni et al., 2006; Li et al., 2000). Its display composition is static and permanent. The main application domains which use such environments are scientific visualization and virtual reality. If interactive applications such as media artwork are run on a multi-display system, then, because of the larger screen size, contents would give greater impact to an audience than when run on a single display. However, if multi-display functionality remains only in forming a larger but single virtual screen, then the user experience with the applications is not expected to differ greatly from those designed for a single display. More interesting representations are possible if we can add additional dynamic features when configuring the display’s layout.
In pursuing the potential of multi-displays as a platform for interactive contents, we decided to seek ways in which the display layout can be reconfigurable interactively, even when applications are running. We believe that such an interaction will bring users new experiences and fun. When we use the word “reconfigurable”, it does not mean merely to change the screen’s arrangement. We also want to add and remove devices freely to and from the multi-display environment. Additionally, we sought interaction such that changing the displays' layout causes an application's instant reaction.
We chose mobile devices such as smartphones and tablet PCs as the ideal hardware platform for our approach because of their mobility and high-resolution display. It is also helpful that many people now own such devices. There are numerous opportunities when not a few devices are gathered at one place simultaneously. Because we do not choose to design a specifically tailored device but instead decided to use ubiquitous devices such as smartphones as our platform, we do not want to attach any extra sensors to them. We want to make the mechanism available casually whenever several devices are brought together, but not make it function only at a specific place with tiresome pre-configuration for network setup. Therefore, we do not employ a server machine for a central role of management architecture. We devised a simple and intuitive interface to accomplish the connection.
Many different approaches and interfaces might realize a dynamically reconfigurable multi-screen system, but we believe that a great discrepancy exists between one that merely achieves the function and one that provides a new experience. The latter cannot be realized merely by designing an interface that is simple and intuitive. More than that, such an interface must be integrated naturally into the usage scenarios and applications. For our endeavor in creating such an interface, we chose to employ a pinching gesture. That pinching gesture is accomplished by putting one’s forefinger and thumb on two annexed screens each, and swiping them just as though stitching the screens together. To break the connection, we chose a shaking or tilting action to accomplish the task. With these gestures used for forming a multi-display environment, we designed “Pinch”-able applications such that each reacts instantly without further prompting other than the actions of connection and disconnection of devices. This arrangement enables a user to have the illusion that the user’s hand has come to possess a magical power that can connect digital contents.
As described in this paper, we present the concept of a reconfigurable multi-display environment, and introduce our interface by explaining its idea and implementation. We also present several sample applications we created to demonstrate our approach’s potential in creating new user experiences.