Abstract
A large percentage of computing tasks in our contemporary environment are spatial problems carried out on mobile or spatially distributed computing devices. In this chapter, the authors evaluate if and how the commonly established metaphor of a windowed user interface may be adapted in order to suit contemporary usability needs. The study observes user behavior and interactions with a set of large-format touch screens in order to assess user's ability to integrate their interpretation and operation with the displayed information across multiple, spatially distributed screens.
Top1. Introduction: Seeing Through The Window
Since almost three decades the Graphical User Interface is the standard for the human-machine interface and whenever we interact with computers we are inclined to expect this interaction to employ Windows, Icons, Menus and a Pointing device. Despite significant technological changes over the course of this timeframe the concepts of how users interact with computers has remained mostly the same. For a core technological function of computers this is a surprisingly long time of constancy and already in the 1990 (Hinckley, 1996) this was regarded as stagnation and an imminent change predicted according to the changes in computing machinery. Myers et al. write that “Virtually all applications today are built using window managers, toolkits, and interface builders that have their roots in the research of the 70’s, 80’s, and 90’s.” (Myers et al., 2000). Again the authors predict that the conventional GUI is ill suited for many contemporary computing applications and thus will be changing soon. Nevertheless, the WIMP concept is still the standard on most of our computers today. While the mouse as a pointing device has in many cases been replaced by trackpad or touch screen based interactions, the window as a conceptual tool to facilitate the user in the negotiation of multiple coexisting contexts and tasks. The window metaphor is established as an easy way to delimit the different contexts of sets of information, applications, files and the various objects we encounter in our work with computers and thus to allow the users to orient themselves and organize their workflow.
The window as a frame to distinguish information contexts goes hand in hand with the development of personal computing. The window was implemented in form of a display screen in Douglas Engelbart’s oNLineSystem (NLS). It was part of a visual interaction system comprising a television monitor together with the mouse, “a hand-held X-Y transducer” constituting a system to “augment the human intellect” (Engelbart, 1968). The original implementation of the NLS used the physical frame of the display screen as a delimitation for the representation of information contexts shown on the screen and required the user to navigate through a hierarchical structure of different ‘screens’ of information representations. In subsequent implementations, though, the idea of framed visual representation and the window as a metaphor of the possibility to look into a different, coexisting world informed equally the software design. Alan Kay adopted the notion of “a general window that viewed a larger virtual world” (Kay, 1993) first in the development of the FLEX machine and later of the Smalltalk language, which was to become at the same time the user interface and the development environment of the Xerox Alto computer. The window metaphor was extended not only to use the display screen as a frame into an information world but it was subdivided and converted into a space in which multiple ‘virtual’ windows could be stacked and overlapped to represent multiple different information contexts. Kay was deeply influenced by the notions of visuality and visual cognition formulated by Seymour Papert and Marvin Minsky (Buchanan, 1974), who stressed the importance of the visual in their research on artificial intelligence (Kay, 1969). In his Ph.D. thesis “The Reactive Engine” Kay imagined large format virtual screens controlled by a windowing system that manages screen subsets and overlaps: “The portion of the display inside the windowed area is only what is transmitted to the CRT. A clipper is used to find the intersection of lines with the window boundaries. Any number of v. screens, windows and viewports may be superimposed.” (Kay, 1969)
Key Terms in this Chapter
Augmented Reality: Is a technique of displaying a representation of a real-world environment with additional information layered on top of it. This technique of stacked information layers aims to augment the view into the real world environment with additional information. The display system of augmented reality is mostly a real-time camera generated image stream with an additional computer generated information layer on top of the camera image.
Human Computer Interaction: Is the study and the design of communicational exchanges between humans and computers. Human computer interactions are generally channeled through an interface layer that allows humans to utter commands to the machine and in exchange the machine is generally equipped with affordances to represent it internal states to the human user.
Graphical User Interface: Screen based human computer interface using visual representations to establish the communication between the computer and the user. GUIs normally use visual objects such as icons, buttons and other so called widgets to construct a visual language.
Ubiquitous Computing: Refers to the concept of making computing resources available anywhere. While computers used to be large and heavy machines, ubiquitous computing relies on the miniaturization of electronics and therefore the possibility of integrating computing technology into small mobile devices and many kinds of other ubiquitously appearing contexts.
Perspective: A technique of representing a three dimensional space on a two-dimensional surface such as the screen or a canvas. Perspective construction assumes a specific point of view for which all the lines characterizing the space are rendered in close approximation to the way the eye would see them in reality. Perspective construction is the underlying notion of computer generated 3D rendering.
Multi-touch: Interaction with a touch sensitive surface that is able to recognize multiple points of contact. Multi-touch technique is mostly used in conjunction with touch screens and allows either input through multiple fingers from one person or multiple people. The touch input can control a cursor in a way similar to the mouse or multi-touch gestures can be interpreted to trigger specific events in a computing program.