Pointing to targets in graphical user interfaces remains a frequent and fundamental necessity in modern computing systems. Yet for millions of people with motor impairments, children, and older users, pointing—whether with a mouse cursor, a stylus, or a finger on a touch screen—remains a major access barrier because of the fine-motor skills required. In a series of projects inspired by and contributing to ability-based design, we have reconsidered the nature and assumptions behind pointing, resulting in changes to how mouse cursors work, the types of targets used, the way interfaces are designed and laid out, and even how input devices are used. The results from these explorations show that people with motor difficulties can acquire targets in graphical user interfaces when interfaces are designed to better match the abilities of their users. Ability-based design, as both a design philosophy and a design approach, provides a route to realizing a future in which people can utilize whatever abilities they have to express themselves not only to machines, but to the world.
TopIntroduction
For many people today, the word “computer” is synonymous with a machine that displays a graphical user interface: depictions on a screen that convey information to a user and enable a user to convey information back to a machine. Although computers existed for decades prior to graphical user interfaces, and although many computers exist today without any visual display—for example, computers embedded in automotive systems, satellites, or home appliances—people’s notions of computers are still dominated by graphical user interfaces. It seems that wherever users go, and on whatever platform users operate, pixels arranged mostly in rectangular shapes are there to greet them. Most users today even carry at least one graphical user interface in their pocket, the immensely popular smartphone that has become more “computer” than ever it was a “telephone.”
Along with the popularity of graphical user interfaces has come the related need for users to point to graphically portrayed objects. An early famous example is Ivan Sutherland’s 1963 Sketchpad, whose direct-pointing approach using a light pen has enjoyed a modern rebirth in the form of stylus- and finger-based direct-touch devices like smartphones (Sutherland, 1963). Douglas Engelbart’s 1968 NLS demo first unveiled the relative-pointing scheme of the mouse (Engelbart, 1963), which was adopted by Xerox PARC for use with their bitmapped graphical displays of the late 1970s (Johnson, Roberts, Verplank, Smith, Irby, Beard, & Mackey, 1989), and later successfully commercialized by the Apple Macintosh in 1984 (Williams, 1984) and Microsoft Windows 1.0 in 1985 (Markoff, 1983).
Despite the revolutionary hardware and software advances that have driven computer evolution, a truth has remained: to operate a graphical user interface, a user must be able to successfully point-and-click on graphical targets rapidly, reliably, and repeatedly. Along with text entry, pointing comprises the essential substrate of interactive computer use. There is precious little way of escaping it, as even command-line aficionados must admit. Studies show that depending on the tasks being performed, 31-65% of computer users’ time is spent using the mouse, and one-third to one-half of that time is spent dragging, a complex human motor operation (Johnson, Dropkin, Hewes, & Rempel, 1993). More recent studies show that mouse usage outweighs keyboard usage by three to five times (Chang, Amick, Menendez, Katz, Johnson, Robertson, & Dennerlein, 2007; Mikkelsen, Vilstrup, Lassen, Kryger, Thomsen, & Andersen, 2007). And yet, despite the inescapable requirement of pointing, it still represents a major obstacle to successful computer use for millions of people with motor impairments and motor-related difficulties (Riviere & Thakor, 1996). Any of the three “r” words above can be significant challenges:
- •
Rapidly: Some people with motor impairments can point only extremely slowly, which means operating a computer can be an excruciating and arduous process.
- •
Reliably: Some people with motor impairments have a great deal of variation in their movements, which means the outcomes of their aimed pointing attempts are neither consistent nor predictable.
- •
Repeatedly: Some people with motor impairments fatigue quickly, which means repeated use degrades their performance before they can accomplish their tasks.