Improving Pointing in Graphical User Interfaces for People with Motor Impairments Through Ability-Based Design

Improving Pointing in Graphical User Interfaces for People with Motor Impairments Through Ability-Based Design

Jacob O. Wobbrock (University of Washington, USA)
Copyright: © 2014 |Pages: 48
DOI: 10.4018/978-1-4666-4438-0.ch008
OnDemand PDF Download:


Pointing to targets in graphical user interfaces remains a frequent and fundamental necessity in modern computing systems. Yet for millions of people with motor impairments, children, and older users, pointing—whether with a mouse cursor, a stylus, or a finger on a touch screen—remains a major access barrier because of the fine-motor skills required. In a series of projects inspired by and contributing to ability-based design, we have reconsidered the nature and assumptions behind pointing, resulting in changes to how mouse cursors work, the types of targets used, the way interfaces are designed and laid out, and even how input devices are used. The results from these explorations show that people with motor difficulties can acquire targets in graphical user interfaces when interfaces are designed to better match the abilities of their users. Ability-based design, as both a design philosophy and a design approach, provides a route to realizing a future in which people can utilize whatever abilities they have to express themselves not only to machines, but to the world.
Chapter Preview


For many people today, the word “computer” is synonymous with a machine that displays a graphical user interface: depictions on a screen that convey information to a user and enable a user to convey information back to a machine. Although computers existed for decades prior to graphical user interfaces, and although many computers exist today without any visual display—for example, computers embedded in automotive systems, satellites, or home appliances—people’s notions of computers are still dominated by graphical user interfaces. It seems that wherever users go, and on whatever platform users operate, pixels arranged mostly in rectangular shapes are there to greet them. Most users today even carry at least one graphical user interface in their pocket, the immensely popular smartphone that has become more “computer” than ever it was a “telephone.”

Along with the popularity of graphical user interfaces has come the related need for users to point to graphically portrayed objects. An early famous example is Ivan Sutherland’s 1963 Sketchpad, whose direct-pointing approach using a light pen has enjoyed a modern rebirth in the form of stylus- and finger-based direct-touch devices like smartphones (Sutherland, 1963). Douglas Engelbart’s 1968 NLS demo first unveiled the relative-pointing scheme of the mouse (Engelbart, 1963), which was adopted by Xerox PARC for use with their bitmapped graphical displays of the late 1970s (Johnson, Roberts, Verplank, Smith, Irby, Beard, & Mackey, 1989), and later successfully commercialized by the Apple Macintosh in 1984 (Williams, 1984) and Microsoft Windows 1.0 in 1985 (Markoff, 1983).

Despite the revolutionary hardware and software advances that have driven computer evolution, a truth has remained: to operate a graphical user interface, a user must be able to successfully point-and-click on graphical targets rapidly, reliably, and repeatedly. Along with text entry, pointing comprises the essential substrate of interactive computer use. There is precious little way of escaping it, as even command-line aficionados must admit. Studies show that depending on the tasks being performed, 31-65% of computer users’ time is spent using the mouse, and one-third to one-half of that time is spent dragging, a complex human motor operation (Johnson, Dropkin, Hewes, & Rempel, 1993). More recent studies show that mouse usage outweighs keyboard usage by three to five times (Chang, Amick, Menendez, Katz, Johnson, Robertson, & Dennerlein, 2007; Mikkelsen, Vilstrup, Lassen, Kryger, Thomsen, & Andersen, 2007). And yet, despite the inescapable requirement of pointing, it still represents a major obstacle to successful computer use for millions of people with motor impairments and motor-related difficulties (Riviere & Thakor, 1996). Any of the three “r” words above can be significant challenges:

  • Rapidly: Some people with motor impairments can point only extremely slowly, which means operating a computer can be an excruciating and arduous process.

  • Reliably: Some people with motor impairments have a great deal of variation in their movements, which means the outcomes of their aimed pointing attempts are neither consistent nor predictable.

  • Repeatedly: Some people with motor impairments fatigue quickly, which means repeated use degrades their performance before they can accomplish their tasks.

Complete Chapter List

Search this Book:
Editorial Advisory Board
Table of Contents
Georgios Kouroupetroglou
Chapter 1
Luis Alberto Rivera, Guilherme N. DeSouza
The goal of this chapter is to explain how haptic and gesture-based assistive technologies work and how people with motor disabilities can interact... Sample PDF
Haptic and Gesture-Based Assistive Technologies for People with Motor Disabilities
Chapter 2
Jason Colman, Paul Gnanayutham
This chapter surveys assistive technologies which make video games more accessible for people who have an Acquired Brain Injury (ABI). As medical... Sample PDF
Assistive Technologies for Brain-Injured Gamers
Chapter 3
Stavroula Ntoa, George Margetis, Margherita Antona, Constantine Stephanidis
Scanning is an interaction method addressing users with severe motor impairments which provides sequential access to the elements of a graphical... Sample PDF
Scanning-Based Interaction Techniques for Motor Impaired Users
Chapter 4
Thies Pfeiffer
The eyes play an important role both in perception and communication. Technical interfaces that make use of their versatility can bring significant... Sample PDF
Gaze-Based Assistive Technologies
Chapter 5
Alexandros Pino
Free assistive technology software can be an answer to various problems of the market, most importantly the products’ high cost. The focus of this... Sample PDF
Free Assistive Technology Software for Persons with Motor Disabilities
Chapter 6
Klaus Miesenberger, Gerhard Nussbaum, Roland Ossmann
The authors outline the potential of sensor technology for people with disabilities and those people with motor disabilities in general. First the... Sample PDF
AsTeRICS: A Framework for Including Sensor Technology into AT Solutions for People with Motor Disabilities
Chapter 7
Chamin Morikawa, Michael J. Lyons
Interaction methods based on computer-vision hold the potential to become the next powerful technology to support breakthroughs in the field of... Sample PDF
Design and Evaluation of Vision-Based Head and Face Tracking Interfaces for Assistive Input
Chapter 8
Jacob O. Wobbrock
Pointing to targets in graphical user interfaces remains a frequent and fundamental necessity in modern computing systems. Yet for millions of... Sample PDF
Improving Pointing in Graphical User Interfaces for People with Motor Impairments Through Ability-Based Design
Chapter 9
Ong Chin Ann, Lau Bee Theng
This work was motivated by the limitations of the existing Assistive and Augmentative Communication tools to help children with Cerebral Palsy who... Sample PDF
A Facial Expression Mediated Natural User Interface Communication Model for Children with Motor Impairments
Chapter 10
Claudia De Pace, Fabrizio Stasolla
When the authors talk about Assistive Technology (AT) they refer to all technological solutions that allow persons with multiple disabilities and... Sample PDF
Promoting Environmental Control, Social Interaction, and Leisure/Academy Engagement Among People with Severe/Profound Multiple Disabilities Through Assistive Technology
Chapter 11
Ben Tran
The purpose of this chapter is to analyze the history of motor disabled assistive technology. Specifically, this chapter will cover motor disability... Sample PDF
Assistive Technology: Impact on Independence, Employment, and Organizations for the Motor Disabled
About the Contributors