Using Multimodal Displays to Signify Critical Handovers of Control to Distracted Autonomous Car Drivers

Using Multimodal Displays to Signify Critical Handovers of Control to Distracted Autonomous Car Drivers

Ioannis Politis, Stephen Brewster, Frank Pollick
Copyright: © 2017 |Pages: 16
DOI: 10.4018/ijmhci.2017070101
(Individual Articles)
No Current Special Offers


Until full autonomy is achieved in cars, drivers will still be expected to take over control of driving, and critical warnings will be essential. This paper presents a comparison of abstract versus language-based multimodal warnings signifying handovers of control in autonomous cars. While using an autonomous car simulator, participants were distracted from the road by playing a game on a tablet. An automation failure together with a car in front braking was then simulated; a rare but very critical situation for a non-attentive driver to be in. Multimodal abstract or language-based warnings signifying this situation were then delivered, either from the simulator or from the tablet, in order to discover the most effective location. Results showed that abstract cues, including audio, and cues delivered from the tablet improved handovers. This indicates the potential of moving simple but salient autonomous car warnings to where a gaming side task takes place.
Article Preview


Autonomous cars are becoming a more and more popular topic of research, although not without concerns from the public over the safety of this new technology (Kyriakidis, Happee, & Winter, 2014). To address such worries, there is careful examination of road accidents involving autonomous vehicles from technology providers (Google, 2015b). This shows the importance of safety while automation is becoming more robust. Car autonomy is a staged rather than binary process, with levels of autonomy increasing as driver involvement decreases (National Highway Traffic Safety Administration, 2013; SAE J3016 & J3016, 2014). Therefore, user interfaces are required that improve safety when driver involvement is reduced but still necessary. The handover, the point of transition of control from the car to the driver, and vice versa, is a critical part of this interaction. An effective warning mechanism for such a critical case is essential, as lack of clarity over who has control of the vehicle at a given moment can be catastrophic, e.g. (Politis, Brewster, & Pollick, 2015a).

In parallel, as vehicle automation increases, drivers are more likely to engage in tasks other than driving. Gaming is a popular activity that drivers are expected to engage in while the car is in autonomous mode, and is a topic of ongoing research, e.g. (Krome, Goddard, Greuter, Walz, & Gerlicher, 2015; Neubauer, Matthews, & Saxby, 2014). Due to the high level of concentration required by a game, a particularly demanding scenario would be attending to a critical handover while gaming. A critical handover often examined is an automation failure, since it happens unexpectedly, leaving little time to react (Gold, Damböck, Lorenz, & Bengler, 2013; Mok et al., 2015; Pfromm, Khan, Oppelt, Abendroth, & Bruder, 2015). Signifying handovers with multimodal warnings (Naujoks, Mai, & Neukum, 2014; Politis et al., 2015a), using varying message contents (Koo et al., 2014) and evaluating transition times (Gold et al., 2013; Christian Gold & Bengler, 2014) are important aspects of this critical case. However, there is no work on how critical handovers can be facilitated by multimodal warnings originating from the game area. In this study, we use an engaging tablet gaming task and test the time required to resume driving during an automation failure. Handover notifications are moved to the tablet and abstract versus language-based multimodal warnings are compared as alerts for this scenario, both being novel interventions.

Complete Article List

Search this Journal:
Volume 15: 1 Issue (2023)
Volume 14: 4 Issues (2022): 1 Released, 3 Forthcoming
Volume 13: 1 Issue (2021)
Volume 12: 3 Issues (2020)
Volume 11: 4 Issues (2019)
Volume 10: 4 Issues (2018)
Volume 9: 4 Issues (2017)
Volume 8: 4 Issues (2016)
Volume 7: 4 Issues (2015)
Volume 6: 4 Issues (2014)
Volume 5: 4 Issues (2013)
Volume 4: 4 Issues (2012)
Volume 3: 4 Issues (2011)
Volume 2: 4 Issues (2010)
Volume 1: 4 Issues (2009)
View Complete Journal Contents Listing