POMDP Models for Assistive Technology

Introduction

A growing area of activity in health technology is support systems for older adults, possibly with cognitive or physical disabilities, who want to continue to live independently in their own homes i.e. age-in-place. Such systems are typically engineered for a certain task to provide guidance, assistance, or emergency response (Mihailidis & Fernie, 2002, LoPresti, Mihailidis, & Kirsch, 2004). However, this approach is labour intensive, and the resulting systems tend to have no capacity to adapt over time or to different users or tasks. In this chapter, we discuss an approach to this problem: a ubiquitous modeling technique that can adapt to users over time. The idea is to have a single model and learning technique that can be easily applied to different tasks, without the need to re-engineer the model.

A typical task requiring assistance consists of four principal elements. We discuss these elements here in the context of the handwashing task for cognitively disabled people, who typically require assistance from a human caregiver to wash their hands. An example of this is shown in Figure 1, which shows key frames from about 15 seconds of a video of a user washing their hands, assisted by a human caregiver. First, the task state is a characterisation of the high-level state of the user, and is related to the goals in the task. For example, handwashing can be described by task states that describe whether the hands are wet or dry, dirty, soapy or clean. In Figure 1, the user’s hands are dirty and wet at frame 1331, but become soapy and wet by frame 1745. Second, the behavior of the user is the course of action the user takes to change the task state. Common behaviors during handwashing may be things like rinsing hands or using soap, as in Figure 1. Third, the caregiver’s action is what the caregiver does to help the user through the task. During handwashing, these actions are typically verbal prompts or reminders such as the “I want you to use some soap now” in Figure 1. However, as we will show, actions can include more general dialogue iterations, calls to other response systems, or physical control of related systems. The fourth element, the users ability, is the cognitive state of the user, such as their level of responsiveness, attention, frustration with the system, and overall level of health. The user’s expeditious reaction to the prompt in Figure 1, for example, might give us an indication that they are responsive, and are attending to the prompting system. Over a longer time period, the user’s overall level of health may change. For example, their medical condition might take a turn for the worse, requiring attention from a professional. Such a change may be noticeable in their responses and behaviors.

Figure 1.

Example sequence in which a user is prompted to put soap on their hands

Our goal is then to design a model of the interactions between these four elements, and to optimize an automated caregiving strategy by maximising (over the actions) some notion of utility over the possible outcomes. The model must be able to deal with uncertainty in the effects of actions and in sensor measurements, it must be able to tailor to specific individuals and circumstances, it must be able to trade off various objective criteria (e.g., task completion, caregiver burden, user frustration and independence), and it must be relatively easy to specify. A partially observable Markov decision process (POMDP), a decision theoretic model which has recently received much attention in the AI community, fulfills these constraints.