POMDP Models for Assistive Technology

POMDP Models for Assistive Technology

Jesse Hoey (University of Waterloo, Canada), Pascal Poupart (University of Waterloo, Canada), Craig Boutilier (University of Toronto, Canada) and Alex Mihailidis (University of Toronto, Canada)
DOI: 10.4018/978-1-4666-4422-9.ch007
OnDemand PDF Download:
No Current Special Offers


This chapter presents a general decision theoretic model of interactions between users and cognitive assistive technologies for various tasks of importance to the elderly population. The model is a partially observable Markov decision process (POMDP) whose goal is to work in conjunction with a user towards the completion of a given activity or task. This requires the model to monitor and assist the user, to maintain indicators of overall user health, and to adapt to changes. The key strengths of the POMDP model are that it is able to deal with uncertainty, it is easy to specify, it can be applied to different tasks with little modification, and it is able to learn and adapt to changing tasks and situations. This chapter describes the model, gives a general learning method which enables the model to be learned from partially labeled data, and shows how the model can be applied within our research program on technologies for wellness. In particular, we show how the model is used in four tasks: assisted handwashing, stroke rehabilitation, health and safety monitoring, and wheelchair mobility. The first two have been fully implemented and tested, and results are summarized. The second two are meant to demonstrate how the POMDP can be applied across a wide variety of domains, but do not have specific implementations or results. The chapter gives an overview of ongoing work into each of these areas, and discusses future directions.
Chapter Preview


A growing area of activity in health technology is support systems for older adults, possibly with cognitive or physical disabilities, who want to continue to live independently in their own homes i.e. age-in-place. Such systems are typically engineered for a certain task to provide guidance, assistance, or emergency response (Mihailidis & Fernie, 2002, LoPresti, Mihailidis, & Kirsch, 2004). However, this approach is labour intensive, and the resulting systems tend to have no capacity to adapt over time or to different users or tasks. In this chapter, we discuss an approach to this problem: a ubiquitous modeling technique that can adapt to users over time. The idea is to have a single model and learning technique that can be easily applied to different tasks, without the need to re-engineer the model.

A typical task requiring assistance consists of four principal elements. We discuss these elements here in the context of the handwashing task for cognitively disabled people, who typically require assistance from a human caregiver to wash their hands. An example of this is shown in Figure 1, which shows key frames from about 15 seconds of a video of a user washing their hands, assisted by a human caregiver. First, the task state is a characterisation of the high-level state of the user, and is related to the goals in the task. For example, handwashing can be described by task states that describe whether the hands are wet or dry, dirty, soapy or clean. In Figure 1, the user’s hands are dirty and wet at frame 1331, but become soapy and wet by frame 1745. Second, the behavior of the user is the course of action the user takes to change the task state. Common behaviors during handwashing may be things like rinsing hands or using soap, as in Figure 1. Third, the caregiver’s action is what the caregiver does to help the user through the task. During handwashing, these actions are typically verbal prompts or reminders such as the I want you to use some soap now in Figure 1. However, as we will show, actions can include more general dialogue iterations, calls to other response systems, or physical control of related systems. The fourth element, the user’s ability, is the cognitive state of the user, such as their level of responsiveness, attention, frustration with the system, and overall level of health. The user’s expeditious reaction to the prompt in Figure 1, for example, might give us an indication that they are responsive, and are attending to the prompting system. Over a longer time period, the user’s overall level of health may change. For example, their medical condition might take a turn for the worse, requiring attention from a professional. Such a change may be noticeable in their responses and behaviors.

Figure 1.

Example sequence in which a user is prompted to put soap on their hands


Our goal is then to design a model of the interactions between these four elements, and to optimize an automated caregiving strategy by maximising (over the actions) some notion of utility over the possible outcomes. The model must be able to deal with uncertainty in the effects of actions and in sensor measurements, it must be able to tailor to specific individuals and circumstances, it must be able to trade off various objective criteria (e.g., task completion, caregiver burden, user frustration and independence), and it must be relatively easy to specify. A partially observable Markov decision process (POMDP), a decision theoretic model which has recently received much attention in the AI community, fulfills these constraints.

Complete Chapter List

Search this Book: