The essence of pervasive computing lies in the creation of smart environments saturated with computing and communication capabilities, yet gracefully integrated with human users (inhabitants). Context Awareness is perhaps the most salient feature of such an intelligent computing environment. An inhabitant’s mobility and activities play a significant role in defining his contexts in and around the home. Although there exists optimal algorithm for location and activity tracking of a single inhabitant, the correlation and dependence between multiple inhabitants’ contexts within the same environment make the location and activity tracking more challenging. In this chapter, we discuss a cooperative reinforcement learning and a non-cooperative Nash H-learning approach for location-aware resource management in multi-inhabitant smart homes that attempts to minimize the joint location uncertainty of inhabitants. Experimental results demonstrate that the proposed framework is capable of adaptively controlling a smart environment, significantly reduces energy consumption and enhances the comfort of the inhabitants. We also present open problems in this area.
TopIntrodcution
Advances in smart devices, mobile wireless communications, sensor networks, pervasive computing, machine learning, middleware and agent technologies, and human computer interfaces have made the dream of smart pervasive environments a reality. According to Cook and Das (Cook & Das, 2004), a “smart environment” is one that is able to autonomously acquire and apply knowledge about its users (or inhabitants) and their surroundings, and adapt to the users’ behavior or preferences with the ultimate goal to improve their experience in that environment. The type of experience that individuals expect from an environment varies with the individual and the type of environment considered. This may include the safety of users, reduction of cost of maintaining the environment, optimization of resources (e.g., energy bills or communication bandwidth), task automation or the promotion of an intelligent independent living environment for health care services and wellness management. An important characteristic of such an intelligent, pervasive computing and communication paradigm lies in the autonomous and pro-active interaction of smart devices used for determining users’ important contexts such as current and near-future locations, activities, or vital signs. In this sense, context awareness is a key issue for enhancing users’ living experience during their daily interaction with smart devices including sensors and computer systems, as only a dynamic adaptation to the task at hand will make the computing environments just user friendly and supportive.
Context awareness is concerned with the situation a device or user is in, and with adapting applications to the current situation. But knowing the current context an application or system is used to and dynamically adapting to it only allows to construct reactive systems, which run after changes in their environment. To maximize usefulness and user support, systems should rather adapt in advance to a new situation and be prepared before they are actually used. This demands the development of proactive systems, i.e., systems which predict changes in their environment and act in advance.
To this end, we strive to develop methods to learn and predict future contexts, thus enabling systems to become proactive. Our goal is to provide applications not only with information about the current user contexts, but also with predictions of future contexts. When equipped with various sensors, a system should classify current situations and, based on those classes, learn the user behaviors and habits by deriving knowledge from historical data. The focus of our work is to forecast future user contexts lucidly by extrapolating the past and derive techniques that enable context prediction in pervasive systems. An instance of such an intelligent indoor environment is a smart home (Das, Cook, Bhattacharya, Heierman & Lin, 2002) that perceives the surroundings through sensors and acts on it with the help of actuators. In this environment, the most important contexts like user mobility and activity create an uncertainty of their locations and hence subsequent activities. In order to be cognizant of such contexts, the smart environment needs to minimize this uncertainty.
Contributions
In this chapter we summarize an information-theoretic learning and prediction framework that minimizes context uncertainty in multi-inhabitant smart homes. This framework is based on Reinforcement Learning (Cooperative) (Roy, N., Roy, A., Basu & Das, ICOST 2005; Roy, N., Roy, A., Basu & Das, MobiQuitous 2005) and Nash Q-learning (Non-cooperative) (Roy, N., Roy, A., & Das, PerCom 2006) algorithms. The novelty of our work lies in the development of learning algorithms that exploit the correlation of mobility patterns across multiple inhabitants and attempts to minimize their joint uncertainty. This is achieved with the help of a joint utility function of entropy. Optimization of this utility function asymptotically converges to Nash Equilibrium (Hu & Wellman, 2003). Minimizing this utility function of uncertainty helps in accurate learning and estimation of inhabitants' contexts (locations and associated activities). Thus, the system can control the operation of automated devices in an adaptive manner, thereby developing an amicable environment inside the home and providing sufficient comfort to the inhabitants. This also aids in context-aware resource management, for example, minimizing the energy usage and hence reduction of overall maintenance cost of the house. We conclude this book chapter with open directions for research. (Details of our approaches are presented in (Roy, 2008)).