Virtual Agent as a User Interface for Home Network System

Virtual Agent as a User Interface for Home Network System

Hiroyasu Horiuchi (Graduate School of System Informatics, Kobe University, Hyogo, Japan), Sachio Saiki (Graduate School of System Informatics, Kobe University, Hyogo, Japan), Shinsuke Matsumoto (Graduate School of System Informatics, Kobe University, Hyogo, Japan) and Masahide Namamura (Graduate School of System Informatics, Kobe University, Hyogo, Japan)
Copyright: © 2015 |Pages: 11
DOI: 10.4018/ijsi.2015040103
OnDemand PDF Download:
$30.00
List Price: $37.50

Abstract

In order to achieve intuitive and easy operations for home network system (HNS), the authors have previously proposed user interface with virtual agent (called HNS virtual agent user interface, HNS-VAUI). The HNS-VAUI was implemented with MMDAgent toolkit. A user can operate appliances and services interactively through dialog with a virtual agent in a screen. However, the previous prototype heavily depends on MMDAgent, which causes a tight coupling between HNS operations and agent behaviors, and poor capability of using external information.To cope with the problem, this paper proposes a service-oriented framework that allows the HNS-VAUI to provide richer interaction. Specifically, the authors decompose the tightly-coupled system into two separate services: MMC Service and MSM service. The MMC service concentrates on controlling detailed behaviors of a virtual agent, whereas the MSM service defines logic of HNS operations and dialog with the agent with richer state machines. The two services are loosely coupled to enable more flexible and sophisticated dialog in the HNS-VAUI. The proposed framework is implemented in a real HNS environment. The authors also conduct a case study with practical service scenarios, to demonstrate effectiveness of the proposed framework.
Article Preview

Introduction

Research and development of home network system (HNS, for short) is recently a hot topic in the area of ubiquitous computing (Igaki, Nakamura, & Matsumoto, 2004; Li, & Zhang, 2004). Orchestrating house-hold appliances (e.g., TVs, DVDs, speakers, air-conditioners, lights, curtains, windows, etc.) and sensors (e.g., temperature, humidity, brightness, human-presence, etc.) via the network, the HNS provides value-added services for home users.

In the HNS, improving the usability is an essential requirement. Since home users operate appliances and services every day, the user interface must be intuitive and easy to learn. Virtual agent user interface (VAUI, for short), which uses virtual agent as an interface, is a promising approach to fulfill the requirement (Ochs, Pelachaud, & Sadek, 2008; Cassell, 2000). Using voice recognition and synthesis technologies, the VAUI allows users to talk to an agent in a screen, to make intuitive operations for the system.

In our research group, we have been studying VAUI for HNS (say, HNS-VAUI) within an actual HNS environment (called CS27-HNS) (Nakamura, Tanaka, Igaki, Tamada, & Matsumoto, 2008). In our previous study (Soda et al., 2012), we have implemented a prototype system of HNS-VAUI, using MMDAgent Toolkit (Mmdagent.jp, 2009). Extending the finite state machine of MMDAgent so as to invoke external Web services, we successfully allowed the virtual agent to operate appliances and services of the HNS. The study showed that home users enjoyed operating the HNS through natural conversations with the agent.

However, the previous prototype system was able to provide quite limited interactions, since the extension was just made locally to the state machine. Thus, the prototype strongly depended on the design of MMDAgent Toolkit, which caused the lack of flexibility and expressivity. More specifically, operations of the HNS and detailed behaviors of the virtual agent were both described in the same state machine. The tight coupling of the operations and the behaviors made the system difficult to manage and extend. Also, the original state machine was so primitive that it could not use information from external Web services. Moreover, it was impossible to update and modify the state machine during run time.

To cope with the problem, we propose a new framework that allows HNS-VAUI to provide richer interactions. Exploiting the concept of service-oriented architecture (SOA) (Papazoglou and Georgakopoulos, 2003), we decompose the tightly-coupled system into two separate services: MMC Service and MSM service. The MMC service concentrates on controlling detailed behaviors of a virtual agent. Wrapping MMDAgent, the MMC service publishes a stateless Web service that simply commands a virtual agent to speak or move. On the other hand, the MSM service manages logic of HNS operations and dialog with the agent using richer state machines. The MSM service publishes a stateful Web service that invokes external Web services (including HNS and MMC services), based on a given input and the current state. The new state machine involved in the MSM service enables the usage of external information as well as the dynamic update of the state machine. Thus, the MMC and MSM services are loosely coupled to enable more flexible and sophisticated dialog in the HNS-VAUI.

In this paper, we implement the proposed framework within the CS27-HNS. The MMC and MSM services are both implemented by the Java Web service (Apache Axis2 on Tomcat) (Axis.apache.org, 2004) (Tomcat.apache.org, 1999). Also, Julius is used for the voice recognition engine (Julius.sourceforge.jp, 2002). Using the developed system, we also conduct a case study to demonstrate effectiveness of the proposed framework. In the case study, we implement three service scenarios: (a) turning on a TV, (b) asking today’s weather, and (c) recommendation of using fan. It is shown that the proposed framework allows HNS-VAUI to provide more advanced and flexible interactions.

Complete Article List

Search this Journal:
Reset
Open Access Articles: Forthcoming
Volume 5: 4 Issues (2017)
Volume 4: 4 Issues (2016)
Volume 3: 4 Issues (2015)
Volume 2: 4 Issues (2014)
Volume 1: 4 Issues (2013)
View Complete Journal Contents Listing