Article Preview
TopIntroduction
Research and development of home network system (HNS, for short) is recently a hot topic in the area of ubiquitous computing (Igaki, Nakamura, & Matsumoto, 2004; Li, & Zhang, 2004). Orchestrating house-hold appliances (e.g., TVs, DVDs, speakers, air-conditioners, lights, curtains, windows, etc.) and sensors (e.g., temperature, humidity, brightness, human-presence, etc.) via the network, the HNS provides value-added services for home users.
In the HNS, improving the usability is an essential requirement. Since home users operate appliances and services every day, the user interface must be intuitive and easy to learn. Virtual agent user interface (VAUI, for short), which uses virtual agent as an interface, is a promising approach to fulfill the requirement (Ochs, Pelachaud, & Sadek, 2008; Cassell, 2000). Using voice recognition and synthesis technologies, the VAUI allows users to talk to an agent in a screen, to make intuitive operations for the system.
In our research group, we have been studying VAUI for HNS (say, HNS-VAUI) within an actual HNS environment (called CS27-HNS) (Nakamura, Tanaka, Igaki, Tamada, & Matsumoto, 2008). In our previous study (Soda et al., 2012), we have implemented a prototype system of HNS-VAUI, using MMDAgent Toolkit (Mmdagent.jp, 2009). Extending the finite state machine of MMDAgent so as to invoke external Web services, we successfully allowed the virtual agent to operate appliances and services of the HNS. The study showed that home users enjoyed operating the HNS through natural conversations with the agent.
However, the previous prototype system was able to provide quite limited interactions, since the extension was just made locally to the state machine. Thus, the prototype strongly depended on the design of MMDAgent Toolkit, which caused the lack of flexibility and expressivity. More specifically, operations of the HNS and detailed behaviors of the virtual agent were both described in the same state machine. The tight coupling of the operations and the behaviors made the system difficult to manage and extend. Also, the original state machine was so primitive that it could not use information from external Web services. Moreover, it was impossible to update and modify the state machine during run time.
To cope with the problem, we propose a new framework that allows HNS-VAUI to provide richer interactions. Exploiting the concept of service-oriented architecture (SOA) (Papazoglou and Georgakopoulos, 2003), we decompose the tightly-coupled system into two separate services: MMC Service and MSM service. The MMC service concentrates on controlling detailed behaviors of a virtual agent. Wrapping MMDAgent, the MMC service publishes a stateless Web service that simply commands a virtual agent to speak or move. On the other hand, the MSM service manages logic of HNS operations and dialog with the agent using richer state machines. The MSM service publishes a stateful Web service that invokes external Web services (including HNS and MMC services), based on a given input and the current state. The new state machine involved in the MSM service enables the usage of external information as well as the dynamic update of the state machine. Thus, the MMC and MSM services are loosely coupled to enable more flexible and sophisticated dialog in the HNS-VAUI.
In this paper, we implement the proposed framework within the CS27-HNS. The MMC and MSM services are both implemented by the Java Web service (Apache Axis2 on Tomcat) (Axis.apache.org, 2004) (Tomcat.apache.org, 1999). Also, Julius is used for the voice recognition engine (Julius.sourceforge.jp, 2002). Using the developed system, we also conduct a case study to demonstrate effectiveness of the proposed framework. In the case study, we implement three service scenarios: (a) turning on a TV, (b) asking today’s weather, and (c) recommendation of using fan. It is shown that the proposed framework allows HNS-VAUI to provide more advanced and flexible interactions.