Understanding, Modeling and Exploiting User Emotions for Brain-Driven Interface Design: Application to an Adaptive-3D-Virtual-Environment

Understanding, Modeling and Exploiting User Emotions for Brain-Driven Interface Design: Application to an Adaptive-3D-Virtual-Environment

Valeria Carofiglio (Dipartimento di Informatica, Universita' di Bari, Bari, Italy) and Fabio Abbattista (Dipartimento di Informatica, Universita' di Bari, Bari, Italy)
Copyright: © 2014 |Pages: 21
DOI: 10.4018/ijpop.2014010101
OnDemand PDF Download:
List Price: $37.50


Innovative applications are often complex systems. In designing this kind of application, usability, perceived usefulness and appropriateness of adaptation are the three most commonly assessed variables. However, in order to obtain a more engaging overall user experience, a good designer should perform proper formative and summative usability tests, based on the user's emotional level, which becomes a user-centered evaluation activity. Moreover, traditional methods are not ideal, as information about the user's emotional state should be captured in an implicit and transparent manner, in order to be non-invasive and more effective. Brain Computer Interface has recently witnessed an explosion of systems for studying human emotion by the acquisition and processing of physiological signals. The authors view Adaptive Virtual Environments, as one of the most representative examples of innovative applications, and also as elicitors of a complex user emotion synthesis. Therefore, in this paper the authors propose a user-centered approach to the design and support of the user experience through an adaptive virtual environment, via brain-computer interface. Firstly, the authors focus on the design of an engaging overall experience for potential users, by exploiting their emotional level as a powerful engine in the interaction experience. Secondly, the author work to enhance the user experience by dynamically adapting the interaction to the user's emotional state, so that there will be a more immersive and satisfying interaction.
Article Preview


In the global race for more intuitive interfaces that must allow non-expert users to operate on increasingly complex technology, we explored 3D Virtual Environments (3DVEs). We focused on the role of emotions in the design and use of such interfaces, by developing a method to design affective interaction with 3DVEs. This is related to the problem of how to manage the user’s emotional level as a User-Centred Evaluation (UCE) activity. In this context, the efficiency of the emotion recognition method is the main factor affecting the research outcome.

Traditional methods, such as self-report or interviews, are not ideal: they rely either on sampling approaches or the user's perception of the environment. Also traditional methods for capturing the interaction experience in an unconscious and continuous approach (e.g. log experience) may be troublesome, as they do not collect subjective feedback from (potential) users.

We chose Brain-Computer Interface (BCIs, (van Gerven, 2009)) as a way to investigate the emotional activity of a subject beyond his conscious and controllable behavior (Bos, 2007; Choppin, 2000; Murugappan et al., 2008). By employing BCI we tried to overcome the cited weaknesses of traditional methods for assessing the user's (emotional) experience.

In this paper, the design of a 3D virtual environment (3DVE) is presented as a way to elicit and synthesize complex user emotions. A 3DVE is one of the most representative examples of an emotional adaptive virtual environment (EAVE). Because it is adaptive, the more the system knows about the user’s detailed emotions, the better it can react by guiding them along well-defined emotional and informative paths (Leite et al., 2010).

Concerning the evaluation of the selected environment, we manage the user’s emotional level by performing (i) formative usability tests, in order to guide the designer in organizing the user-system dialog, according to the potential emotional activity of users. This is linked to the possibility of implementing the opportune interface to collect additional information on the user’s needs and preferences; (ii) summative usability tests, in order to evaluate the user-system dialog, in terms of dynamic increase of the emotionally-driven interaction customization. This is linked to the possibility of using real-time acquisition of information about the emotional state of the user to adapt the characteristics of the interaction, to reach the intended emotional effects on each individual user better.

Complete Article List

Search this Journal:
Open Access Articles: Forthcoming
Volume 6: 2 Issues (2017): 1 Released, 1 Forthcoming
Volume 5: 1 Issue (2016)
Volume 4: 2 Issues (2015)
Volume 3: 2 Issues (2014)
Volume 2: 2 Issues (2012)
Volume 1: 2 Issues (2011)
View Complete Journal Contents Listing