Article Preview
TopIntroduction
People with profound and multiple learning disabilities (PMLD) present severe learning difficulties that significantly affect their skills to communicate their feelings and requirements as well as to be independent. Their capacity to perceive and interact in common situations is significantly and severely diminished. They express their needs with non-symbolic behaviors, which are constituted by a number of unconventional reactions like body movements or vocalizations. The interaction with these patients is done by understanding these emotional non-symbolic comportments and it requires a constant support by a group of professional caregivers (Antón, Maña, Muñoz, & Koshutanski, 2011; Antón, Muñoz, Maña, & Koshutanski, 2012).
The goal of the INSENSION project is to design and develop an intelligent platform that enables persons with PMLD to enhance the quality of their life and increase their ability to self-determination with digital applications and services.
The platform aims to advance and integrate several specialized technologies to provide means of natural interaction of people affected by PMLD with others, using their own expressions.
Current trends in technologies, such as artificial intelligence (AI) and internet of things (IoT), enables the development of smart solutions for recognizing behavioral communication signals and collecting data about the context of these behaviors. The final goal is to find meaningful patterns to interpret this information such as specific intents of communications from an individual with PMLD. The non-symbolic actions that will be recognized and monitored by the platform include gestures, facial expressions, vocalization, and unobtrusive physiological parameters. The recognition of these specific reactions creates additional challenges. This is concerned with the particular character of these behaviors, due to the fact that each individual uses different signals to communicate certain needs. Hence, the same necessity can be expressed in several ways depending on the person. Thus, an individualized system is required in order to facilitate the understanding between caregivers and people with PMLD. For this reason, assuring a robust identification process becomes a crucial first stage in the whole system.
Figure 1. General scheme of the personalized intelligent platform for enabling interaction with digital services to individuals with PMLD
The INSENSION platform will be composed by a set of cameras and sensors that collect information from the environment of each person with PMLD. These data will be analyzed by diverse AI algorithms, which final aim is to automatically associate a specific signal of a person to its meaning, providing these results through a digital service to the people in charge of them.
Figure 1 shows the general structure of the INSENSION platform. Following this diagram, three stages can be distinguished in order to design the whole solution for automatically detecting, in a personalized way, non-symbolic signals in people with PMLD: i) person identification system, ii) behavior pattern recognition, i.e., gestures, facial expressions, vocalization, and physiological parameters, and iii) decision making within the interaction decision support service.
The person identification system is based on a facial recognition method which classifies who is the person monitored from a set of live video streams. Once the identity of the person is obtained, the second stage consists of recognizing non-symbolic behavioral signals automatically. Based on previous knowledge about each person, it is possible to associate expressions of each one with their meanings. Finally, the decision making stage computes this matching and obtains an interaction result, i.e., the meaning of a non-behavioral signal for each person affected by PMLD.