Mobile Technology of Learning and Communication for Students With Disabilities

Mobile Technology of Learning and Communication for Students With Disabilities

Sayaka Matsumoto (Otsuma Women's University, Japan), Kuniomi Shibata (Tsuda University, Japan) and Akira Hattori (Komazawa University, Japan)
DOI: 10.4018/978-1-7998-1400-9.ch012


This chapter presents a mobile application to support children and students with hearing difficulties and/or developmental disorders in interacting, understanding, and preparing for learning in their daily, school, and social lives. The authors think that a learning and communication support technique using a combination of characters, pictograms, and photographs is effective for people with disabilities. The application recognizes words spoken by their communication partners by using a speech recognition system and arranges the pictures corresponding to the words in a sequence. The results of two demonstration experiments show that PICTONGUE promotes mutual understanding between students who have hearing difficulties and/or developmental disorders by utilizing pictograms and photographs in combination with speech. The authors suggest that PICTONGUE supports visualization and feedback for planning and learning as a new way of communication for children and students with hearing difficulties and developmental disorders.
Chapter Preview


Even though differently abled people can often think very clearly, they experience difficulties in expressing their thoughts and intentions, with verbal and nonverbal communication being a major challenge. Gaining an understanding by listening to a conversation is difficult for people with hearing disabilities. Moreover, people with developmental disorders have difficulty in grasping the context of a conversation. Even planning daily actions can be challenging. For instance, the following thought process is considered “natural” for most people:

“I need to go to school today. Therefore, I must wash my face by 7:30 am and finish eating my breakfast by 8:00 am.”

However, people with disabilities often cannot think like this. When an unplanned incident occurs, panic overwhelms their faculties, and they dread the prospect of taking subsequent action. Such obstacles hinder their fundamental day-to-day actions such as learning and communication and interfere with their analytical skills and execution capabilities, often resulting in responses inferior to their actual capabilities. These factors have a negative influence on their self-confidence and interpersonal relationships and affect their self-esteem in the long term; they create a downward spiral in personal development, lowering the quality of life for differently abled people and simultaneously inconvenience the support staff and people around them. If these problems can be solved, people with disabilities will be equipped to be much more successful in their pursuits.

So far, the authors have investigated problem creation as a topic for discussion in relation to a person with disabilities. Meanwhile, mobile technology has revolutionized our lives. Most smartphone users use applications to support communication and a majority of them are text-based. Because it is difficult for us to express our feelings solely via text, the use of decorated letters and photographs has already been implemented. Various tools for communication support have been developed, as applications can currently be developed by anyone, even on a small budget.

As the mobile phone is a useful tool for people with disabilities, a basic application that targets this group of people was developed.


Focus Of The Chapter

The authors propose an application that supports visualization and feedback for learning, planning, and communication on a mobile terminal. Through experimentation, the authors examine the effectiveness of the application in the life of a differently abled person. The authors had developed a prototype of this application prior to the current research and had designed the interface specifically to support the communication and scheduling of actions for a person with disabilities. The questions addressed by the authors are as follows: 1) What kinds of revision are necessary to develop this prototype as an application suitable for use in real life? 2) What effects are palpable for a user of this application regarding his communication or activities? 3) In which situations does a user experience a significant effect of this application (Shibata et al. 2018)? The authors collected the data necessary to analyze these questions. The participants in this study were students with extensive knowledge about the lives of people with disabilities, and they offered detailed feedback about the application. This paper analyzes their responses to ensure that the necessary improvements are clearly understood and discusses the aspects of life in which a differently abled person might find the application useful.

Key Terms in this Chapter

Developmental Disorders: The disorders of psychological development which comprise learning disorders, autism spectrum disorders and other neurodevelopmental disorders.

Visualization: Express a language with a picture and a photograph.

Hearing Difficulties: The problems of hearing system and related structures and functions. It is known as deaf, hearing loss, or heard of hearing.

Communication Support: Support conversations depending on the characteristic of people.

Mobile App: Applications of smartphone and tablet.

Speech Recognition: The function to recognize a voice and to convert into letters.

Learning Support: The systems and services for students which aims to acquire knowledge and promote understanding.

Complete Chapter List

Search this Book: