Assistive Technology-Based Solution for Hearing Impairment Using Smartphones

Assistive Technology-Based Solution for Hearing Impairment Using Smartphones

Vasu Mehra, Dhiraj Pandey
Copyright: © 2022 |Pages: 17
DOI: 10.4018/IJSI.292024
OnDemand:
(Individual Articles)
Available
$37.50
No Current Special Offers
TOTAL SAVINGS: $37.50

Abstract

Developing a system for sign language recognition becomes essential for the deaf as well as a mute person. The recognition system acts as a translator between a disabled and an able person. This eliminates the hindrances in the exchange of ideas. Most of the existing systems are very poorly designed with limited support for the needs of their day to day facilities. The proposed system embedded with gesture recognition capability has been introduced here which extracts signs from a video sequence and displays them on screen. On the other hand, a speech to text as well as text to speech system is also introduced to further facilitate the grieved people. To get the best out of a human-computer relationship, the proposed solution consists of various cutting-edge technologies and Machine Learning based sign recognition models that have been trained by using TensorFlow and Keras library. The proposed architecture works better than several gesture recognition techniques like background elimination and conversion to HSV
Article Preview
Top

Introduction

In this fast-moving world of cut-throat competition, deaf-mute people often find themselves on the disadvantageous side of any competition. There is an immediate need to develop feasible and technologically advanced solutions for these people so that they can face the ferocious competition without their natural state becoming any hindrance. Communication has been a basic need for a human being to exchange ideas and give or retrieve information. Any communication process has three parts such as sender, channel, and recipient. Communication might look like a simple process but in reality, there is so much to it. One cannot ignore the emotions or cultural messages proper communication can transmit. All these factors make people attracted towards individuals who can convey a clear and concise message while keeping in mind all the factors (Shrawankar & Dixit, 2016). However, this process is not always so straightforward for everyone. The World Health Organization estimates around 2 billion people around the globe with some kind of disability. World Health Organization defines a disabled person who has “a problem in body function or structure, an activity limitation, has a difficulty in executing a task or action; with a participation restriction”.

Figure 1.

Disabled population by type of disability in India-Census, 2011

IJSI.292024.f01

Figure 1 shows the percent-wise types of disabilities in India. According to a survey conducted by the Ministry of Statistics and Programme Implementation, Government of India, in 2016 there are around 26.8 million deaf-mute people which constitute about 2.21% of India’s total population. And surprisingly for all the scientific developments, there was an increase in this number per year. World Health Organization attributes this dangerous trend to the tectonic shift of the population pyramid towards older age groups along with the introduction of various types of chronic diseases being developed into humans (Mishra et al., 2018).

Going with the same flow, an effort has been made to design and develop a solution that may help any deaf-mute person to bridge the gap and communicate normally with a person. An abled person often finds it difficult to converse because of his/her inability to understand sign language. This gives rise to the need for translators who can translate the messages to one another. But they come at a cost. Firstly, they cannot be available all the time, second, they can be monetarily costly, and third, they will breach privacy. Hence in a series of solutions, this work focuses upon building software interpreters which can save the day and eliminate the shortcomings mentioned above (Singha & Das, 2015).

Figure 2 shows various signs for alphabets and numerals for American Sign Language. However, sign language is known by a handful of people. And to make matters worse, there is no global standard for sign language. According to different languages and dialects, the sign language also changes for the region. A deaf-mute person might use alphabet books, common gestures, hand drawing, or make notes but they cannot be relied upon for official communication or when an emergency arises.

Hence in this series of solutions, the next is the speech analysis and conversion technology which can be brought to use for Text speech conversion as well as for Speech to text conversion. For the sake of simplicity, these have been implemented and hosted in a single application to do away with cumbersome and multiple apps. These work on Android’s ‘TTS’ and ‘speechtotext’ library respectively developed by the Google community. Various algorithms and how they are implemented differently in varying conditions have been discussed at lengths in this paper. These technological aids have been developed and discussed for deaf-mute people in hope that these make their lives simpler and better. This research work has further been divided into six sections, namely Related Work, Implementation, Results, Future Scope, and Conclusion.

Figure 2.

ASL Tactile Sign Language

IJSI.292024.f02

Complete Article List

Search this Journal:
Reset
Volume 12: 1 Issue (2024)
Volume 11: 1 Issue (2023)
Volume 10: 4 Issues (2022): 2 Released, 2 Forthcoming
Volume 9: 4 Issues (2021)
Volume 8: 4 Issues (2020)
Volume 7: 4 Issues (2019)
Volume 6: 4 Issues (2018)
Volume 5: 4 Issues (2017)
Volume 4: 4 Issues (2016)
Volume 3: 4 Issues (2015)
Volume 2: 4 Issues (2014)
Volume 1: 4 Issues (2013)
View Complete Journal Contents Listing