Article Preview
Top1. Introduction
The World Health Organization (WHO) estimates that around 1.3 billion people, or 16% of the global population, live with significant disabilities (World Health Organization [WHO], 2024). About 43 million of them are blind (International Agency for the Prevention of Blindness, 2020) and 430 million of them are deaf (World Health Organization [WHO], 2024). Facilities for the disabled are nowadays lively developed and extremely commonly deployed, bringing more convenience into their daily life, like blind sidewalk, braille button and sound traffic light. But physical difficulties aside, the mental health of the disabled has always been a significant problem that should not be ignored (Salako, I., 2017). Therefore, convenient facilities and equipment targeting their mortal life are also wealth to be attached importance to, like helping the blind to visit a museum or the deaf to attend a concert. Deafness and hearing loss are found worldwide in all regions and countries. Approximately one-fifth of the global population are living with hearing loss and 430 million of them are suffering disabling hearing loss (World Health Organization [WHO], 2024). It is expected that there could be over 700 million people with disabling hearing loss by 2050 (International Agency for the Prevention of Blindness, 2020). 34 million children have deafness or hearing loss and about 30% of people over 60 have hearing loss (World Health Organization [WHO], 2024). Besides, by the growth of age, the ability of hearing low magnitude or high frequency voice will degenerate significantly, (Oh, Lee, Park, Kim, Chung, Kim, & Yeo, 2014) which will also impact the ability to sense the music.
Employing medical technique is one of the solutions. Equipment like bone-anchored hearing devices (Baker, S., Centric, A., & Chennupati, S. K., 2015) can be deployed by some specific surgery procedures. But on one hand, implants have been proved not to be able to provide enough assist to the hearing impacted caused by all the reasons (Lupo, J. E., Biever, A., & Kelsall, D. C., 2020), on the other hand, the cost of implant surgery is not easy for every family to afford. Another family of solutions to help people with hearing loss to enjoy music is to transform the music into other communication modalities more familiar to deaf people, which shows more potential for higher availability and expansibility with the support of rapidly developing advancements.
Music information can be roughly separated into two properties, physical and perceptual. The physical part consists of amplitude, frequency spectrum and timbre, while the perceptual part represents the emotion it carries. To achieve the target of passing this information to the deaf, to extract these characteristics from the audio steam is the key. The most well-known physical feature representing system is pitch + magnitude + duration, which is utilized in the famous Musical Instrument Digital Interface (MIDI) filespec (MIDI Manufacturers Association., 2023). In the MER field, Russell (Russell, J. A., 1980) proposed circumplex model consists of two dimensions: valence and arousal, which is then adapted and improved by Thayer (Thayer, R., 1989), forming the widely used model of valence and arousal dimensions. Valence shows the pleasure level of the mood (displeasure to pleasure) and arousal shows the excited degree of it (clam to excited).