Human-Centred Acoustic Detection for Smartphone Application in Healthcare

Introduction

Among the various sensing technologies, such as those based on sound, light, and electromagnetism, sound-based sensing has a unique position, particularly with the rapid development and widespread use of smartphones. On the one hand, it can preserve users' privacy without taking images or videos; on the other hand, it can use inherent sound components in the smartphone, such as speakers and microphones, to simplify the system operation. Among the widespread applications based on acoustic sensing, human-centred application, mainly via smartphones, is emerging as a vital direction, partly because of the need for assisted living by those living alone or living with difficulties, and easy access to mobile phones for healthcare aid. Because of their widespread use, smartphones have become a potential tool for people's daily life to support broad applications such as home care (Hori et al., 2004) and clinical monitoring (Qi et al., 2014b). Related research has been increased significantly recently to facilitate healthcare, such as Parkinson's patient analysis (Pepa et al., 2015), diagnosis of physical abnormalities (Rodriguez et al., 2015), hazard warning systems (Unni & Pati, 2018), AI-enabled healthcare system (Song et al., 2021) and intelligent COVID-19 diagnosis (Chen et al., 2022).

Looking back on the history of acoustic sensing, approximately seven stages, namely, obstacle detection, position detection of a single person, motion detection of a single person, motion detection with hybrid sensing, body segments detection, event motion detection, and detection of activities for multiple persons, are illustrated in Figure 1. As early as 2002, researchers started exploring the possibility of utilising an

Figure 1.

Development of human-centred acoustic detection

ultrasonic sensor for obstacle detection (Huang et al., 2002; Zeng, 2002). Owing to the limitations of sound components and sampling rate, currently at 44.1kHz, high-frequency ultrasound sensing techniques cannot be applied to smartphones. On the other hand, the frequency cannot be too low, which would interfere with the human voice. Usually, the frequency band between 17kHz and 22kHz is widely used for smartphone applications. In addition, due to the varying quality of the sound components in different smartphones, new challenges are raised in using traditional sound detection techniques.

In this chapter, we discuss from the vertical and horizontal directions to introduce human-centred acoustic sensing and analyse the latest development and healthcare application comprehensively. The vertical analysis contains an in-depth discussion of technological development and fundamentals in this field. The horizontal analysis provides a systematically parallel comparison and summary. An extended analysis of the breadth of research is conducted in this field, including the detected object differences, methods, hybrid acoustic sensing, advantages and disadvantages of human-centred acoustic detection, and the emerging problems to be solved. In addition, smartphone-based acoustic sensing with innovative healthcare-based applications is investigated and presented in this chapter, along with the derived artificial intelligence algorithms. The authors also predict the trend in technology developments and provide suggestions for future research.

Terminology Definition

In order to avoid confusion, the main terminologies used in this chapter are defined here, where the detected object is an essential classification criterion. For motion detection, single-person detection and multiple human detection are different branches. As shown in Figure 2, six attributes are used for performing the categorization. However, the research is not restricted to them. Static detection refers to the measurement at the static state where the position of a person is detected with a fixed behaviour. The behaviour and posture of a single person can be found in motion detection. Apart from that, events and activities are also essential features in human motion detection.

Figure 2.

List of attributes for detection