Article Preview
TopIntroduction
This study is a series of experiments, which is part of the project funded by Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2016S1A2A2912583). The project “Effect of olfactory stimulation on extending concentration behaviour patterns in high school students” aimed to 1) investigate the effect of aromatherapy with essentials oils on students during the lessons at school 2) examine how presence of aromatherapy affects the quality of movement of the students and 3) application of computer vision techniques for data processing and analysis for comparison of movement behaviour with and without aromatherapy.
The project is based on a tightly coupled interaction between technical and pedagogical partners, to implement software modules that will be useful in detecting the difference in movement behaviour affected by aromatherapy. This paper summarizes two steps to test the feasibility of using computer vision techniques in a high school classroom. The first step was a pilot program focused on testing only 4 participants during class (on 2 days) followed by the second step which focused on the application of using these techniques for video data from 2 different classes. There were a total of 12 fifty-minute class periods for 3 high school subjects (social science, mathematics, and geography), spread out over 4 days (Class A without aroma during both days (control group) and the Class B without aroma on the first day and with aroma on the second day (experimental group). These preliminary studies allowed us to progress to a series of pilot tests using computer vision techniques applied to subjects during actual high school classes. A further aim is to expand the investigation to different approaches, which will be implemented in the framework of the Project “Effect of olfactory stimulation on extending concentration behaviour patterns in high school students”.
Education in Korea is very highly ranked among the OECD countries (Fair Reporters 2015), however it is also well known that the educational environment creates high levels of stress for the students, which can lead to various mental issues (Lee, Puig, et al., 2010; Lee, Puig, et al., 2013). In addition to educating, teachers at schools are expected to alert parents/guardians if any student has any emerging mental issues, shown by adverse classroom behaviour, that affects their learning (Lee, Hong, Espelage, 2010; Park, Schepp, et al., 2006). As computer vision techniques can detect differences in emotions and behaviours (Camurri, Canepa et al., 2016), we aimed to investigate if similar computer vision techniques, applied to video recordings of an actual classroom in high school, can detect differences in behaviour and movement characteristics of the students.
It is reported that understanding and quantifying human behaviour and movements are key in the early diagnosis of both mental and physical health problems (Beniczky, Polster et al., 2013; Bustamante-Bello, Ruiz-Soto, Ramírez-Mendoza, 2016; Kim, 2017). However, even though some wearable devices have been developed for early detection of diabetes, Parkinson, breast cancer (Cyrcadia Health ltd., USA), hypoglycemia (HealthPatch from VitalConnect, USA), heart disease (Polar, USA) many of these devices are invasive and relatively expensive. In this study, the experimental settings were based on non-intrusive data collection, i.e. through HD video camera recordings. For post processing of the recordings, we used OpenPose, a computer vision technique developed in Carnegie-Mellon University (Zha, Taniguchi et al., 2009; Ramakrishna, Munoz et al., 2014) for estimation of position of joints from 2D video data. Matlab was then used for processing the positional data and tracking of the subjects, and EyesWeb was used to extract certain movement features. Finally, we performed a statistical analysis on the extracted movement features in order to determine the difference between conditions with and without aromatherapy. This article describes the methods, results and discusses the limitations of using computer vision techniques for an automated student’s behaviour analysis.