Virtual Reality and Augmented Reality Head-Mounted Displays (HMDs) have been emerging in the last years. These technologies sound like the new hot topic for the next years. Head-Mounted Displays have been developed for many different purposes. Users have the opportunity to enjoy these technologies for entertainment, work tasks, and many other daily activities. Despite the recent release of many AR and VR HMDs, two major problems are hindering the AR HMDs from reaching the mainstream market: the extremely high costs and the user experience issues. In order to minimize these problems, we have developed an AR HMD prototype based on a smartphone and on other low-cost materials. The prototype is capable of running Eye Tracking algorithms, which can be used to improve user interaction and user experience. To assess our AR HMD prototype, we choose a state-of-the-art method for eye center location found in the literature and evaluate its real-time performance in different development boards.
TopIntroduction
Head-Mounted Displays (HMDs) that provide Virtual Reality (VR) and Augmented Reality (AR) capabilities have been emerging in the last years. These technologies sound like the new hot topic for the next years. Users have had the opportunity to enjoy these technologies for entertainment, work tasks, retrieving health and body information, and other daily activities.
For years, engineers and researchers have been developing technologies to build the Head-Mounted Displays for many different purposes. Manufacturers are seeking to design hardware and software that improve the user experience and the user immersion in augmented and virtual worlds, in order to increase the user adoption and transform these devices into products of the mainstream market.
Despite the recent release of many AR and VR HMDs, two major problems are hindering the AR HMDs from reaching the mainstream market: the extremely high costs (from US$ 800 to US$ 3000 (Mirza & Sarayeddine, 2015), (Microsoft, 2016)) and the user experience issues such as the vergence-accommodation conflict (Kramida, 2016).
In order to minimize these problems, we have developed a simple AR HMD prototype based on a smartphone and on other low-cost materials, such as a beam splitter, a webcam (pointed to the user's eye, in order to perform the eye center location task) and a development board capable of running digital image processing algorithms.
The smartphone generates the 3-D (stereoscopic) virtual objects images and displays these images into the beam splitter. The beam splitter combines the virtual objects, formed by the smartphone, with the real ones, which are in the real environment. Therefore, the smartphone and the beam splitter are responsible for providing the Augmented Reality visualization to the user.
The prototype is also capable of running an eye center location algorithm, which can be used to improve the user experience. The eye center location information can be used to correct the right and left images' position of the virtual object in the stereoscopic view generated by the smartphone application. The correction of the image position using this approach guarantees the successful 3-D visualization of the virtual object independently of the user’s gaze, improving the user experience.
Furthermore, the prototype was built thinking on its application in industrial environment scenario. General AR HMDs are used in several industrial applications. Thus, the prototype was developed in such a way that it can be coupled to a safety helmet, which is in accordance with the industrial environment.
We implemented the state-of-the-art algorithm proposed by Valenti and Gevers (2012), which performs the eye center location task in low resolution images. We had to adapt that algorithm to work with the HMD setup since the original version did not perform well in this scenario. A sequential version and a parallel version of the algorithm were developed. Both versions of the algorithm were evaluated in different embedded platforms. The results show that our implementation of the algorithm is in accordance with the system requirements, but it still needs some adjustments.
This work is based on an earlier work: Towards a Low-Cost Augmented Reality Head-Mounted Display with Real-Time Eye Center Location Capability (D’Angelo, Delabrida, Oliveira, & Loureiro, 2016), in Proceedings of the 6th Brazilian Symposium on Computing System Engineering. ©IEEE, 2016.
The main contributions of this work are:
- •
An overview of the main concepts of HMD development.
- •
A review of methods for HMD development which were found in the literature.
- •
To introduce a low-cost AR HMD prototype with eye center location capability.
- •
To perform an evaluation of different hardware platforms for the eye center location algorithm.