In large-scale immersive virtual reality (VR) environments, such as a CAVE, one of the most common problems is tracking the position of the user’s head while he or she is immersed in this environment to reflect perspective changes in the synthetic stereoscopic images. In this paper, the authors describe the theoretical foundations and engineering approach adopted in the development of an infrared-optical tracking system designed for large scale immersive Virtual Environments (VE) or Augmented Reality (AR) settings. The system is capable of tracking independent retro-reflective markers arranged in a 3D structure in real time, recovering all possible 6DOF. These artefacts can be adjusted to the user’s stereo glasses to track his or her head while immersed or used as a 3D input device for rich human-computer interaction (HCI). The hardware configuration consists of 4 shutter-synchronized cameras attached with band-pass infrared filters and illuminated by infrared array-emitters. Pilot lab results have shown a latency of 40 ms when simultaneously tracking the pose of two artefacts with 4 infrared markers, achieving a frame-rate of 24.80 fps and showing a mean accuracy of 0.93mm/0.51º and a mean precision of 0.19mm/0.04º, respectively, in overall translation/rotation, fulfilling the requirements initially defined.
TopIntroduction
During the last decade, Virtual and Augmented Reality technologies have became widely used in several scientific and industrial fields. Large immersive virtual environments like the CAVE (Cave Automatic Virtual Environment) (Cruz-Neira, 1992), can deliver to the users unique immersive virtual experiences with rich HCI modalities that other environments cannot achieve. In this context, through a collaboration of a large group of Portuguese and Brazilian entities namely, institutional (Portuguese Ministry of Science, Grândola City Hall), academic (ISCTE-IUL, IST, FCUL, PUC Rio), and industrial (SAPEC, Fundação Frederic Velge, Petrobrás, Microsoft), the first large scale immersive virtual environment installed in Portugal, “CaveHollowspace of Lousal” or “CaveH” in short (Dias, 2007), started its operation in the end of 2007. The CaveH of Lousal (situated in the south of Portugal, near Grândola) is part of an initiative of the National Agency for the Scientific and Technological Culture, in the framework of the Live Science Centres network. CaveH aims at bringing to the educational, scientific and industrial sectors in Portugal, the benefits of advanced technologies such as, immersive virtual reality, digital mock-up and real-size interactive simulation. Its physical configuration is a 4-sided cave assembling six projection planes (2 for the floor, 2 for the front, and 1 for each side) in a U topology with 5.6 m wide, 2.7 m height and 3.4 m in each side (Figure 1), giving a field of view of more than 180º, with a resolution of 8.2 mega pixel in stereoscopy for an audience of up to 12 persons (where one is being tracked). It is worth mentioning that each front and floor projection planes of the installation have 2 x 2 associated projectors (in a passive stereo set-up), with a blending zone that creates the impression of a single projection plane each.
Figure 1. CaveHollowspace of Lousal physical set-up
The range of critical industries where simulation and real-size digital mock-up observation are imperative, extends the CaveH applicability to several fields such as, entertainment, aerospace, natural resources exploration (oil, mining), industrial product design (automotive, architecture, aeronautics), therapy (phobia, stress), etc. In most of these applications, the interaction with the user is crucial to fulfill the application purpose.
A CAVE is an immersive virtual environment, which largely benefits from the ability of human brain to process the depth perception. However this awareness can be lost and the full immersion can be compromised, if the several 3D view frustums of the set-up (truncated pyramids usually associated to visualization volume of a virtual camera in rigorous perspective), and corresponding viewports, are not correctly adjusted to the main user’s head position, while he/she is in inside the physical space of the immersive space (Figure 2) during interactive sessions.
Figure 2. Dynamically adjustment of view frustums and viewpoints required to correctly display images on several projection planes, when the user moves from a) to b)