Multimedia Sensory Cue Processing in the FIVIS Simulation Environment

Introduction

For almost all modern means of transportation (car, train, airplane) training simulators exist that provide realistic models of the vehicle’s dynamics, interiors and the outer environment. They allow for complex and potentially dangerous situations to be simulated under defined and safe laboratory conditions. For many years, these simulators have been successfully applied for driver training and education, and as a consequence have contributed to the overall safety in the respective fields. In general, simulators allow for more cost-effective training in comparison to real vehicles, especially for expensive aircrafts. Sometimes, the vehicles being simulated do not even exist at that point in time.

Unfortunately, there is no such commonly available simulation system for bicycles, although the number of bicycle accidents in Germany has been increasing against the common trend (according to the Federal Statistical Office). Hence, the objective of the FIVIS project at the Bonn-Rhein-Sieg University of Applied Sciences is to develop a bicycle simulator that is embedded into an immersive visualization environment. The immersive visualization environment provides visual input cues also to peripheral areas of the visual field of the trainee. Visualizing three-dimensional content in such an environment allows spectators to become part of the computer-generated world and experience depth and three-dimensional space almost like they are used to in the real world ((Herpers2008a) ((Herpers2008b) ((Herpers2009a). Moreover, the FIVIS environment is used for conducting perception research experiments within the context of multimedia sensory cue processing, all integrated in one coherent simulation framework.

In the FIVIS project, a real bicycle is equipped with sensors and actuators (see figure 1), which are connected to a simulation computer. It computes the bicycle dynamics, interactions with the virtual world and also graphically renders the three-dimensional immersive visualization of the environment. The computer-generated images are projected onto three seamlessly joined screens that surround the bicycle rider (see figure 1), using back projection with mirrors to make the system more compact. The screens cover a wide range of the visual field and therefore allow for a high degree of immersion. Using an optical infrared tracking system, the rider’s head and body parts are tracked in order to adjust the virtual camera view according to the most recent head position. Hand signs given by the rider are detected, evaluated, and might influence the further processing of the simulation. Optionally, the FIVIS simulation and visualization system can be extended by a hydraulic 6DOF motion platform ((Schulzyk2007), which allows for simulating several forces acting upon the bicycle (see figure 2).

Figure 1.

The FIVIS bicycle simulator running a simulation of an urban environment for traffic training. Elementary school children were trained within this environment to get prepared for their bicycle driver’s license examination

Figure 2.

The FIVIS bicycle simulator extended by a hydraulic 6DOF motion platform. An elementary school child is practicing the correct behavior in urban traffic scenarios within a virtual model of his hometown environment

Computation Of The Bicycle Simulation And Its Visualization

The task of the simulation software is to perform the actual simulation of the virtual world. Therefore, a flexible simulation model for objects such as the bicycle, buildings, traffic lights and other road users has been developed. Additionally, the software has to read data from the bicycle’s sensors and transfer it to the simulation model. Moreover, the immersive 3D visualization has to be computed in real-time.