Multimedia Sensory Cue Processing in the FIVIS Simulation Environment

Multimedia Sensory Cue Processing in the FIVIS Simulation Environment

Rainer Herpers (Bonn-Rhein-Sieg University of Applied Sciences, Germany; York University, Canada, & University of New Brunswick, Canada), David Scherfgen (Bonn-Rhein-Sieg University of Applied Sciences, Germany), Michael Kutz (Bonn-Rhein-Sieg University of Applied Sciences, Germany), Jens Bongartz (RheinAhrCampus Koblenz University of Applied Sciences, Germany), Ulrich Hartmann (RheinAhrCampus Koblenz University of Applied Sciences, Germany), Oliver Schulzyk (RheinAhrCampus Koblenz University of Applied Sciences, Germany), Sandra Boronas (Bonn-Rhein-Sieg University of Applied Sciences, Germany), Timur Saitov (Bonn-Rhein-Sieg University of Applied Sciences, Germany), Holger Steiner (Bonn-Rhein-Sieg University of Applied Sciences, Germany & Institute for Occupational Safety and Health of the German Social Accident Insurance, Germany) and Dietmar Reinert (Bonn-Rhein-Sieg University of Applied Sciences, Germany &Institute for Occupational Safety and Health of the German Social Accident Insurance, Germany)
DOI: 10.4018/978-1-60960-821-7.ch011
OnDemand PDF Download:
No Current Special Offers


The FIVIS simulator system addresses the classical visual and acoustical cues as well as vestibular and further physiological cues. Sensory feedback from skin, muscles, and joints are integrated within this virtual reality visualization environment. By doing this it allows for simulating otherwise dangerous traffic situations in a controlled laboratory environment. The system has been successfully applied for road safety education applications of school children. In further research studies it is applied to perform multimedia perception experiments. It has been shown, that visual cues dominate by far the perception of visual depth in the majority of applications but the quality of depth perception might depend on the availability of other sensory information. This however, needs to be investigated in more detail in the future.
Chapter Preview


For almost all modern means of transportation (car, train, airplane) training simulators exist that provide realistic models of the vehicle’s dynamics, interiors and the outer environment. They allow for complex and potentially dangerous situations to be simulated under defined and safe laboratory conditions. For many years, these simulators have been successfully applied for driver training and education, and as a consequence have contributed to the overall safety in the respective fields. In general, simulators allow for more cost-effective training in comparison to real vehicles, especially for expensive aircrafts. Sometimes, the vehicles being simulated do not even exist at that point in time.

Unfortunately, there is no such commonly available simulation system for bicycles, although the number of bicycle accidents in Germany has been increasing against the common trend (according to the Federal Statistical Office). Hence, the objective of the FIVIS project at the Bonn-Rhein-Sieg University of Applied Sciences is to develop a bicycle simulator that is embedded into an immersive visualization environment. The immersive visualization environment provides visual input cues also to peripheral areas of the visual field of the trainee. Visualizing three-dimensional content in such an environment allows spectators to become part of the computer-generated world and experience depth and three-dimensional space almost like they are used to in the real world ((Herpers2008a) ((Herpers2008b) ((Herpers2009a). Moreover, the FIVIS environment is used for conducting perception research experiments within the context of multimedia sensory cue processing, all integrated in one coherent simulation framework.

In the FIVIS project, a real bicycle is equipped with sensors and actuators (see figure 1), which are connected to a simulation computer. It computes the bicycle dynamics, interactions with the virtual world and also graphically renders the three-dimensional immersive visualization of the environment. The computer-generated images are projected onto three seamlessly joined screens that surround the bicycle rider (see figure 1), using back projection with mirrors to make the system more compact. The screens cover a wide range of the visual field and therefore allow for a high degree of immersion. Using an optical infrared tracking system, the rider’s head and body parts are tracked in order to adjust the virtual camera view according to the most recent head position. Hand signs given by the rider are detected, evaluated, and might influence the further processing of the simulation. Optionally, the FIVIS simulation and visualization system can be extended by a hydraulic 6DOF motion platform ((Schulzyk2007), which allows for simulating several forces acting upon the bicycle (see figure 2).

Figure 1.

The FIVIS bicycle simulator running a simulation of an urban environment for traffic training. Elementary school children were trained within this environment to get prepared for their bicycle driver’s license examination

Figure 2.

The FIVIS bicycle simulator extended by a hydraulic 6DOF motion platform. An elementary school child is practicing the correct behavior in urban traffic scenarios within a virtual model of his hometown environment


Computation Of The Bicycle Simulation And Its Visualization

The task of the simulation software is to perform the actual simulation of the virtual world. Therefore, a flexible simulation model for objects such as the bicycle, buildings, traffic lights and other road users has been developed. Additionally, the software has to read data from the bicycle’s sensors and transfer it to the simulation model. Moreover, the immersive 3D visualization has to be computed in real-time.

Complete Chapter List

Search this Book: