Abstract
This chapter presents an overview of the design of an interactive medical/biological training environment using a multi-modal user interface. We describe the software architecture required to develop such environment. Then we introduce the physics-based models of the objects interacting in the virtual scenes. We discuss the implementation of the dexterity enhancing training tasks combined with the associated definitions of metrics which can be used as a part of score keeping operation. A virtual mentoring agent was used throughout the training tasks for guidance in terms of multi-modal feedback including graphics, haptic and audio feedback cues. A fuzzy logic based method was used to evaluate and compare the performance metrics of the trainee in relationship to both novice and expert user.
TopIntroduction
This chapter presents and demonstrates the design and development of a virtual environment that can be used as an interactive gaming platform for certain biological and medical health delivery tasks which require the use of hand-held tools. Game-based interactive training environments have emerged over the past decade as a potential tool for training surgeons in various procedures in both invasive and minimally invasive surgeries. In addition, as a part of earlier training and education curriculums, such virtual interactive environments have been gaining acceptance as an effective hands-on approach for teaching various biology related subjects. These environments can be used to train subjects in mastering various hands-eye coordination tasks to more advanced tissue and organ manipulation such as dissections. The interactive gaming environments consist of deformable objects where the trainee (player) through a multimodal interface can interact with the environment using the graphical objects. Each training task has associated metrics for performance measures to classify player’s performance.
Design and development of a virtual environment that can be used in the context of gaming (serious games) to enhance dexterity skills in various manual tool handling tasks related to various biological and medical sciences is the main motivation of this chapter. The notion of developing a training environment to enhance the manual dexterity of users has been one of the intriguing topics for research and development in various disciplines. Enhancing such training environment with score keeping and performance evaluation has been one of the main challenges. This is even more pronounce when the user interacts with the virtual environment through multi-modal user interface devices. One of the earliest training environment for enhancing the manual dexterity of the user is in nuclear waste management. For example, a system for automating the handling of the nuclear waste using automation and robotics is proposed by (Seward, 2005). It was also concluded by various nuclear training agencies (NEC report, 2011) that one of the key objectives of training environment must be for operators who need to remotely manipulate special long toolings attached to a the end of the robotic manipulator. Similarly, a system for supporting spatial awareness using an intelligent tutoring for training astronauts involve in telemanipulation is proposed by (Roy, 2004), (Belghith, 2012). The proposed system includes various performance measures and devices for user interface which can guide the player at various performance levels.
For the past two decades, both academic institutions and industrial developers have contributed to the new design and improvement of virtual training environment associated with the medical and surgical education (Downes,1998; Payandeh, 2002; Lap Mentor, 2014; Virtual lab, 2014; Ribaupierre, 2014). This new interactive paradigm is referred to as Serious Games. These are simulations of physical word models that can be used to train users in addressing a problem. They can also be designed to have some entertaining factors but they are mainly designed for education and training. Similar to the utilization of real physical objects (e.g. animals or cadavers), as part of enhancing various skills in manual dexterity and/or hand-eye coordination, virtual environment can also offer similar benefits. They can be designed to increase the sense of presence of the user by including both sense of (haptic) touch feedback and simulation and visualization of realistic behavior of virtual objects. In addition, by modeling and computing complex behavior of objects during the interaction tasks (such as cutting or suturing) it is possible to present the user with the increased sense of realism and acceptance. Dexterity training environment can also be designed in order to offer the user mentoring and guidance in the form of force feedback, graphical cues and audio feedback. Combined with the various multi-modal scores to measure and asses the performance of the user, such virtual training environment can eventually be a natural complement or even a replacement to the physical training set-up. Such replacement needs to be further validated for any integration into the exiting teaching curricula (Graafland 2012).
Key Terms in this Chapter
Haptic Rendering: The computational model which allows the creation of reaction forces between the virtual tool being manipulated by the user and the physics-based object. Such computed reaction forces are then send back to the haptic device for creation of sense of touch at the hand of the user. In case of virtual deformable object, such reaction forces on the object may also cause the object to deform or tear apart.
Multimodal Interface: Ability for the user to interact with the virtual environment through various sensing modality such as sense of touch, hearing (audio) and visual (graphics). This type of interface is accomplished with various computer supported external devices.
Haptic Feedback: The ability to create a sense of touch at the hand of the user while interacting with the virtual environment. This is usually done through control of electromechanical devices which are manipulated by the user.
Remote Medical or Biological Manipulation Training: Surgical and biological tasks that require the user to manipulate an environment through usage of tools are referred to as the remote manipulation. In such training tasks, it is required for the user to gain demanding remote dexterity skills or tedious eye-hand coordination expertise.
Virtual Fixtures: Guidance of the user while manipulating virtual objects can be facilitated by creating artificial constraints in the virtual scene. These constraint surfaces can be activated in order to create various multimodal cues. These cues can be in the form of the guiding or reactive forces, graphical images or auditory output.
User Performance Evaluation: Design of serious games requires evaluation and guidance of the user both during the gaming session and after the end of a session. During the gaming, the performance can be communicated in terms of number of successes or failures during a procedure. After a session, various metrics which are more representative of a given session can be compiled and compared in relation to the other players or expert player of a given gaming session that can be used to characterized the performance of the user.
Physics-Based Deformable Objects: In order for the graphical object to have physical attributes in the virtual environment, computational models which can represent the physical of the real object are included in their simulation models.