ENT Endoscopic Surgery and Mixed Reality: Application Development and Integration

ENT Endoscopic Surgery and Mixed Reality: Application Development and Integration

Elmer Jeto Gomes Ataide (Otto-von-Guericke-Universität, Germany), Holger Fritzsche (Otto-von-Guericke Universität, Germany), Marco Filax (Otto-von-Guericke Universität, Germany), Dinesh Chittamuri (Otto-von-Guericke Universität, Germany), Lakshmi Sampath Potluri (Otto-von-Guericke Universität, Germany) and Michael Friebe (Otto-von-Guericke Universität, Germany)
Copyright: © 2020 |Pages: 13
DOI: 10.4018/978-1-7998-0326-3.ch002


Minimally invasive otorhinolaryngology surgery uses a system that consists of an endoscope, microscope, high-resolution display, and several surgical tools to perform procedures of the Ear, Nose and Throat (ENT) up to the upper Oesophagus. The complexity, and number of systems used, forces the surgeon to focus on multiple factors rather than exclusively on the procedure. This chapter focuses on the development of a system integrating the endoscopic feed with a Mixed Reality (MR) headset. For that, the visual data stream from an endoscopy system is integrated with an MR head-mounted device. An application was developed using Unity, Visual Studio, and Windows 10 SDK. The application also had the ability to access pre-operative images through its Graphical User Interface, and was integrated with the endoscopic feed wirelessly over a local area network. The application was tested in an educational abdominal phantom. The goal was to streamline the surgeon's focus more on the patient and to provide access to pre-operative images for in-procedure comparison at their fingertips.
Chapter Preview


Otorhinolaryngology or Ear Nose and Throat (ENT) Endoscopic surgery is a procedure that allows for the examination of the middle ear, nasal passage and openings to the sinus and the upper section of the oesophagus through the nasal and oral cavities. ENT Endoscopic surgery benefit the patient in terms of minimal invasiveness as only the natural orifices (Ear, nose and throat) are used. This causes less pain in patients and decreases the amount of time taken for them to heal from the surgical procedure (Sher 1986). ENT endoscopy is a form of image guided minimal invasive surgery for observation, manipulation and resection resulting in minimal trauma and faster patient recovery (Reuter 1999). As per iData Research Inc. in the year 2018 there were close to 23.8 million Endoscopic surgeries performed in the United States. An ENT Endoscopic surgery is performed with the aid of an endoscope, microscope, high resolution display and endoscopic tools. There is little integration between these instruments, which makes navigation cumbersome. The high-resolution display provides a rather limited field of view of the patient anatomy and its placement causes the surgeon to divert attention from the patient to the screen. (Nicolau 2011). Additionally, this involves the constant movement of the surgeon’s head and can cause discomfort as the number of procedures a day increases. Image guided surgeries often require the surgeon to have ready access to the patient’s pre-operative images. Currently this is achieved by accessing the images on an individual system located within the surgical suite. The introduction of an integrated system that reduces the stress on surgeons and enables the access of pre-operative images could prove to be beneficial as well as time-efficient. These are some of the pains or unmet clinical needs that were observed during surgery visits and clinical input. Mixed reality (MR) in the surgical suite could be a solution to address these issues. Mixed reality helps bridge the gap between the unmodelled real world and a virtual modelled environment (Fullum, 2009) (Figure 1). It allows for seamless integration of virtual objects into the real world. The virtual aspect has the capability to react to human movements and gestures (Rodrigues 2017).

Figure 1.

Simplified representation of a ‘Virtuality Continuum’ depicting the position of AR

(Milgram 1994)

Complete Chapter List

Search this Book: