Kinect Applications in Healthcare

Introduction

Microsoft Kinect is one of the most popular inexpensive gadgets released in recent years. Kinect is equipped with a color camera, a depth camera, and a microphone array. The device allows users to interact with a computer via a natural user interface in terms of gestures or voice commands. So far, Microsoft has released three versions of Kinect sensors. The first two, typically referred to as Kinect for Xbox and Kinect for Windows, are rather similar except that Kinect for Windows is capable of doing near-mode tracking. Because both releases use the same depth sensing technology, they are collectively referred to as Kinect version 1, or Kinect v1 for short. The most recent Kinect sensor was released together with the new game console, Xbox One. Hence, it is some times referred to as Kinect for Xbox One. It uses a completely different depth sensing technology, which is based on time-of-flight measurement. As such, the new sensor is also referred to as Kinect version 2, or Kinect v2 for short. Kinect v2 has made drastic improvements over Kinect v1, including 1080p resolution for the color camera, and higher actual depth sensing resolution (the nominal resolution for the depth sensing resolution is actually lower than that for Kinect v1). Images of the two versions of Kinect sensors are shown in Figure 1.

Figure 1.

Two versions of Microsoft Kinect sensors. On the left is the Kinect v1. On the right is the Kinect v2.

The low-cost and the availability of Software Development Kit (SDK) for Microsoft Kinect \ has attracted many researchers to investigate its applications in many areas, particular in the healthcare realm. As the aging population rapidly grows in the United States, demands of healthcare services, especially physical therapy and rehabilitation services, have grown enormously in recent years. To meet the increasing demands and reduce the cost of services, providers are often looking for computers and other equipment that can assist them in providing services to patients in an affordable, convenient, and user-friendly environment. As a low-cost, portable, accurate, nonintrusive, and easily set up motion detecting sensor, Kinect enables researchers to develop computer-based vision control without using traditional input devices, e.g. mouse, keyboard, or joystick. This revolutionary technology makes it possible for Kinect to meet the challenge of providing high quality evaluations and interventions at an affordable price for healthcare services, as seen from the works surveyed in this chapter.

Background

Microsoft Kinect provides several streams of information to a software developer. The most common streams include: (1) A stream of 2D color image frames; (2) A stream of 3D depth image frames; and (3) A stream of 3D skeletal frames for at least one human subject in the view. A skeletal frame may contain the 3D position information for various number of joints. The availability of the skeletal frames has greatly facilitated Kinect application development because it frees the application developers from dealing with the complicated task of human pose estimation.

In Kinect v1, the depth of each pixel is calculated via a structured light method, which enables the use of a single infrared (IR) emitter and a single depth sensor to calculate the depth of each pixel. While this is a very clever scheme, the fidelity of the depth measurement is quite low because for the depth sensing to work perfectly, there has to be a visible unique pattern for each pixel. Because there has to be some space between two adjacent dots as part of the structured light and this space has to be wide enough for the depth sensor to distinguish, only about 1 in every 20 pixels has a true depth measurement in typical situations and the depths for other pixels must be interpolated. Hence, the depth sensing resolution is actually significantly below the nominal 640x480 for Kinect v1. The depth-sensing technology used in Kinect v2 is completely different and the depth is calculated based on time of flight. The depth of each pixel can be calculated based on the phase shift between the emitted light and the redirected light.

Due to the maturity of the Kinect technology, developers can now publish Kinect v2 applications to the Windows store. Kinect v2 SDK also provides oial support for Unity, which is a development platform for 3D games. Finally, Kinect v2 SDK provides a tool to develop gesture recognition based on machine learning.