In the previous chapters of the book, you will have seen multiple applications for using (and the benefits of using) a gaze tracker. In this chapter, you will be given more insight into how an eye tracker operates. Not only can this aid in understanding the eye tracker better, it also gives important information about how future applications might improve on current ones, by using more of the information available from the eye tracker: as we shall see, an eye tracker can often provide you with more information than just coordinates on a screen. This chapter gives an overview of the components of an eye tracker and introduces basics of gaze modelling. It helps in understanding the following chapters which each provide some details of how to build an eye tracker. This section has technical content, but it is our hope that also readers not particularly interested in the details of eye and gaze trackers will gain some useful insights.
TopIntroduction
In this chapter, you will be given more insight into how an eye tracker operates. This section has technical content, but it is our hope that also readers not particularly interested in the details of eye and gaze trackers will gain some useful insights. Each chapter provides some details of how to build an eye tracker, but for a comprehensive review of different techniques for eye and gaze tracking we refer the reader to the work of Hansen and Ji (2010).
A person’s gaze direction is determined by eyeball position and orientation. A person can change his or her gaze direction by rotating the eyeball (and consequently also the pupil) while keeping the head stationary. Similarly, a person can change gaze direction by moving the head while keeping the eye stationary relative to the head. Usually a person moves the head to a comfortable position before orienting the eye. Head pose therefore determines the coarse-scale gaze direction while the eyeball orientation determines the local and detailed gaze direction Hansen and Ji (2010).
A gaze tracker is a device for detecting and tracking eye movements and may provide information about the point of regard (PoR) or gaze direction. The PoR is the intersection of the gaze direction with an object in space, such as the screen. While eye trackers vary in the technologies they employ, in this chapter we will focus only on eye detection and gaze tracking in video-based eye trackers (a.k.a. video-oculography).
A general overview of the components of eye and gaze trackers is given in Figure 1. Eye and gaze tracking systems obtain information from one or more cameras (image data). The eye location in the image is detected and is either used directly in the application or subsequently tracked over frames. The gaze direction or PoR can be estimated on the basis of the information obtained from the eye region and possibly, but not necessarily, through head pose data. This information is then used by gaze-based or gaze-attenuated applications in, for example, positioning of a mouse pointer for gaze-driven control.
Figure 1. Components of video-based eye detection and gaze tracking. Adapted from the work of Hansen & Ji (2010)
In addition to choice of the most appropriate hardware, there are two main considerations for eye and gaze tracking – namely, eye localisation in the image and gaze estimation. For eye detection there are three sub-tasks to consider. One is to detect the presence of eyes in the image, another is to interpret eye positions in the images accurately, and the third (for video images) is to track the detected eyes from frame to frame. The eye position is commonly measured according to the pupil or iris centre. The gaze estimation method uses the information obtained in the images of the eyes to estimate and track where a person is looking (e.g., on the screen). In the subsequent discussion, we will use the terms ‘eye detection’ and ‘gaze tracking’ to differentiate between the components, where eye detection represents eye localisation in the image while gaze tracking means estimating gaze paths. Calibration is an important issue when one is using a gaze tracker and is performed by having the user look at a set of on-screen targets. The purpose of calibration is to infer the parameters needed to estimate gaze. The main reason for having calibration is that our eyes are inherently different and the parameters for each person need to be determined if one is to obtain reliable results from the system. Another reason is that information about the geometric relationships of hardware components needs to be encoded for gaze estimation. In some gaze estimation methods, most of the parameters for encoding geometric relationships are known prior to use. This is the most likely situation in commercial gaze trackers. The producers have spent time and money to fix the locations of the hardware such that their geometric relationships are known. If these parameters are known, then fewer parameters need to be inferred by the calibration process. However, the number of calibration targets may change on account of the actual gaze estimation method employed and for robustness considerations. In cases where the locations of the hardware components are unknown prior to use, more calibration targets usually are required. However, although such solutions often offer more flexibility in terms of moving hardware components around, they are often less accurate.