Welcome to the InfoSci Platform
How to Use Manual Labelers in the Evaluation of Lip Analysis Systems?

How to Use Manual Labelers in the Evaluation of Lip Analysis Systems?

Shafiq ur Réhman, Li Liu, Haibo Li
Copyright: © 2009 |Pages: 21
ISBN13: 9781605661865|ISBN10: 1605661864|ISBN13 Softcover: 9781616925338|EISBN13: 9781605661872
DOI: 10.4018/978-1-60566-186-5.ch008
Cite Chapter Cite Chapter

MLA

ur Réhman, Shafiq, et al. "How to Use Manual Labelers in the Evaluation of Lip Analysis Systems?." Visual Speech Recognition: Lip Segmentation and Mapping, edited by Alan Wee-Chung Liew and Shilin Wang, IGI Global, 2009, pp. 239-259. https://doi.org/10.4018/978-1-60566-186-5.ch008

APA

ur Réhman, S., Liu, L., & Li, H. (2009). How to Use Manual Labelers in the Evaluation of Lip Analysis Systems?. In A. Liew & S. Wang (Eds.), Visual Speech Recognition: Lip Segmentation and Mapping (pp. 239-259). IGI Global. https://doi.org/10.4018/978-1-60566-186-5.ch008

Chicago

ur Réhman, Shafiq, Li Liu, and Haibo Li. "How to Use Manual Labelers in the Evaluation of Lip Analysis Systems?." In Visual Speech Recognition: Lip Segmentation and Mapping, edited by Alan Wee-Chung Liew and Shilin Wang, 239-259. Hershey, PA: IGI Global, 2009. https://doi.org/10.4018/978-1-60566-186-5.ch008

Export Reference

Mendeley
Favorite

Abstract

The purpose of this chapter is not to describe any lip analysis algorithms but rather to discuss some of the issues involved in evaluating and calibrating labeled lip features from human operators. In the chapter we question the common practice in the field: using manual lip labels directly as the ground truth for the evaluation of lip analysis algorithms. Our empirical results using an Expectation-Maximization procedure show that subjective noise in manual labelers can be quite significant in terms of quantifying both human and algorithm extraction performance. To train and evaluate a lip analysis system one can measure the performance of human operators and infer the “ground truth” from the manual labelers, simultaneously.

Request Access

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.