The overall coverage of the chapter is about moving face recognition out of the comfort zone and dramatically improving the current performance of existing biometric tools by fusing the rich spatial, temporal, and contextual information available from the multiple views made available by video (rather than still images) in the wild and operational real-world problems. Instead of relying on a “single best frame approach,” one must confront uncontrolled settings by exploiting all available imagery to allow the addition of new evidence, graceful degradation, and re-identification. Uncontrolled settings are all-encompassing and include Aging-Pose, Illumination, and Expression (A-PIE), denial and deception characteristic of incomplete and uncertain information, uncooperative users, and unconstrained data collection, scenarios, and sensors. The challenges are many: most important among them lack of persistence for biometric data, adversarial biometrics, open rather than closed set recognition, covariate shift, cross-dataset generalization, alignment and registration, interoperability, scalability, and last but not least, the deployment of full-fledged biometrics that include detection, authentication, informative sampling, and tracking. The overall recommendations are synergetic and should consider for implementation and processing purposes the regularization, statistical learning, and boosting triad complemented by sparsity and grouping (feature sharing) to deal with high-dimensional data and enhanced generalization. The recurring theme is that of a unified framework that involves multi-task and transfer learning using metric learning and side information.
TopIntroduction
This chapter discusses face recognition methods for the intertwined objectives of interoperability and enhanced generalization for the joint purposes of biometric authentication and identity management applications as encountered during real-world security operations confounded by uncontrolled settings. Current and emerging biometric protocols for both identity management and performance evaluation are critically assessed. Protocols are needed to enforce regulations, track the use and misuse of biometric systems, e.g., functional creep where data collected for one purpose is used for a different one, and safeguard overall security, integrity, and privacy. Performance and validation, which should be metered using uncontrolled-settings and interoperability, determine the robustness and reliability of the authentication decisions made and their side-effects, if any. Robustness is about high sensitivity and high specificity when coping with adversarial information, e.g., biometric variability including but not limited to data capture during image formation, and incomplete (“occlusion”) and corrupt (“disguise”) biometric footprints. Reliability is about consistency (in the limit) and stability of the predictions made during authentication. Both robustness and reliability are mostly a function of information uncertainty and variability, with learning and adaptation the methods of choice to enhance them. Uncontrolled-settings are characteristics of real life scenarios, which go beyond mass screening to include surveillance using smart camera networks, tagging driven by social networks and/or Internet, and biometric management of crowds. Performance for uncontrolled settings is hard to come by, is usually anecdotal, and is definitely significantly less than the performance advertised for large scale but tightly controlled FRVT biometric evaluations. Furthermore, ground truth is not available, false positives are not disclosed, and false negatives are not available. Policymakers should insist on adequate and enforceable performance metrics that can be independently corroborated and don’t casually take “an unbiased look at data sets bias” (Torralba, 2011). Interoperability and validation are further paramount for the effective and ethical use of biometrics.
Biometric identity, for all practical purposes, is about information rather than data. Data are confined to qualitative or quantitative attributes of a variable or set of variables. Data, typically the results of measurements, are often viewed as the lowest level of abstraction from which information and then knowledge are derived. Raw biometric data, i.e. unprocessed identity data, refers to a mere collection of images. Information, however, stands for data together with their implicit or explicit reference assuming the status of evidence. The etymology and semiotics of the word “information” are clear on this aspect. They involve communication and meaning, and their hallmarks are those of entropy and uncertainty. Last but not least “information” has value and establishes context to inform and instruct, and ultimately to control behavior using meta-planning and meta-recognition. Knowledge is the next stage of abstraction, with information management operating at a higher level of categorization, where generalization and predictions can now be made, e.g., demographics and stratification including gender, race, age, disabilities, mobility, and employment status. Public policy can be ultimately influenced by such demographics and their perceived dynamics. Towards that end, the problem of identification bears not only on criminology but also on civil identification (Gates, 2011).
The Quo Vadis for this chapter is about how to move forward on face recognition under uncontrolled settings. It is about making the case for leveraging similarity, sparsity, and statistical learning for the joint purposes of encoding, interoperability, learning, and ultimate validation. The case is made throughout that multiple-views and multiple cues need to be integrated in order to accrue evidence that overcomes the inherent uncertainty and adversarial conditions encountered during enrollment and querying. The objectives are many and go beyond traditional still-frame authentication. They also include new and all encompassing biometric models for representation, training, and matching; open set recognition; protocols; and re-identification. Three roadmaps cover our discussion: biometrics challenges, current authentication methods built around regularization, and solutions and venues for future research built around multi-task learning.