State of the Art

3D Face addresses a large number of knowledge domains, e.g. 3D sensors issues, 3D face recognition algorithms, multimodal fusion and template protection methods. To reach the goals summarized in the projects Objectives, advanced research and development in each single technology is necessary, but in addition the optimal combination and alignment of the different technologies is of absolute importance.

The state of the art with respect to the following know-how fields is presented in the following chapters:.

1 Technologies

1.1 3D Sensors

Different methods for designing a 3D sensor are well known, such as stereovision, time-of-flight (TOF) and modulation methods, structured light, triangulation principle, shape-from-shading, shape from focus and others. Coded structured light is considered one of the most reliable techniques for recovering the surface of objects. Different coding strategies for structured light are well-known in the literature. Time multiplexing coding strategies generate the code for each observed pixel by combining multiple patterns projected separately onto the object. Spatial neighbourhood methods combine the projected pattern in the local neighbourhood. These methods require more complex patterns which are often based on colour encoding. Direct coding strategies try to encode the information directly in the colour or grey tone. With current 3d acquisition devices, the best accuracies can be achieved by capturing still images from rigid objects in controlled lighting situations. Hence the research in this work package concentrates on the challenging topic to reach an accuracy of 0.1mm for non-rigid, moving objects with a video like frame rate.

1.2 3D face recognition

Algorithms to exploit the shape of a person’s face as a biometric characteristic have been developed since the late 80's. But looking at the spectrum of R&D in facial recognition, only a very small fraction of the effort has been invested into the research on 3D methods.

A number of general classes for 3D facial recognition algorithms can be separated. The first family of algorithms analyses the shape or surface geometry explicitly, e.g. by measuring the distance between 2D sections of the 3D shapes . An extension of this approach called ICP (iterative closest points) makes use of the full 3D shape. For comparison two faces are aligned using an optimisation technique based on a global distance measure. One main drawback of ICP is the need of extensive computing power for a quick comparison. ICP does not allow the representation of facial geometry in a compact template. To circumvent these problems some algorithms extract features from the shape and then compute absolute or relative distances, angles or curvatures at and between predefined characteristic points (e.g. nose and eyes). Once these quantities are computed they form a compact template that can be used for the recognition.

A second popular family of algorithms is an extension of the well known Eigenfaces approach developed originally for 2D face recognition . The depth map of the shape is treated like a 2D greyscale image and is processed in a similar way to the original approach.

Elastic graph matching is a well known method for 2D face recognition and was extended by Viisage to cover both the texture and the shape of the face. The graph is automatically matched to the 2D and 3D description of the face and extracts characteristic features in 2D and/or 3D. This technique is known to be very accurate in 2D and has the universality to be extended to 3D.

The verification performance of current 2D and 3D Face recognition has been reviewed widely. The results of the Face Recognition Grand Challenge (FRGC) test give an indication of the improvements. Furthermore, the results are very much depending on the operating conditions (lighting, pose, expressions, etc.) under which the face recognition algorithms are tested.

The first main issue for performing 3D Face recognition is the face and pose detection: once the pose is detected, a normalisation can be used (rotation, translation) for aligning the 3D models before comparing, whatever the comparison method. For this normalisation, characteristic points and general landmarks have to be detected.

The second important point is the selection of the feature extraction method and such the features. For that, many methods can be proposed. One possibility is to try to extend the 2D face recognition method. Another way is to find original features from 2D information.

The third point is the way of comparing the features. Some comparison method include complex operation: geometrical transformation, etc. Some other methods are on the contrary very simple: comparison of two feature vectors.

Lastly, it is very important to optimize the speed of each step (localisation, normalisation, extraction, comparison), and to find the functioning points that can guarantee the better trade-off between accuracy and operational constraints (CPU, memory, speed…°)

1.3 Multimodal

Research on multimodal biometric systems has been going on for a number of years and more recently work has begun on exploring multimodal systems that include 3D facial information. However, the scope for multimodal integration is much greater than just adding 2D information and in this work package a range of options will be explored. Also existing multimodal strategies may have to be modified and adapted to be suited to the 3D framework that is the cornerstone of this project as well as to the particular application scenarios that the project is aiming to address.

Score and decision fusion are two popular methods for combining classification outputs. Prior to obtaining scores and decisions, 2D information can also be helpful in the 3D acquisition process. The first possibility is to combine 2D and 3D information in order to enhance the acquisition step, by having a better control on the parameters (pose, lighting, etc.).The second possibility is at the face and pose detection step: information coming from 2D can improve the 3D algorithm, and inversely. The third is at the normalization step: 3D can help to normalize the pose of the subject on the 2D face portrait by synthesizing the correct pose. That can also be extended to more complex parameters: mimics, shadows, etc.

These and other approaches to multimodality involving high resolution face texture, multiple 3D samples and combinations of multiple 3D recognition algorithms have so far not been studied by previous investigators. Template Protection Enabling privacy protection is synonymous with protecting the template information stored in a database. In order to achieve this it seems natural to apply cryptographic techniques. This is, however, non-trivial as it can be understood from the following construction: During enrolment the administrator stores all the data in a database in encrypted form. When a user presents himself at the control unit (sensor) for authentication, the sensor encrypts the measured data with the appropriate key and sends the encrypted data to the verifier. As biometric measurements are noisy by nature, the verifier can only compare the templates by decrypting them. This implies that the verifier needs a key for decryption and hence has access to all enrolled biometric data in its database. Consequently the privacy is not guaranteed. In recent literature there are attempts to protect the privacy of biometric templates. Examples are private biometrics , fuzzy commitment , cancellable biometrics , fuzzy vault , quantized secret extraction and secret extraction from significant components. All these systems are based on the use of a one-way function to achieve biometric template protection. This is similar to the password check in computer systems. When a computer verifies a password it does not verify the password with the password stored in a database. Instead the password is processed by a cryptographic one-way function F and the outcome is compared to a hashed version of the stored password. Translations of these principles to 3D Face biometrics do not exist and is not straightforward as biometric data is noisy by nature. Comparing cryptographic hashed versions of noisy measurements will in general not lead to positive authentication.

2. Biometric Standardisation for Facial Recognition

The standardisation of biometric technologies is coordinated at the ISO/IEC JTC1 level by the Sub-Committee 37 (SC37). SC37 was established in June 2002. The first plenary meeting held in Orlando (USA) in December 2002. The US delegation are the main contributors in the majority of topics. Major standards are based on current ANSI standards, promoted by the US M1 committee.

The Working Group 3 of SC37 is addressing data interchange format. The project 19794-5 is dealing with the face image standard. This very new standard is dealing only 2D facial images. The facial image size is commonly around 15 Kbytes. The process consists basically to identify the position of the eyes, then to compress the face using JPEG2000. The 2005 SC37 ballot has resulted in the decision to amend the 19794-5 standard for 3D face data and assigned the editor role for this new standard to key members of the 3D Face project consortium.

The Working Group 5 of SC37 is dealing with testing and performance. They are 4 projects on-going. It should be noted that most of the standards are based on the experience acquired in the fingerprint fields. Specificity of facial recognition on the current standard process definition is not significant.

The Working Group 6 of SC37 is dealing with Cross-Jurisdictional and societal aspects. It is addressing mainly privacy, accessibility and societal issues involved in the biometric technologies. Currently, a Technical Report (TR) is in the draft stage and the WG is regularly attended by a significant number of international delegates including researchers from Europe, North-America, Asia, Africa and Australia.

The Working Group 2 of SC37 has an ongoing work on a Technical Report related to multimodal biometrics which in time may lead to standardisation in this area.

ICAO, a United Nation Agency, contains a governmental Working Group dedicated to New Technologies (NTWG). This group, linked with the ISO/IEC JTC1 SC17 WG3, is drafting recommendation for travel documents. ICAO recommendation, referenced 9303 are endorsed by ISO, referenced 7501. A couple of years ago, this NTWG of ICAO has selected for the new generation of travel documents contact less technology, including biometry. The only mandatory biometric feature requested by ICAO for the new generation of passport is facial picture. The facial picture standard is the ISO 19794-5 standard. Fingerprints and iris are optional for ICAO.

The CEN Focus Group on Biometrics has also begun recently to coordinate European interest in biometric standardisation.