Identifying a face from a single picture
Law enforcement officers around the world can be happy today. A computer scientist and PhD candidate from the Umeå University in Sweden has developed algorithms that give a computer the possibility of recognizing a face, even if only one picture exists in the database used to identify criminals or suspects. The software can synthesize other images of a single face using various angles, light conditions or facial expressions. This means that at a security control, a police officer should be able to compare an image taken by a surveillance camera with all the variants of the images contained in their databases. This is at the same time brilliant and frightening. But read more...
You can see above some examples of synthesized images by this software. The middle column contains original photos while the left and right sections show synthesized images from different angles. (Credit: Hung-Son Le, Umeå University) the algorithms can also generate images under different light conditions or with various facial expressions. This means that you don't have a single picture in your database, but maybe thousands of it.
These algorithms have been developed by Hung-Son Le, a doctorate student who wrote his PhD disserattion under the supervision of Professor Haibo Li, Head of the Digital Media Lab (DML) at the Umeå University, Sweden
According to the computer scientist, here are some of the reasons why curent face recognition systems are not really efficient. "Systems that can identify different faces are normally trained through a database with a large collection of face images in different illumination and pose. Nevertheless to collect such a large number of face images for each person is difficult and quite often expensive. Moreover these systems have problems due to the bad quality of the pictures, as well as facial expressions, the variety of angles and the different illuminations. These problems are now over."
This last sentence looks very affirmative. So is there any truth behind this assertion? "The effective algorithms developed by Hung-Son Le make it possible to have a system that can identify a face even when there is only one picture in the database for each person. Moreover, the effectiveness of the system is a considerable improvement when taking into account light conditions, or facial expressions. His algorithms use a method than improves contrast in underexposed and overexposed pictures. Thus details can be made visible which otherwise would be difficult for a computer to identify."
Hung-Son Le will defend his dissertation named "Face Recognition: A Single View Based HMM Approach" on February 1, 2008. Please note that HMM stands for "Hidden Markov Model." Here are two links to the abstract and to the full text (PDF format, 174 pages, 10.1 MB) of Hung-Son Le's thesis.
Here is how the researcher introduces the concepts that guided him. "Here we present effective algorithms that deal with single image per person database, despite issues with illumination, face expression and pose variation. Illumination changes the appearance of a face in images. Thus, we use a new pyramid based fusion method for face recognition under arbitrary unknown lighting. This extended approach with logarithmic transform works efficiently with a single image. The produced image has better contrast at both low and high ranges, i.e. has more visible details than the original one. An improved method works with high dynamic range images, useful for outdoor face images."
Here is a short description of the model he used. "Face expressions also modify the images’ appearance. An extended Hidden Markov Models (HMM) with a flexible encoding scheme treats images as an ensemble of horizontal and vertical strips. Each person is modeled by Joint Multiple Hidden Markov Models (JM-HMMs). This approach offers computational advantages and the good learning ability from just a single sample per class. A fast method simulated JM-HMM functionality is then derived. The new method with abstract observations and a simplified similarity measurement does not require retraining HMMs for new images or subjects."
Now, let's jump to a specific paragraph in the conclusions of this document describing how these algorithms could be used. "Our current system deals only with the identification problem. It can be integrated into a full identification system. The current method may serve as a filter for the best matched face or a short list of most similar faces to the query/unknown face. Based on this list, a specific verification module can give final judgement about who is who. Alternatively, the full identification system may rely completely on classification-based techniques if we can generate somehow a set of dummy faces that represent the 'complementary face space' of the real reference faces."
For your information, the illustration above has been picked from page 132 of this dissertation. And for fun, you might want to test the Photo Album Search Prototype, a demo on the AR publicly available face database available at Purdue University.
Finally, be awre that commercial applications based on these algorithmms are under development. So the next time you cross a border, it will not matter if you smile or not, a software based on this research might identify you -- and maybe as someone else. This software development is very clever, but its applications might be flawed and dangerous for many travelers. Please tell me what you think.
Sources: Umeå University news release, January 21, 2008; and various websites
You'll find related stories by following the links below.