Hairstyles for games and movies
U.S researchers have announced at the SIGGRAPH 2008 conference that they have developed a new method for accurately capturing the look of a person's hairstyle for use in animated films and video games. The research team used multiple cameras, light sources and projectors. Then, the team created 'algorithms to automatically 'fill in the blanks' and generate photo-realistic images of the hairstyles from new angles and new lighting situations.' As said one member of the team, 'we want to give movie and video game makers the tools necessary to animate actors and have their hair look and behave as it would in the real world.' But read more...
You can see above how the method works. Here is what the research team says. "Our geometry reconstruction pipeline computes both a volumetric representation of hair position (bottom) and hair orientation (top). Our algorithm takes as input 16 uniformly images lit with all 150 LED lights turned on (one such image from each camera), 36 864 stripe images with a single line of light projected (768 lines per projector and 3 projectors), and 3 648 tracking images with all three projectors turned on (one such image every 10 stripe images). Hair fibers are then grown inside the volume and along the orientation, starting from the scalp." (Credit: Adobe/MIT/UC San Diego)
This research work has been done by Sylvain Paris and Wojciech Matusik from Adobe Systems, Inc. The UC San Diego researchers associated to this project are Will Chang, Wojciech Jarosz and Matthias Zwicker. The last two researchers, Oleg Kozhushnyan and Frédo Durand work at the MIT.
On the left, you can see "side-by-side comparisons for various hairstyles. For fair comparison, the reference photographs were removed from the data set used by our image-based rendering method." (Credit: Adobe/MIT/UC San Diego)
The team said they gave hair 'The Matrix Treatment.' "The makers of the movie 'The Matrix' used digital face replacement to generate realistic images of human faces even though they had no photographs from these angles, explained Zwicker. [...] If you had an infinite number of cameras and light sources, there would be no angles, views or shots that need to be computer generated. 'But this is totally impractical,' said Zwicker."
So what was the solution developed by the computer scientists? "Instead, for each of the hairstyles that received 'The Matrix treatment,' the researchers captured about 2,500 real-world images using 16 cameras, 150 light sources and three projectors arranged in a dome setup. With all this data, the computer scientists determined the physical position and orientation of all visible strands of hair. The algorithms then generate complex hair models, producing on the order of 100,000 hair strands."
Now, what's next? "One possible extension of this work: making an animated character's hair realistically blow in the wind. This is possible because the researchers also developed a way to calculate what individual hair fibers are doing between the hairstyle surface and the scalp. They call this finding the 'hidden geometry' of hair."
For more information, this research work has been published in the ACM SIGGRAPH 2008 papers under the name "Hair photobooth: geometric and photometric acquisition of real hairstyles." Here is a link to the abstract. "We accurately capture the shape and appearance of a person's hairstyle. We use triangulation and a sweep with planes of light for the geometry. Multiple projectors and cameras address the challenges raised by the reflectance and intricate geometry of hair. We introduce the use of structure tensors to infer the hidden geometry between the hair surface and the scalp. Our triangulation approach affords substantial accuracy improvement and we are able to measure elaborate hair geometry including complex curls and concavities. To reproduce the hair appearance, we capture a six-dimensional reflectance field. We introduce a new reflectance interpolation technique that leverages an analytical reflectance model to alleviate cross-fading artifacts caused by linear methods. Our results closely match the real hairstyles and can be used for animation."
And here is a link to the full paper (PDF format, 9 pages, 4.50 MB), from which the pictures in this post have been extracted. Here are the conclusions of this paper. "We have described a method to capture both the geometry and the appearance of a person's hairstyle. Our capture setup and analysis algorithm significantly improve the accuracy of the acquired geometry. We have complemented the acquisition with a method to infer the hidden geometry of hair. We have also shown the gain brought by incorporating a priori knowledge in the image-based rendering. As a result, we are able to capture real hairstyles with their full complexity."
Finally, if you want to see movies showing how this technique work, here is a link provided by Sylvain Paris. But be warned: some of the files are quite huge.
Sources: Jacobs School of Engineering news release, University of California at San Diego, August 13, 2008; and various websites
You'll find related stories by following the links below.