It's a tech demo that doesn't look like a tech demo. The clip was just a woman talking. Pedestrian stuff. That is, until the woman's face changed colors. Literally. Months back a facial animation clip called "Emily" popped up online, showing off the strides that its software maker Image Metrics has made. "Our recent Emily project is something we're all proud of," says Image Metrics co-founder Kevin Walker. Damn well they should be.
That tech demo wasn't the first time Joe and Jane Q. Public had seen what Image Metrics was capable of — nor is it the end of it. Image Metrics did face work for Grand Theft Auto IV, Unreal Tournament II as well as Gnarls Barkley's "Who's Gonna Save My Soul" and Rock Band 2.
Traditional face motion capture uses traditional tools. Ditching pricey, special equipment or markers, the Image Metrics solution is software-based. All that's needed is a well-lit video of the actor's performance. Company CEO Kelvin Duckett explains: " Image Metrics provides markerless performance-driven facial animation using its proprietary technology. Our solution starts with video footage of an actor's performance, which is then analyzed pixel-by-pixel. The resulting data is transferred back onto the CG facial model." That's it, folks.
The tech is the fruits of Walker's studying computer vision at Manchester University. After completing his PhD, Walker and a colleague created what would become Image Metrics on the hopes of getting closer to the industries that their science had so much to offer." We were convinced that this was the best way to really understand and solve the problems that had prevented computer vision from becoming integrated into the many fields it could benefit," says Walker.
A lifelong gamer who lost a large part of his childhood to playing Elite on my BBC model B, Walker could see how gaming would only benefit from better computer vision. Better computer vision meant better rendered faces. "As computer vision can be applied to many fields, when developing the technology, we were looking for the most difficult task to test it with," says Walker. "Facial animation was the perfect challenge. Faces, for the most part, are quite similar, but the small subtle differences are what make us so unique. Teaching a computer to learn to recognize those minute elements that define our personas was quite a challenge."
Coupled with the tech challenges, Walker and his colleagues didn't have a background in animation. It was an uphill climb, there were obstacles, but they lucked out and met industry types who could see the inherent value in Image Metrics and were willing to fill in the gaps and plug the leaks. Image Metrics did the only smart they could with these experienced industry vets: Hired them.
With the new tech in hand, the differences were startling. Traditional mocap can only capture markers, losing whatever is in the space between the markers. Image Metrics captures the entire face and thus, renders a more emotive mug as seen in the the Emily demo. "While we're always working on improving our technology, I think the Emily demo shows how real we can get with it right now," says Walker. For film and television, the company is able to produce photo-real facial animation like in the Gnarls Barkley video. "We can produce Emily-quality animation for games as well, but it just can't work in a real-time gaming environment," says Walker.
We're venturing into Uncanny Valley territory here. That's a robotics theory that humans are repulsed when robots and androids look and act too much like real people. First hypothesized in 1970, the theory has since been expanded to encompass photo-realistic animation as well. While it's often taken as a scientific law and not a theory, roboticist David Hanson, best known for his realistic humanoid faces (pictured), reminds: "The Uncanny Valley Theory is just that: A theory." Walker agrees, seeing that point of heighten realism as something Image Metrics should strive for and not avoid. "If we cross it," he says, "we know that we've achieved something outstanding.