Apparently Sega is test-marketing a software tool called the Magical V Engine that allows for advanced facial animation from voice recordings alone. By animating based upon not only the phonetic lip syncing but expression (we're guessing tone), the software can emulate the four basic human emotions—happiness, sadness, anger and surprise—in full facial expression.
The software is said to be between 70% and 80% accurate, with levels approaching 100% with a little tweaking. We're guessing such tools could save developers a lot of time, you know, if they work as well as advertised.
Sega Testing New Voice-Recognized Facial Animation Tool [gamasutra]









Comments
Oh lord. Is that Max Headroom?
Max Headroom 2 here we come!!!!
Cyberpunk at its finest! ;D
Wow... expect a rating system to be invented that will be used to cataloge the voice-actors according to the synchronisity between the voice and the facial expressions.
This could be terrifying, with every person looking and talking like the G-man. Even women.
Now all we need is a real time version of this that will emote your character when you speak in multi player games.
Sweet cabinet making Jesus! thats creepy...
Seaman 3?
I want to see the surprise on the guards' faces as I stab them in the neck in AC whilst shouting "Surprise cockfag!".
But sounds like cool stuff if it works. It would be good for something like Elder Scrolls V.
@Guild_Navigator: that would be fantastic! little details like that are what make a good game into a great game
Good ol' Matt Frewer. I used to love that guy.
Didn't Sega start this with Shenmue? That game, if I remember correctly, had automatic lip-sync. Worked very well. Adding emotion to that could be amazing.
This is the kinda thing that I've always seen coming into play heavily in gaming's future. The next big thing being combining this with NPCs that understand natural language, typed or spoken. My tech-sense tells me that this is just the first baby steps into that territory, though. I'd be surprised to see this sort of thing working in the next 10 years.
hmmm wasn't Valve working on this a long time ago and they where going to add it to Team Fortress 2?
@KaneTaker: If the audio is pre-recorded, the lip sync doesn't need to be automatic. They can pre-calculate it all and put it on the disc instead of using CPU time to figure out what it is.
If this does get used to lip sync voice chat in real time, it will be funny to see how it handles non-voice noises. Having your character mouth out the dog running through the kitchen might be interesting.
@Guild_Navigator: I thought some version of Rainbow Six already did this for multiplayer? I could be mistaken though.
Regarding saving the developers time, I think all that time saved on syncing movement to speech will have to be spent finding voice actors that are worth a damn. No more calling in the janitor "in a pinch" unless you want every single character he voices to not just sound drunk, but look it too...
That is awesome.
The way I see, they'll be able to easily have almost every random NPC in a game has the same spot on lip and facial movement as the games important characters and conversations.
Case in point, Mass Effect. Lip syncing for the main characters was great most of the time, most other NPCs were kind of spotty at times.
Nope, not Max Headroom. The dude in the pic is the announcer from a mid-90s PC racing game. There were weapons... powerups... sort of F-Zero-ish... fuck, what was the name! Something "m" sounding. Oh this is going to piss me off.
Max Headroom game CONFIRMED, Yes the 80's have made such a resurgence this game was needed.
@el_gordo: erm....[en.wikipedia.org]
@rususeruru: It's not real facial animations, just the mouth opening and closing, but its hilarious either way.
Chocolate Milk (Contains Harsh language)
That's pretty cool actually. Hopefully it works out for Sega. I still want to see them do something great...Again.
Oh SHIT! Do I look like a douche. Well, still! There was this GAME! In the 90s! I was just a kid. But it was awesome! You guys! You gotta believe me!
Samuel L is screwed if they ever use this. It'll only be a mad shouting face
sega needs to make great games!! (sonic anyone?) by the way MAX HEADROOM FOR PRES 08! we need more stuttering futurisic and incromprehensive characters
That has to be the best picture posted to accompany an article ever. It's so awesome, I might say it even distracts from the article. I read the headline, I see the picture, I get it. Visual Communication at its finest!
The source engine has a similar thing to this. You can input an audio file into the editor where it will identify the phenomes (you can assign them as well) and the model's face will 'move to the beat' as it were...
Shame that the source engine was never really taken great advantage of....
Sega and testing, that's an oxymoron!
@el_gordo: Don't worry I believe you, because I played it too :)
Megarace
Holy crap, Max Headroom?!
What ever happened to Matt Frewer anyway...?
All I remember is Lawnmower Man 2 and the crappy Dawn of the Dead remake...
Bring back max bring back max
@uvadave: megarace was awesome, i got that shit with my 2x cd rom drive !
@Guild_Navigator: Only if an angry 13 year old, in fact, looks like an angry 13 year old. That way I can laugh at how the voice recognizer/facial expression thing has determined him to be an angry prepubescent and has thus shown him to be such.
Wow.....who the fuck is that dude up there? lol, I have never seen that creepy/fake ass looking dude before in my life, I am clearly missing something here.....I got some chills/creepy feelings when I looked at that fool, anyone elaborate?
@rususeruru:
Well, I know that there are a few multi player games that do have characters flap their mouths to your voice but as far as showing expressions with it, I haven't seen any.
@Dexor: Matt Frewer currently plays Jim Taggart on "Eureka" on SciFi. Great show. Also, he is playing Moloch in "Watchmen" according to [imdb.com]
If Sega makes a Max Headroom game with this, I refuse to buy another Sega game for at least a year. >_>
Odd, that freakishly looks like Tony Snow...
The amount of people here that don't know who Max Headroom is makes me feel really old. On top of that, the cafe 80s scene in Back to the Future 2 must have been really confusing for some people.
@Guild_Navigator: I agree....I'm only 21 too.
@sir_blazeAlot: If you think that guy's creepy, DEFINITELY do not search Youtube for "Max Headroom Video Pirate".
I would imagine many developers would use this for less important charaters in a game. For close-ups and in-the-character's-face scenes I would imagine devs would rather do some parts of the emotion expression on the character manually; just to get the right amount of wrinkles and such. It would save work and time for the devs with no doubt. I am interested to see if this surfaces and becomes widely used. I am sitting and I am watching. .
....
This may take some time.
Blue. Blue! BLUE! DAMMIT!
Fuck all this fancy-shmancy shit. I still have my Teddy Ruxpin and he works just fine. I can put my MC Hammer tape in his back and he sings right along. I think he's up there with some of the other past technological wonders like the aqueduct and the Sphinx.
@Torgen: Automated and pre-calculated are not opposites. It's nice to have at least part of the syncing process automated, even if you can them to the disc instead of running the simulation realtime during the gameplay.
They tested this with recordings of Mariah Carey, and every thing she said produced the same "come hither" look on her fembot, even when she was doing anti-Mariah.
@gatsuuga: [www.youtube.com]
holyshit that was creepy. D:
Cut the crappy experiments already Sega, where the hell is my Dreamcast 2?
Or for that matter, a decent Sonic game?
Start a discussion:
Login with your username and password below. Or comment on this post via email.
Forgot your username or password? New User?