When Xbox 360's controller-free Kinect add-on hits stores later this week it has the potential to redefine gaming in ways even the Nintendo Wii hasn't yet done. It could just as easily fall flat on its face.
While Nintendo's motion-sensing Wii changed the way we use controllers, Microsoft's Kinect completely removes the need to hold a controller for its Xbox 360 games. But with that potential comes some very big problems, according to experts on digital media and user interface.
"Removing the physical controller takes away a major piece of context from the experience," said Noah Wardrip-Fruin, an Associate Professor of Computer Science at the University of California, Santa Cruz, where he co-directs the Expressive Intelligence Studio, one of the world's largest technical research groups focused on games. "If I hand you something with buttons, you know you're supposed to press the buttons. So the game has to do more work to tell me what to do, and give me feedback if I'm doing it wrong, if there's no physical controller.
"But along with that extra burden comes new power. Moving our bodies in different ways conveys attitudes and emotions - and there's evidence it helps us feel them as well. We could become involved in games in a whole new way."
Kinect uses a set of cameras and microphones packed into a single rectangular piece of plastic resting over or under your TV to watch and listen to the player. It then translates that data into controls for motion-based games like bowling, kick ball and dancing.
"The user interface is the most important part of any experience," said Kinect Creative Director Kudo Tsunoda. "It is the entire way you are interacting with anything you do. It is as important as your five senses are to human beings interacting
with the world. I think this is why people are so excited about Kinect. It is a fundamentally new way of interacting with your games and entertainment. It is a new way to play. And this gives consumers and the makers of entertainment an entirely new palette of toys to play with."
But while Kinect promise to "make you the controller," that doesn't necessarily mean that the experience will be easier or more immersive than using the traditional Xbox 360 controller, the Wii remote or the PS3 Move.
"Removing a physical controller doesn't inherently make an experience with computation more or less anything, it introduces new interaction challenges and opportunities that the designer has to address, so it all depends on how the designer makes use of the new affordances provided to them," said Carl DiSalvo, assistant professor at Georgia Institute of Technology's School of Literature, Communication and Culture.
While the notion of standing in front of your TV and moving to play a game may sound easier, that's not always true. That's because people tend to become blind to whatever controller they use to interact with technology over time. Spend enough time typing, using a mouse, a TV remote or game controller and eventually you almost forget it's there. That is until there's a problem.
"We typically only notice a controller when it breaks—then it becomes present-to-hand," said Katherine Isbister, Director of the Social Gaming Lab at the Polytechnic Institute of NYU. "But most times, it's an extension of our own fingers—a prosthetic part of us—it's ready to hand. It's a really cool trick the human mind has that is very useful, but that can trip us up when we shift tools."
Because of this, shifting from controllers designed for gamers or the Wii remote, designed to look like a TV remote, to the seemingly more natural controls of just moving, can actually be confusing and confounding if not done right.
"Not having to use a controller, and being able to use your own movements directly to engage the game, should offer a big leap in immersion," Isbister said. "However, the Kinect is still tethered to screen-based output, so the feedback part of things is not as fluid and natural as the input part. And also, as with all movement technologies, there are constraints around what can be recognized and how to design for this."
And they way we control games is as important to gaming as are graphics, sound, the cast of characters, the plot and the game play mechanics.
Dropping the controller all together could actually make gamers feel less connected to the games they're playing, creating a sort of kinetic dissonance.