When Xbox 360's controller-free Kinect add-on hits stores later this week it has the potential to redefine gaming in ways even the Nintendo Wii hasn't yet done. It could just as easily fall flat on its face.
While Nintendo's motion-sensing Wii changed the way we use controllers, Microsoft's Kinect completely removes the need to hold a controller for its Xbox 360 games. But with that potential comes some very big problems, according to experts on digital media and user interface.
"Removing the physical controller takes away a major piece of context from the experience," said Noah Wardrip-Fruin, an Associate Professor of Computer Science at the University of California, Santa Cruz, where he co-directs the Expressive Intelligence Studio, one of the world's largest technical research groups focused on games. "If I hand you something with buttons, you know you're supposed to press the buttons. So the game has to do more work to tell me what to do, and give me feedback if I'm doing it wrong, if there's no physical controller.
"But along with that extra burden comes new power. Moving our bodies in different ways conveys attitudes and emotions - and there's evidence it helps us feel them as well. We could become involved in games in a whole new way."
Kinect uses a set of cameras and microphones packed into a single rectangular piece of plastic resting over or under your TV to watch and listen to the player. It then translates that data into controls for motion-based games like bowling, kick ball and dancing.
"The user interface is the most important part of any experience," said Kinect Creative Director Kudo Tsunoda. "It is the entire way you are interacting with anything you do. It is as important as your five senses are to human beings interacting
with the world. I think this is why people are so excited about Kinect. It is a fundamentally new way of interacting with your games and entertainment. It is a new way to play. And this gives consumers and the makers of entertainment an entirely new palette of toys to play with."
But while Kinect promise to "make you the controller," that doesn't necessarily mean that the experience will be easier or more immersive than using the traditional Xbox 360 controller, the Wii remote or the PS3 Move.
"Removing a physical controller doesn't inherently make an experience with computation more or less anything, it introduces new interaction challenges and opportunities that the designer has to address, so it all depends on how the designer makes use of the new affordances provided to them," said Carl DiSalvo, assistant professor at Georgia Institute of Technology's School of Literature, Communication and Culture.
While the notion of standing in front of your TV and moving to play a game may sound easier, that's not always true. That's because people tend to become blind to whatever controller they use to interact with technology over time. Spend enough time typing, using a mouse, a TV remote or game controller and eventually you almost forget it's there. That is until there's a problem.
"We typically only notice a controller when it breaks—then it becomes present-to-hand," said Katherine Isbister, Director of the Social Gaming Lab at the Polytechnic Institute of NYU. "But most times, it's an extension of our own fingers—a prosthetic part of us—it's ready to hand. It's a really cool trick the human mind has that is very useful, but that can trip us up when we shift tools."
Because of this, shifting from controllers designed for gamers or the Wii remote, designed to look like a TV remote, to the seemingly more natural controls of just moving, can actually be confusing and confounding if not done right.
"Not having to use a controller, and being able to use your own movements directly to engage the game, should offer a big leap in immersion," Isbister said. "However, the Kinect is still tethered to screen-based output, so the feedback part of things is not as fluid and natural as the input part. And also, as with all movement technologies, there are constraints around what can be recognized and how to design for this."
And they way we control games is as important to gaming as are graphics, sound, the cast of characters, the plot and the game play mechanics.
Dropping the controller all together could actually make gamers feel less connected to the games they're playing, creating a sort of kinetic dissonance.
That's because people playing a game without a controller will subconsciously expect the game to track their real-world natural movements and expect real world results from those movements. But this new wave of motion gaming is still nowhere near approaching realistic movement dynamics.
Most motion-based games use "movement metaphors," Isbister said.
"That is to say they don't really totally mimic the real movement, instead they evoke it," she said. "The best games capture the most fun aspect or essence of a movement, without bogging you down in all the intricacies of the real embodied activity."
So when you're playing Star Wars: The Force Unleashed, you can pick up and fling enemies across the screen with a flick of the thumbsticks.
"What my lab is realizing is that the trick is to shape expectations in the player, right when you offer the interaction technology—to offer them such a strong path of action and set of constraints that they are really clear on how far the metaphor extends," Isbister said. "There's an art to it, and it's an important part of designing these kinds of applications."
"The player will have a better experience with stylized control approaches - gesture vocabularies that may start with our understanding of how something is done in the physical world, but then simplified and exaggerated to become a control language," said Wardrip-Fruin. "This is also better suited to where we are technologically."
If the interface, the connection between player and game, is perfected, Kinect's greatest potential is how it could redefine the nature of play.
"I think the nature of play is that you draw a 'magic circle' around you (and whoever you are with) where the rules are different and the stakes are not 'real' (or at least not *as* real)," Isbister said. "You can do this by playing paper football at the lunch table at work, really. You don't have to be in your living room or in a park.
"I think Kinect will actually provide a really cool new space for playful experience that offers a different set of design options for game developers that we haven't had yet."
Wardrip-Fruin said that traditionally physical activity was the root of both work and play.
"Now we have work and play that both involve sitting in front of a computer and getting repetitive stress injuries," he said. "Freeing my hands from the same repetitive movements is not going to make Kinect games seem more like work.
"It will open the possibility for new experiences, some of which will feel magical."
Well Played is an internationally syndicated weekly news and opinion column about the big stories of the week in the gaming industry and its bigger impact on things to come. Feel free to join in the discussion.