For a week in June, Team Kotaku played Microsoft's hands-free control system Kinect, standing up. Kinect has problems with seated players, we reported. Yesterday, I tried doing a push-up in front of it.
I was trying — no, testing — the Kinect-enabled version of EA Sports Active 2 at a showcase event in New York City.
I had been startled at E3 in June that even Microsoft's car-driving and TV-watching Kinect demos didn't permit players to sit. You had to do all your Kinect stuff standing, which made sense when the task was playing tennis but not when the mission was to fast forward part of a feature film with a hand-swipe. Company spokespeople promised sitting would be supported in some way — at least for the TV-watching features — but I needed to see Kinect working with a seated person to believe it.
I wondered if EA Sports Active 2 could solve the sitting mystery or at least its cousin, the push-up puzzle.
From afar, yesterday, I had seen an EA-hired trainer doing push-ups in front of the Kinect. If the sensor system couldn't read a seated person well enough for Microsoft to demonstrate that configuration yet, how could it see the push-up? The postures seemed related, at least because my hunch was that Kinect wanted to see a full body in a flat vertical plane, a position you get from standing but neither from sitting nor from push-ups.
The EA trainer showed me some dumbbell exercises first, emphasizing that the Kinect frees the player from having to hold any gadgets. Liberated, you can add your home weights to the routine. On the Wii, by contrast, you need a Wii Remote in your hands. Encumbered with only a small heartrate monitor strapped to his arm, the trainer lifted his arms to raise the weights.
I asked to see push-ups but, as unreleased software good and bad sometimes does, the game crashed. The Xbox 360 had to be re-booted.
In the process of powering the Xbox 360 back up, we briefly turned on the Kinect visualizer. This program is not a part of EA Sports Active nor something customers will likely see. But it is a neat program that shows in three on-screen windows what the Kinect actually sees, a perfect partial visual explanation for reporters of how the magic trick works. One of the windows shows the 19 or so joints the sensor sees on the body of the person or people standing in front of the Kinect. It connects those points with lines, creating a stick figure that moves exactly as you move your body, a reflection of your human self as a stick man. I stood in front of this and lowered my body to a push-up position.
As I lowered my body I kept my head raised so I could see the TV. My head, shoulders, arms and hands still seemed to be tracked accurately. The top half of my stick figure self had lowered itself to the floor. But my legs and its legs did not match. My legs were behind me, extended for a push-up. The stick figure's legs dangled below the stick figure's torso, as if fallen through a hole in the floor.
I did a push-up, still craning my neck. The Kinect continued to capture most of the top half of my body — now the front half — with no trouble. I realized that this was why EA Sports Active 2 can count a Kinect user's push-ups. All it needs to know is that the head has been lowered, the elbows bent, the shoulders dropped to the hands.... that's the ready position... then up!... that's the up position. One push-up done. EA Sports Active 2 wouldn't need to care about your legs while you are doing a push-up. So, no problem, right? Not for this fitness program's push-ups, it seems.
I don't know what to make of the leg-dangling thing and wonder if that is a hint to the Kinect's limitations. Can the Kinect not read body parts that recede into depth, away from where they started in the standing position? Or can it just not understand a full body whose joints are not stacked more or less in the vertical order of a standing person.
The Kinect's limits continue to have me scratching my head and leave me eager to see Microsoft demonstrate a Kinect used in more postures than upright standing.
I guess I should have tried doing a sit-up.
EA Sports Active 2 ships this November for the Xbox 360 (Kinect only), PS3 and Wii as a full workout motion-sensitive workout program. All versions include a heartrate monitor and are promised to provide a rigorous workout if so desired. EA is also planning iPhone and iPod Touch versions of the software.
PIC: Tim Rogers and Robert Pelloni doing push-ups at last month's E3, no Kinect involved.