My room had a private balcony, a calming view of the Pacific, and beaches a few hundred feet away. Like dozens of guests who stayed at a Santa Barbara resort in October, I was there to play a video game.
Over the course of two days, I gazed upon California's beaches from a distance. I walked by the swimming pool and hot tub a dozen times. I ate at the Fess Parker Doubletree Resort's finest restaurant. But unlike the resort's guests there for a getaway, I never left the hotel grounds, never set foot on the beach or hit the tennis court. Instead, I spent every waking hour playing Call of Duty: Modern Warfare 2, eyes glued to a television screen.
This was Modern Warfare 2's "review event," a three-day affair at which members of the media played through the game's single-player campaign and multiplayer components.
I'd driven a hundred miles to get there, to play Infinity Ward and Activision's blockbuster shooter for a review. It was my second time sequestering myself in a hotel room to evaluate a video game. I reviewed Rockstar Games' Grand Theft Auto IV under similar conditions at a hotel in San Francisco over the course of four days. Other members of Kotaku have attended invite-only review events for titles such as Halo 3 ODST.
Typically, those review event conditions range from comfortable to exceedingly comfortable. In the case of Modern Warfare 2, rooms were outfitted with large screen televisions, thundering surround sound systems, optional professional quality headphones and robes and slippers embroidered with the game's logo.
While uncommon, the prevalence of these types of events is increasing. Bungie and Microsoft held similar review events for their most recent Halo releases. Events have been held for BioShock, Gears of War 2 and Metal Gear Solid 4. And publisher Electronic Arts recently invited the media to play cooperative military shooter Army of Two: The 40th Day, at which reviewers could get early access to a build of the game, but only after playing an organized co-op session in New York City.
Review events are not the norm. Typically, publishers and PR firms mail reviewers a copy of the game—some copies only work on special "debug" game consoles—and a letter that entrusts us to hold off on running the review until a certain date.
The practice of making the reviewer go to the game, at a review event, rather than send that game to the reviewer, has raised ethical concerns that such events influence the opinions of the reviewers who take part. Some say the events make it difficult to form an impartial critical opinion of a game under such circumstances. Whether through direct interference or as a byproduct of being catered to with travel, accommodation and the free gifts commonly given at such events, allegations that reviews are "bought and sold" at events paid for by publishers can tarnish the credibility of a review at best, mar the reputation of a publication or video game at worst.
So why do publishers and developers hold these sometimes contentious review events?
Halo developer Bungie, which held review events for Halo 3 ODST in advance of the game's retail release, cites multiple reasons for hosting such things.
"Looking back at Halo 2, my first Bungie review title, we built these events around a desire to deliver the best possible experience to players," says Brian Jarrard, community lead at Bungie. "We had nice HD televisions, 5.1 surround sound and a comfortable environment. It was a way to help make sure reviewers experienced the game the way Bungie intended."
(Rockstar Games and Infinity Ward did not respond to requests for comment about their respective review events. Activision reps declined comment when asked.)
A comfortable, high-fidelity experience may be important to game creators. But so is control.
"Usually, the cited reason for a review event is disc security," says Jeff Gerstmann, co-founder of Giant Bomb and former editorial director at Gamespot. He's participated in and approved coverage of numerous publisher-held review events over his course of his game review career.
"I understand the desire to keep control of all copies of a game, as pre-release piracy is certainly a real issue," Gerstmann says. "But considering that games get leaked out ahead of time even when these events happen, I think it's safe to say that the media isn't out there leaking copies to pirate groups, especially in cases where we're receiving discs that only run on debug consoles."
Bungie's concerns go beyond illegal distribution of its games. Jarrard says that content leaks and story spoilers are something developers work hard to protect.
"I'm not sure we would have been able to keep [the surprise of] playing as the Arbiter a secret if we mailed out hundreds of discs weeks before the game shipped," he theorizes, referring to a key Halo 2 plot point. "The same could be said for plot twists found in Halo 3 and ODST. We spend up to three years crafting these stories and experiences, and it's very important to the team that fans can enjoy the game and story the way we intended."
In the case of Modern Warfare 2, some of the game's content had already been leaked onto the internet via YouTube by the time we'd showed up to review it. When I reached the game's infamous "No Russian" chapter, I'd known about the content—but not the context—of that mission for the past 24 hours. The game wouldn't ship to stores for another two weeks.
But that didn't stop Infinity Ward and Activision from taking security precautions to prevent further leaks. The review version of Modern Warfare 2 I played was hand-delivered to reviewer's rooms with four employees in tow, the hard drive installed and physically locked to the console.
My Grand Theft Auto IV experience saw similar protections, with an executable running from the hard drive that required a special memory unit to play. Rockstar Games reps asked us to carry the memory unit with us whenever we left the Xbox 360 and PlayStation 3 unattended.
Jarrard said that he finds the "vast majority" of game reviewers to be trustworthy, honoring agreements designed to prevent leaks and piracy, "but it's still easier to control embargoes and protect story elements when everyone is there in person with discs that are all accounted for."
Having dozens of video game reviewers in the same room brings with it benefits to games with multiplayer components. Jarrard points to the difficulty in organizing multiplayer game sessions in advance of widespread releases, sessions that sometimes require dozens of players.
"That would become very challenging to do in a world where a few hundred people were sitting in their own offices all over the world trying to coordinate a game when outward facing Xbox LIVE support isn't even turned on yet," he says.
In Modern Warfare 2's case, Activision and Infinity Ward assumed control of one of the resort's ballrooms, offering reviewers a public space at which to play the game's multiplayer and Spec-Ops cooperative modes during day two of the event. My experience with GTA IV's multiplayer was more private, playing solo against Rockstar quality assurance team at a remote location at the review event and later against game journalists during Rockstar-scheduled sessions from home.
Gerstmann believes—and I agree—that multiplayer sessions held at remote review events often provide a more accurate, more efficient early look at a game's online component than the alternative.
"Sometimes these sessions work out great," Gerstmann says, "Sometimes they're sparsely attended or not entirely representative of the average user experience. I played Call of Duty: World at War online with a couple of journalists and a handful of QA guys who were instructed to take it easy on us. That didn't feel terribly 'real' to me."
But there's a catch. "The downside of testing multiplayer in a controlled environment (like a review event) is that you have no idea how the game will react in an actual real world situation, where ping times and the quality of the average player's broadband connection are a major factor."
Fortunately, many publishers distribute retail copies of games experienced at review events before the retail release. In the case of Modern Warfare 2, which wasn't sent to reviewers in advance, early distribution and broken street dates offered Kotaku and other outlets a chance to play substantial portions of the game at home.
Gerstmann stressed the importance of "never letting the event be my final time with a game before running a review."
"The events I've attended—Halo 3, Halo 3: ODST, Gears of War 2, and Modern Warfare 2—gave me plenty of time to see everything," he says. "But in all of those cases, I replayed the single-player again, start to finish, at home before writing a review. In all cases I was also able to get in substantial multiplayer time thanks to the street dates being broken and average consumers getting their hands on the game ahead of schedule."
MTV Multiplayer's Russ Frushtick, who participated in one of Bungie's Halo 3 ODST review events, feels the practice is ultimately damaging to a reviewer's opinion of a game.
"While reviewing games can be enjoyable at times, nothing makes it feel more like work than having to plow through a game over the course of 24 or 48 hours," Frushtick says. "And, on top of that, you're forced out of the comfort of your home and office, at times traveling across the country, to slog through an experience that most people will play over the course of a few weeks."
"Unfortunately the importance of having a review as close to a game's release often trumps a reviewer's desire for comfort, but it's the sort of decision that needn't be made," he says.
Bungie's Brian Jarrard sees another potential negative aspect to hosting such events.
"Reality is that these events are often only reserved for 'top tier' media due to space and cost which means a lot of the little guys are left out," Jarrard says. "I'd wager that these fansites, bloggers and more community-oriented outlets probably don't like being last to review a big title because they can't get an invite to the event or get an early copy of the game. Perhaps that leaves a bad taste in their mouth."
Dan "Shoe" Hsu, former editor in chief of EGM and Bitmob.com co-founder—who has since returned to the revived Electronic Gaming Monthly print magazine—has his own concerns about the nature of review events. He's personally participated in review events "under a 'controlled environment'" for 2K Games' BioShock and Bungie's Halo 2.
"I have very mixed feelings about this," Hsu says. "At EGM, we would never, ever want to taint the reviews process in any way. Our readers don't play games under these conditions — in nice hotel suites with developers, producers, and PR nearby. Why would we as professional critics? Can we fairly review something if we're playing under different circumstances than our audience?"
"At the same time, who would we be hurting if we banned such events? Those same readers," Hsu says. "Now we can't get a review out to them in time for it to be relevant."
The perception that review events are score-inflating wine and dine affairs isn't difficult to understand. Activision and Infinity Ward shouldered the cost of hotel rooms—including ours—and travel expenses for some of the other attendees of the Modern Warfare 2 review event. It also paid for the meal we enjoyed at the aforementioned fancy restaurant. The robe and slippers, too, but those we gave away for charitable purposes.
In the case of our extended Grand Theft Auto IV review event, our parent company paid for our flight, hotel room and meals. But Rockstar reps joined us for a dinner.
Did those so-called perks influence my opinion of either game? I'd like to think they didn't, confident that I can separate external factors from the core experience. I personally find the practice of video game review events an inconvenience. They provide an opportunity to remain focused on the task and title at hand, but traveling to play a game for days on end is not my preferred method of review.
Veteran game reviewer Jeff Gerstmann has similar thoughts about the "misconception" that these assignments are "cushy" or "lavish."
"There's nothing lavish about being cooped up in a dark hotel room for two days. It's annoying," Gerstmann says. "And unless you have direct questions about a game, the company reps at the event usually just stay out of your way. Most of the time they don't even ask us what we thought of the game. They just sort of hang back and, I guess, hope for the best."
"I sort of get the impression that no one involved on either side really enjoys these events," Gerstmann adds. "But between us wanting timely coverage and publishers feeling protective of their biggest releases, I don't see these events going away anytime soon, either."
Bitmob's Dan Hsu may say it best.
"As long as the game reviewers can treat the product fairly and objectively, the same as if he were playing at home or in his own office, I don't see a big problem with this," Hsu says of the conditions. "It's either that, or if you want a truly untainted review, stop listening to the professionals and get your feedback from the community instead."