Danc at Lost Garden has another take on the utility (or lack thereof) of game reviews in today's gaming landscape, this one looking at the 'expertise bias.' He points out the disparity between reviewers and players when it comes to looking at new games — especially ones that have a gentle difficulty curve. His basic operating premise is that because game reviewers have plowed through so many titles and mechanics, they're looking at 'difficulty' in an entirely different light than vast portions of the audience. What will the future look like? He posits observation of other players is going to become increasingly important to developers, and if reviews can't keep up with that, they will really fall by the wayside:
If you are serious about providing objective insight into a game, either a title you are building or one your are reviewing, your expertise is not enough. In fact, your vast mastery of game related skills is mostly likely causing a giant bias in your judgments. You need to fight this bias by observing other players over and over again. They will do things with the game that are a source of wondrous insight. Your expertise becomes a tool for making great changes based off these insights, not one for predicting a priori exactly how all users will react to the game.
As for the current review industry, it is built on the unstable foundation of expert opinion in the absence of actual player observation. As games evolve and become ever more about first time learning experiences, the traditional game review will become increasingly irrelevant. It is arguable that they've already stopped informing most buying decisions and now serve as little more than entertainment for the hardcore niche. As the value proposition of reviews falter, the vast, churning, capitalist forces of creative destruction will replace them with a much richer set of game criticism that offers real value to its readers.
We've heard a lot about why the reviewing structure is broken, but this is an especially thoughtful take on the problem. I'm not sure it's one that can really be gotten around (critics — of the game, film, food, or book variety — tend to get those positions by being 'experts'), but it makes for interesting reading to be sure.