Do gamers really want unbiased reviews?

Gamers say they want unbiased reviews, but are they being truthful? Outage over some recent reviews suggests that gamers want confirmation, not criticism.

The average game review score on Metacritic is currently 72.3. Of all the games listed on the site, the current average score for all of them is just slightly better than 7 out of 10. That’s a solid three and a half stars out of five.

Take a visit to any gaming forum and you’ll likely find a thread moaning about this fact. Some forums have entire threads dedicated to complaints about the terrible state of gaming journalism and the positive bias that makes reviews difficult to take seriously. That hasn’t been helped by the recent furor brought on by the article written by Rab Florence and published by Eurogamer criticizing the state of gaming journalism, which then turned into news when one of the subjects mentioned by name, gaming writer Lauren Wainwright, threatened to sue for defamation after Florence quoted her tweets verbatim. Eurogamer relented due to the UK’s bewildering libel laws, but the story shone a light on the ethics of the gaming industry. 

Florence’s article argued that the industry and those covering it were too tightly entwined, which led to a natural, if not deliberate bias in both publications and game publishers alike. Publications have little choice but to strike ad deals with the companies they’re meant to criticize. Game publishers try to win over journalists by flying them to exciting events or, if that fails, subtly giving trouble-makers the cold shoulder by refusing to send out early press copies and  slapping a lock on previews.

Both practices are worthy of a raised eyebrow, but are they the root of the problem? Or do gamers only have themselves to blame?

Is there an industry conspiracy?

The relationship between publications and publishers is a problem, but it’s not limited to the gaming industry. Almost every form of media that’s frequently reviewed, from movies to music, faces the same issues. Metacritic’s average score for movies is currently a 61.4, the average for TV is a 61.3 and the average for music is 71.6. Positive bias abound – granted, not as much as that found in game reviews, but it is still there.

Another problem with blaming the industry is the review score average of the most popular and influential sites. These sites work most closely with game publishers and rely heavily on their ads, so if industry incest was the problem, we’d expect them to post exceptionally high scores. They don’t. The ten most active game review sites score about 4.5 points lower than the industry-wide average (on a 1-100 scale).

Both of these points make it hard to believe that positive bias in game reviews is the result of an industry conspiracy. So what’s the problem?

It’s the gamers.

Gamers don’t want criticism

This is a thought that has bounced around my head over the past year, and Tom Chick’s review of Halo 4 crystallized it.

Mr. Chick’s blog, Quarter To Three, has a reputation for being critical. This is reflected in its Metacritic average of 62, which is the 9^th lowest average score among all publications the site has ever listed. No one was surprised to see him give Halo 4 one star (out of five), which translates to a Metacritic score of 20. In his review he slams the latest entry in Microsoft’s headline franchise for being “ponderous, familiar, and disappointing.”

Gamer response was quick and vicious. Over 1000 comments, the vast majority of them critical, were posted within 24 hours of the review being listed on Metacritic. The flames range from the dramatic, “Your cynicism is only rivaled by your ignorance” to morally outraged, “You are having a very negative impact on the years of effort that went into this game” to dismissive, “Oh look, a meaningless website is giving a meaningless score, trying to be special.”

The punchline? Most of these comments were published late on November 4^th or on November 5^th. while Halo 4 was not released until November 6^th. Mr. Chick was the only person in the conversation who’d actually played the game, not counting the handful of gamers that got a hold of leaked copies.

Quarter To Three did not catch flak from rabid fanboys alone. BigPond GameArena and EGM, which posted respective scores of 55 and 70, also were overrun with gamers bashing their less-than-stellar reviews. The scores on both of these sites are far from the average score of 87. Again, many of the comments were posted before the release of Halo 4. DT’s own freelancer Adam Rosenberg gave the game a very respectable 4.5 out of 5 for G4, which immediately drew the ire of fans, and fairly vicious complaints like “a 4.5? no wonder x-play is going under just give it the dam  [sic] 5 out of 5.” Our own review, which scored Halo 4 well at 8.5 out of 10, fared little better.

This is not a trend confined to Halo fans. Mr. Chick received the same hate (at lower volume) over his negative reviews of The Secret World, Max Payne 3, and Journey. Nor is this trend an artifact of Mr. Chick’s writing. Take a look at the comments left on any review that posts a score more than 20 points under the Metacritic average. Vitriol is almost guaranteed.

Confirmation bias

After reading critical reviews – and the response to them – I can’t help but feel that what most gamers really want in a review is confirmation. They want to know that Big Name Title X, the game they’ve looked forwarded to or have already invested hours in, is really all that.

I must admit that I’ve occasionally read reviews of a game after I played it. Why? Because I wanted to read the stories of others who enjoyed the same experience. Or because I wanted to see if anyone else had been critical of a game I hated. In essence, I wanted to see if people agreed with me. And while I rarely leave comments on reviews, I admit that a negative review of a game I love does stoke the flame of fanboy rage inside me.

These reactions are likely a part of human nature, but that doesn’t mean knee-jerk anger should be unleashed the moment it’s felt. Serious conversation requires serious restraint. That’s a trait many gamers seem to lack.

The bias gamers have towards confirmation sends a clear message. Publish a consensus review or shut up. Journalistic integrity is great in theory, but why should publications be expected to pursue it if their readers are consistently, vocally against any review that does not fall in line? It’s easy to talk about honesty and integrity when you’re not the journalist whose reputation is being attacked. It’s harder to believe in these ideals when gamers attack you for practicing them.

Like many of gaming’s large problems – including sexism, racism, and sequel-itus – the blame for positive bias can be aimed at the community just as easily as the industry. Gamers have no reward for honest critics. So why are we surprised that they’re so few?