Are Video Game Reviews Broken?

[Vampyr]

Originally Posted by Typhoid

But how is it fair to compare a Wii game to a PS3/360 game?

All of those download games (like Braid etc) don't get low scores [if/when reviewed] because they have bad graphics. If anything the graphics usually (from what I've seen) are a forgotten factor which is trumped by "fun".

I'm not one to say that Raffi is as good as The Beatles - but I don't think it would be fair to compare the musical stylings of Raffi to that of The Beatles simply because they both make music.

Braid is a fantastic looking game. The artistic values are high, and it's in HD.

If it were a Wii game, it would look blurry and be full of jaggies. If it had been released on 360 and the Wii, the Wii version should get a lower score because of that.

To give them the same score would be to say to the consumer "These are two equal products, in every category", which just isn't true.

[BreakABone]

Originally Posted by Vampyr

It's my opinion that Wii games should almost always get low scores on graphics, unless they are very artistically unique.

They shouldn't get a free pass because Nintendo flaked out on HD.

I wonder how many people would take me seriously if I said this:

It's my opinion that Xbox 360 games should almost always get low scores on graphics, unless they are very artistically unique.

They shouldn't get a free pass because Microsoft flaked out on 3D.

I mean it sounds almost as absurd to me.

The problem is, and you kind of covered this, when its a game on multiple consoles, graphics sometime factor into the score.

And in general, you will see that reflected in Wii scores
Star Wars, Call of Duty, WWE games, all score lower on the Wii than their HD counter parts. But as you figured, very little crossover between HD games and Wii games.

And the ones that kind of rely on HD graphics and advanced AI generally don't make it because of that.

Anyhow, another mini-rant, but one this reminded me of.

Is I loathe the idea of comparing review scores, which again always happens, but to save myself the time, just gonna quote myself.

Why is it then, the first thing that happens when a video game review is posted is searching for other reviews either from the website/publication, the reviewer or similar genres. And usually without doing anything more than looking at the final score, we begin to have all sorts of rants and raves fill up the internet.
“I can’t believe they gave game A a 10 in graphics, but game B only a 9.9 it looks so much better.”
“Really Game B has a better story than Game A? I don’t think so!”
“I can’t believe they say Game A is the best game on the console, but it scored lower than Game B!”
Those are just blanket statement, though I would be lying if I did say they weren’t based on actual comments I read in regards to games released in the past year! The fact of the matter is game reviewing doesn’t exist in a bubble. What was a 10 2 months ago, may not be considered a 10 today. Hell, what was a 10 last week may not be considered a 10 this week. That’s how fast things change. New games, new ideas, new levels are achieved at a rapid pace. There’s no way to get a uniformed system that allows for a clear progression of quality because honestly it doesn’t exist.

Even taking games from the same series may yield wildly different results. Perhaps, the 2nd game did something remarkable, but in the 2 years between sequels has become commonplace or maybe it builds on the strength of the last game without really adding anything new to the formula. Do you grade it up because it is just as good as the last one but doesn’t really re-invent the wheel or do you grade it down because it plays it safe?

End of the day, because I realize this is getting long-winded, there’s no way to compare review scores. Even if you were to line up perfectly and get the same reviewer, reviewing the same genre, on the same platform and released the same day there’s no way to say that Game A is better than Game B based on a score. And that’s all I want gamers to take away from this, stop bitching and moaning about your soon to be favorite game not getting the respect it deserves based on a random number!

[gekko]

Certainly the publisher provides the journalists with the copies of the game before it's released, but it's not with any agreement that the game will receive a high score. Reviews are advertisement, and almost certainly ensure when the game releases there's an article on the front page of every site featuring it. Sure a good review is always what they want, but this is video games, not politics. It's not ALL corrupt.

Yes, there is certainly a relationship there. IGN would be nothing if it couldn't get the news from publishers, but IGN is also the largest gaming site on the web, and publishers want their game on the front page. Larger sites like IGN have strict policies to prevent any possible issues. IGN has policies against journalists receiving gifts from publishers, and the journalists are also deliberately not told about advertisement deals. For example, the journalist reviewing Gears 3 isn't allowed to know that the advertising department was planning to give Gears 3 the full-page ad this week because it might influence the score. If the game for a 5/10, that ad would still have gone up.

Now there are certainly examples of corruption in the past, but that kind of thing can easily kill a journalist's career and severely damage the reputation of a company. For a fan site or blog, that means nothing. For a site like IGN, those guys (some trained journalists, some not) are making a living doing that job, so that's a huge risk. Also it's a big risk to News Corp. if they get caught. There's famous stories of this kind of thing happening to companies as big as CBS (with the famous Wigand story), so I'm not saying it's out of the question. However, it's certainly not the norm.

[Angrist]

But you're never saying that in your review. What you're saying is "these are the differences, but based on what the console is capable of, this is the score."

[Typhoid]

To give them the same score would be to say to the consumer "These are two equal products, in every category", which just isn't true.

I didn't mean to insinuate that 1 game for multiple platforms should get the same score regardless, I'm saying that the Wii version (or any version of any game) shouldn't get docked marks because of a version (or a like game, for that matter) for a different console.

To say "This game is good, but it's better on (system)" isn't a review. Or to say "This got a 9 on the 360, but I'll give the Wii version a 7 because the graphics are worse" is flawed to begin with because it's entirely irrelevant to the game itself. It should be compared to other games for the same system, not the same game for other systems, or other games for other systems. If I'm reviewing a country singer, I'm not comparing it to jazz-fusion. I'm comparing that country singer to other country singers, or to that specific country singers previous endeavors.

Now, to jump ahead here; say that country singer also released a jazz-fusion album. I still wouldn't compare the two to each other because they're not comparable, despite both being music. Sure, you can compare them - but the comparison wouldn't be accurate to begin with, not to mention highly subjective. Which is a whole other can of donkeys.

But you're never saying that in your review. What you're saying is "these are the differences, but based on what the console is capable of, this is the score."

I completely agree.

[Vampyr]

Originally Posted by Typhoid

I didn't mean to insinuate that 1 game for multiple platforms should get the same score regardless, I'm saying that the Wii version (or any version of any game) shouldn't get docked marks because of a version (or a like game, for that matter) for a different console.

To say "This game is good, but it's better on (system)" isn't a review. Or to say "This got a 9 on the 360, but I'll give the Wii version a 7 because the graphics are worse" is flawed to begin with because it's entirely irrelevant to the game itself. It should be compared to other games for the same system, not the same game for other systems, or other games for other systems. If I'm reviewing a country singer, I'm not comparing it to jazz-fusion. I'm comparing that country singer to other country singers, or to that specific country singers previous endeavors.

Now, to jump ahead here; say that country singer also released a jazz-fusion album. I still wouldn't compare the two to each other because they're not comparable, despite both being music. Sure, you can compare them - but the comparison wouldn't be accurate to begin with, not to mention highly subjective. Which is a whole other can of donkeys.



I completely agree.

That analogy doesn't make any sense. A better analogy is to say "I'm going to review this country singer's album, but I'm only going to compare each song to other songs on the album, instead of taking all the other country album's into consideration."

Sure, you can say one song is better than another on the album, but would it be fair to give a song on the album a 10/10 just because it's better than every other song on the album, even if the singer doesn't do some things as well as another singer?

Reviews only serve one purpose: making a recommendation to a consumer on whether they should purchase a game or not. If a game is released for the Wii and the 360, and is exactly the same in every regard except for the fact it isn't in HD on the Wii, and is blurry with a lot of jaggies, then the reviewer HAS to recommend the superior version to the consumer - and by doing so the reviewer has to give the higher score to the non-Wii game.

If the game is only released on the Wii, then the artistic value of the graphics have to be taken into account. Sometimes that might be enough to bump the score up, but more often or not the reviewer is going to have to dock some points because this Wii FPS doesn't look as good as other FPS's right now, or this Wii RPG doesn't look as good as other RPG's out right now. The whole point of a review and the points on it are to make a recommendation to a buyer.

But this is one of the main reasons I like Giant Bomb reviews. They don't break things down into categories like graphics or gameplay. It has one rating out of five stars, and those stars of all encompassing. Depending on the game, graphics might not even factor into the star rating.

[Typhoid]

The reason I think it's retarded to compare a game to other versions of itself is because they're not interchangeable with each other. It's not like an attachment for your car where you can just get a different looking cupholder if the one you got doesn't suit your needs. It's an entirely different form of media you need to play the two/three/four different versions of the game that has a multi-hundred dollar difference that is solely hinged on a handful of people saying "Well, we've got some jaggies here."

[Vampyr]

Originally Posted by Typhoid

The reason I think it's retarded to compare a game to other versions of itself is because they're not interchangeable with each other. It's not like an attachment for your car where you can just get a different looking cupholder if the one you got doesn't suit your needs. It's an entirely different form of media you need to play the two/three/four different versions of the game that has a multi-hundred dollar difference that is solely hinged on a handful of people saying "Well, we've got some jaggies here."

How are they different? Other than the fact that you may have to waggle a controller on the Wii version, they are interchangeable.

[BreakABone]

Not to point fingers, but I think the discussion with Angrist, Vamp and Typhoid exhibits the problem I have with reviews (and this does extend beyond video games), but you folks are arguing the technicalities of the numerical system/score, but at the end of the day its an arbitrary number.

And I think too many people focus on the score (which was part of the article I linked to), more so then the actual content provided in a review.

I don't care if a game gets a 5 or 10 in graphics, I assume somewhere in the written word, someone will explain how it negatively effects the experience or not, and that's more important.

Anyhow, I will say that there are some shady practices in the game industry, but part of it deals with the fact that the gaming press acts as glorified PR reps, and that's because you need that coverage for hits and for money. So its a vicious cycle where you want to keep them happy so that they keep you happy.

[Typhoid]

but you folks are arguing the technicalities of the numerical system/score, but at the end of the day its an arbitrary number.

I'm just articulating my point very poorly.
Wii games should only be scored against Wii games (as with all other systems). Therefore a Wii game that gets an 8 isn't on par with a PS3 game that gets an 8 - because (in my mind) they shouldn't be rated on the grand scheme of "video games", rather what system the game is for; if the game is for multiple systems, review them separately.


But I agree with everything else; the scale is broken, not enough people read the content, and the whole system is just a long, slow circle-jerk.

[Angrist]

Originally Posted by BreakABone

Not to point fingers, but I think the discussion with Angrist, Vamp and Typhoid exhibits the problem I have with reviews (and this does extend beyond video games), but you folks are arguing the technicalities of the numerical system/score, but at the end of the day its an arbitrary number.

And I think too many people focus on the score (which was part of the article I linked to), more so then the actual content provided in a review.

I don't care if a game gets a 5 or 10 in graphics, I assume somewhere in the written word, someone will explain how it negatively effects the experience or not, and that's more important.

Anyhow, I will say that there are some shady practices in the game industry, but part of it deals with the fact that the gaming press acts as glorified PR reps, and that's because you need that coverage for hits and for money. So its a vicious cycle where you want to keep them happy so that they keep you happy.

All I read of that was "5 or 10".