The Power of Reviews

Kalah · Unread postby **Kalah** » 05 Oct 2012, 22:27

<img src="/images/news/PAreport.jpg" align=right vspace=5 hspace=5>The <a href="/http://penny-arcade.com/report/editoria ... nk1">Penny Arcade Report</a> have dragged up something interesting. PA have had a sit-down with the game research company <a href="/http://www.eedar.com/">EEDAR</a> and gotten some data on the importance of positive reviews:

"...the results are startling: a game with an average review score in the 90s will sell three times as many copies as a game rated in the 80s. Lower scores can sink a game. The influence of game reviews also extends into word of mouth and user reviews. The critics have more power than we thought."

Read the whole article <a href="/http://penny-arcade.com/report/editoria ... ere</a>.

If you would like to take a look at the original page visit this link:
https://www.celestialheavens.com/1349476045

hellegennes · Unread postby **hellegennes** » 06 Oct 2012, 12:45

This sounds 100% unscientific. In order to determine the influence of reviews, they should have measured games that would have gotten good reviews but never got any reviews at all.

Their conclusion is flawed. It's more likely that games get good reviews and have high sales because they're good. How do they establish the connection between reviews and sales? How do they know how much of the sales owe to good reviews? The correlation seems weak. For example, they say they compare the sales of games that lets you customise X VS games that don't. But that's absurd because you are measuring just one fact and relate it to sales, which in reality may have no effect at all (or worse, it could have a negative impact). That's like saying Red cars are involved in accidents more often. But that's an observation with no statistical value; the colour of the car plays no part in the probability.

So, is it that games have high sales because they get good reviews or is it that games which are good have high sales and in the same time get good reviews?

To see if there is a real connection between reviews and sales, you have got to check the list of each year's best sellers and compare it with their reviews. Wii Sports is the best-selling game of all time, yet review aggregator, Metacritic, gives it an aggregate score of 76%. If you actually take a look at Metacritic's highest and lowest rating game lists, you'll see there's not much connection between their scores and their sales. Some games get good reviews and high sales, but others have either bad critical reception or low sales. I mean, Myst was one of the worst rated games of all time, yet it became the highest grossing PC game of all time (until Sims surpassed it, in 2002). After the initial release, reviews became more favourable FOLLOWING sales.

Telumehtar · Unread postby **Telumehtar** » 07 Oct 2012, 21:27

I think you're right, but not entirely. If you drive a red car you actually have less chances to have an accident...

I was always against numbers in reviews. A system of "followers" would be much more appropriate.

mr.hackcrag · Unread postby **mr.hackcrag** » 08 Oct 2012, 02:27

All hellgen is saying is that correlation does not imply causation. The only way to make that conclusion (which I shouldn't even be talking about since I didn't read the article, just infered it based on last two posts) is by experimentation, which was most likely not done.

hellegennes · Unread postby **hellegennes** » 08 Oct 2012, 10:52

@mr.hackcrag:

Well, according to the article, they only collect data, so the basis of their assumption is the comparison between sales and review scores.

Now, reviews can certainly influence sales, but is it measurable this way? No. So, yes, in order to determine the real relation between sales and reviews you have to go by experiment. One way to do it is to establish two teams of people who have heard of a game and are equally likely to buy it, then have one team read good reviews and the others read bad ones (the teams need to be statistically balanced). You then ask them if they are going to buy the game and if and how the reviews changed their mind.
Edited on Mon, Oct 08 2012, 06:53 by hellegennes

Angelspit · Unread postby **Angelspit** » 08 Oct 2012, 18:29

hellegennes wrote:You then ask them if they are going to buy the game and if and how the reviews changed their mind.

And they'll tell you they'll just download the game because they had no intention of buying it in the first place.

hellegennes · Unread postby **hellegennes** » 08 Oct 2012, 19:20

Right. So I'm guessing the $65 billion in earnings comes from the 1% that doesn't download their games. I see... the gaming industry should yield $6.5 trillion globally, or around 10% of the world's GDP.

Campaigner · Unread postby **Campaigner** » 23 Oct 2012, 18:45

That's disturbing....I believe it inflates scores (just like they weren't high enough already....).

I don't trust big reviewers anyway so doesn't affect me. I just hope that the small (and honest ones) can resist the temptation to go big and get $._.$

The Heroes Round Table