Placebo Effect in Games

[{CH}ArticleBot]

Here's a small and interesting story: New Scientist have a report from an experiment done by Paul Cairns, a professor of human-computer interaction at the University of York. The idea was to see if the placebo effect which has been tested in other areas works the say way in computer gaming.

To test this, his team had several players play Don't Starve and told them that while the first map would be random, the second map would be using an adaptive AI that changed the difficulty level according to the player's skill. The better the player, the harder the map would become, and vice versa. In the survey the players filled out after finishing the game, players revealed the placebo effect. According to the article:

"[N]either game used AI – both versions of the game were identically random. But when players thought that they were playing with AI, they rated the game as more immersive and more entertaining."

The results are backed up by psychologist Walter Boot at Florida State University, who concludes that the experiments clearly show how "expectations influence people’s gaming experiences".

 

I find it an interesting read and it makes me wonder how many Heroes VII players have their playing experience marred by having their expectations lowered by reading reviews beforehand. After all, as the experiment clearly shows: if you expect the AI to be bad because people have told you it is, you may experience it as such no matter the actual quality. What if you're a medium to low-level player who doesn't care too much about the AI - maybe it would be good enough if it was of a medium quality? A competitive player would find it inadequate. Reading a review written by the latter could lower expectations even for players who have lower requirements.

Something I will definitely keep in mind when writing our review.
Read full article...

[cjlee]

My god, you could be an apologist for Ubisoft. The reviews were bad, so many buyers didn't like H7! Blame the fans once again!

If GM could get away with blaming drivers, Macdonalds' get away with blaming diners, Samsung get away with blaming handphone users...

[Kalah]

I'm offended by that. Reading the second paragraph here (where you can see that I agree with the fans who say that the current state of the game is poor), the criticism in my Heroes VI review as well as a plethora of critical articles I have written over the past several years, it should be clear as day that any claim that CH are anything but independent are ridiculous.

Were I to answer in kind, I may as well claim that your stance the last five years has solidified to the point at which you are unable to see any positive aspects of Ubisoft's M&M production.

We here at Celestial Heavens are trying to see both the good and the bad. The biggest difference between you and I is that I rend to remain hopeful rather than despondent and dejected.

[Erwinner]

why not just strive to be very honest though, instead of artificially going out of one's way to be either hopeful or dejected, or dig for good things to say if they are few and far between? you've heard of the false balance fallacy, right?

I don't think someone like cjlee could possibly have any agenda to gain from by criticizing Ubisoft, as if this is party politics or something, I believe it is usually just his legitimate reaction to their screw ups,

and hey maybe he just subscribes to a different brand of hope to you, maybe he thinks harsher, more brutally direct criticism instead of platitudes could speed up a change in Ubisoft practices, and expedite a better game? (is not to suggest you are using platitudes, but I couldn't entirely blame him if he thinks this article screams "our review will be level-headed, all of you filthy peasants' opinions are invalid") lol

[ToweringAmishPlumber]

Wow! The truth hurts. I think the article is very accurate. I can't count the number of times
someone has told me they didn't like a particular movie, book, game, whatever, only to be told when asked if they had seen the movie, read the book, etc.answered no, they didn't need to. A friend, critic, whatever had told them it was bad. Therefore, they didn't need to partake, and if they did, they view was almost always negative. I will bet that if a person's expectations are low, that that will almost always start something with a negative attitude that will be almost impossible to change. There's also the old "stuck in the past" attitude where something from the past is ALWAYS, automatically better - can you say Heroes III?

[cjlee]

Everyone here has a point.

However if you read my H7 comments made in the interim when Ubisoft started announcing things like they'd changed back to the old resource system, before H7 actually came out (especially those made in the first half of this year), I was actually quite positive about H7 before I knew enough about it.

Ubisoft had done a very good job, prior to June this year, of RAISING my expectations. They really looked like they had learned from past lessons, and I said as much.

[vicheron]

So the reason why the AI never waits, always goes after the weakest stack (even when there's only 1 creature in it or if it puts them in a bad position), has archers defend as soon as a melee unit gets next to them (even if there's only 1 creature in that stack), gets confused by cover and run around behind it rather than move forward, immediately sends its creatures out of the walls when defending in a siege, send in a bunch of heroes with no armies that don't actually take anything, doesn't pick up half the resources around the map, and doesn't build up its cities very well was all due to my low expectations? I must have psychic abilities that can magically make the game's AI dumber. Maybe if I expect my lottery numbers to be the winning one, I'll be a millionaire by next week.

Also, the game used in that study is an action adventure. The AI doesn't have to do anything complex like base building, resource management, coordinate troop movement, manage hero skills, etc. I'm pretty sure that if they had done this study with a strategy game like Civilization, Starcraft, or chess, people would notice the difference. If a chess AI was just randomly moving a knight in a circle, it'd be pretty easy to figure out that it's just doing random stuff.

[Fnord]

Interesting study, but wondering if the conclusions may be flawed. They didn't say if players had ever played the game before (guessing probably not), so the fact that they enjoyed it and found it more immersive the second time they played could simply be that they were over the learning curve and at the stage where they were enjoying the game more and finding more depth in the game play.

I wonder what the results would have been if they had told them that a third game to follow would go back to a random map, or perhaps better, if some players were told that the second game would be using the adaptive AI and some were told that the third would.

As far as Heroes goes, I'm sure expectations play a part in one's experience. On the other hand, haven't you ever watched a movie or TV show or even read a book that you heard might not super great but were pleasantly surprised? I'm sure that applies to some games too. So I think expectations can go the opposite way as well: having high expectations can sometimes lead to easier or faster disappointment and low expectations can do the opposite.

[mns3dhm]

So write your review already Kalah; Celestial Heavens used to be a thought leader concerning 'all things Might and Magic'. The notion that waiting to post a review until the game is 'patched', 'bug fixed', 'balanced' or 'stabilized' is a bad idea. Stand up and say what you think or by the time you do post a review, the vast majority of gamers won't care.

[Kalah]

@mns3dhm: I appreciate your thoughts, but I prefer to wait - and I don't mind telling you why:

Like I said when I wrote my H6 review, I think it's a real problem that the media (are forced to) write reviews hastily and not take their time to evaluate all aspects of the game. I mean, one of the most important things of the Heroes games is the editor. How much time have the reviewers spent on testing that?

I think we can all agree that a good game is something you play for a long time. People in here still play H3 and H4. Be honest: if you had based your opinion of H4 solely on the early reviews and the early state of the game, would you still play it? When H4 was first released, it was horrible in many ways - it was only with its patching and eventual completion that it turned into a game people would play for decades. Now, I'm not going to wait several years until I review H7, but I don't want to review an early and buggy version either, because that's not the game people will be playing a year from now.

[cjlee]

Fan goodwill exists, but it is a reservoir that WILL be depleted if you disappoint them too much.

When I got H4, I knew there were tons of bad reviews. But I was willing to give 3DO a chance. I did not even want to wait 1 week for a patch.

And while I was not satisfied by game stability at all (and back then it took forever to download patches on our slow internet), I eventually loved H4.

Right now Ubisoft has managed to damage my goodwill so badly, I'm not going to trouble myself with getting a copy until people start praising the latest patch. I don't even want to waste my HD space on a "free" torrented copy. Shix doesn't start smelling good just because it's free.

[cjlee]

Darn, I can't edit my last post. I wanted to say, Kalah, why don't you wait till a definitive patch is posted.

You guys imagine I am super prejudiced against H7, but the fact is, if they produced Super Patch tomorrow and an experienced and respected member of the community like Kalah, Jeff or PandaTar could review it and give an unstinted thumbs up, I WOULD buy H7 at full price.

As I said in the forums, miracles happen. Germany beat Brazil 7-1 in World Cup. On this entire planet, it was reported that NO ONE won a single football bet from all legit bookies because no one had betted on this outcome.

[Karmakeld]

I can relate to Kalah's point of view. If ppl. were to review a bands demo tape you would likely get comments like, "the sound is bad, it seems unpolished, songs seems unfinished etc" ofcourse -it's a demo/rough cuts. Now H7 is no longer on beta, and surely the product should've been polished. I for one was looking forward to this one, I believe that the editor (from what I've seen so far), could actually make H7 a game that would be played for years to come. Therefor I support Kalah's decision to way for an improved version. But it should be fair to do a review after first patch, as it would be expected to fix atleast +90% of the bugs. But knowing you'd have to wait for patch 2 or 3 before getting a bugfree game, would be a failure declaration in my optic and should result in critical review, simply because it's embarresing towards your customers to release an unfinished product.
But like Cjlee, I've decided to postpone my buy, untill I know it's worth buying. Atleast with the cost it have atm. compared to game experience and performance.

[Panda Tar]

The only difference between waiting for H4 and waiting for H7 is that in H4 the additions and new features felt fun even in the awful state of the game. In H7, there's nothing really new and the recycled stuff feel bland. Still, I concede that I cannot manage to play H4 without Equilibris anymore. Call me spoiled panda.

[vicheron]

Heroes 4's early reviews were significantly better than Heroes 7's. People were willing to give NWC a chance because Heroes 1 - 3 were great games and didn't have huge problems and bugs when they were released. Whereas Heroes 5 and 6 were both released in terrible states and took a long time to fix, with Heroes 6 never getting fully fixed. It makes sense that people would trust a company with a good track record more than a company with a bad one.

Also, people who paid full price for a game have legitimate reasons to complain if it's not finished. People who pay $50 for a game are expecting a finished product. They're not paying to be beta testers for future customers.

[igoraki]

Wont go into discussion about Heroes 7, but have to say i do agree with findings in the article since i caught myself in how others ppl opinions or reviews influence mine perception of the product.Most noticeable with movies, since i dont play too much games lately.If the buzz is positive am usually having big expectations, there is a tendency to call movie bad quite easily if it turn out to be less then expected but if there is not hype or its even negative, equally bad movie will be labeled as average, not so bad or not great, but ok

Unfortunately we are too influenced by marketing in this day and age, guess we can blame the internet

[ywhtptgtfo]

My god, you could be an apologist for Ubisoft. The reviews were bad, so many buyers didn't like H7! Blame the fans once again!

If GM could get away with blaming drivers, Macdonalds' get away with blaming diners, Samsung get away with blaming handphone users...

I notice this trend too... with reviewers blaming fans for having rigid expectations and wanting carbon copies of Heroes 3 in sequels.

This does not address the fact that the bulk of the fury tend to involve the serious stability, performance, and gameplay issues. Is it so much to ask for to have a Heroes game that runs relatively fast, that doesn't crash or glitch out every now and then, and to have an A.I. that doesn't walk around aimlessly or sit in its own castle for the entire duration of the game?

[cjlee]

Is it so much to ask for to have a Heroes game that runs relatively fast, that doesn't crash or glitch out every now and then, and to have an A.I. that doesn't walk around aimlessly or sit in its own castle for the entire duration of the game?

YES, IT IS TOO MUCH TO ASK.

Fans have unrealistic expectations now. Games are all very complex, so they run slow, have glitches, and cannot have good AI.

Fans should stop thinking that a Heroes game should be smooth, stable, have a challenging AI and relatively free of game breaking bugs. You don't use Windows 95 anymore, do you?

Signed, Ubisoft Corporate Flunky

ps please don't invoke Blizzard as an example. If you want to go the way of Blizzard it will take you 20 years to get Heroes of Might and Magic 8.

[Kalah]

@cjlee: Just for clarity's sake, you're not actually quoting an Ubisoft rep. there, right?

[Karmakeld]

sniff sniff.. What's that smell..? Did someone just spill their gallbladder