Question about AI Quality

[Gaidal Cain]

Specifically reading Alamar’s post, I think he is definitely wrong when he says the goal should be to teach the AI to play the game like a human would. The reason for that is simply, that the goal of the AI is NOT winning ( at least not always; I thought that was clear now); the goal of the AI is making things difficult for the human player(s).

That doesn't ring quite true- if the game is a free for all, the way that the AI would make things most difficult for the Human would be either to join and rush the human (probably not very fun), or to simply hole up in their towns after having them built- again, not very fun. AI:s can't emulate humans fully, but it should act as it has somewhat similar goals and means- in the grail example, that might well be a viable human tactic- wait until someone else goes for the grail and then pounce. The impossibilty of reaching the state were the AI manages the Heroes equivalent of the Turing test doesn't mean that that is in general a good direction to aim. When playing an unscripted AI, I want to feel like I'm playing someone who's also trying to win- not just my way as hard as possible.

I simply don't think it's necessary. If you look at a Warlock for example and the skill probabilities then the heroes should develop well most of the time IF the correct abilities are taken for the skills picked.

The problem is that with H5, building combos of skills is required to make a real good hero. That means planning ahead, something which is very hard to do unless there is some underlying plan. If you wish to reduce tedium, make some 10 preset builds for each hero type with a few conditionals - some smaller, for skills like navigation or pathfinding, and some larger, for situations where the AI for example has easy acces to a certain magic school, and make the AI randomly progress towards that goal- which means that half-built heroes might differ very much even if they have the same goal. One can even prioritise some of the skills (let the Warlock pick Destruction roughly every third level up) so as to make more "even" heroes. Yes, it's quite a task, but less so than the alternative - which would be the emulating of a human you don't believe in.

[ThunderTitan]

The AI should be able to at least play the game without any cheats or handicaps in such a manner that he wouldn't defeat itself. It's not much of a victory when all you have to do is wait him out.

[Jolly Joker]

The problem is that with H5, building combos of skills is required to make a real good hero. That means planning ahead, something which is very hard to do unless there is some underlying plan. If you wish to reduce tedium, make some 10 preset builds for each hero type with a few conditionals - some smaller, for skills like navigation or pathfinding, and some larger, for situations where the AI for example has easy acces to a certain magic school, and make the AI randomly progress towards that goal- which means that half-built heroes might differ very much even if they have the same goal. One can even prioritise some of the skills (let the Warlock pick Destruction roughly every third level up) so as to make more "even" heroes. Yes, it's quite a task, but less so than the alternative - which would be the emulating of a human you don't believe in.

I think that's only true if you really have to go through the motions with all the neutrals. If you get an advantage - and I don't see a reason why the AI should have to play against the same amounts of neutrals than the human on Heroic difficulty - only the end result counts.

[Gaidal Cain]

I think that's only true if you really have to go through
the motions with all the neutrals. If you get an advantage - and I don't see a reason why the AI should have to play against the same amounts of neutrals than the human on Heroic difficulty - only the end result counts.

What the heck has neutrals to do with a good hero? The player is going to run into AI heroes. Seeing AI heroes with good skills makes for diversity (which I noted you were big on earlier), which is a Good Thing (tm). The game is filled with interesting abilities, and it would be a waste not to have to face them once in a while. The advantages of a good hero instead of a mediocre one and, say, a thousand extra gold/day might not be very big when it comes to challenge level, but the former will both feel more like playing an opponent that plays the same game, and for more variation.

[Jolly Joker]

From a human point of view a good hero is a hero that can handle the tasks. For example, a Warlock is good, if he gets Attack and Tactics early - but only because of neutral fighting.
So, GC, then go ahead and tell me what heroes are good at each and any point.

[Gaidal Cain]

A good hero is one where some thought has been put into the skills. Exactly what that happens to be depends heavily upon the type of map, hero type and how much time has passed, but to take an easy example: Warlock, fairly large/xp rich map.
Levels 1-5 should see one or two levels go to destruction (NB:I'll always be including abilities with the main skills) and probably one or two to IM, plus for example luck, logistics, Sorcery, summoning, attack or enlightenment. Next 5 is similar, but with some more focus on abilities. By level 15, Expert Destruction and IM should have been reached, along with a couple of extra skills, and a couple of "advanced" abilties (for example TA, Warlock's Luck, Secrets of Destruction).

The general idea would be to try and build a hero who'd look similar to what a good human player would have made (with perhaps a few exceptions of skills that isn't needed for the AI as scouting, unless it's needed for a particular ability, or there is no support in the AI code for the use of it). I don't see what the problem with that kind of approach would be; there is really no large difference between what the AI can do with a particular set of abilities and what a human can do

[Jolly Joker]

A good hero is one where some thought has been put into the skills. Exactly what that happens to be depends heavily upon the type of map, hero type and how much time has passed, but to take an easy example: Warlock, fairly large/xp rich map.
Levels 1-5 should see one or two levels go to destruction (NB:I'll always be including abilities with the main skills) and probably one or two to IM, plus for example luck, logistics, Sorcery, summoning, attack or enlightenment. Next 5 is similar, but with some more focus on abilities. By level 15, Expert Destruction and IM should have been reached, along with a couple of extra skills, and a couple of "advanced" abilties (for example TA, Warlock's Luck, Secrets of Destruction).

The general idea would be to try and build a hero who'd look similar to what a good human player would have made (with perhaps a few exceptions of skills that isn't needed for the AI as scouting, unless it's needed for a particular ability, or there is no support in the AI code for the use of it). I don't see what the problem with that kind of approach would be; there is really no large difference between what the AI can do with a particular set of abilities and what a human can do

Disagree completely. Your Warlock sucks. The skills you picked won't do anything for the hero.

[Alamar]

A good hero is one where some thought has been put into the skills. Exactly what that happens to be depends heavily upon the type of map, hero type and how much time has passed, but to take an easy example: Warlock, fairly large/xp rich map.
Levels 1-5 should see one or two levels go to destruction (NB:I'll always be including abilities with the main skills) and probably one or two to IM, plus for example luck, logistics, Sorcery, summoning, attack or enlightenment. Next 5 is similar, but with some more focus on abilities. By level 15, Expert Destruction and IM should have been reached, along with a couple of extra skills, and a couple of "advanced" abilties (for example TA, Warlock's Luck, Secrets of Destruction).

The general idea would be to try and build a hero who'd look similar to what a good human player would have made (with perhaps a few exceptions of skills that isn't needed for the AI as scouting, unless it's needed for a particular ability, or there is no support in the AI code for the use of it). I don't see what the problem with that kind of approach would be; there is really no large difference between what the AI can do with a particular set of abilities and what a human can do

Disagree completely. Your Warlock sucks. The skills you picked won't do anything for the hero.

How does that response add to the quality of the discussion? .... Did I miss your sarcasm?

[Sir_Toejam]

As for dig the tear VC,you could simply put in a failsafe that AI has to visit 90%/75%/55%/30% of all the obilisks on the map(depending on difficulty level)before it can start digging.

bit of clarification:

that is already programmed into the difficulty levels for the AI.

It has to find far less obelisks on impossible than on easy.

[Gaidal Cain]

Disagree completely. Your Warlock sucks. The skills you
picked won't do anything for the hero.

Yeah? How so? If the AI can't utilize skills, there's a problem with it that also would need fixing. I'd be willing to think it OK if it couldn't use for example TA effectively, but if it can't use something as simple as Destruction magic, then there are serious problems that needs correcting.

[Jolly Joker]

From a human point of view Destructive and IM do nothing for the hero within the first level. Destructive Magic would only have priority if you got BOTH Stone Spikes AND Lightning Bolt in the guild (25% chance). IM does nothing as well. For Empowered spells are most probably the spell points missing, for Elemental Vision is the building missing and Dark Ritual hasn't got preference in the first week.
Those skills won't "run away
Later on they will surely become useful, but a level 2-6 Dungeon AI hero that a human player would meet with this skills would NOT be good.
A preset building of heroes could only guarantee a good hero later on, at high level, not at ANY level. Furthermore it couldn't guarantee an optimal handling of maps, so what you call a better hero building would still not allow playing human-like and handling neutrals human-like.
Now, if you look at the skill probabilities for a Dungeon hero you see
Attack and Destructive at 15%.
IM, Sorcery and Summoning at 10% (with IM being there anyway)
and Logictics, Luck, War Machines and Eblightenment at 8%.
This leaves Defense, Leadership, Dark and Light at 2%.
If you program the AI to always take the ability as soon as the number of skill levels is under 40% (or so), and have a bit trust in probabilities, you'll get average good heroes. The only thing that would have to be taken care of is a handling of the more advanced abilities.

[Khelavaster]

You know, we played a hotseat 2v2 game on Hard yesterday with a friend that doesn't have the game but comes to play now and then since HOMM2.

The Yellow computer had locked itself in its castle (we never saw it until we reached it at the end of the game). We tried to come up with reasons for this behavior, map design, blockages, etc. but there was no one we could fathom. Broken.

What really irked my friend was the amount of free resources lying about that hadn't been picked up, including piles in the middle of the cobbled road. I didn't want to ruin the game for him by telling him that the computer CHEATS and that's why it doesn't need it.

The AI actually seems to get worse the more I play.

Khel.

[Khelavaster]

Forgot to add.

Green had captured three extra castles. For some reason, by month 2, week 3, none of these extra castles had been developed beyond Village Hall (Fortifications: NONE) so taking them was extremely easy.

Suggestion to Ubival: GIVE UP. PLEASE HIRE GUS SMEDSTAD.

Khel.

[PhoenixReborn]

Edit: Nevermind.

Err, when I play on hard the a.i. comes gunning for me with a huge army. They build up more than on normal too. Oh well.

[Khelavaster]

I was expecting the usual rush + moronic-beeline-to-your-castle too. Who knows.

Hence the "seems to get worse the more I play" line.

Oh well. Glad the AI at least suits your needs.
Khel.

[Alamar]

@JJ: We're just always going to disagree aren't we?

Could you be a little more specific in what you mean by "how"? Do you need to see a SPECIFIC algorithm, source code, or what?

I just mean some basic considerations about how the AI should act in practise, the way I did.

Well I'll try to be MORE specific but I'm not sure what it will accomplish. I will try to use general terms so that everyone understands [hopefully] what I'm trying to say.

On day 1 each AI faction gets its own independent AI thread so it can process its data and plan its moves accordingly. The point of spawning threads is that so when it's the player's turn the AI has some extra time to plan in the background to figure out what it should be doing.

While it's the player's turn I have the AI looking over variables like map size, guardian strength, starting hero levels, starting army strength, etc. Then you need to develope a time estimate for how long the various guardians are going to keep you away from the other factions AND how long the guardians will keep the other factions from getting to you.

This can be used to tell you if you should plan for a "big fight" in week 1, 2, 3, etc. This info will then be used to guide things like castle build priorities, hero build scripts to choose from, how much time you can afford to spend safely maximizing your economy / leveling your hero / etc. before you have to get to the payoff.

Once you know how much time you have then you can begin looking over "your territory" and deciding what things you can take so that your total army strength is maximized at the time you expect the big battle(s) to happen. Please note the AI needs to come up with a path / plan that lets it get to the most things of the highest value in the least amount of time. THIS IS NOT AN EASY PROBLEM per-se. If the AI isn't doing a good job we'll just have to program some "rules of thumb" and hope it gets close enough.

If [when] the AI has problems coping and not playing up to a challenging level then this is where you design a set of bonuses/cheats that will help make up for its deficiencies.

Through iterative testing and tweaking you should come up with a reasonable result.

May I ask why you see predictability as an issue? Could you elaborate on that please?
IMHO with multiple scripts allowed per hero [say 3-5] you would not see too much of a predictability problem. Why? Assuming you play with random heroes you'll have about 10 different heroes to choose from. With 3-5 different hero builds you'd have to play at least 30-50 games before you saw each pattern. Before you saw patterns repeat you'd have to play 60-100 games [probably much more!] before you'd notice the pattern per-se. By the time heroes became "predictable & boring" you're looking at [realistically] 200 games minimum!! [assuming use of random heroes]

Also, because I as a human tend to develope a given hero almost identically every time I play, why is it a problem if the AI does this? Why not use a system that will dramatically increase hero power thus leading to better fights & more competition with a human?

If you are worried about a human hiring a well-built AI hero that they could actually use then I don't see that as an AI problem .... I see that as a game design issue. Note your issue is just as large of an issue whether the AI develops a good hero who is defeated then hired vs. a human developing the good hero who is then defeated then hired.

[Sarcasm warning] Lord forbid we have AI heroes with good skills and abilities that match their spells fairly well. Lord forbid that we fix one of the BIGGEST AI problems that the HoMM series has had.[Sarcasm over]

I simply don't think it's necessary. If you look at a Warlock for example and the skill probabilities then the heroes should develop well most of the time IF the correct abilities are taken for the skills picked. Since I don't think that in battle AI against neutrals things are considered like whether the AI has Tactics or not (and I don't think it should be considered it's not necessary to pre-define hero development at all. I actually think it's against the spirit of the game because it's highly map-dependable what skills are the best set, really (Navigation, and Estates are examples for abilities that might be of use or may never be of use for the AI). The only thing that I somehow would like to make sure is that IF the AI has a magic skill it should get the abilities that fit to their spells as well.
The probabilities SHOULD do an adequate job now.

For now I think that we agree that H5 does a poor job [considering the skills & abilities available] at building a hero.

IF we rely on probability to give us what choices the AI can pick from we also need to make sure that when the AI is presented with choices it makes appropriate ones. Right now that "choice making AI" simply isn't there. FYI a crude "make sure X% of choices are abilities" is not NEARLY good enough. Whatever is done should [theoretically] fit in with some plan / goal the AI has at building the most appropriate hero given the circumstances that it finds itself in.

IF the skills choosing problem is too hard to solve then it makes sense to me to just remove the hard problem of "choosing the best from a set of random skills/abilities" to just choosing the best set of pre-canned scripts from a list of hero development scripts and just going from there.

As an aside I think one thing that may help the AI is to have a very simple to use but powerful set of tools in the map editor to help give the AI hints as to how the map maker would like to see them develope. This could include "mandatory skill choices" [Logistics then Navigation for example], suggested goals & priorities, etc.

As a matter of fact I believe that it would be a good idea to be able to give mapmakers HUGE control over things like AI bonuses and the like so that they can EASILY make sure they get the desired behaviour from the AI.

We clearly disagree on the purpose of an AI. To state it again the CORE principal is to develope an AI that is a "fun opponnent to play with" for a wide variety of players, with a wide variety of skills, on a wide variety of scenarios / maps.

Are you in any way saying that the above core value is wrong? If so then please explain why the goal of the AI should be "a fun opponnent"?

If you agree with the core principal then which of my specific goals based on that core value to you disagree with and why?

1. The AI should be competitive for a wide variety of players in a wide variety of scenarios.

2. The game experience, whether playing AI or human, should be as similar as possible.

3. The AI should not give us [I.E. players with many differing sensibilities] reasons to be upset with its behaviour.

Based on the above why is TRYING to teach the AI to play by the rules [not necessarily play like a human] a bad thing?

I disagree with point 1. IMO, the AI should be competetive NO MATTER MAP OR PLAYER OR VICTORY CONDITION.

I believe that you do not understand what I was trying to communicate. Your statement is a little too broad and general for what I was trying to get across .....

If I as a map maker decide that I want a deliberately weak faction [for story purposes] then I shouldn't have to go to great lengths to fight the game's inherent bonuses that allow the AI to "break my map" because it is too strong.

The AI should be strong when the map is set up in a manner that allows it to be strong [most situations]. However if I as a map maker design a side to be weak then it should be weak no matter WHO controls that side.

Point 2 is something I CLEARLY disagree with, simply because it is in direct contradiction to point 1. The AI cannot copy human behaviour in any way that would make it competetive.

Actually it does not HAVE to be in direct contradiction with point 1 at all. You ASSUME that the AI has to copy human behaviour in order to make sure gameplay experiences are similar. It doesn't have to do this although IF you could make that happen it would be great.

All you need to make gameplay experience similar is:

1. Have the heroes that the AI builds basically look like the sort of heroes a human might build [even if the builds have to be scripted] under similar circumstances.

2. Have the army sizes of the AI look roughly [maybe a little larger] than what a human would build over that time.

3. Have the AI on a time schedule that is not too dissimilar from a human.

4. Have the AI use some basic goals that humans might share.

Note you could "cheat like the dikens" where the AI doesn't have to do ANYTHING in a manner that a human would do it and still end up with a situation that would make the gameplay experience reasonably similar.

Right now H5 shatters 1-3 above and isn't even close.

Point 3 is phrased so wide that I can't agree with it either. Upset? Depends on what people upsets. It's basically impossible to please everyone and no matter how you do things there will alyways people that are upset about something.

Therefore that's why you have to allow people to customize the behaviour of the AI to suit their particular desires!!!

I feel confident that the developers of H5 didn't care about point 3 which is why everyone is crying about the AI because it gives LOTS of people a LOT of reason to complain. I contend that if you look at the AI power curve that if you didn't know that you would get complaints about "the cheating AI" then you didn't do your job to make sure the AI is a "fun" opponnent.

In the end you want an AI that is FUN first. Everything else comes second. If (to be fun) it needs to compromise other basic principles in certain circumstances then so be it .....

[Gaidal Cain]

From a human point of view Destructive and IM do nothing for the hero within the first level. Destructive Magic would only have priority if you got BOTH Stone Spikes AND Lightning Bolt in the guild (25% chance). IM does nothing as well. For Empowered spells are most probably the spell points missing, for Elemental Vision is the building missing and Dark Ritual hasn't got preference in the first week.
Those skills won't "run away

Hmm. I'm inclined to agree about IM - with the exception about Dark Ritual- by third level, you could have basic destruction, DR and Secrets of Destruction, which would be quite potent. It was just an example of one theoretically could build a hero. The details aren't as important as the idea- one could just as well build one that focuses more on attack, or luck.

If you program the AI to always take the ability as soon as the number of skill levels is under 40% (or so), and have a bit trust in
probabilities, you'll get average good heroes. The only thing that would have to be taken care of is a handling of the more advanced abilities.

Nope. Not true- you'll get subpar heroes on average, as a good Warlock (for example's sake) is one that (assuming that there's nothing special about the map, which would make this point even more valid) never has Dark or Light magic, and always has destruction. Trusting in chance means that you'll occasionally get heroes that doesn't fulfill this, and thus are subpar. The AI should actively pick good skills and abilities (somehow- it could do it the same way as humans, but that'd still require somewhat preset paths), and not trust in chance. (Just the same as the AI (if put under some real monetary constraints) shouldn't just build whatever's available but sometimes conserve resources for for example a capitol or higher-level dwelling).

[Jolly Joker]

@ Alamar
First, thanks for the lengthy answer; it makes things - your pov - a lot clearer for me and makes it indeed possible for me to point out where exactly I disagree with.
The main thing is this here:

All you need to make gameplay experience similar is:
1. Have the heroes that the AI builds basically look like the sort of heroes a human might build [even if the builds have to be scripted] under similar circumstances.
2. Have the army sizes of the AI look roughly [maybe a little larger] than what a human would build over that time.
3. Have the AI on a time schedule that is not too dissimilar from a human.
4. Have the AI use some basic goals that humans might share.
Note you could "cheat like the dikens" where the AI doesn't have to do ANYTHING in a manner that a human would do it and still end up with a situation that would make the gameplay experience reasonably similar.

With this kind of priorities I don't see a challenge anywhere, at least not a heroic challenge. To makeit a heroic challenge you would have to assume that the AI does everything BETTER than a human - a LOT better. Because you can safely say that the battle AI will never be able to be as good as a human. Heroic should be a challenge for veteran players as well, otherwise there is no long time motivation. There is nothing wrong with a non-veteran player being beaten by the AI on heroic. There is not even something wrong with a veteran player being beaten by the AI on heroic.
So it's simply not enough to do things roughly the way a human would do - which is not possible anyway considering the wide variety of possible victory conditions and tactics.
Strictly spoken, a heroic challenge on a map where the initial position is equal for all (a typicl MP map) means that the AI players must be stronger than the human player and I don't think this is possible with your basic concept.

[Gaidal Cain]

I think Alamar's definition is quite OK for what a "bare" Ai shopuld look like- one used on normal or maybe even hard. On heroic, I'd have no problem with giving it some cheats, as double gold production or extra creatures. 1 is something that I think would mean a good improvement over the current AI though.

[Jolly Joker]

If you program the AI to always take the ability as soon as the number of skill levels is under 40% (or so), and have a bit trust in
probabilities, you'll get average good heroes. The only thing that would have to be taken care of is a handling of the more advanced abilities.

Nope. Not true- you'll get subpar heroes on average, as a good Warlock (for example's sake) is one that (assuming that there's nothing special about the map, which would make this point even more valid) never has Dark or Light magic, and always has destruction. Trusting in chance means that you'll occasionally get heroes that doesn't fulfill this, and thus are subpar. The AI should actively pick good skills and abilities (somehow- it could do it the same way as humans, but that'd still require somewhat preset paths), and not trust in chance. (Just the same as the AI (if put under some real monetary constraints) shouldn't just build whatever's available but sometimes conserve resources for for example a capitol or higher-level dwelling).

No, GC. Not true. Destructive Magic isn't necessary to produce a good Warlock.
Give him:
Irresistable Magic
Attack
Logistics
Summoning
and two out of Luck, Enlightenment and Sorcery.
Now give him Ice Bolt and if he has Sorcery give him Circle of Winter as well. He WILL have SOME destructive power, but Summoning won't be that bad an alternative to the missing level 4/5 destructive spells: Phantome Forces on the Grim Raiders, Summon Phoenix, a Firewall at the right place...
So, no. A Warlock doesn't NEED destructive to excel. That's the beauty of the game.