Wrangling the Heroes IV AI

Part 1: Saddling Up

Since all of my readers may not be lucky enough to live in Texas, maybe I should start by explaining why I chose the term wrangling, which is usually defined as ‘to herd’. When wrangling cattle, the cowboy is working to keep the herd together by rounding up strays and bringing them back to the main group. This job is complicated by the fact that cattle are independent minded and tend to graze off on their own, where they become susceptible to predators. But they also have a herding instinct that keeps them moving in the same direction, more or less, and that’s where the wrangling comes in.

I’m suggesting that the Heroes IV AI is something like a herd of cattle. If you understand the nature of the beast, you can get it to follow your general plan for the game. But you have to keep your eye on the strays or else the herd will go off in all directions.

After deciding on ‘Wrangling the Heroes IV AI’ as my topic, I looked up wrangle and found some other meanings which may or may not apply here as well. Incidentally, the origin of the term is unknown, but it may derive from Caballerango, which is Spanish for horse groom. Originally, the wrangler was the one on the trail drive who herded the horses, but over time the term was also applied to herding cattle. (Just as the caballero, which means horseman, became known as a cowboy.)

Other definitions:

Wrangle: to contend in argument; to discuss; to reason; to debate; to altercate. (This may be a corruption of ‘Wangle’ which has the same general meaning.)

Wrangler: one of those who stand in the first rank of honors in the University of Cambridge, England. They are called, according to their rank, senior wrangler, second wrangler, third wrangler, etc.

Does the AI Cheat?

Sometimes I hear people say that the Heroes IV AI cheats. Personally, I’ve never seen this and if someone could give me a verifiable instance, I’d like to take a look. I surely could be wrong, but as far as I know, none of the Heroes AIs have ever cheated. The closest thing is the resource bonus for computer players on higher difficulty levels, but this is just a way of handicapping play, not a violation of rules that would otherwise apply to human players.

What may look like cheating could be places where machine logic and plentiful CPU cycles allow computer players (and neutrals) to play with a cold precision that humans lack.

One example is the way a computer player always seems to be able to get a defending army back to town just before an attack. It does not look to me as though it is deliberately keeping an army within movement range of the town, but that it is responding to a threat as soon as it detects it. My theory is that when an opposing army comes into its field of vision, the computer player assesses the threat and dispatches any armies in the area to move back to town. Clearly, there is some homing routine involved, since computer players regularly return to their towns to pick up reinforcements.

Another example is how the computer player always seems to know when a town is undefended and makes a beeline to attack it. However, I think this can be explained by a combination of getting a ‘right-click’ assessment and giving a high priority to the prize. In the first version of Heroes IV, it was possible to trick computer players into foolish attacks by ‘hiding’ a major force in the garrison slots. It looks as though this was fixed.

Yet another example is the computer player’s ability to defeat neutrals with such ease. I’ve seen this happen many times when playing with a computer ally, where I can see the composition of the ally’s army. Sometimes, my computer ally sends its armies up against foes that I would not dream of attacking—and often comes away with a victory. I think the reason for this is that when computer players fight against neutrals or each other, they use a form of ‘quick combat’ to resolve battles—they do not play out turns on the battlefield. In such battles, their overall strengths are compared to determine the winner, so particularly hard neutrals are defeated as easily as average ones. For instance, where I would hesitate to attack a gold mine guarded by Cyclops, a computer player would wade right in because in quick combat they would be no more difficult to defeat than Ogre Mages. We mere human players are only allowed to use quick combat in battles where we have an overwhelming advantage, so we don’t get the benefit of this leveling effect. You might call this cheating, but since weak neutrals are also treated as average, it ultimately evens out.

To come at the question from the other direction, if computer players cheat, they are not doing a very good job of it because of some of the mistakes they make. Quite often a computer-controlled army will just waltz up and attack a superior force. There seems to be some ‘crazy eddie’ factor that leads them into such follies--I don’t think they are sacrificing armies to get accurate readings of opponent strength. My theory is that the computer player’s pathfinding logic does not take into account zones of control not visible at the start of the turn. Where a human player’s travel would stop upon entering a ‘yellow’ square that has just come into view, the computer player already has its full path laid in and sometimes gets surprised. By the time it gets to where it can see the ‘yellow’ area, it may be too late to get out of range of the opponent and with combat inevitable, it prefers to take the offensive. Or more likely there is some random desperado element in play.

What does the AI do poorly?

It’s possible that someone has gone to the trouble of itemizing the deficiencies of the Heroes IV AI and if so, it would be interesting to exchange notes. What you have here are my own observations, deductions, and theories of the behavior of Heroes IV computer players and neutrals. Any correspondence to reality is purely coincidental. Here’s what I think the worst problems are:

- Goal setting
- Army formation
- Spell selection
- Siege defense
- Town development

I wish that the AI performed better in these areas—but have managed to reconcile myself to the flaws and started to think of ways to deal with them. That will come in Part 2.

Goal Setting

The most conspicuous and disturbing behavior of computer players is the way they pass up unguarded treasures and fail to flag resources after defeating their guards. Clearly there is some problem with goal setting here. It’s one thing to pass up an unneeded resource in order to get where you are going and another to fight your way to a gold mine and then not flag it. No human would play like that.

What I think is happening is a side effect of the way objectives are set. There has to be some way to identify possible targets and to choose among the available alternatives. It looks to me like this takes place at the start of the turn and all of the computer player’s armies use the same evaluation data, since they all can ‘see’ the same illuminated area, just as a human player can. However, not all the visible will be reachable, so these need to be eliminated from consideration. The remaining targets are evaluated--who knows how—and assigned priority values. Then when each army’s turn to move comes, it pursues the highest valued objective within its range of movement.

The behavior flaw we see could result from such an arrangement. At the beginning of the turn, the guarded gold mine is classified as unreachable since all paths to it lead into the zone-of-control of the guard stationed outside. So when it moves in that direction, it turns out that the computer player is not fighting its way to the gold mine after all—it’s fighting the guard to gain experience! Fighting would have a high priority value in the target selection decision since removing blocking armies would often be necessary in order to gain access to more territory. The AI seems to do this without regard to whether the guards are actually guarding anything or blocking the way. It just fights to fight. (Rule Number One: Don’t talk about Fight Club.)

So when the computer player has defeated the guard, just for the sake of the fight, it looks around for a new target. But it does not have a priority value for the gold mine since it was not accessible at the beginning of the turn. And now that it is in the computer player’s field of vision, the AI does not take time to reassess it—there’s no time to recheck everything. Perversely, this means the computer player will not be able to target the gold mine until it falls out of the computer player’s field of vision and then back in again. By then, the computer player’s army may have moved on to pursue other, less desirable objectives. Eventually, another of the computer player’s armies may get around to capturing the gold mine--or maybe not if there is nothing else to attract an army to the area.

Finally, computer players tend to evaluate possible objectives, um, objectively in that they do not seem to know what they need or what fits in with what they already have. A human player would give priority to capturing creature dwellings that match ones already possessed, planning on aggregating armies from both sources. A human knows that Gold Mines are extremely valuable and tries to capture them ASAP. A human player with Life Magic would naturally make a Tome of Life a priority over other artifacts at the same level, but such affinities are ignored by computer players.

Army formation

The second most obvious problem with the Heroes IV AI is the way it assembles armies. Sometimes it sends out powerful armies without heroes. Other times it sends out armies with only heroes--or maybe with a few weak creatures that slow the heroes down. Of course, the hero-less armies cannot capture anything, though they can wreak havoc with scouts, caravan routes, etc. The other problem is that any experience they gain from fighting goes to waste. The hero-heavy armies often struggle, since it takes some care and feeding to develop heroes to the point where they can fight alone.

What I think is going on here is that the AI spends its money like a sailor: it buys whatever it can, whenever it can. When there is a hero available, it always buys it. If there are no supporting troops available, it sends the hero out alone. Then as it accumulates resources during the week between heroes, it buys creatures and sends them out. Occasionally, two armies meet and merge, but the farther they get from town the less likely this becomes. On the other hand, starting armies put together by the mapmaker almost always stay intact. At least the Heroes IV AI does not have the maddening tendency of the Heroes II AI to dismiss a superior force because it has happened across some freebies.

The AI does not mix its armies like humans do. It sometimes includes slow, weak creatures with other fast armies. It pays no attention to Morale. And it never seems to dismiss an army, which can result in scouts tying up too many of the 8 slots.

Another factor here is that the AI is coo-coo for caravans, as players discover when they capture a town controlled by a computer player. There are usually several caravans in route, each with what must be a day’s worth of some creature. Caravanned creatures seem to accumulate in town until a build occurs and then are added to the new force. (Incidentally, players, this is not such a bad plan. When your Stealth hero manages to flag an unguarded creature dwelling, you can caravan the occupants back to town and let the hero carry on alone.)

Finally here, computer players do not seem to understand the value of developing multiple towns the same way. A human player who already has an Asylum producing Medusas will want to capture another one and build a Statuary Garden there with an eye on eventually merging the output of both towns into a formidable army. Computer players do not really care what kind of towns they attack and even though they will merge armies when they can, they do not go out of their way to do so. And where a human player might send out reinforcements with creatures that match the ones in the main army, it is by accident when computer players manage to reinforce armies in the field. To be fair, they do return their heroes to town to pick up reinforcements, but this is obviously less efficient than sending the reinforcements up to the front.

Spell selection

We’ve all seen cases where computer heroes fail to cast what are obviously the best spells for the situation. This also applies to creatures with spell-casting powers. For instance, they rarely Exorcise disabling spells. One the other hand, they often cast essentially useless spells, such as Slow on a shooter. Spell targeting is not so bad, since the AI usually targets the strongest opposing army while also trying for outright kills and attacks without retaliation. But the AI never uses little tricks that human players know, like using Displacement or Teleport to bring a target into range for a shooter. Or laying down quicksand to slow down opponents.

What’s going on here anyway? It looks as though Quicksand creates a little man with a wheelbarrow that he dumps along the way. It is possible that an entire army could be stuck in one wheelbarrow full of sand? And how come it works on a ship?

The cause of the problem is that when a computer player has the opportunity to cast a spell, it does so at random. If there are four spells available, the chances are equal that any will be cast, not including those that would have no effect. So sometimes, the computer player looks smart and other times it looks dumb.

A related problem is that computer players are not aggressive in developing magic-using heroes. When they advance, they seem to make a random selection from whatever skill choices are presented, rather than working heroes up in their schools of magic. This problem is exacerbated by the way the game pushes Combat skills at every opportunity. This results in most every computer hero having Combat as one of its classes, so you never see a dual-magic hero like a Demonologist.

Finally, computer heroes seem to be incapable of using Scrolls, Parchments, or Potions to expand their choices of spells. In fact, they seem to collect them just to provide treasure for the opponent. I’ve obtained as many as 70 of these from a single defeated hero.

Siege Defense

Computer players seem particularly stupid when it comes to siege defense. As best I can tell, the shooters and spell casters take to the platforms and fire on the enemy until they are killed. Melee creatures and heroes step back from the wall and turn their asses to the enemy until the gate is broken. Then they rush the enemy. Fliers wait for an opposing army to move into range and then attack on their own. If the opponent can cast Poison, Teleport, or Displacement, the town may fall without ever having the gate open. Personally, I find it much easier to beat a computer player in town than on the open map, since my shooters can concentrate first on the defending shooters while the other defenders stand around doing nothing. Then with Teleport or Displacement, I can bring armies into firing range one by one and never have to face the all-out rush that would have been made in a regular battle.

I’m sure the thinking here was that the strong advantage given to the platform shooters would motivate the attacker to storm the gate. But with spells like Wasp Swarm and Displacement, it is just too easy to neutralize the shooters. Seems to me the Heroes III AI played a more aggressive defense than what we see in Heroes IV.

Town Development

There are so many problems with town development that it’s hard to know where to begin.

Have you noticed how often computer players choose the weaker of the two dwellings available at levels 2, 3, and 4? Though there was an attempt to balance the creatures, and there will be occasions when the weaker creature is desirable, most of the time it does not pay to develop Venom Spawn instead of Vampires, Nagas instead of Genies, or Hydras instead of Black Dragons. Yet computer players do this with dogged regularity. I’m certain the reason for this is that the better creatures require a large quantity of some particular resource and rather than wait (or trade or fight) to accumulate enough, as a human player would, the computer player builds whatever it can as soon as it can.

The most conspicuous error is the neglect of the Creature Portal. Computer players will build it, but never recruit the creatures it makes available. Personally, I always go after Preserves when I play, because I know when I take one there will be a big batch of Mantises, Water Elementals, etc. there for me to recruit.

Computer players make other dubious choices in town development. For instance, they place a high priority on building up Mage Guilds, even though they have no heroes capable of casting the higher level spells. And they build their Citadels and Castles sooner than most players would--unless faced with an immediate threat--which leaves them short in other areas.

Gross stupidity

There are some other behaviors that I find incomprehensible and can only attribute to oversights by the programmers. One is the way computer players will strand their own armies by casting Summon Boat while an army is on an island. Another is the way they neglect Taverns, even when they need a hero to escape the three-day countdown. And we’ve all seen how they sometimes leave towns virtually unguarded. And Altars being bypassed since computer heroes want to be Combat combos.

When faced with elimination, computer players become extremely aggressive, even to the point of launching foolhardy attacks against vastly superior forces. It appears that when faced with the three-day countdown, other objectives are forgotten and every effort is made to take a town. Sadly, these efforts are wasted, since the three-day countdown does not kick in until the computer player has lost all its heroes. And even when it might be able to visit a Tavern or Prison to obtain a new hero, it never does since the armies are all bent on attack.

Similarly, when the last town is threatened, a computer player will exhaust itself by sending out small armies with heroes that retreat when they encounter the opponent. I’ve seen the same army come back as many as four times. Once this was one with Diplomacy, who seemed determined to buy me off with repeated surrenders.

What does it do well?

For all that, there are some aspects of the game where the computer players do quite well, all things considered.

- Field combat
- Might hero development
- Exploration
- Opportunistic attacks
- Creature gathering

Field combat

Since Gus wrote this himself, you expect it to be good, and given the complexity of Heroes IV, you could say it really is great. To be sure, there are some mistakes and some opportunities that are overlooked, but overall I’ve found computer players to put up good fights on the open battlefield. Each army seems to do the right thing from its own perspective, though there is little cooperation among the armies. This tells me that the AI evaluates the situation as each army’s turn comes up and chooses its best move. The result can be a failure to concentrate fire or target opposing heroes.

Movement on the battlefield is one of the hardest routines to get right. By and large, the Heroes IV does OK when it comes to covering lines of sight to its back row shooters and it is deadly when it comes to putting its armies in the right location to foul opposing shooters. It knows how to use the Wait function to get a last-first move combo for fast armies, but it does not know to Wait so that another army can clear a path.

Might heroes

As I said before, computer players tend to develop Might heroes more often than Magic heroes. Advancement is more reasonable in that they tend to concentrate on Combat skills that will make them more effective. They are also played better than Magic heroes, since the only options available are to shoot, charge, or move. The exception here is the Thief, which never seems to do any thieving—as if the computer players were unaware of how Stealth works.

Exploration

Computer players are pretty good about exploring the territory around their starting towns and their explorers are pretty good about picking up unguarded treasure chests and resources they may come across. However, the goal seems to be to increase the size of the area they can see, rather than to discover ways to reach visible, but inaccessible goals. Computer player scouts are not limited by a human’s inability to see objects hidden behind others, so they often find things than human players would overlook.

Opportunistic attacks

Though it does not seize every opportunity that presents itself, a computer player finds enough opportunities to attack weak towns and armies that the human players have to be careful. It’s actually good that they do not attack at every opportunity or it would be too easy to draw them into ambushes. What the computer players lack is the ability to stage two-stage attacks where a low-priority objective must be cleared in order to get to a more desirable one.

Computer players are particularly good at fighting neutrals. I believe this is because they ‘know’ that they will be able to use ‘quick combat’ even when force levels are about equal, so that when they have the stronger force they will be bound to win. Computer players do sometimes come too close to a ‘wandering’ neutral, especially when the neutral’s patrol area exceeds its zone of control. But they compensate for this by having a more accurate and complete view of potential threats than human players do. And they seem to have a very human way of stumbling into trouble and not having enough movement left to get away.

Creature gathering

Computer players are quite good about aggregating creatures for their armies. First, they are much more aggressive in attacking creature generators than most humans would be. I’ve often marveled that a computer player has been able to take a level-4 dwelling like a Dragon Cave when I would not dream of attacking it. Another case of ‘quick combat’ between balanced forces, I suspect.

Computer players are also adept at Caravanning and never seem to revisit creature dwellings once they have been captured. This is why there always seem to be a few weak armies being caravanned to towns you have just captured. The computer player is monitoring every dwelling and as soon as a creature is available, it buys it and caravans it to its nearest town.

Also, computer players do a good job of merging armies, something they rarely did in Heroes III. They combine armies of like creatures and add heroes when slots are available. And the armies are arranged the same way humans would—shooters in back and walkers in front. However, its decision on joining armies does not seem to take into account the loss of movement when an exhausted army combines with a fresh one. In the original version of Heroes IV, you could see this often with armies that spawned every day—the first one would walk as far as it could and stop, then the next day another one would appear and move over to join with the first, and it would go like that again and again. I have not seen that happen in a long while, so something may have been done about it.

And most impressively, computer players will bring their armies back to town to pick up creatures that have accumulated there. This could be a carry-over from Heroes III, where everyone headed home for the Day 1 build. This makes for more substantial forces later in the game, an improvement over Heroes III where every computer-played army tends to be about a week’s worth of armies.

What it does better than I do

On balance, computer players are not as strong as human opponents, but there are a few things it does better humans—at least better than this human. Computer players have intimate knowledge of the way the game engine works. They see things that humans don’t. They judge enemy forces precisely.

But the main advantage is that computer players do not play emotionally or form attachments to particular towns or heroes. Though flawed, their goal setting is based on cold calculations. Capture a town and it becomes just another possible target—as if they had never held it. Take a prisoner and the computer player will make no effort to recover it.

In Part 2, we will look at some ways to overcome the main limitations of computer players and to take advantage of their strengths. That is, how to wrangle the Heroes IV AI.