Auto-translated
Nival Interactive programmer Timur Bukharaev shares the experience gained while developing artificial intelligence for the popular project Heroes of Might & Magic V. The article does not reveal details regarding the specific inner workings of the "Heroes" AI, but instead offers a series of general recommendations on organizing the process of development, debugging, and testing AI:
Timur Bukharaev
Born in Kazan. Graduated from the Faculty of Radio Engineering and Cybernetics at MIPT. Has been working as a programmer at Nival Interactive since 2004.
Introduction
In this article, I will try to share the experience gained during the development of artificial intelligence (hereinafter AI) in the Heroes of Might & Magic V project. However, not a single word will be said here about the design of that very intelligence. Why?
First and foremost—to concentrate attention on the topic of organizing the process of AI development, debugging, and testing. The thing is, the design of the AI in "Heroes" is quite heavily tied to the fact that it is specifically "Heroes," and not some other game. It is not certain that knowledge of the AI design in the Heroes of Might & Magic V project would be useful to developers of a racing game featuring flying scrap-heaps.
At the same time, the ideas set out below may be useful for the development of any game AI, regardless of genre, rules, and software architecture.
Historical Digression
It should be noted that there are situations where it's not a matter of how to write an AI correctly. Sometimes it's a matter of writing one at all. In any satisfactory form. For instance, toward the end of the project, our publisher, Ubisoft, sounded the alarm: the game has no AI! However, when our managers informed me of this, I was surprised. After all, it had been developed by my predecessor for a year and a half and was considered finished.
The code dedicated to the AI was enormous. More than ten thousand lines described how it should develop, build, fight, scout, dig up treasures, and much more. That is, judging by the code, it could do everything—except perhaps compose poetry.
However, stepping away from current tasks and taking a look at it, I discovered a rather sad sight: the computer hero would leave the Castle, crawl around a bit collecting chests, after which it would die upon bumping into neutral creatures. Then the virtual opponent would buy the next hero and repeat the process. Diagnosis: the patient is more dead than alive.
Not only was the preliminary assessment of the battle outcome broken. Practically everything was broken. Building castles, choosing targets on the map, interacting with the opponent. Most of what those thousands and thousands of lines of program code were dedicated to. And their author was no longer with us on the team by that time...
Why did this happen? Was it due to the unprofessionalism of my predecessors? I don't think so. The programmers who conceived the AI ideology and implemented it were good professionals. And most importantly, everything they did worked once. Probably worked...
Imagine you are riding a train. You are bored, and you decide to entertain yourself by building a house of cards on the table. A big, beautiful, multi-story one. But here's the catch: the train shakes. With your tongue sticking out, you painstakingly assemble this fragile airy construction, but the first serious vibration collapses it to its foundations.
It is the same with AI: it is a very fragile and delicate system. And the project was rushing forward. A huge project, with a large number of developers on board, not all of whom possessed extensive experience. Dozens of changes were made to the code and data every day. Designers tweaked the game balance. Parameters and abilities of creatures and heroes changed constantly. Even basic game rules and formulas changed. Therefore, it is no surprise that, returning suddenly to see how our little house was doing, we found only a pile of cards scattered chaotically across the table. And so we took glue and got to work.
Giving in beautifully to the player
The wisdom that Nival programmers learned from designers sounds as follows:
The main task of the AI is not to win against the player, but to give in to them beautifully.
The primary goal of a game is to entertain a person, to make them feel good. And people love to win; they love to feel their superiority, even over an electronic dummy. Therefore, an AI, which also works for the entertainment industry, should resist for show first, and then lose.
It should be noted that although the principle itself is very simple and clear, it initially causes contradiction among programmers. They don't want to make an AI that gives in beautifully. They want to make an AI that makes the player weep from their own helplessness. But one should not do that.
Not caught, not a thief
Should the AI cheat? Ideally, of course, no. But even creating an AI that doesn't just win, but provides worthy resistance to a good player, is also a very difficult task. And this difficulty absolutely does not correspond to the time allocated for it. Therefore, shortcuts must be used to increase the strength of the AI.
In principle, there is nothing wrong with this. It is not necessary to create a worthy opponent. It is enough to create the illusion of a worthy opponent. And if you deceive them, but aren't caught red-handed—then you've won.
I must admit that not all cheats succeeded. One of them turned out to be very crude, and players noticed it immediately and began to complain. On the other hand, they don't even suspect some others.
Particularly curious players actively dig through game resources looking for cheats, trying to track them using script commands. Moreover, suspicions and accusations of cheating occur even where there actually are none. For example, some suspicious players mistook certain open constants of the AI evaluation function for cheats and even created their own mod that zeros them out. This mod, which broke the evaluation function, was posted on the internet and solemnly named anti-cheat.
The conclusions are these: you can cheat, but invisibly. Moreover, it is better to make many small cheats than one big one.
Concentration on the task
Although we had a mini-team of programmers with the proud name AI-Team, it didn't devote very much time to the AI itself. The main work it performed was: development of high-level code and data for game mechanics, expanding the set of script functions, binding art assets to game logic, and creating and supporting the automatic camera system in battle.
When I was tasked with bringing the AI back to life, as the team leader, I took on this tasty and interesting task personally. The working day time devoted to AI resuscitation was distributed roughly like this:
30% - sorting and distributing incoming tasks and bugs
10% - code review of others' changes
20% - finishing ghost mode
10% - visits to designers to admire a new "assert"
10% - answering questions about what is located where and how it works
10% - fixing minor errors that are faster to fix than to reassign
5% - figuring out why the artist's new camera in the game is backward
5% - desperate straining of brain cells while loading AI code into the mind
By the way, my predecessor wrote it under roughly the same conditions.
As a result, just a week later, I came to the conclusion that one cannot live like this, and I delegated the AI coding to one of my subordinates, leaving only the control over ideas to myself. And I have never regretted it since.
The conclusion from this is:
Artificial intelligence is a very complex task that requires high tension and maximum concentration of the programmer's mental abilities. And the main necessary condition for the successful solution of this task is to completely shield this programmer from any other problems.
Regressive Testing
What is a good AI? What should it even be doing? How do you formalize requirements? How do you check that it is ready and working?
Unfortunately, discussions about AI often lack specificity. And as for the bugs that sometimes come in: "AI tupit". Without any additional explanations. And what are you supposed to do with that?
Therefore, both during requirement gathering and during testing, it is necessary to collect use cases. These are certain game situations in which the behavior of the AI is unambiguously determined by common sense. Consequently, a deviation of actual behavior from the specified behavior clearly indicates an error.
In "Heroes," we organized such test situations in the form of a set of maps. The requirements for tests are:
Expected behavior
An unambiguous "yes" or "no" answer to whether the observed behavior matches the expected behavior
Fast reproduction of behavior for analysis
Small size
To illustrate, I will provide an example of one such map:
Heroes of Might and Magic V: Tribes of the East - Artificial Intelligence in Heroes of Might and Magic V
Two computer heroes stand near their fully built castles (the screenshot was taken from the editor, which displays all castles as unbuilt). Before them is a choice between a crystal mine and a group of three other mines. For the demon lord, all mines are equal, and based on common sense, he should first go to the group of three mines. But for the knight, archangels are sold for these crystals in the castle, and he should choose the crystal mine first.
If the choice is made incorrectly, the test fails. This means that the AI resource evaluation system is broken.
Test checks should be automated using scripts and run regularly. It should be noted that some test maps broke repeatedly during code changes. Unfortunately, we only started making test maps in December 2005, meaning toward the end of the project, and the automation of launching and checking was implemented only after the game went to print.
Thus, regressive tests are a vital means of maintaining the functionality of a system as complex as artificial intelligence.
Special Mode for AI Debugging
Suppose, for example, it is required to find out how the AI leveled up its hero or built its castle. But how? Dive into the debugger? Create script commands for this and write them in the console? Possible, but slow and inconvenient. Therefore, one of the first pieces of work on AI resuscitation was dedicated to creating a special game mode in which one could observe the AI in first person and arrange AI games with each other without human participation.
From then on, most observations of AI behavior could be conducted via the standard user interface of the game, which is much more comfortable than setting a breakpoint in the game and looking at variable values.
Additionally, I placed a second computer next to me and arranged regular battles between AIs on it. Observing this process with peripheral vision, I was sometimes able to catch errors (by the way, not only related to AI) and fix them while they were still fresh.
So do not be lazy in providing yourself with convenient tools for AI development.
AI Visualization
But not everything can be learned even through the standard interface. A very important question that a developer must be able to answer while looking at AI behavior is: why did it act this way and not another? Why, instead of attacking my weak hero, did this fool gallop past for another chest? This question can undoubtedly be answered by running the game under a debugger. But that is slow, not operational, and incredibly tedious.
On the other hand, a clear visual representation of the processes occurring in the AI will significantly ease the process of getting an answer to the "why" question and the subsequent correction of the error.
I will provide several examples of such representation:
Object evaluations on the map. The AI wants to capture small green columns less, and large red ones more
Threat zone. Red dots mark the area around a scary enemy hero, where one should not go. This same screenshot shows that tooltips can also be used for AI debugging
Conclusion: visualization reduces the time for debugging and fixing AI errors manifold!
Logging
Another very useful method for AI debugging and testing is the creation of logs, i.e., text files in which the AI writes a report on the work performed. It can be run overnight, and in the morning you can read what was done.
It should be noted that logs turn out quite large, and not all of their content is interesting at any particular moment. Therefore, a small trick was used: recording logs in a tab-separated format. Then the log was viewed in Excel, and unnecessary information was filtered out.
Here is an example of such a log:
Playing with AI
One of the great omissions was that the developers almost never played against their own AI. Testers played. Reports were written. Feedback came from all sorts of people. But we ourselves personally played only a handful of times. A full game of "Heroes" takes almost a whole day, and where do you get that when there is so much to do and deadlines are looming?
Meanwhile, personal gaming experience actually cannot be replaced by anything. No bug reports, reports, or feedback can replace the feeling of hair standing on end from a personal look at what it actually does sometimes.
Therefore, you must definitely find time to play with the AI personally.
Conclusion
That's probably all. If anyone wishes to know how the AI is designed in Heroes V, it's better to talk about that separately. Here, we only remain to summarize in the form of a short list of recipes for preparing AI in computer games. I hope someone will be able to use them for their benefit:
The main task of the AI is not to win against the player, but to give in to them beautifully.
You can cheat, but invisibly. It is better to make many small cheats than one big one.
The main necessary condition for creating AI is the concentration of efforts on the task and detachment from all other problems.
Regressive tests are a vital means of maintaining AI functionality.
Do not be lazy in providing yourself with convenient tools for AI development.
Visualization reduces the time for debugging and fixing AI errors manifold.
You must definitely play with the AI personally.
The article was written based on a presentation at CRI-2006.
Timur Bukharaev
Born in Kazan. Graduated from the Faculty of Radio Engineering and Cybernetics at MIPT. Has been working as a programmer at Nival Interactive since 2004.
Introduction
In this article, I will try to share the experience gained during the development of artificial intelligence (hereinafter AI) in the Heroes of Might & Magic V project. However, not a single word will be said here about the design of that very intelligence. Why?
First and foremost—to concentrate attention on the topic of organizing the process of AI development, debugging, and testing. The thing is, the design of the AI in "Heroes" is quite heavily tied to the fact that it is specifically "Heroes," and not some other game. It is not certain that knowledge of the AI design in the Heroes of Might & Magic V project would be useful to developers of a racing game featuring flying scrap-heaps.
At the same time, the ideas set out below may be useful for the development of any game AI, regardless of genre, rules, and software architecture.
Historical Digression
It should be noted that there are situations where it's not a matter of how to write an AI correctly. Sometimes it's a matter of writing one at all. In any satisfactory form. For instance, toward the end of the project, our publisher, Ubisoft, sounded the alarm: the game has no AI! However, when our managers informed me of this, I was surprised. After all, it had been developed by my predecessor for a year and a half and was considered finished.
The code dedicated to the AI was enormous. More than ten thousand lines described how it should develop, build, fight, scout, dig up treasures, and much more. That is, judging by the code, it could do everything—except perhaps compose poetry.
However, stepping away from current tasks and taking a look at it, I discovered a rather sad sight: the computer hero would leave the Castle, crawl around a bit collecting chests, after which it would die upon bumping into neutral creatures. Then the virtual opponent would buy the next hero and repeat the process. Diagnosis: the patient is more dead than alive.
Not only was the preliminary assessment of the battle outcome broken. Practically everything was broken. Building castles, choosing targets on the map, interacting with the opponent. Most of what those thousands and thousands of lines of program code were dedicated to. And their author was no longer with us on the team by that time...
Why did this happen? Was it due to the unprofessionalism of my predecessors? I don't think so. The programmers who conceived the AI ideology and implemented it were good professionals. And most importantly, everything they did worked once. Probably worked...
Imagine you are riding a train. You are bored, and you decide to entertain yourself by building a house of cards on the table. A big, beautiful, multi-story one. But here's the catch: the train shakes. With your tongue sticking out, you painstakingly assemble this fragile airy construction, but the first serious vibration collapses it to its foundations.
It is the same with AI: it is a very fragile and delicate system. And the project was rushing forward. A huge project, with a large number of developers on board, not all of whom possessed extensive experience. Dozens of changes were made to the code and data every day. Designers tweaked the game balance. Parameters and abilities of creatures and heroes changed constantly. Even basic game rules and formulas changed. Therefore, it is no surprise that, returning suddenly to see how our little house was doing, we found only a pile of cards scattered chaotically across the table. And so we took glue and got to work.
Giving in beautifully to the player
The wisdom that Nival programmers learned from designers sounds as follows:
The main task of the AI is not to win against the player, but to give in to them beautifully.
The primary goal of a game is to entertain a person, to make them feel good. And people love to win; they love to feel their superiority, even over an electronic dummy. Therefore, an AI, which also works for the entertainment industry, should resist for show first, and then lose.
It should be noted that although the principle itself is very simple and clear, it initially causes contradiction among programmers. They don't want to make an AI that gives in beautifully. They want to make an AI that makes the player weep from their own helplessness. But one should not do that.
Not caught, not a thief
Should the AI cheat? Ideally, of course, no. But even creating an AI that doesn't just win, but provides worthy resistance to a good player, is also a very difficult task. And this difficulty absolutely does not correspond to the time allocated for it. Therefore, shortcuts must be used to increase the strength of the AI.
In principle, there is nothing wrong with this. It is not necessary to create a worthy opponent. It is enough to create the illusion of a worthy opponent. And if you deceive them, but aren't caught red-handed—then you've won.
I must admit that not all cheats succeeded. One of them turned out to be very crude, and players noticed it immediately and began to complain. On the other hand, they don't even suspect some others.
Particularly curious players actively dig through game resources looking for cheats, trying to track them using script commands. Moreover, suspicions and accusations of cheating occur even where there actually are none. For example, some suspicious players mistook certain open constants of the AI evaluation function for cheats and even created their own mod that zeros them out. This mod, which broke the evaluation function, was posted on the internet and solemnly named anti-cheat.
The conclusions are these: you can cheat, but invisibly. Moreover, it is better to make many small cheats than one big one.
Concentration on the task
Although we had a mini-team of programmers with the proud name AI-Team, it didn't devote very much time to the AI itself. The main work it performed was: development of high-level code and data for game mechanics, expanding the set of script functions, binding art assets to game logic, and creating and supporting the automatic camera system in battle.
When I was tasked with bringing the AI back to life, as the team leader, I took on this tasty and interesting task personally. The working day time devoted to AI resuscitation was distributed roughly like this:
30% - sorting and distributing incoming tasks and bugs
10% - code review of others' changes
20% - finishing ghost mode
10% - visits to designers to admire a new "assert"
10% - answering questions about what is located where and how it works
10% - fixing minor errors that are faster to fix than to reassign
5% - figuring out why the artist's new camera in the game is backward
5% - desperate straining of brain cells while loading AI code into the mind
By the way, my predecessor wrote it under roughly the same conditions.
As a result, just a week later, I came to the conclusion that one cannot live like this, and I delegated the AI coding to one of my subordinates, leaving only the control over ideas to myself. And I have never regretted it since.
The conclusion from this is:
Artificial intelligence is a very complex task that requires high tension and maximum concentration of the programmer's mental abilities. And the main necessary condition for the successful solution of this task is to completely shield this programmer from any other problems.
Regressive Testing
What is a good AI? What should it even be doing? How do you formalize requirements? How do you check that it is ready and working?
Unfortunately, discussions about AI often lack specificity. And as for the bugs that sometimes come in: "AI tupit". Without any additional explanations. And what are you supposed to do with that?
Therefore, both during requirement gathering and during testing, it is necessary to collect use cases. These are certain game situations in which the behavior of the AI is unambiguously determined by common sense. Consequently, a deviation of actual behavior from the specified behavior clearly indicates an error.
In "Heroes," we organized such test situations in the form of a set of maps. The requirements for tests are:
Expected behavior
An unambiguous "yes" or "no" answer to whether the observed behavior matches the expected behavior
Fast reproduction of behavior for analysis
Small size
To illustrate, I will provide an example of one such map:
Heroes of Might and Magic V: Tribes of the East - Artificial Intelligence in Heroes of Might and Magic V
Two computer heroes stand near their fully built castles (the screenshot was taken from the editor, which displays all castles as unbuilt). Before them is a choice between a crystal mine and a group of three other mines. For the demon lord, all mines are equal, and based on common sense, he should first go to the group of three mines. But for the knight, archangels are sold for these crystals in the castle, and he should choose the crystal mine first.
If the choice is made incorrectly, the test fails. This means that the AI resource evaluation system is broken.
Test checks should be automated using scripts and run regularly. It should be noted that some test maps broke repeatedly during code changes. Unfortunately, we only started making test maps in December 2005, meaning toward the end of the project, and the automation of launching and checking was implemented only after the game went to print.
Thus, regressive tests are a vital means of maintaining the functionality of a system as complex as artificial intelligence.
Special Mode for AI Debugging
Suppose, for example, it is required to find out how the AI leveled up its hero or built its castle. But how? Dive into the debugger? Create script commands for this and write them in the console? Possible, but slow and inconvenient. Therefore, one of the first pieces of work on AI resuscitation was dedicated to creating a special game mode in which one could observe the AI in first person and arrange AI games with each other without human participation.
From then on, most observations of AI behavior could be conducted via the standard user interface of the game, which is much more comfortable than setting a breakpoint in the game and looking at variable values.
Additionally, I placed a second computer next to me and arranged regular battles between AIs on it. Observing this process with peripheral vision, I was sometimes able to catch errors (by the way, not only related to AI) and fix them while they were still fresh.
So do not be lazy in providing yourself with convenient tools for AI development.
AI Visualization
But not everything can be learned even through the standard interface. A very important question that a developer must be able to answer while looking at AI behavior is: why did it act this way and not another? Why, instead of attacking my weak hero, did this fool gallop past for another chest? This question can undoubtedly be answered by running the game under a debugger. But that is slow, not operational, and incredibly tedious.
On the other hand, a clear visual representation of the processes occurring in the AI will significantly ease the process of getting an answer to the "why" question and the subsequent correction of the error.
I will provide several examples of such representation:
Object evaluations on the map. The AI wants to capture small green columns less, and large red ones more
Threat zone. Red dots mark the area around a scary enemy hero, where one should not go. This same screenshot shows that tooltips can also be used for AI debugging
Conclusion: visualization reduces the time for debugging and fixing AI errors manifold!
Logging
Another very useful method for AI debugging and testing is the creation of logs, i.e., text files in which the AI writes a report on the work performed. It can be run overnight, and in the morning you can read what was done.
It should be noted that logs turn out quite large, and not all of their content is interesting at any particular moment. Therefore, a small trick was used: recording logs in a tab-separated format. Then the log was viewed in Excel, and unnecessary information was filtered out.
Here is an example of such a log:
Playing with AI
One of the great omissions was that the developers almost never played against their own AI. Testers played. Reports were written. Feedback came from all sorts of people. But we ourselves personally played only a handful of times. A full game of "Heroes" takes almost a whole day, and where do you get that when there is so much to do and deadlines are looming?
Meanwhile, personal gaming experience actually cannot be replaced by anything. No bug reports, reports, or feedback can replace the feeling of hair standing on end from a personal look at what it actually does sometimes.
Therefore, you must definitely find time to play with the AI personally.
Conclusion
That's probably all. If anyone wishes to know how the AI is designed in Heroes V, it's better to talk about that separately. Here, we only remain to summarize in the form of a short list of recipes for preparing AI in computer games. I hope someone will be able to use them for their benefit:
The main task of the AI is not to win against the player, but to give in to them beautifully.
You can cheat, but invisibly. It is better to make many small cheats than one big one.
The main necessary condition for creating AI is the concentration of efforts on the task and detachment from all other problems.
Regressive tests are a vital means of maintaining AI functionality.
Do not be lazy in providing yourself with convenient tools for AI development.
Visualization reduces the time for debugging and fixing AI errors manifold.
You must definitely play with the AI personally.
The article was written based on a presentation at CRI-2006.
