Tanta Simulator

Update (in case anyone is interested). 1. I reworked a lot of things because initially, I designed everything around Tic-Tac-Toe (for example, I was evaluating based on the consequence of an action, rather than the state-action combination). I also extracted a good chunk of code into the "agent," and now there are many agents, interesting and diverse ones (some controlled by the computer, some controlled by Minimax, and some controlled by a QLearningAI based on either a lookup table, a map, or a neural network). 2. I separated "training" and "gameplay." Now, you can train gradually, saving intermediate results to disk. Accordingly, if something goes wrong, you can simply load the previous result and start again. You can also control how much to train in the current session. 3. There was a problem where the neural network would eventually degenerate into NaN (basically, it would become irreparably corrupted), so I changed the activation function from ReLU to Sigmoid in the first layer. It learns a bit slower, but it's already reached 4 million iterations, and it's still working, with no dependencies. 4. I extracted the game result, but I haven't yet implemented the conversion of the result directly into the neural network's output, but there are no obstacles to doing so. 5. I have an idea to try a convolutional layer, but it would be helpful for Tic-Tac-Toe, but not for Heroes (although if I further complicate the neural network's structure, I could pass the field analysis through it, and the analysis of the rest – magic, hero stats, etc. – through a dense layer). In short, that's what I've been working on. I'll be presenting my progress on Tic-Tac-Toe at my company on September 25th. Before that, I'll polish what I have and create a presentation. If anyone wants to, they can start implementing the state for Heroes during this time (I'm enticing you :D). If not, I'll probably start working on Heroes myself in October. 3x3 learns in 1.5 million iterations to the point where it doesn't lose to Minimax (i.e., it plays perfectly). Below are some images to grab your attention (showing how 5x5 and 4-in-a-row are learned).