After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 735412 - [PATCH] Correction to the main loop of the AI.
[PATCH] Correction to the main loop of the AI.
Status: RESOLVED FIXED
Product: iagno
Classification: Applications
Component: general
git master
Other Linux
: Normal normal
: ---
Assigned To: iagno-maint
iagno-maint
Depends on:
Blocks:
 
 
Reported: 2014-08-25 21:04 UTC by Arnaud B.
Modified: 2014-08-27 15:18 UTC
See Also:
GNOME target: ---
GNOME version: 3.13/3.14


Attachments
Correction to the main loop of the AI. (2.86 KB, patch)
2014-08-25 21:04 UTC, Arnaud B.
reviewed Details | Review
Correction to the main loop of the AI. (2.95 KB, patch)
2014-08-25 22:58 UTC, Arnaud B.
reviewed Details | Review
Correction to the main loop of the AI. (3.43 KB, patch)
2014-08-26 21:18 UTC, Arnaud B.
none Details | Review
Correction to the main loop of the AI. (3.43 KB, patch)
2014-08-26 21:19 UTC, Arnaud B.
reviewed Details | Review
Use a negamax notation. (4.98 KB, patch)
2014-08-26 21:25 UTC, Arnaud B.
reviewed Details | Review
Comment for when there's no move. (951 bytes, patch)
2014-08-26 21:38 UTC, Arnaud B.
accepted-commit_now Details | Review
Correction to the main loop of the AI. (2.67 KB, patch)
2014-08-27 00:53 UTC, Arnaud B.
committed Details | Review
Use a negamax notation. (5.27 KB, patch)
2014-08-27 01:11 UTC, Arnaud B.
reviewed Details | Review
Use a negamax notation. (5.36 KB, patch)
2014-08-27 05:01 UTC, Arnaud B.
none Details | Review
Comment for when there's no move. (997 bytes, patch)
2014-08-27 05:02 UTC, Arnaud B.
committed Details | Review
Use a negamax notation. (5.23 KB, patch)
2014-08-27 07:28 UTC, Arnaud B.
committed Details | Review

Description Arnaud B. 2014-08-25 21:04:24 UTC
Created attachment 284452 [details] [review]
Correction to the main loop of the AI.

There’s (at least) one bug in the main loop on the AI. Since commit 769c6cb1576cd707e51278065ca2c9be481d3bd5 the program stops estimating the choices of move if he falls on a finish of the game, even if it means he loses the game. It should instead continue to search for a better solution…

Here is a patch that (I think, needs verification) should correct the bug. It only tests during the search of moves if it’s a perfect game-end, and handle the other cases at the beginning of the function (more usual in reversi’s algorithms).
Comment 1 Michael Catanzaro 2014-08-25 21:55:22 UTC
Review of attachment 284452 [details] [review]:

It looks good to me.  Let's just play a few games on -f fast mode to make sure it doesn't start crashing or anything, since this code is really hard to edit without breaking things.

::: src/computer-player.vala
@@ +127,2 @@
+                /* if it's a perfect game-end, don't try to find better */
+                if (g.count_tiles (Player.flip_color (g.current_color)) == 0)

I think this whole branch becomes a small optimization, correct? I'd rather remove it to simplify the code.
Comment 2 Arnaud B. 2014-08-25 22:58:49 UTC
Created attachment 284457 [details] [review]
Correction to the main loop of the AI.

That’s in fact a big optimization, and I think it’s quite clear as it’s logical to the “if win” code to be there. Here is a patch that places all the cases where it wins at the same place, is it better?
Comment 3 Michael Catanzaro 2014-08-26 03:56:09 UTC
Review of attachment 284457 [details] [review]:

::: src/computer-player.vala
@@ +101,3 @@
+        /* An end of the game where he loses */
+        if (g.is_complete ())
+            return p > 0 ? int.MIN + g.count_tiles (Player.flip_color (g.current_color)) : int.MAX - g.count_tiles (g.current_color);

This does not look easier to read than the previous patch. :p  I'd rather this be written out with if statements.

@@ +120,1 @@
                 if (g.is_complete ())

I was hoping I'd convinced you to get rid of this condition, but I'll accept it either way.
Comment 4 Michael Catanzaro 2014-08-26 15:31:58 UTC
(In reply to comment #2)
> That’s in fact a big optimization, and I think it’s quite clear as it’s logical
> to the “if win” code to be there. Here is a patch that places all the cases
> where it wins at the same place, is it better?

Well I'm still not really convinced. The moves are already ordered such that the moves are expanded in order of how many pieces you won, so a move that results in you winning is probably going to be one of the first nodes expanded.  And I don't think speed is really a consideration anyway, since it moves so fast that we have to inject artificial delay.
Comment 5 Arnaud B. 2014-08-26 21:18:17 UTC
Created attachment 284550 [details] [review]
Correction to the main loop of the AI.

That’s not so important for now, here is a patch that puts all in the beginning of the function.
Comment 6 Arnaud B. 2014-08-26 21:19:44 UTC
Created attachment 284551 [details] [review]
Correction to the main loop of the AI.

Oops, wrong patch.
Comment 7 Arnaud B. 2014-08-26 21:25:36 UTC
Created attachment 284553 [details] [review]
Use a negamax notation.

Here is a second patch that makes use of the negamax notation. One little note: I think the commit 5375dd4ce7254a2c5b2063d1e4eae473242e160a introduced a regression in the algorithm (accepting moves with the same score but ordered after in the list, so with less tiles taken…), so while rewriting, I’ve done it another way.
Comment 8 Arnaud B. 2014-08-26 21:38:23 UTC
Created attachment 284556 [details] [review]
Comment for when there's no move.

Here is a little clarification…
Comment 9 Michael Catanzaro 2014-08-26 23:02:35 UTC
Review of attachment 284551 [details] [review]:

Super, thanks! But please add a paragraph to the commit message to explain what's going on here. Thanks!

::: src/computer-player.vala
@@ +162,3 @@
                 int next_x_move = 0, next_y_move = 0;
                 var a_new = search (g, strategy, depth - 1, a, b, -p, ref next_x_move, ref next_y_move);
+                if (a_new > a)

This change looks right, but is it related to this commit?
Comment 10 Michael Catanzaro 2014-08-26 23:03:21 UTC
Review of attachment 284556 [details] [review]:

Yup.
Comment 11 Michael Catanzaro 2014-08-26 23:12:21 UTC
Review of attachment 284551 [details] [review]:

::: src/computer-player.vala
@@ +162,3 @@
                 int next_x_move = 0, next_y_move = 0;
                 var a_new = search (g, strategy, depth - 1, a, b, -p, ref next_x_move, ref next_y_move);
+                if (a_new > a)

Damn good thing I left this comment:

152	            /*			
153	             * We have to update the principle variant if the new alpha/beta is			
154	             * equal to the previous one, since we use (0, 0) as our default			
155	             * move, and a search could otherwise select it even if invalid			
156	             * when the search reaches the end of the game.			
157	             */

It doesn't look like anything has changed to make this safe; can you explain?  (And if it is safe now, you should remove that comment too.)
Comment 12 Michael Catanzaro 2014-08-26 23:14:39 UTC
Review of attachment 284553 [details] [review]:

Yay, this looks simpler.

There's probably some horrible bug in this commit, but if so I couldn't find it, so looking good.

::: src/computer-player.vala
@@ +90,3 @@
          * strategy. */
         int x = 0, y = 0;
+        search (new Game.copy (game), strategy, depth, -10000, 10000, ref x, ref y);

What's this change for? I think int.MIN and int.MAX is more clear; is there a reason we can't use them anymore?  If so, please define these as static constants.

@@ +98,2 @@
     {
         /* End of the game, return a near-infinite evaluation */

This comment would need to be updated
Comment 13 Arnaud B. 2014-08-27 00:53:46 UTC
Created attachment 284568 [details] [review]
Correction to the main loop of the AI.

The ‘oops wrong patch’ was already because of the ‘if (a_new > a)’ thing, looks like I forgot two times to format-patch without checking. ^^’ This patch should be the good (I hope).

This ‘if (a_new > a)’ thing is what I explain briefly at comment 7, it prefers moves with less tiles taken (or whatever less optimal depending of what compare_move() does) if the score is the same, that should be corrected in the ‘negamax’ patch.

The problem with int.MIN and int.MAX is that, for what I know, int.MIN ≠ - int.MAX; with the negamax algorithm, we invert many times these numbers. That’s why I choosed a big number (10000), that is near-infinite when you look at the other numbers we use. I’m going to make a patch with static constant for that.
Comment 14 Arnaud B. 2014-08-27 01:11:12 UTC
Created attachment 284569 [details] [review]
Use a negamax notation.
Comment 15 Michael Catanzaro 2014-08-27 03:25:30 UTC
Review of attachment 284568 [details] [review]:

We're going to break something. I just know it.
Comment 16 Michael Catanzaro 2014-08-27 03:36:03 UTC
Review of attachment 284569 [details] [review]:

Nits:

::: src/computer-player.vala
@@ +33,3 @@
     }
 
+    private static const int positive_infinity = 10000;

Add a comment here, otherwise someone is going to change this to int.MAX and int.MIN.  Likely me, after I forget why we did this.

Oh and we usually use ALL_CAPS for our constants.

@@ +137,3 @@
+         * We use (0, 0) as our default move; if we don't change that,
+         * a search could select it even if invalid at the end of the game.
+         */

OK, this makes more sense to me now that everything's in the proper patch. I think it's right. It looks right. Make sure all the AI tests pass, if you haven't already (run 'make check').

If you could move the comment into the else block instead of separating the if from the else, that would be more typical/readable.

@@ +155,2 @@
+            int next_x_move = 0, next_y_move = 0;
+            var a_new = - search (g, strategy, depth - 1, -b, -a, ref next_x_move, ref next_y_move);

For clarity, I'd write -1 * search (...
Comment 17 Michael Catanzaro 2014-08-27 03:40:18 UTC
(In reply to comment #13)
> This ‘if (a_new > a)’ thing is what I explain briefly at comment 7, it prefers
> moves with less tiles taken (or whatever less optimal depending of what
> compare_move() does) if the score is the same, that should be corrected in the
> ‘negamax’ patch.

The number of tiles on the board is just for ordering the moves to maximize pruning.  If two moves have the same heuristic value, and one results in more tiles, they're still equally good (one has tiles in more advantageous positions, closer to the edge of the board). So we can pick either one.

So the <= is not necessary and you'd normally use a < instead, but <= doesn't harm anything at all, it just slows things down very very slightly.  But < is slightly more clear to read, too, so that's the form I'd prefer. <= it was just a hack to get around the crash which I *think* you patch will avoid. It's really hard to reason about this code, though; we might discover otherwise the hard way. :(

> The problem with int.MIN and int.MAX is that, for what I know, int.MIN ≠ -
> int.MAX; with the negamax algorithm, we invert many times these numbers. That’s
> why I choosed a big number (10000), that is near-infinite when you look at the
> other numbers we use. I’m going to make a patch with static constant for that.

Ooooh good point.
Comment 18 Arnaud B. 2014-08-27 05:01:32 UTC
Created attachment 284577 [details] [review]
Use a negamax notation.

Looks like `make check` says all is OK.

> If two moves have the same heuristic value, and one results in more
> tiles, they're still equally good (one has tiles in more advantageous
> positions, closer to the edge of the board). So we can pick either one.
That’s (at least) theorically correct.
Comment 19 Arnaud B. 2014-08-27 05:02:05 UTC
Created attachment 284578 [details] [review]
Comment for when there's no move.
Comment 20 Arnaud B. 2014-08-27 07:28:28 UTC
Created attachment 284580 [details] [review]
Use a negamax notation.

I finally understood a little thing that didn’t make sense for me in Iagno’s AI code. Here’s a correction of my ‘negamax’ patch.
Comment 21 Michael Catanzaro 2014-08-27 13:00:54 UTC
Review of attachment 284580 [details] [review]:

Hoping for the best. I'm clearly very optimistic about this. :p
Comment 22 Michael Catanzaro 2014-08-27 13:01:35 UTC
Review of attachment 284578 [details] [review]:

OK
Comment 23 Michael Catanzaro 2014-08-27 13:05:35 UTC
Attachment 284568 [details] pushed as 270e0d1 - Correction to the main loop of the AI.
Attachment 284578 [details] pushed as 26fbd47 - Comment for when there's no move.
Attachment 284580 [details] pushed as aa969f3 - Use a negamax notation.
Comment 24 Arnaud B. 2014-08-27 15:18:59 UTC
Thanks!