Reduction of the search space to find perfect play of 6 × 6 board Othello

In 1993, mathematician Feinstein found out perfect play on 6×6 board Othello gives 16-20 loss for the first player by using computer. He reported on the Web that it took two weeks to search forty billion positions in order to obtain the result. In our previous papers, we confirmed the perfect play he found is correct. And we also found another perfect play different from the one he found to search 884 billion positions. In order to search efficiently, we attempted to reduce the search space to find some perfect play. In this paper, we introduce some techniques to solve 6×6 board Othello by searching about nine billion positions.


Introduction
Othello is a board game derived from Reversi by Goro Hasegawa (JPN) in 1973.Othello rules 1 are completely unified, whereas Reversi has many local rules.According to the Japan Othello Association established by Hasegawa, World Championship has been held every year since 1977, the tournament is held in Japan, the United States and European countries.
Othello is categorized into two-player zero-sum finite deterministic games of perfect information 2 .Games in this class are possible to look ahead in theory, thus if both players choose constantly the best move, these are classified into win, loss or draw game for the first player 3 (the sequence obtained in this way is called perfect play).
6×6 board Othello is solved by mathematician Joel Feinstein in 1993.He reported on the Web that perfect play of 6×6 board Othello gives 16-20 loss for the first player 4 .He also reported, he needed two weeks to obtain his perfect play by searching 40 billion positions.On the other hand, standard 8×8 board Othello is still open in spite of the fact that over twenty years have passed since he solved 6×6 board Othello.

P -675
In our previous papers 5,6 , we examined his results and showed that the perfect play he found is correct.In addition, we also found new perfect play.However, we needed five and a half days to find it by searching 884 billion positions.
In the future, in order to solve standard 8×8 board Othello, it is necessary to greatly reduce the search space.Therefore, in this paper, we introduce some techniques to solve 6×6 board Othello by searching about nine billion positions.Moreover, we explain an idea that solves by dividing the problem of large search space into multiple sub-problems using 4×12 board Othello.

Othello *
Othello always begins with the setup as shown in Fig. 1.One player uses the black side of the pieces (circular chips), the other the white side.Black always moves first.
Each player puts a piece of own color to an empty square in own turn.A player's move consists of outflanking the pieces of his opponent, he flips outflanked pieces to his color.To outflank means to place the piece on the board so that his opponent's rows of the piece are bordered at each end by the piece of his color.If a player cannot make any move, then he has to pass.If he is able to make a valid move, then passing is not allowed.The game ends when neither player can make any valid move.The winner is the player who has more pieces than his opponent.

Techniques
Perfect analysis of Othello is useful to refer thinking routines of game program.This is because end-game routine of game program is the perfect analysis exactly, and evaluation function in the middle-game routine is available for the ordering of the search in perfect analysis.
* Othello is a registered trademark.

MiniMax with Alpha-Beta Pruning
As the technique in computer Othello, regardless of the middle or the end, MiniMax with Alpha-Beta pruning is commonly used.MiniMax algorithm achieves the best moves by both players, Alpha-Beta method cuts unnecessary search in MiniMax.These are commonly used together since MiniMax with Alpha-Beta pruning gives the same results as the simple MiniMax search.

Move Ordering
Appropriate order in search causes pruning by Alpha-Beta method.In our program, all positions progressed one move (from current position) are extracted, and then they are sorted.To realize this, we calculate the evaluation values by machine learning in advance.Therefore, the higher the accuracy of machine learning, the higher the reduction effect of search space.

Machine Learning
We adopt reinforcement learning using Monte Carlo method.First, it starts at root node of the tree, several moves of random selection are running.Second, they select optimal child nodes until a leaf node is reached.Third, computer uses the final disc difference as a reward, it updates evaluation value of the position while returning moves.In our program, one cycle including the above three steps is repeated a million times.

Evaluation function
The evaluation value is the sum of evaluation values of several patterns.For example, in 6×6 board Othello, it consists of the patterns shown in Fig. 2.

Adding Hash Table
To cut the search of duplication positions, we added a hash table.However, since the same positions hardly occur in rectangular boards, we only implemented on 6×6 board Othello.By this revision, we were able to reduce the search space of 6×6 board Othello by 40 percent.

Experiments
Table 1 shows number of final positions for 6×6 board Othello.From Table 1, comparing the first row with the third row, we can see that our current program has reduced Feinstein's search space by about 78%.Besides, comparing the second row with the third row, we can see that our current program has reduced the search space by about 99%.This is due to the implementation effect of the Move Ordering described Section 3.2.

4×12 board Othello
We found an initial position of 4×12 board Othello as shown in Fig. 3.This position gives win for the first player.The sequence is perfect play which is consistent with 4×8 board and 4×10 board.This is why we choose the position.If the sequence of Fig. 3 is the best moves for both players, 4×12 board Othello gives win for the first player.Therefore, it is necessary to check some positions which branched from white turn.
Fig. 4, Fig. 5 and Fig. 6 are three positions that must be checked in order to prove that gives win for the first player on 4×12 board.The Greek alphabets in these figures mean all of places that the second player can put in his turn.'F' in each figure means 'Finish', this is a place which already finished the search when we check in descending order (Depth 6, Depth 4 and Depth 2).
Table 2 shows a part of the position fixed from move 1 to move 6 in 4×12 board Othello.These are currently executing sub-problems, however, blacked out cells are already finished.For every element in Table 2, the first Greek alphabets show the places as mentioned above.The following character strings are a sequence to move 6 corresponding to the coordinates in Fig. 7.For further details on the sub-problems, refer to reference 8. Table 2. Executing sub-problems to prove our perfect play.As we can see (Table 2), eleven problems have already finished, the rest is three problems.If their results are all the black win, it will show that the first player wins in 4×12 board Othello.

P -677
Firstly, in rectangle board, it is reasonable to expect number of final position in 4×12 board is about 10 3 times larger than 4×10 board.Now, given that the execution time in 4×10 board is four hours, the execution time in 4×12 board is considered to be about four thousand hours (when we analyze perfectly from move 1).
Secondly, in square board, number of final position increased about 10 8 times in the transition from 4×4 board to 6×6 board.From this fact, we assume that number of final position in 8×8 board is about 10 18 at least.Now, given that the execution time in 6×6 board is 2 hours, we know that perfect analysis in 8×8 board is impossible as it stands.

Conclusion
In this paper, we attempted to reduce the search space of 6×6 board Othello by implementing Move Ordering (Section 3.2) and Hash Table (Section 3.3).As a result, we succeeded in reducing Feinstein's search space by about 78%.In addition, the technique described in Section 4.1 is available for all boards.Therefore, it could be one key factor for solving even larger boards.

Future work
To challenge 6×8 board, 4×14 board and 8×8 board, it is necessary to reduce the search space by exponential order.For that purpose, it is most effective to improve accuracy of Move Ordering.Specifically, we strengthen reinforcement learning and improve search algorithm.

Fig. 8
Fig. 8 shows the number of final position for miniature Othello solved by us.The horizontal axis shows each board size and the vertical one represents number of final position by the exponent.The elements in Fig. 8 mean that circles are square boards and the triangles are rectangular boards.

Table 1 .
Number of final position for 6×6 board Othello.