Strategy Analysis of RoboCup Soccer Teams Using Self-Organizing Map

In the soccer games, the player’s behavior changes depending on the game situation such as winning or losing, score gap, remaining time. The players act more offensive when their team is losing, or more defensive when their team is winning with minimum score difference. Currently most of robot teams keep the same strategy during the game so that the result of game depends on the ability of each player like speed of robot, quality of localization, obstacle avoidance and ball handling. Next issue for the robot intelligence is collaborated team behavior and strategy. The autonomous soccer robots are required to behave cooperatively and make decision using environment and game situation. In this research, the team strategy is analyzed based on parameters such as the positions of human player, time and actions using Self-Organizing Map (SOM).


Introduction
Multi-agent System (MAS) is one of the solutions to the problems that are difficult or impossible for an individual agent or a monolithic system.As human solutions to the applications that require multiple humans to works together, the use of robot teams would be considerable solution for complex tasks.From the point of total cost, the development of multiple simple robots may be possible to reduce the cost rather than a monolithic robot.Asama proposed the robotic systems composed of multiple agent and discussed the importance of "distributed autonomy" in terms of autonomy and cooperativeness 1 .An autonomous and decentralized robot system called ACTRESS is developed and applied into multiple objects pushing problems and showed that the system can solve by P -421 parallel and independent action by each robot 2 .Parker proposed a software architecture for MAS called ALLIANCE, which facilitates fault tolerant and robust mobile robot cooperation and demonstrated the feasibility by hazardous waste cleanup using a physical robot team 3 .RoboCup Soccer is a landmark project with the goal of realizing an autonomous robots team that can win the champion of Soccer World Cup until the 2050 4,5 .RoboCup Soccer has been attracting attentions as a standard problem of MAS as it is composed of multiple interacting intelligent agents within an environment.Human teams cannot win soccer games depending on the ability of individual players, and the co-operation with teammates, team behavior, is the important solution.Especially in RoboCup middle size league (MSL), autonomous mobile robots are affected physically by not only teammate and opponent agents but also their environment.It is difficult to describe all behaviors by if-then rules, learning methods such as reinforcement learning 6 , Genetic Algorithms 7 have been tried to environmental recognitions.Takahashi and Asada 8 developed a multi-layered learning system using reinforcement learning and applied into RoboCup tasks which are to chase the ball and shoot into the goal with avoiding opponent robot.Suzuki 9 applied Genetic Algorithm into real time localization problem of RoboCup robots.In this research, we introduce Self-Organizing Map (SOM) 10 into the classification of RoboCup team strategy.SOM is a topologically correct feature map proposed by Kohonen and is well known as an attractive tool for extracting the characteristics of data and map the data into lower-dimensional space through its selforganizing process.We aim at developing a strategy decision system depending on the situation and environment.As the first step, the classification method of each robot using SOM is discussed.

Autonomous Mobile Robot Platform for RoboCup
After a decade pass from RoboCup started in 1997, the quality of the game has risen with the development of information and computer technology.We also organize a RoboCup team "Hibikino-Musashi", and have joined to the MSL in the RoboCup Tournament.In recent years, most of robots have the similar shape including the ball handling mechanism, the kick mechanism capable of adjusting the trajectory of the ball, environment recognition system using an omni-directional mirror and omni-wheels that can be moved in the all direction.Our autonomous mobile robot platform is shown in Fig. 1.Currently most of robot teams keep the same strategy during the game so that the result of game depends on the ability of each player like speed of robot, quality of localization, obstacle avoidance and ball handling.Next issue for the robot intelligence is collaborated team behavior and strategy.As the next step, we have been trying to implement a strategy selection algorithm of the team's behavior.

SOM for Classify Situation
The learning step of SOM is discussed in the following steps 11 , and the conceptual diagram of SOM is shown in Fig. 2. Suffix i denotes the number of input data, and j indicates the number of unit.
Step 1: Assign the weight vectors (wij) with random values and normalize the input in the range of [0, 1].
Step 2: Compute the Euclidean distance (dj) between a series of input vectors (xi) and all the weight vectors at iteration by eq.(1).Step 3: Find the winner unit Uc (Best Matching Unit, BMU) which has the minimum Euclidean distance.
Step 6: The time increases to +1.If < T then go to step 2; otherwise stop the training.Where is the training step index, ( )is the learning rate, and ℎ , ( ) is defined as the neighborhood kernel function, as expressed in Eq. ( 5) and Eq. ( 6). and positions of nodes BMU and on the SOM grid.Both the learning rate and the neighborhood radius decrease monotonically with time .

Teaching Data Set
As the input vectors (teaching data), we had observed and analyzed the state variables of a human soccer (futsal) game.The human soccer player performed the game 5 to 5 at the futsal court and we observed the positions of all the players and the ball in the field with the coordinates of the grid.(See Fig. 3) Input vectors are not affect behavior selection.That is, the input vector is given by Eq. (7).
, is the position of the player k, and is the position of the ball.Suffix indicates the i-th data set.We set 2 players (k = 2) and the number of input data is 100 sets (maximum i = 100).

Classification of Human Behavior
In the method described above, SOM's feature map was made from the position information of the actually observed player and ball.Each parameter of SOM algorithm is shown in Table .2. In this paper, we focus on 2 players and display the results.One is an experienced futsal player and the other is inexperienced player.Fig. 4-a shows the distance between each player and the ball, Fig. 4-b shows the distance between each player and the goal, and Fig. 4-c shows the distance between the ball and the goal in color.
If the color of the unit is blue, it means that the evaluation factor is far.On the other hand the color of unit becomes near as the color becomes red.As you can see from Fig. 4, we can see that they are similar distance relationship in all factors, although there are rotated in map.In other words, the position of the players behavior are almost same.

P -423
Experienced Player Inexperienced Player Fig. 4-a In fact, it does not include input about the circumstances and environment around the players.Therefore, it is considered that it was not located in feature map considering the relationship with other people and it did not make a difference.
In future works, we will conduct experiments with inputs including relationships with others, situations in games and the surrounded environments factors.

Conclusion
In this research, the team strategy is analyzed based on parameters such as the positions of human player, time and actions using Self-Organizing Map (SOM).We developed the SOM map from the position of each player and ball on the field.The colors of SOM map represent the distance relationships of the player, the ball, and the goal, and which kind of differences occurred among players is observed.As a result, there was the difference between experienced and inexperienced people.As a next problem, we aim to analyze the positioning of players including relations with others.

Fig. 1 .
Fig. 1.Soccer robot of "Hibikino-Musashi" : A color-coded SOM of the distance between the ball and each player Experienced Player Inexperienced Player Fig. 4-b: A color-coded SOM of the distance between the goal and each player Experienced Player Inexperienced Player Fig. 4-b: A color-coded SOM of the distance between the ball and the goal Near Far Fig. 4: SOM of soccer game by human However, the point of the view of inputs, only the position of the ball and their own position are inputted.