in Games

.

the strategy and the opponent's reactions.The action transitions of a good boxer need to be flexible and contain great variety to achieve the optimal outcome.Such information often serves as an important indicator in assessing the skill level of a player, and the same principle applies to many other sports such as basketball [4] and fencing [5] .Unfortunately, automatic systems for analyzing and evaluating sports motions at such a high level is very limited.
In this paper, we propose a robust visualization system to address the above limitations, by represent motions as an interactive graph of high-level features, including the flexibility and richness of the actions as well as the transitions of actions.Although we use boxing as a demonstration in this paper, our method is generic and can be applied to different sports.Our approach starts with capturing the shadow boxing training motion of a boxer, in which the boxer performs boxing with an imaginary opponent.An experienced coach can effectively assess the boxer's skill by watching the shadowing boxing motions.As a positive side effect, this method of motion analysis greatly reduces the complexity of motion capture due to occlusion and collision and has shown to be very effective in our system.The motion data is then processed and visualized in two different graphs: the posture-based graph and the actionbased graph, for performance analysis.
In the posture-based graph, the semantic actions segmented from the captured motion are grouped into clusters based on a customized distance function that considers action specific features.Our system then automatically generates a motion graph structure known as Fat Graph [6]

Sports visualization
Helping athletes on skill improving via the visualization of sport motions is a field that has not been fully explored in the field of sports science.Existing research [9,10] mainly focuses on the appearance changes of motions when body and motion parameters are changed.For example, Yeadon [9,10] has done research on how diving and somersault motions change when the motions are launched at different timings by using physical simulation.Although such tools are useful for the athletes to interactively visualize possible results under different parameters, they can only evaluate the performance of sports that do not require complex maneuvers and strategies, such as jumping, high jumping, sky jumping, or somersaults.In many sports games, the performance depends not only on physical factors such as velocity, power and strength, but also on flexibility to switch from one motion to another and richness of the player's motions.This high-level information has not been used to visualize the skills of the athlete in previous research and it is the major difference between our work and the afore-mentioned ones.In this research, we combine the approaches of motion graph [11-13] and dimensionality reduction [14,15] to visualize high-level skills information of the athletes for the skill assessments.

Motion graphs for motion modeling
The Motion Graph approach [11-13,16-19] is a method to interactively reproduce continuous motions based on a graph generated from captured motion data.Reitsma and Pollard [20] compared different motion graph techniques comprehensively.Heck et al.
[21] further parametrized the motion space to control how the motions are generated by blending samples in the motion graph.Such an approach can be used for interactive character control such as that in computer games.When it comes to graph construction, [16,17] are the ones most similar to our method.Min et al. [16] grouped similar postures and transitions into nodes and edges.Their focus was the motion variety of synthesized motions so they used generative models to fit the posture and motion data.Our focus is on skill visualization through the analysis of postures and motions so we can afford simpler and faster methods of analysis.Beaudoin et al. [17] cluster postures first then find motion motifs by converting the motion matching task into a string matching problem.Their priority was to find motifs that were representative while our focus is to visualize motion details and statistics to help people assess the skills.Xia et al. [22] constructed a series of local mixtures of autoregressive models (MAR) for modeling the style variations among different motions for real-time style transfer.They demonstrated style-rich motions can be generated by combining their method and motion graph.
Since the Motion Graph produces a lot of edges and nodes without any context, it becomes difficult to control generated motion as the user wishes.Safonova and Hodgins [23] optimized the graph structure by combining motion graph and interpolation techniques to improve performance.On the other hand, works to resolve this problem by introducing a hierarchical structure were proposed [6] .These approaches add topological structures into the continuous unstructured data so that the motion synthesis can be done at a higher level.In a sport like boxing, it is possible to create a motion graph of semantic actions such as attack and defence, which is known as the action-level motion graph [24,25] .A recent work by Hyun et al. [4] proposed Motion Grammars to specify how character animations are generated by high-level symbolic description.Such an approach can be used with existing animation systems which are built based on motion graphs.Ho  [26] built a finite state machine (FSM) based on Topology Coordinates [27] for synthesizing two-character close interactions.The sparse graph structure can be used for controlling the movement of virtual wrestlers in computer games.The purpose of these approaches, however, is motion generation rather than the visualization of the player's skill.
In our research, we adapted a hierarchical motion graph structure called the Fat Graph [6] on the action level to analyze the connectivity and the variety of a captured motion set.In a fat graph, similar nodes are grouped together as fat nodes, and similar edges are grouped as fat edges, allowing better organization of motion data.The filtered motion graph is a variation of the Fat Graph, in which the temporal relationship between poses are considered [28] .Such a structure, however, is targeted for motion reconstruction and analysis rather than visualization [29] .

Statistical motion modeling
Dimensionality reduction methods have been proposed to visualize the overall structure of captured motions.Grochow et al [14] proposed a method to project the 3D motions of a human onto a 2D plane, and further reconstruct 3D motions by mapping arbitrary points from the 2D plane back onto 3D joint space.PCA [15] and ISOMAP [30] are proposed to map the motions onto 2D planes.Due to the high variation of human motion, local PCA that considers only a relevant subset of the whole motion database in order to generate a locally linear space is proposed [31,32] .One can generate motions from arbitrary points on the plane by interpolating the postures of the original motion.Meanwhile, non-linear methods [33,34] and Deep Learning [35] have also been used to reduce the dimensionality of motions.The Gaussian Process [36] and the mixture of Gaussian Processes [36] can be used to represent a set of human postures with a small number of Gaussian parameters.However, such methodologies do not take into the account the connectivity structure of the motions.We apply dimensionality reduction to our graph structure to visualize the connectivity structure of captured motions on a 2D plane.
Other researchers have focused on the connectivities of motion/actions by methods such as Markov models.Hidden Markov Model (HMM) [7] has been widely used in analyzing and synthesizing human motion.Typically, the hidden states of the HMM are the distribution of body poses and the dynamics of the motions are represented by the transitions between the hidden states.The parameters of the HMM can then be learned from training data using the Expectation-Maximization (EM) algorithm.Hara et al. [37] proposed to model daily activities using HMM in intelligent house.
Françoise et al. [38] proposed to use HMM models for analyzing Tai Chi motion sequences.An early work proposed by Brand and Hertzmann [39] proposed to learn the dynamics of human motion using HMM in their motion style synthesis model.Tango and Hilton [40] proposed to learn a HMM model from captured human motion for synthesizing in-between frames in keyframe animation.Ren et al. [41] presented a data-driven approach for quantifying naturalness of human motion including those synthesized by HMM.While existing work focuses on finding statistical distributions of motions, our focus is on visualizing the motion richness and the transition dynamics for skill assessments.

Motion capture and organization
We first capture the motion required for analysis using motion capture systems.Then, we propose an automatic system to segment long sequences of captured motion into meaningful actions, which are used as building blocks of our posture-based and actionbased graphs.Here, we follow the definition from [25] , in which a motion is 232 considered to be a raw sequence of captured human movement, 233 and an action is considered to be a short, meaningful segment of 234 movement within a motion.In the field of boxing, an action can 235 be an attack (such as a "left straight", "jab" or a "right kick"), a 236 defense (such as "parries", "blocking" or "ducking") , a transition 237 (such as "stepping to the left", "stepping forward" or "back step"), 238 or any combination of them.

239
Postures and actions are good entities for skill visualization, as 240 sports players typically plan their strategies and evaluate their per-241 formances with such terms.For example, a boxer typically thinks 242 about what sort of attack/defense/transition should be launched 243 during a match.A coach typically evaluates the overall strategy in 244 the action level, as well as how well individual postures and ac-245 tions are performed.Although it would be best to capture the motions of all players 248 in multi-player sports because the data would reflect the features 249 of the motions, capturing multiple players remains difficult due 250 to the occlusions and collisions among players.Fortunately, it is 251 possible to only capture individual motions for our purposes with-252 out compromising the true motion characteristics.In boxing or any 253 other martial arts, there is a training practice called "shadow box-254 ing".The boxer imagines a boxing session with another boxer, and 255 launches boxing actions to interact with such an imaginary oppo-256 nent.The boxer launches not only offensive actions such as punch-257 ing, but also defence, stepping, and the consecutive combination of 258 all such actions.There are similar practice methods in basketball 259 and soccer as well, in which players use the ball to conduct var-260 ious techniques in the court, imagining that their opponents are 261 trying to take the ball away from them.The players thus perform 262 various actions to keep the ball and trick an imaginary opponent.263 This technique has also been used by coaches for skill assessment 264 hence is suitable for our analysis.We employed an optical motion 265 capture system to acquire the performed motion as shown in Fig. 1 266 as it was less intrusive and highly accurate.Also, we preferred to 267 capture long and continuous clips of motion, such that the player 268 could perform the motion in a natural manner.fective joints that contribute the most to the semantic meaning of the actions.
For boxing motions, we observed that actions normally start and end in a double supporting state (i.e. both feet touching the floor), as the state is usually dynamically stable.We detect such a state by monitoring the feet height and velocity and setting corresponding thresholds.This allows us to segment the raw captured motion into a set of movement segments , which are the periods between every two successive double supporting states, as visualized in Fig. 2 Upper.
We also observed that actions normally require a relatively larger force to be performed, such as a punch or a step.We define periods with a high-level of force exertion as the activity segments .Since force is proportional to acceleration, these segments can be found when the sum of squares of acceleration of all joints is above a threshold, as visualized in Fig. 2 Middle.The threshold is statistically obtained from the acceleration profile of the motion.
Finally, the actions are composed by using the movement segments as the building blocks.The timing and the duration of the activity segments are used to determine if the movement segments should be merged together to form longer segments.Regarding the relationship of the movement segments and the activity segments, there could be three possible cases: (1) There is no activity segment inside a movement segment.In this case, the movement segment becomes a single action of pure body transition.(2) There is one activity segment inside a movement segment.In this case, this movement segment becomes an action with a special activity.
(3) There are one or more activity segments lying across successive movement segments.In this case, the movement segments containing activity segments at the border are merged to form an action as visualized in Fig. 2 Lower.Note that due to this merging process, the resulting action may contain multiple activity segments.In our system, we implement an optional step to filter very short actions that are likely to be generated due to the noise of the supporting feet.
We define the effective joints to be the set of joints to represent an activity segment.In case (1) above, since the actions contain no special activities, the pelvis is considered to be the effective joint.
In case ( 2) and ( 3), the effective joint is the joint that contributes the most to the sum of squares of the acceleration in the activity segment.In more complicated actions such as left-right combo punches, there may be multiple effective joints as there are multiple activity segments.Such joints are used in later processes to evaluate the similarity of actions.

Posture-based graph 317
The posture-based graph focuses on evaluating the common 318 postures that are used to start and end actions.In such a graph, 319 the nodes represent similar postures and the edges represent sim-320 ilar actions.It allows us to evaluate the consistency of common 321 postures and the diversity of actions.We adopt a Fat Graph structure [6] in the action level [25] to 324 generate the posture-based graph, as it can effectively simplify the 325 graph representation by grouping similar postures and actions to-326 gether.The Fat Graph was originally proposed for motion synthesis, 327 and thus it is not optimized for skill visualization.We redesign the 328 algorithms to generate nodes and edges in the Fat Graph for our 329 purpose.In our system, the nodes of the Fat Graph, known as Fat Nodes, 332 are the common starting or ending postures of the actions.We 333 design an unsupervised clustering scheme for grouping all start-334 ing/ending postures into a finite set of posture groups, which 335 avoids additional labour for posture labelling and grouping.Specif-336 ically, we used k-means to cluster postures.The distance between 337 two postures P 0 and P 1 is defined as: 338 where θ 0 ( i ) and θ 1 ( i ) represent the 3D joint angle of the joint i 339 in posture P 0 and P 1 respectively, and i total is the total number of 340 joints.Regarding the cluster number k , a large k would result in 341 many clusters (Fat Nodes), which unnecessarily increases the com-342 plexity of the graph.A small k will cluster very different postures 343 into the same node, defeating the purpose of the graph.Therefore, 344 we set up a posture difference threshold empirically based on ex-345 perts' suggestions.Then, we iteratively search for a proper k by 346 initially setting k = 1 and incrementing k by 1 until we find the 347 first value of k that does not violate the distance threshold.Formally, the distance between two actions A 0 and A 1 is defined as where A 0 ( j )( f ) and A 1 ( j )( f ) represent the 3D positions of effective joint j in frame f in the action A − 0 and A 1 respectively, j total is the total number of effective joints in the actions, f start and f end are the starting frame and ending frame of the considering effective joint.In case two effective joints with different duration are to be compared, the shorter one is linearly scaled to the duration of the longer one.
In the field of boxing, a Fat Edge typically contains a set of actions with basic attacks or defences such as "straight punch", "hook punch", "parry", or a set of complex actions combining several attacks and defences.Since member actions in a Fat Edges have to share the same starting and ending Fat Nodes, if an action group contains multiple starting or ending poses, it is sub-divided into multiple Fat Edges.
Again, by only looking at Fat Edges, one can tell the differences between experienced and novice players.Experienced players normally have Fat Edges with similar numbers of actions, as they have mastered a large variety of boxing actions and can switch between them effectively using a small number of stable transition maneuvers.Novice boxers tend to have a larger number of Fat Edges but each with a small number of actions, due to the inability to reproduce boxing actions consistently.Figure 3 shows the relationship of Fat Nodes and Fat Edges.

The connectivity index
It requires deep knowledge and years of experience to assess one's skills in sports.Here, we make use of the posture-based graph and define an index representing the skill level, allowing more objective and efficient skill assessment.In many types of sports, there are two important skill indica-407 tors.The first one is the richness of the actions that indicates the 408 resourcefulness of a player.The other is the flexibility of transi-409 tions between states so that the player can switch between differ-410 ent states at will.Our posture-based graph captures both of the 411 indicators.The richness can be represented by the number of Fat 412 Edges, indicating how many kinds of maneuvers the player has.413 The flexibility is indicated by the connectivity of the graph, which 414 is inversely proportional to the number of Fat Nodes.A fully con-415 nected graph shows great flexibility because there are transitions 416 between any two nodes.

417
Notice that these two factors are somehow contradicting.In 418 general, the richer the actions are, the greater the number of differ-419 ent starting and ending poses is hence the poorer the connectivity 420 of actions is.Independently considering either of them would not 421 suffice.We therefore define a Connectivity Index that evaluates both 422 the action richness and the action flexibility of a player To accurately reflect the skill level of a player, in our imple-424 mentation, we do not consider Fat Nodes that are not intentionally 425 created.For example, one of our boxers tripped over during a ses-426 sion.While it is good that our system can objectively pick up the 427 posture generated by the accident, we do not include the corre-428 sponding Fat Nodes when calculating the Skill Index.Also, we only 429 consider Fat Edges that are consistently performed, as those having 430 only a small number of member actions could be randomly per-431 formed actions.Empirically, we consider edges having more than 432 2 member actions.Here, we describe the design of our visualization system to vi-435 sualize the posture-based graph in an effective manner.We also 436 introduce interactive features for the user to view the graph with 437 different levels of details.

438
The posture-based graph consists of high dimensional Fat Nodes 439 (groups of similar postures of many degrees of freedom) and Fat 440 Edges (groups of similar actions in the spatial-temporal domain), 441 which presents a challenge for visualization.To reduce the dimen-442 sionality for better visualization, we propose two different schemes 443 for nodes and edges due to their different nature in this graph.4 4 4 Specifically, we project the Fat Nodes on a 2D space using Principal 445 Component Analysis (PCA) as it creates a more consistent low di-446 mensional space compared with other methods.We represent Fat 447 Edges with 2D curves and augment the curves with a combination 448 of geometric primitives to visualize the action features.We render the mean posture of each Fat Node onto a 2D X-Z plan.This allows the user to identify inappropriately performed postures.In boxing, novice boxers sometimes lose track of their boxing rhythm, and hence start or end an action with an inappropriate posture.We use the fatness of the character to represent the number of member postures in the node, as shown in Fig. 4 .This allows the user to easily observe the postures that the player usually uses to start actions.

Visualizing Fat Edges
Here, we explain how to visualize the Fat Edges, which contain information of groups of similar actions.
We do not apply dimensionality reduction techniques directly on the action data itself because the low dimensional projection would be very complex.Instead, we propose to visualize each Fat Edge by a 2D curve that represents its mean action on the X -Z plane.We optimize the angle and sign of these curves to minimize occlusion.For edges with a starting node different from the ending node, the edge angle is fixed.The only adjustable variable is the bending side of the curves, which is essentially the sign of the curves.For those with the same starting and ending node, both edge angle and bending side can be controlled.We optimize the signs and angles of the edges in a greedy manner such that they would blend towards a less dense region of the graph.
To visually distinguish between different Fat Edges, we add some geometric patterns to the 2D curves.We collect the highenergy frames of all actions and project them onto a 1D space using the PCA system explained in Section 4.3.1 .Since the highenergy frames of different actions are typically distinguishing pos- We further represent the number of member actions in the 503 edge by the thickness of the curve.This allows the user to iden-504 tify the player's preferred actions.For instance, if a boxer relies 505 heavily on single straight punches, the Fat Edge for such action 506 will be unreasonably thick, while edges for other attacks will be 507 relatively thin, which demonstrates a potential lack of diversity at-508 tacking strategies.

Interactive features 515
We integrate some interactive features in our system to dis-516 play relevant information based on user input.When the user se-517 lects any specific entities in the graph, related information will be 518 shown.

519
When a Fat Node is selected, its corresponding Fat Edges will 520 be highlighted for easier observation.Information about the num-521 ber of members in that node, number of outgoing edges, and num-522 ber of incoming edges are displayed in a sub window.When   we render the member actions included, such that the user can understand the content of the edge.
As an example, in Fig. 6  In addition, an edge can be smooth like a circle or bumpy with geometric patterns.A single pattern means one activity segment such as a single punch, while multiple patterns indicate a series of activities such as a combo attack.Our system also supports interactive features.Fig. 6 is the result when the user selects Node 1.
All the edges starting from this node are highlighted, each with a small character performing the action on it.

Action-based graph
The action-based graph focuses on evaluating the transition probability from one action class to another.In such a graph, the nodes represent groups of action with similar activity segments.
The edges represent the transition probability between two action groups.It allows us to evaluate the pattern of launching actions and extract the strategy of the boxer.

Graph construction
We use the hidden Markov model (HMM) to organize the captured motion, as it has been shown effective in modelling human motion.In the domain of character animation, HMM has been mostly used in the posture level to create motion graphs [12] .We adapt the graph into the action level such that we can visualize the transition probability among actions.
The nodes of the graph represent different action groups.We apply Eq. ( 2) to group the captured actions into a number of action groups with k-means clustering.The process is similar to that in Section 4.1 , in which we define a threshold based on expert knowledge, and then incrementally increase the number of classes until the threshold is met.We denote k as the total number of groups, | G i | as the number of actions in the i th action group (which is used in the visualization system for visualizing the fatness and the placement of the node and will be described later).
The edges of the graph represent transitional probability from one action group to another.To obtain the transitional probability, we go through the sequence of actions in the captured motion and count the number of occurrences for an action belonging to group i to be followed by another belonging to group j , which is denoted as c ij .The transition probability of action group i to action group j is defined as 572 where the denominator represents the total number of transition 573 in the whole motion.Notice that i may be equal to j .In such a case, 574 two actions of the same action group are launched successively.

575
The concept of the action-based graph is shown in Fig. 7 .In 576 general, experienced boxers tend to have a more evenly distributed 577 transitional probability across all actions, which means that there 578 should be edges connecting all the nodes.This indicates that the 579 boxer's pattern is dynamic and cannot be easily predicted by an 580 opponent.Conversely, novice boxers may have limited edges and 581 some thick edges connecting two nodes, which means a high prob-582 ability to launch those two groups action consecutively.An oppo-583 nent may discover such a pattern and counter-act in advance when 584 the first action is observed.585

The action strategy index 586
In many sports, the unpredictability of action patterns is an im-587 portant skill indicator.Experienced players would diversify their 588 action patterns such that their opponents cannot predict the next 589 action.However, novice players tend to perform actions based on 590 predictable patterns (i.e. the sequence of actions to be launched 591 continuously), which can be easily identified.For example, a novice 592 boxer usually perform two straight punches successively.This is 593 because the boxer is not able to link different types of punches 594 fluently, and therefore would perform the simplest punches again 595 and again.The proposed action-level graph allows easy observa-596 tion of boxing patterns, as we can visualize the transitional proba-597 bility among actions.We further propose the Action Strategy Index , 598 which evaluates the unpredictability of action pattern.We obtain 599 the number of outgoing HMM edges for each HMM node, forming 600 a set that is denoted as where k is the total 601 number of HMM nodes.Skillful players would have similar values 602 in the e set, while novice players would have very different values.603 We therefore define the Action Strategy Index as the precision of 604 e , that is, the reciprocal of its standard deviation

Visualization system
Here, we explain the visualization system for the action-level graph.The system allows easy observation of the preference of action and the boxing pattern.Both are very important aspects to evaluate the high-level strategy of a boxer.

Visualizing HMM Nodes
Each action group is represented by its corresponding median action, which is the action that is the closest to the mean value of the action group during k-means clustering.We render the nodes where G i is the number of member actions in a node as defined in Section 5.1 , a is a preset frequency threshold.Fig. 8 shows the result of setting different values of a .We find that setting a = 2 generates the best results.
We place the nodes belonging to the frequent class at an inner circle, and those belonging to the rare class at an outer circle, such that the user can identify them easily and decide what to focus on.For the inner circle, nodes are ordered according to the corresponding value of | G i |, and are placed evenly at a circle with a smaller radius.For the outer circle, to minimize edge crossing, we place the nodes at a position on a circle with a larger radius that is the closest to the nodes with incoming and outgoing edges.To implement this, we develop a simple optimization algorithm that optimizes the position of the nodes.During the optimization, we constrain the position to be at the circle and not overlapping with existing nodes.We then minimize the sum of distance with respect to the nodes connecting to the current one.
By default, we render the HMM node belonging to the frequent class with solid colors, and those belonging to the rare class in semi-transparent colors.This further avoids the user being distracted by the rarely performed actions.

Visualizing HMM Edges
We visualize the edges using 2D curves.While we can render the edges with straight lines, the resultant group would be difficult to observe as the lines overlap significantly.We augmented the edges with a small random curvature to solve the problem.We also render the edges as semi-transparent such that the users can see through partially overlapped edges.The thickness of the edge is proportional to T ij calculated in Eq. ( 4) .As a result, a thicker 656 edge connecting node i to node j indicates that the boxer often 657 launches action group j after action group i .The color of the edges 658 are decided based on that of the source node.This helps the user 659 to identify which action groups the boxer may launch after a par-660 ticular one. 661

Interactive features 662
We also implement some interactive features such that the user 663 can select what to view.The most important component of the 664 action-based graph is the action itself.Therefore, we implement 665 an interactive system such that when a user clicks on a particular 666 HMM node, the median action of the corresponding action group is 667 displayed.We also highlight the outgoing edges from such a node.668 This allows the user to examine individual action group together 669 with the transition probability to the next groups.The information 670 of the node, such as the number of member actions and the num-671 ber of out-going HMM edges, are displayed on a separate window.672 As an example, in Fig. 9 , there are 5 HMM nodes belonging to 673 the frequent class including node 1 and 2. These nodes are vi-674 sualized with more muscular characters, meaning that the boxer 675 performs them more frequently.There are 3 HMM nodes belong-676 ing to the rare class including node 5 and 6, which are visualized 677 with thinner characters.Node 1 has 5 outgoing HMM edges, in 678 which edge 3 point towards another node, while edge 4 is a self-679 connecting edge.Edge 4 is thicker than the others, indicating that 680 the boxer performs successive actions belonging to node 1 very 681 frequently.The screen is captured when the user selects node 1, 682 and as a result, all outgoing edges of node 1 are highlighted, and 683 the character representing node 1 performs the corresponding me-684 dian action.685

686
In this section, we present experimental results.We captured 687 the motions of four boxers with varying skill levels.We first give 688  as skillful, medium, medium and novice respectively, and were denoted as S, M1, M2 and N.

Boxer evaluation
The boxers' posture-based and action-based graphs are shown in Fig. 10 , in which letter annotations are given to help explain the graphs.These graphs allow the users to assess boxing skills even if they are not familiar with boxing.

Boxer S
The first row of images in Fig. 10

Boxer M2
For boxer M2's posture-based graph, there is a main standard posture (a) launching the majority of actions (b).There are, how- In the posture-based graph of the novice boxer N, there are two 781 major standard postures (a) and (b) instead of one.There are a 782 large number of self-connecting actions (c) and (d) for both pos-783 tures, as well as a lot of actions (e) connecting the two.This shows 784 that the boxer is highly inconsistent in the boxing postures.Pos-785 ture (a), the more relatively frequently used one, is inferior to pos-786 ture (b), due to its wider foot distance.It does not allow the boxer 787 to step freely.Posture (f), (g) and (h) are all secondary postures 788 with different posture variations.They are all not well performed 789 due to the low arm positions limiting blocking capability, and the 790 wide foot width limiting movement capability.

791
The corresponding action-based graph shows some actions in 792 the frequent class (a) but a large number of actions in the rare 793 class (b).This means that the novice boxer cannot perform actions 794 consistently.The action in the rare class (b) are mainly very long 795 combo that are randomly combined and cannot be reproduced.The 796 main action (c) is a movement action.Such an action cannot con-797 nect to a number of others in the rare class (b), and many ac-798 tions in the rare class (b) are not well connected.This means that 799 the boxer's action is more predictable, which is bad in a match as 800 the opponent can guess what the boxer may launch next.The two 801 more frequently used attack action (d) and (e) are left-right combo 802 and left punch respectively, showing that the boxer tends to start 803 an attack with a left punch.tured from 4 boxers with varying skill levels.The evaluations from our system aligns with that of a professional boxing coach.
Although we use boxing as our target sport in the experimentation section, the underpinning theoretical development can be applied to most sports that require swiftness, flexibility and creativity, such as tennis, fencing and basketball.The adaptation of the proposed system to these sports and the comparison of the system performances on different sports remain as future work.
We focus on analyzing the skill level of the boxers in terms of high-level motion behaviour such as the richness of the action, the transition of action and the unpredictability of boxing patterns.We do not evaluate the lower-level parameters such as the speed of the punches, which has been explored in previous works.It is an interesting future direction to combine both high-level and lowlevel evaluation in order to have a full assessment of the boxers.
There are limitations to our method.First, our method is based on the assumption that the sports skills mainly consist of a finite number of key postures and key actions.Admittedly, not all sports follow this pattern.Second, the visualization and skill assessment is based on an individual athlete, not considering skills related to collaborations such as those in group sports, in which the assessment might need to employ different criteria.
We argue that novice boxers tend to have different posturebased graphs, while experienced boxers tend to have graphs of a similar topology.This is because unlike experience boxers who have only 1 to 2 main postures nodes, novice boxers tend to have more nodes, resulting in a much larger variation on the graph topology.As a future work, we would like to utilize the stem to evaluate a large number of boxers in different skill levels to verify this argument.
In the future, we wish to extend the proposed algorithm to the field of computer animation.Currently, when synthesizing animations by motion graphs, experienced animators are required to tell what motions are missed or badly captured.With our system, it is possible to analyze the connectivity and variety of a motion set, which are two critical factors in motion synthesis.However, how to generalize these findings to give high-level suggestion, such as proposing the motions to capture, remains an open problem.In addition, we would like to develop a visualization system to take the adversarial nature of sports.For instance, although two boxers might have roughly the same skill level, in a match, one's skill composition might give him/her advantages over the other.This kind of analysis would be very useful in preparation for a game or predicting the result.
, which uses nodes to represent https://doi.org/10.1016/j.cag.2017.09.007 0097-8493/© 2017 The Author(s).Published by Elsevier Ltd.This is an open access article under the CC BY license.( http://creativecommons.org/licenses/by/4.0/ ) Please cite this article as: Y. Shen et al., Posture-based and Action-based Graphs for Boxing Skill Visualization, Computers & Graphics (2017), https://doi.org/10.1016/j.cag.2017.09.007 100 two graphs are presented in an organized and effective manner.101 The rest of this paper is organized as follows.Related works 102 are reviewed in Section 2 .The details of motion capture and or-103 ganization are given in Section 3 .In Sections 4 and 5 , we explain 104 the design and implementation of the posture-based graph and the 105 action-based graph respectively.Related experiments can be found 106 in Section 6 .The paper is concluded in Section 7 with future re-107 search directions discussed. 108

Fig. 1 .
Fig. 1.The shadow boxing motions of several boxers were captured using an optical motion capture system.

269 3 . 2 .Fig. 2 .
Fig. 2. Upper:The movement segment is defined as the period between two double support supporting phases.Middle: The activity segment is defined as the period with high acceleration.Lower: The action is the combination of movement segment and activity segment.

330 4 .
1.1.Fat Nodes 331 After 348 clustering, we use the mean posture of a group to represent the 349 corresponding Fat Node.The nodes in the graph represent the set 350 of standard postures which the player starts various action from.In 351 the case of boxing, they are usually the fighting postures that the 352 Please cite this article as: Y. Shen et al., Posture-based and Action-based Graphs for Boxing Skill Visualization, Computers & Graphics (2017), https://doi.org/10.1016/j.cag.2017.09.007ARTICLE IN PRESS JID: CAG [m5G; October 16, 2017;22:13 ] boxer uses to guard his/her face against the opponent, with both feet landing on the ground and keeping shoulder width apart.By evaluating the Fat Nodes alone, we can already tell if a boxer has multiple unnecessary standard postures, or if any standard postures contain potential weakness.In general, experience players have fewer Fat Nodes, such that they can start actions in a standard posture effectively without the needs of shifting to other ones.Novice players sometimes may have a particular Fat Node for some particular actions.This is discouraged in boxing training as such postures hint the opponent as to what actions are going to be launched.4.1.2.Fat Edges We design the edges of a Fat Graph, known as Fat Edges, as directional edges that represent groups of similar actions.Each edge points from the Fat Node representing the starting posture to that representing the ending posture.Similar to the Fat Nodes, we implement an unsupervised clustering algorithm to group similar actions into Fat Edges.We use k-means to cluster the actions and search for the smallest acceptable k for a given distance threshold.We define the actions distance according to the trajectory of the effective joints as explained in Section 3.2 .This allows accurate clustering of actions and ensures that the effects of the effective joints are not smoothed out by other joints.

Fig. 3 .
Fig. 3.The Fat Node represents the standard fighting pose.The three outgoing Fat Edges represent different action groups.

449Fig. 4 .
Fig. 4. From left to right, the character becomes larger as the size of the nodes increases.

Fig. 5 .
Fig. 5.The geometric patterns for landmark values between -1 and 1.Each pattern represents a landmark posture in an action.(Lower) Comparison of visualization without/with the patterns.Each curve represents a group of action.The right image shows the uses of landmark patterns to identify different types of action.

509
Through the comparison between Fig.5Lower Left and Lower 510 Right, it shows that adding the geometric patterns gives a better 511 visualization of actions in the edges.This strategy presents an intu-512 itive way to show the players preferences over actions of different 513 complexity. 514
, there are three Fat Nodes indicated by red arrows and numbered as 1, 2 and 3, each visualized as a character with a mean posture in the node.The sizes of the nodes are indicated by the body fatness.Node 1 is represented by the most muscular character, which indicates the largest node size.Nodes 2 and 3 are far thinner.Fat Edges are rendered as curves between nodes such as the ones shown by 4 and 5.The thicknesses of the edges indicate the frequency of the actions taken.Edge 5 is thicker than edge 4, suggesting that this boxer takes action 5 more often.

Fig. 7 .
Fig. 7.The three HMM nodes represent action groups.The HMM edges represent transitional probability between them.

Fig. 8 .
Fig. 8. Action-based graphs of the same boxer generated by setting the frequency threshold as (a) 0, (b) 1 and (c) 2. The red shade indicates the inner circle covering nodes of the frequent class, and the blue shade indicate the outer circle covering nodes of the rare class.
using human characters with the starting posture of the median action.The number of actions in each action group is visualized using the fatness of the corresponding character.The color of the nodes are randomized.As mentioned in Section 4.1.2, we observe that some boxers, especially novices, may produce random actions that are not repeatable.Such actions may generate a large number of thin nodes, which distract the user from evaluating the actions that are often launched.Therefore, we classify the action groups with | G i | > a into the frequent class , and groups with | G i | ≤ a into the rare class ,

Fig. 9 .
Fig. 9.The action-based graph of the Boxer S. 1, 2 are HMM nodes belonging to the frequent class.3, 4 are outgoing HMM edges from the node 1. 5, 6 are HMM nodes belonging to the rare class.
shows the graphs of boxer S. The posture-based graphs shows a main standard posture (a) to start and end actions, which is good for boxing as it allows the boxer to transit from one action to another effectively through the standard posture.A large variety of actions (b) can be produced from such a posture.There is a secondary posture in which the arms are further apart (c).This should be avoided as such a posture is weak in blocking attacks.Posture (d) is generated because the boxer trips over during the training.Our system can pick up and visualize such a mistake accurately.The action-based graph of boxer S shows there there are many actions in the frequent group (a) and only a few in the rare group (b).This shows that the boxer is experienced and his actions are consistent.There is a major movement action (c) in the frequent group (a), and such an action has good connections to many of the others.This is good as experienced boxers typically use movement actions to adjust their position relative to their opponent, and launch attacks when the time is right.Other actions in the frequent group (a) are variations of attacks.For example, the more frequently used action (d) is a right-left combo and action (e) is a single right punch, which show that the boxer tends to start an attack with the right punch.It is good to see that attacking actions may connect to each other, which enhance the unpredictability of the boxer.6.3.Boxer M1 Next, we evaluate the posture-based graph of boxer M1.The boxer has a main standard posture (a) to launch most of the actions (b).However, he has a secondary posture (c) for launching some attacks, and another (d) for launching a turning action.In both postures, the arms are in a low position and cannot guard the boxer well from the opponent.More importantly, the relatively more frequently used secondary posture (c) is performed with the foot distance much wider than the shoulder width.This means the boxer has limited mobility in this posture, as the legs must move towards each other before another stepping action can be performed.These observations show that the boxer is not as experienced and consistent as boxer S. The corresponding action-based graph shows that there are fewer frequent class actions (a) but more rare class ones (b) compared to boxer S.This means that that the boxing action of boxer M1 is less consistent.The boxer has a large number of movement actions (c) that are connected to all the rest of the action nodes.He also has a variety of attack actions as shown in other actions in the frequent class (a).In particular, action (d) is a left-right combo and action (e) is a left punch, showing that the boxer tends to start an attack with the left punch.Overall, there is an acceptable number of connections among attacks, demonstrating the acceptable unpredictability of the boxer.

Table 1
Statistics of the boxing motions.SL: Skill Level evaluated by a professional boxing coach.PN: Posture Number (for starting and ending actions).AN: Action Number., we can find that the posture is performed as a subtle 759 movement to prepare various left punches.This should be avoid 760 as the opponent can tell the moves whenever seeing such a pos-761 ture.Postures (g) and (h) are very different from the rest, and are 762 geometrically far from the other postures.These two postures are 763 performed because the boxer unintentionally raises the arms dur-764 ing the capture.Our system can pick up the mistake and visualize 765 it in the graph.-based graph, it can be observed that 767 there are relatively fewer actions in the frequent class (a), but a 768 large number of actions in the rare class combining (b) and (c).769 This shows that the boxer is quite inconsistent in the boxing ac-770 tions, and could be because of the lack of training and experience.771 Different from the boxers discussed, boxer M2 has a largest action 772 node (d) of left punch.The second largest action node (e) is a dou-773 ble left punch.The movement action node (f) is relatively small.774 This shows that boxer M2 has a different boxing style to use left 775 punch as a major action to connect to other actions and his left 776 punch is dominant.Such a boxing style is not advised as a punch-777 ing action, comparing to a movement one, consume more energy 778 and expose a larger risk of being attacked.