Shape correspondence for cel animation based on a shape association graph and spectral matching

We present an effective spectral matching method based on a shape association graph for finding region correspondences between two cel animation keyframes. We formulate the correspondence problem as an adapted quadratic assignment problem, which comprehensively considers both the intrinsic geometric and topology of regions to find the globally optimal correspondence. To simultaneously represent the geometric and topological similarities between regions, we propose a shape association graph (SAG), whose node attributes indicate the geometric distance between regions, and whose edge attributes indicate the topological distance between combined region pairs. We convert topological distance to geometric distance between geometric objects with topological features of the pairs, and introduce Kendall shape space to calculate the intrinsic geometric distance. By utilizing the spectral properties of the affinity matrix induced by the SAG, our approach can efficiently extract globally optimal region correspondences, even if shapes have inconsistent topology and severe deformation. It is also robust to shapes undergoing similarity transformations, and compatible with parallel computing techniques.

production. In the digital era, almost all the possible areas of cel animation production are shifting to sophisticated software tools that have dramatically reduced labor costs [1]. However, many tasks in the production process still require more efficient techniques for increasing productivity, such as inbetweening, auto-coloring, occlusion handling, and notably shape correspondence determination. Accurately finding shape correspondences between two keyframes provides a basis for many functions and distinctly improves their efficiency [1].
In cel animation, a region is a closed area composed of finite strokes. An individual shape is the set of all regions with topological relationships in a keyframe, which is what we call a cel animation shape (or cel shape). In this paper, the shape correspondence problem in cel animation is to match all regions between two cel shapes under some constraints, which usually are the geometric and topological similarities of shapes. It is a serious challenge because a keyframe in cel animation usually consists of an enormous number of regions which require significant computation and time to process. Furthermore, the regions in different cel animation shapes tend to have elastic deformations and inconsistent topology, which means that only limited geometric and topological feature information can be extracted and utilized. Topology usually changes within a certain range following animation principles, as shown in Fig. 1.
Several previous methods have used various shape descriptors and geometric similarity measures as the unary mapping constraint between regions to address the challenges [2][3][4][5]. These methods rely greatly on the geometric features of shapes, and they can fail due to insufficient discriminativeness of feature attributes, or loss of feature attributes when the shape deforms. Other existing methods add Examples in which it is difficult to find correspondences. (a) Significant deformation exists, and a similarity transformation caused by camera zooming between shape 1 and shape 2. (b) Topology inconsistencies are caused by partial occlusions: the newly generated region 2 in shape 4 interferes with the region correspondence process.
neighbor relationship information between regions into the matching framework [6][7][8][9][10][11]. Qiu et al. [6,7] improved matching accuracy by taking account of adjacency and hierarchical topological relationships of regions. These methods only focus on local topological information and often result in poor correspondence results when regions undergo large topological changes. Liu et al. [10] used geometric, local topological, and global topological information in a greedy algorithm, which can reduce time costs while ensuring accuracy. However, it relies too much on seed region selection and cannot handle correspondences of separate region groups. The method of Sato et al. [8] regards topological similarity in terms of differences in area, position, and angle between region pairs, and finds a globally optimal solution through quadratic programming. However, it is limited to the extrinsic similarity measure method and fails when regions have distinct similarity transformations.
To resolve these issues, this paper regards the shape correspondence problem in cel animation as an adapted quadratic assignment problem (QAP), which can incorporate geometric and topology constraints to find globally optimal solutions. Regions in the cel shape are assigned to other regions in such a way that the cost of their geometric and topological deformation is minimized. To simultaneously represent the assignment's geometric and topological similarities, we embed all candidate assignments of regions into an association graph. The nodes of the association graph represent all the assignments, and the edges represent topological relationship compatibilities between two candidates. The topological distances and geometric distances on the edges and nodes are obtained from the intrinsic shape similarity measure in Kendall shape space. Ultimately, the globally optimal correspondence can be effectively extracted by ranking and selecting nodes from the graph according to the spectral property of an affinity matrix induced by the association graph.
The main contributions of this paper may be summarized as follows: • We propose a globally optimal method for cel shape correspondence, which utilizes geometric and topological information of regions to measure their intrinsic similarities. We formulate the correspondence problem as an adapted QAP and introduce a spectral matching method to address it. Our method accommodates shape correspondences involving significant deformation and topological change. • We present a shape association graph (SAG) to represent all assignments of regions with geometric and topological attributes between two cel shapes. It represents how well the geometry matches in a region mapping and how consistent the topological relationships of any two combined region pairs are. Furthermore, it can be constructed in a parallel manner, and effectively induces the affinity matrix for spectral matching. • We utilize Kendall shape space to measure geometric and topological similarity of regions. Topological similarity is implemented by calculating the geometric distance of objects generated by merging and segmenting combined regions, and can accurately represent various types of complex adjacency relationships between regions. The geometric similarity measure method utilizes intrinsic geometric information, making it more accurate and discriminative. The method is invariant under similarity transformations of cel shapes. We demonstrate the accuracy and efficiency of our approach by qualitative evaluation on several representative cases: we achieved comparable or superior performance to existing methods.

Related work
We loosely classify previous works into two categories: those using geometric constraints only, and those using geometric and topological constraints.

Methods using geometric constraints
Early researches used the geometric similarity of the region as the constraint for shape correspondence. Madeira et al. [2] encoded the contours of the regions in a string and measured the region similarity in string metric space. Firstly they filtered the region matching candidate with a set of heuristics and then minimized the sum of the similarity measures by linear optimization to find the correspondence. Kanamori [3] extracted the region correspondence by deriving a matching cost between regions and minimizing the total matching cost matrix. The cost function is based on proxy ellipses that can approximately represent the orientation, scale, and position of each region. After that, Kanamori [4] evaluated the cel shape matching performance under the bipartite graph matching framework with three different shape descriptions, which are proxy ellipses, Fourier descriptors, and shape context. Trigo et al. [5] implemented a forward matching algorithm to find the correspondence. A comparison cost function consisting of five sub-functions was proposed to measure the various geometric features between two regions. These sub-functions are combined by a series of weights.
These methods regarded shape correspondence problem as a linear assignment problem with only geometric constraint, which can easily be resolved. However, their shape descriptions and similarity measure method were sensitive to local geometric features. They could not handle situations when the geometric features changed or disappeared when a shape had an exaggerated distortion. These methods also ignored topological deformation of shapes that would lead to failure when the shapes had occlusion. In contrast to these methods, our geometric similarity measurement does not rely much on the local characteristic points of regions. Furthermore, we can effectively combine the intrinsic geometric attribute and topological attribute of regions without redundant weights, which is robust to similarity transformation and unstable topology.

Methods using geometric and topological constraints
In contrast to the above methods, other works add topological constraints to their frameworks to improve output correspondences. Qiu et al. [6] defined a measure of similarity between regions at different scales. They ignored a region's actual coordinates and only considered topology and relative positions along contours of regions. The method determines the best region correspondence using a threshold and searches for the best-matching regions through a greedy algorithm. Their later extensions to this work tried to utilize topological information from region hierarchies [7] and skeletons [12] to estimate correspondences between different regions. Chang and Lee [13] presented a method to search for region correspondences in existing frames based on geometric and topological feature similarity of regions. The method first determines correspondence candidates according to geometric features of the regions. Then, if the measured topological similarities are within acceptable thresholds, the candidate is accepted. Sýkora et al. [9] determined shape correspondence by combining locally optimal block matching with as-rigid-as-possible shape regularization [14]. Other similar dense correspondence methods have been proposed that assume additional semantic information about the image [15], or utilize patchbased sampling [16]. Zhu et al. [11] developed a global shape correspondence approach that utilizes the spatiotemporal transformation of all regions in an entire animation sequence. The correspondence problem is formulated as a network flow problem: the flow is regarded as a potential correspondence trajectory. A similarity method was proposed by Zhang et al. [17]. Liu et al. [10] regarded region correspondence between two shapes as a bipartite graph perfect matching, in which the edges of the graph are encoded with geometric similarity, local topological, and global topological information. The global topological information is added to the graph according to a heuristic traversal path. Then, a greedy algorithm is used to solve the graph matching problem by updating the scores of seed pairs. A similarity method was presented by Miyauchi et al. [18], utilizing positional relationships and depth context relative to a seed pair. Sato et al. [8] constructed region correspondences for manga characters using quadratic programming and positional connection relationships of regions, but did not consider shape similarity between any two regions. Maejima et al. [19] formulated anime colorization as a graph matching problem and used spectral matching to solve it.
These methods' common limitation is that the analysis of geometric similarity of regions is too coarse and uncertain, and the topology of the shape is not fully utilized due to insufficient discrimination provided by topological similarityit reduces accuracy when the shape correspondence has serious topology inconsistency. Some methods overly rely on various parameters or seed regions. Some strictly require that the input shape is a pixel image or a frame sequence with many shapes. Furthermore, some methods use greedy algorithms in their frameworks, which often fail to find globally optimal solutions. Our method can handle incompatible topology in shape correspondences even if the topology is not connected: it extracts global topological information about shapes more reliably. In addition, there are no seed regions or redundant parameters in our method. While many state-ofthe-art works [20][21][22] focus on learning cel shape correspondences at the pixel-level, this requires more time and reference data for training.

Problem formulation
We assume that we are given two cel shapes S 1 and S 2 from two animation keyframes. We regard S 1 and S 2 as being composed of two sets of regions S 1 = R 1 1 , · · · , R 1 n and S 2 = R 2 1 , · · · , R 2 m , where R i k is the kth region of shape S i . n and m are the numbers of regions, of each shape, and we assume n = m without loss of generality. The correspondence between S 1 and S 2 can be regarded as finding a oneto-one mapping X : where the value 1 indicates a correspondence between regions in S 1 and S 2 , and a value 0 indicates the regions do not correspond. If m = n, some regions will not find their correspondence.
To ensure the correspondence is an injective map, we always regard the input cel shape with fewer regions as the original shape S 1 . Therefore, determining cel shape correspondence can be posed as maximizing an energy as Eq. (1): For accurate similarity measurement, we simultaneously use both comprehensive geometric and topological information about regions More precisely, to represent the geometric constraint, we define the geometric distance of regions in Fig. 2. To represent the topological constraint, we define the combined region pairs obtained by their topological relationships, as also shown in Fig. 2. The topological distance between P 1 ij and P 2 i j measures how compatible the topological relationship of P 1 ij with that of P 2 i j , which we denote W iji j Topo . Then, the matching similarity coefficient C ii jj is defined using the geometric and topological differences: where f is a function of W ii Geom , W jj Geom , and W iji j Topo . Actually, there are nm candidate assignments in total, which can be expressed as We rank all candidate assignments in a fixed order and define the ranked set as T = {τ 1 , · · · , τ nm }, where τ i is the ith assignment of the ranked set. Therefore, the correspondence Candidate assignment of regions, and a combined region pair.
problem can be formulated as an adapted QAP, in which the weights simultaneously contain both the intrinsic geometric distance and the accurate topological distance between regions. Equation (1) is equivalent to the integer quadratic score function in Eq. (3): where M ij is the matching similarity coefficient of τ i and τ j . vec(X) is the column-wise vector of X using the same rank order as for T . X * is the final binary matrix that indicates the correspondence. Constructing M and finding the solution of Eq. (3) are the key tasks of the correspondence problem. In this paper, we propose a shape association graph to help build M and we apply a spectral matching method to find the solution of Eq. (3) by relaxing X to be the eigenvector of the principal eigenvalue of M . The optimal X can be computed as the leading eigenvector of M . Details are discussed in Sections 6 and 7.

Framework
As Fig. 3 shows, our framework includes three modules: the shape representation module, the shape association graph (SAG) construction module, and the spectral matching module. Our framework uses 2D vectorized hand-drawn shapes as input because they are precise, editable, and manageable. More crucially, they are widely used in animation industry software tools like CACANi (Computer Assisted Cel Animation), Toon Boom Harmony, etc. [1].
Given two different cel shapes as input, first, we extract and store their topological and geometric information through the shape representation module. It defines and classifies the topological relationships of each combined region pair, and provides a geometric representation for the topological and geometric similarity measurement.
Next, we construct the SAG in the shape association graph construction module and embed the similarity of topology and geometry between regions pairwise as attributes into the nodes and edges of the graph. This module uses a geometric and topological similarity measurement method in Kendall shape space to set the attributes. It also efficiently translates the graph into an affinity matrix for spectral matching.
We then run spectral decomposition for the matrix in the spectral matching module and optimize the matching constraints to obtain the global optimal cel shape correspondence; this is done by sorting according to properties induced by spectral decomposition. Finally, we filter and select the consistent region correspondence according to the mapping constraints.
More details of each module are explained in the following sections.

Shape representation
A cel shape contains topological and geometric information. We now introduce the representation used to extract and store it.

Topological representation
As defined earlier, a cel shape consists of regions and there is a topological relationship between any two regions in the shape. Given two regions R i and R j in a cel shape, P ij = R i , R j is the combined region pair. The topology of a cel animation shape is defined as the topological relationships of all combined region pairs in the shape. We propose an improved region adjacency graph (RAG) [15] for shape topology representation, as Fig. 4(a) shows. Unlike traditional RAGs, our RAG is a disconnected attribute planar graph that can specifically reflect the topological relationship between an isolated region and all other regions in a cel shape, such as region R 11 in Fig. 4(a).
Let G = {V R , E, A E } be an RAG, where each node V R represents a region and each edge E indicates that there is a boundary connecting two regions. A E represents the topological relationship information attached to the edge, that is, the type of topological relationship between the two regions. There are many ways to extract regions and construct a traditional RAG [23][24][25]. We follow the RAG extraction method proposed by Liu et al. [10].
To set A E , we refine the type of topological relationship on the basis of a 9-intersection model [26] according to the regions' connectivity and spatial relationship. We consider the following topological relationships when defining A E : (a) Connected with shared points. R i touches R j at a finite number of intersection vertices, like R 1 and R 3 in Fig. 5(a). We call these intersections shared points. (b) Connected with shared edges. R i touches R j along a finite number of strokes, like R 1 and R 2 in Fig. 5(b). We call these strokes shared edges.  (c) Connected with shared holes. R i touches R j at shared boundaries, which may include multiple shared points or non-connected shared edges. One or more holes are formed between the boundaries. See, for example R 1 and R 4 in Fig. 5

(c). (d) Inner & interior connected.
There is no boundary between R i and R j . One region is located inside the other, or one region encompasses the other, like R 2 and R 7 in Fig. 5(d). (e) Non-connected. There is no boundary between R i and R j and there is no inner & interior connected relationship: see R 3 and R 11 in Fig. 5(e). We have found that topological transformations usually follows animation principles [27]: topological connectivity is generally consistent. Therefore, we divide the five topological relationship types into three classes based on connectivity: inner & interior, connected, and unconnected, as shown in Fig. 5. We posit that if two topological relationships belong to the same class, we can measure their topological similarity by a quantitative method.

Geometric representation
The geometric representation of a cel shape is regarded as the geometric representations of all of its regions. Each region is represented using a finite number of discrete points on its contour, as the contour can comprehensively describe the geometry of a region, as Fig. 4(c) shows. Let R = {(x 1 , y 1 ) , · · · , (x n , y n )} indicate a region via the coordinates of discrete points on its contour. Since the contour is a closed curve, the point list needs to be ordered and the starting point of the point list can be arbitrarily chosen. We can easily sample any number of discrete points in counterclockwise order as needed, because our input region is vectorized.
Contour points uniformly sampled. Note that the contour must not branch. The more points that are sampled, the more detailed the description of the region; however, using large numbers of sample points increases the computing time of our method. Our geometric representation allows us to focus on the intrinsic geometry of a region, that geometrical information which remains when location, scale, and orientation effects are removed from the region.

Shape association graph
To build the affinity matrix M in Eq. (3), we use a shape association graph (SAG) to organize all candidate assignments of regions from the two cel shapes. The SAG is an attributed undirected graph, which can be constructed from the individual RAGs of the shapes to be matched. The SAG reflects the geometric and topological similarity of candidate assignments. Let , where the node V SAG represents the set of bound nodes {V ii | i = 1, · · · , n, i = 1, · · · , m}, which indicates the set of candidate assignments i j belong to the same topological relationship class (see Section 5.1). The node attribute giving the topological distance between P 1 ij and P 2 i j . Details of SAG construction follow.

Construction of SAG nodes
We consider areas of regions when building nodes V ii . We require that if two nodes v i and v i form a node V ii in the SAG, then R 1 i and R 2 i must meet the area constraint. Let the areas of R i and R i be A i and A i . The total areas of all regions in S 1 and S 2 are A s 1 and A s 2 . The area constraint is that the difference between the ratio of A i to A s 1 and the ratio of A i to A s 2 must be lower than a threshold W a , as Eq. (4): where The area of each region can be easily calculated using our geometric representation, and we can improve the accuracy and efficiency of the algorithm given the area constraint.

Construction of SAG edges
V ii and V jj can be used to construct an edge E (V ii ,V jj ) as long as the combined region pairs P 1 ij and P 2 i j satisfy two conditions, as shown in Fig. 7(c). Firstly, the topological relationships of P 1 ij and P 2 i j must belong to the same class. Secondly, their topological relationship type should not be non- connected. For example, if φ 1 ij is of type connected with shared point and φ 2 i j is of type connected with shared edges type, we will generate an edge E (V ii ,V jj ) between V ii and V jj , because these two types belong to the same topological relationship class, the connected class. If φ 1 ij or φ 2 i j is of type nonconnected, an edge cannot be generated. Using E SAG , an adjacency matrix M A = {M A kt } ∈ {0, 1} nm×nm can be extracted from G SAG , where k and t index the nodes V ii and V jj , k, t ∈ {1, · · · , nm}. We rank all nodes in a fixed order to give the ranked set T defined in Section 3: see Fig. 6(f). Then M kt can be represented as

Setting node attributes
We use the geometric distance W ii Geom between the two regions as the attribute of V ii , and use the shape similarity measure in Kendall shape space, which is a shape analysis method on Riemannian manifolds [28] for accurately describing the intrinsic geometry of shapes. The shape similarity measure regards a region in a cel shape as a point on a 2κ−4 dimensional manifold S 2κ−4 , where κ represents the number of contour points. We first concatenate the contour points into a 2κ-vector. Then, we construct the Kendall shape space, by making region information similarity transformation invariant, i.e. under translation, rotation, and scaling [10,29], as Fig. 8(c) shows. After doing so, a region can be regarded as a point on a 2κ − 4 dimensional hypersphere, and the similarity of two regions can be described as the geodesic distance between two points on the hyper-sphere [30], which is equivalent to the angle between two 2κ-vectors, as Fig. 8(d) shows. We formulate W ii Geom using Kendall shape space as represents the Kendall distance between regions R i , R i . Z (i) , Z (i ) are the shape representations of R i , R i after removing the similarity transformation. S 2κ−4 is the 2κ − 4 dimensional hyper-sphere. d S 2κ−4 is the geodesic distance between Z (i) , Z (i ) on S 2κ−4 , and can be obtained by calculating the angle between the two 2κ-vectors. Our method can not only measure the similarity of closed contours, but also open curves. The difference is that the starting point of the contour needs to be iteratively decided for closed contours, but not for an open curve [10]. Unlike the method in Euclidean space, our method can provide a robust intrinsic similarity measure for closed or open 2D curves without worrying about the influence of similarity transformations.

Approach
We use the topological distance W iji j Topo to represent the attribute A iji j E SAG of an edge. During edge construction, we distinguish topology similarity based on the consistency of topological relationships. However, the distinction is qualitative, and we need a quantitative method to measure the topological distance between P 1 ij and P 2 i j when their topological relationships are in the same class. To do so, we convert the topological distance into a geometric distance with topological identification, such as spatial relationship, scale relationship, and boundary in a combined region pair. Then, we can utilize the Kendall model to measure the converted geometric similarity. The key challenge is to transform combined region pairs into measurable geometric objects, while preserving topological features and intersecting boundary information about the pairs. Therefore, we propose a merging operation and a segmentation operation for a combined region pair that can glue a pair into a closed curve or split it into two open curves. Then, we can obtain the value of W iji j Topo by calculating the geometric distance of these geometric objects.
Given P ij and P i j , the mathematical representation of W iji j Topo is defined as where U ij merg and U i j merg are the closed curves obtained by performing merging operations on P ij and P i j . U ij seg1 , U ij seg2 , U i j seg1 , and U i j seg2 are the open curves obtained by performing segmenting operations on P ij and P i j . λ is a weight. Combined region pairs with different type of topological relationships have different merging and segmenting methods.

Merging operation
The merging operation merges two regions R 1 i and R 1 j into one closed curve U ij merg based on their topological relationship φ 1 ij , as illustrated in Fig. 8(b). We extract and sample the outer contours of the two regions as a new contour (a closed curve) according to the intersection and boundary information (shared point, shared edge, shared hole). Different combined region pairs with different topological relationship types have different merging methods. Different topological relationship types have different contour sampling methods, as shown in Fig. 8(b).
If the topological relationship belongs to the connected class (see Section 5.1), we sample counterclockwise along the outer contour after removing shared points, edges, or holes. If the topological relationship belongs to the inner & interior class, we first sample the contour of the external region and then find the point nearest to the starting point of the external region as the starting point and continue to sample the inner region. If the topological relationship is in the non-connected class, we do not merge them. U ij merg implies the topological relationship information of the relative position, relative size, and orientation of the two regions.

Segmentation operation
A segmentation operation is used to split R 1 i and R 1 j into U ij seg1 and U ij seg2 by removing their intersecting parts. Like the merging operation, segmentation also requires different sampling methods based on φ 1 ij . If φ 1 ij belongs to the connected class, we pick a shared point on the outer contour as a starting point, and then sample along the outer contour until we encounter the next shared point. Since we sample the curves in different directions from the starting point, we will get two segments, U ij seg1 and U ij seg2 . We define U ij seg1 as the segment sampled clockwise and U ij seg2 as the opposite. If the topological relationship belongs to the inner & interior class, U ij seg1 and U ij seg2 are the contours sampled from the two regions themselves. We ignore the non-connected type. U ij seg1 and U ij seg2 contain more geometric information after removing the topological relationship.

Spectral matching
The analysis in Section 3 shows that the region correspondence problem in Eq. (1) can be transformed into the QAP problem in Eq. (3). In this section, we give a spectral matching method for solving Eq. (3). Let M V = λV , an eigenvalue problem. If we left multiply both sides V T , and ensure that V T V = 1, so that V is a unit vector, we have where λ max is the maximum eigenvalue of M . Let V * be the eigenvector corresponding to λ max , i.e., λ max = V * T M V * . The difference between Eq. (8) and Eq. (3) is that vec(X * ) is constrained while V * ∈ R nm is an eigenvector. We thus use the spectral matching method [31][32][33] to solve the problem in Eq. (3) by relaxing vec(X * ) to be an eigenvector. Firstly, we find the largest eigenvalue λ max of M and the corresponding eigenvector V * . Since the elements of vec(X * ) are 0 or 1, but V * is a real-valued vector, we separate V * into two classes that belong to 0 and 1 using the constraints of vec(X * ) and obtain the final results of our correspondence problem. We may summarize the spectral matching method as the following steps. 1. Build the symmetric non-negative affinity matrix M = {M kt } ∈ R nm×nm using the SAG. M kt is the similarity between the kth candidate assignment R 1 i , R 2 i and tth candidate assignment R 1 j , R 2 j . Since the attributes of nodes and edges in SAG are defined using distances, we need to convert distances into similarities. Thus, M is defined as (1) For k = t, set M kt = exp(−(5W ii Geom ) 2 ). (2) For k = t, using both geometric and topological information, set Perform spectral decomposition on M and obtain its eigenvalues {λ 1 , · · · , λ n×m }. Calculate the principal eigenvector V * corresponding to the largest eigenvalue λ max . 3. Sort V * in descending order to separate it into two clusters in the following steps. Initialize the solution vector vec(X * ) as a nm × 1 zero vector. 4. Check if the largest element V * (i) of V * belongs to 1. In this process, if the candidate assignment that corresponds to V * (i) conflicts with an already chosen candidate assignment, we ignore that candidate assignment based on the oneto-one correspondence constraint in Eq. (3). Otherwise, we set vec(X * ) ij = 1, which means this candidate assignment is chosen. 5. If all regions in the original cel shape have found a correspondence, return the vector vec(X * ) and go to Step 6; otherwise, find the next greatest value in V * and continue at Step 4. 6. Transform the vector vec(X * ) into an n × m matrix according to the sorting of the candidate assignments T . The transformed result is the final matching X * . The spectral matching method avoids the combinatorial explosion inherent in the corresponding problem and is robust to noise. It has a complexity of O n 3/2 .

Data and setting
To evaluate the efficacy of our method, we recorded its performance when applied to shape correspondence and to auto-coloring on ten cel animation cases. The ten cases in Fig. 10 and Fig. 11 provided by professional animators raised typical shape correspondence issues in 2D animation industrial production, as shown in previous works [10,34], and are considered great challenges for cel shape correspondence methods. Table 1, Fig. 10, and Fig. 11 show the topological and geometric differences between the original cel shapes and the target cel shapes. All have significant deformations and topological inconsistencies. The cases of Solider, Jumping girl, and Walk forward contain a large number of regions (more than 65). In the most challenging case, Walk forward, regions with topological changes account for 75% of the cel shape. The Dwarf and Singer cases are both roughly symmetrical. The arms and legs of the cel shapes are extremely deformed. Both the Head case and the Hat case undergo 3D rotations, leading to many newly generated or degraded regions. The Head case contains regions with large numbers of inner or interior topological relationships. As in the Jumping girl case, the overall shape in the Dragon case is seriously deformed, and the mouth and tail have topological inconsistencies. The case further contains a large number of similar and trivial shapes, such as the bony plates on the dragon's back. In the Angry man case, one aggregate region group is divided into two region groups during animation. In the Dog case, 57% of the regions topologically change due to occlusion of the right arm, and many of them undergo similarity transformations.
We deployed our algorithm on a graphics workstation with a 2.4 GHz Intel Xeon CPU and 16 GB of memory, with an NVIDIA Quadro M4000 graphics card. In our testing, when the sampling rate is 100, and λ in Eq. (7) is set to 0.25-0.35, the efficiency of the algorithm is the highest. The ratio of the number of correctly matched regions to the number of regions of the input shape is used as the matching accuracy measure. The efficiency of our method is evaluated in terms of matching accuracy and time cost.

Variants
We first conducted a qualitative experiment to demonstrate the applicability of our algorithm, using four variants, as follows:   The experimental results in Figs. 10 and 11 allow us to analyze which similarity contributes more to our method. In situation (a), the average accuracy of all cases is less than 60%. Correct region correspondence cannot be found in the geometrically and topologically inconsistent positions in these cases, such as the right arm in Solider, and the trunk in Dragon. We can see that the similarity measurement based on Kendall shape space can effectively distinguish geometric shape differences. Nevertheless, our method cannot extract all correct correspondences using only geometric similarity measurements.
Compared to (a), the accuracy is greatly improved in (b) (64.39%) and (c) (79.23%), showing that topological attributes of shapes can help our method to effectively improve matching accuracy. However, only using topological similarity has yet to meet our requirements.
In method (d), the matching accuracy goes up to 94.86%, and in 60% of cases, 100% accuracy is achieved. In the Solider and Head cases, due to our one-to-one correspondence framework, most of the degraded regions and new regions generated due to topological changes resulted in mis-correspondences, as shown in Fig. 9(a). However, in animation production practice, these regions would be corrected by animators. In the Jumping girl case, most incorrect correspondences are around the girl's apron (see Fig. 9(b)). This is because in our method, if a region has both great geometric deformation and topological change, as do its adjacent regions, then the region is likely to find an incorrect correspondence due to the lack of correct geometric and topological clues. The lowest accuracy occurs for the Walk forward case, 73.6%-see Fig. 9. In this case, our method did well in the head area, but the farther away from the head, the more likely mis-correspondences are to occur. This is because only 25% of the regions have consistent geometry and topology, and these are located around the head. 27% of the original regions are degraded, particularly the shoes and the right leg of the cel shape.

Comparison to state-of-the-art
We also compared our algorithm to various other state-of-the-art algorithms. We considered five methods: SCM [35], CARM [6], HRM [7], QPM [8], and KRAGM [10]. Figures 12 and 13 summarize the results of different algorithms when applied to the above cases.
In addition to applying these methods to all cases, we evaluate the effectiveness of these methods in the situation where input shapes undergo a similarity transformation, which is very common in animation production, as shown in Figs. 14 and 15. We randomly moved, rotated, and scaled the target shapes in all cases.
From Table 2, SCM has the lowest accuracy among these methods. The reason is that it relies excessively on geometric similarity and ignores topological information so that correspondences of regions with     great deformation cannot be found. CARM is only better than SCM in terms of accuracy. We found that this method cannot adequately handle regions with ambiguous feature points along the contours. In addition, its topology similarity method is too coarse, and it cannot use topological information effectively. HRM is slightly better than CARM both in terms of accuracy and speed because it first layers regions hierarchically and then matches them layer by layer, which can effectively utilize the inner & interior topological information. However, if there are errors in the root region correspondences, then its child regions cannot find correct correspondences, like the regions on the Jumping girl's face in Fig. 13(h). QPM is more accurate than the above three methods, but is the slowest. It considers the differences between all potential matching pairs without the help of topological relationships. Therefore, its calculation of the affinity matrix is complex and increases time cost. In addition, the similarity calculation of this method depends on extrinsic features of the region, which may lead to a large number of errors when the correspondences undergo similarity transformation. For example, all the regions in the Dog case in Fig. 13(i) have rotation and translation, and it is difficult for QPM to achieve acceptable accuracy. Indeed, Table 2 shows that all methods except KRAGM and ours are severely affected by similarity transformations, especially QPM. Among all methods, KRAGM is the fastest and has an accuracy only second to ours. However, it relies heavily on choice of the seed region, and cannot find all correspondences when shapes are not connected. For example, in the case of Angry man, the hat is separated from the body in the target shape. The seed region is assigned to the hat. Once the hat has been matched, the algorithm will not continue to match the body because of change in topological connectivity. Besides, it will also fail if there are newly generated regions or regions that have great geometric and topological variations splitting the cel shape, such as the apron in the Jumping girl case in Fig. 9(b) or the microphone in the Singer case in Fig. 11(f). Our method has the best accuracy. This comparison shows that our method can handle the region correspondence problem for cel shapes with dramatic shape changes, inconsistent topology, and disconnected topology. Our method is efficient even when the number of input cel shapes is small or the cel shapes undergo similarity transformations. Even though our method is slow, it is readily parallelized. Table 1 shows that it is much faster when multi-threading is used.

Limitations and future work
Despite working well in many common scenarios, some limitations still call for attention. Our method still performs unsatisfactorily in one-to-many and many-to-many correspondence cases. We also cannot find correspondences for regions all of whose adjacent regions have great geometric and topological changes. Our method cannot handle correspondences involving multiple derived or degraded regions. In addition, our current method is limited to vectorized input and cannot handle rasterized input data directly. In future work, we hope to work on these limitations, and use GPU methods to speed up our algorithm. We also hope to improve and extend our approach to multi-dimensional correspondences that can find stroke and region correspondences simultaneously.

Conclusions
This paper proposed a spectral matching method for region correspondence in cel animation, which can greatly reduce the labor of tasks such as coloring, inbetweening, etc. The cel shape correspondence problem was formulated as an adapted QAP, whose affinity matrix reflects the intrinsic geometric distance between two regions and the accurate topological distance between two combined region pairs. In constructing the matrix, we proposed a quantitative method to measure topological similarity and introduced Kendall shape space to measure intrinsic geometric similarity. We used spectral matching to analyze the matrix induced by the SAG to find a globally optimal regional correspondence. Compared to previous work, our method can better deal with situations where cel shapes have inconsistent topology, exaggerated deformation, and similarity transformations. Numerous experimental results show that our approach outperforms existing methods. Xingce Wang is a professor in the School of Artificial Intelligence, Beijing Normal University. She majored in 3D modeling and 3D visualization. Her current research interests include computer graphics, medical imaging, artificial intelligence, and machine learning. Xiangyuan Liu is studying for a Ph.D. degree in the School of Artificial Intelligence, Beijing Normal University. Her research interests include image processing, medical imaging, shape analysis, computer graphics, discrete differential geometry, and Riemannian geometry.