An Evaluation of Canonical Forms for Non-Rigid 3D Shape Retrieval

Canonical forms attempt to factor out a non-rigid shape’s pose, giving a pose-neutral shape. This opens up the possibility of using methods originally designed for rigid shape retrieval for the task of non-rigid shape retrieval. We extend our recent benchmark for testing canonical form algorithms. Our new benchmark is used to evaluate a greater number of state-of-the-art canonical forms, on ﬁve recent non-rigid retrieval datasets, within two diﬀerent retrieval frameworks. A total of ﬁfteen diﬀerent canonical form methods are compared. We ﬁnd that the diﬀerence in retrieval accuracy between diﬀerent canonical form methods is small, but varies signiﬁcantly across diﬀerent datasets. We also ﬁnd that eﬃciency is the main diﬀerence between the methods.


Introduction
The ability to recognise a deformable object's shape, regardless of the pose of the object, is an important requirement for modern shape retrieval methods. One approach to this problem is to transform each deforming model into a canonical form which (ideally) factors out the pose, leaving a standard pose-independent version of the shape. This allows rigid shape retrieval algorithms to be used for non-rigid shape retrieval. Many different methods have been proposed for automatically computing a canonical form from a 3D mesh. Methods using such approaches along with rigid retrieval systems have performed well on shape retrieval benchmarks [21]. However, most of these methods have not been compared using the same dataset, or used for retrieval within the same rigid retrieval system, so their relative performance is unclear. We recently proposed a new benchmark to provide a meaningful comparison of existing and new canonical form methods for non-rigid shape retrieval [34]. In this paper we extend our previous work to test a larger number of recent canonical form methods, allowing us to more reliably characterize the state-of-the-art performance. Our previous benchmark used a small dataset. In this work we test the performance of all canonical form methods on three larger recent non-rigid object datasets. The algorithms were tested by using their output canonical forms within two different rigid shape retrieval frameworks, and evaluating the retrieval performance.
Our paper is structured as follows. Section 2 describes the datasets used. Section 3 outlines all the canonical form methods tested. Section 4 outlines the retrieval frameworks used in our experiments. Our experiments are presented in Section 5, and finally we conclude in Section 6.

Datasets
We performed experiments on five different datasets. Two of these datasets contain only non-rigid human models (see Sections 2.1 and 2.2), and the third contains a variety of different non-rigid shapes (see Section 2.3). We also have non-rigid shape dataset TOSCA and for robustness test we make the noisy version of TOSCA.

Real Human Dataset
The Real human dataset [30,31] was built from point-clouds contained within the Civilian American and European Surface Anthropometry Resource (CAESAR) [7]. Meshes were fitted to these pointclouds, and a data-driven technique was used to create a set of poses for each human subject. The Test set contains 400 models, representing 40 human subjects (20 male, 20 female), each in 10 different poses. The Training set contains 100 models, representing 10 human subjects (5 male, 5 female), again in 10 different poses. None of the training subjects or poses are present in the test set. The meshes each have approximately 15,000 vertices, varying slightly from mesh to mesh. A selection of these models is shown in Fig. 1(a). This dataset is freely available to download [29].

Synthetic Human Dataset
The Synthetic human dataset [30,31] was created using a parameterized human model, where parameters control body shape, provided as part of the DAZ Studio [8] 3D modelling and animation software. Different poses were created using a palette of poses also provided with DAZ Studio. The Test set contains 300 models, representing 15 different human subjects (5 male, 5 female, 5 child), each in 20 different poses. The Training set contains 45 models, representing 9 human subjects (3 male, 3 female, 3 child) in 5 different poses. None of the training subjects or poses is present in the test set. The meshes have approximately 60,000 vertices, again varying slightly. A selection of these models is shown in Fig. 1(b). This dataset is also freely available to download [29].

Non-Rigid Shapes Dataset
The non-rigid shapes dataset [22] contains a greater variety of shape classes, and therefore we also performed experiments on this dataset. It contains 1,200 meshes, split into 50 different shape classes. Each shape class contains 24 meshes of an object in different non-rigid poses. Four meshes in each shape class contain topological errors, such as disconnected components, or unwanted connections. The average number of vertices per mesh is 9,607. A selection of these models is shown in Fig. 1(c). This dataset is also freely available to download 1 .

TOSCA and TOSCA with noise
TOSCA [5] contains hi-resolution three-dimensional nonrigid shapes in a variety of poses for nonrigid shape similarity and correspondence experiments. The database contains a total of 80 objects,  including 11 cats, 9 dogs, 3 wolves, 8 horses, 6 centaurs, 4 gorillas, 12 female figures, and two  different male figures, containing 7 and 20 poses. We also built a noisy version of TOSCA by adding random turbulence to vertices of the mesh. This is to test the robustness of the methods.

Canonical Form Methods
We now briefly describe each of the canonical form methods compared in our study. Many are variants of the multidimensional scaling (MDS) and geodesic distance based method proposed by Elad and Kimmel [11], but some replace either or both of these components.

Background and the Classical Multidimensional Scaling
The key observation is that the pairwise geodesic distances remain the same for the same shape with two different poses (known as isometric deformation). Transforming the geodesic distances to an Euclidean space could eliminate the differences of pose. The classical MDS method takes as input the pairwise distances, and produces a configuration (or coordinates) of all the vertices that best preserves the provided distances.
Elad and Kimmel [11] proposed computing a canonical form of a mesh by mapping the geodesic distance between all pairs of vertices to three-dimensional Euclidean distances. As the geodesic distances are pose invariant, the Euclidean embedding is a pose invariant representation of the shape.
We first compute the all-pairs geodesic distance matrix of the mesh using the fast marching method [18]. The Euclidean embedding of these distances is then computed using classical MDS. We do this by first calculating where D is the matrix of pairwise squared geodesic distances, and N is the number of mesh vertices. I is an N × N identity matrix, J is a centralization operator and B is the centralized squared geodesic distance matrix. We then compute the four largest eigenvalues (λ 0 ≥ λ 1 ≥ λ 2 ≥ λ 3 ) and the associated eigenvectors (φ 0 , φ 1 , φ 2 , φ 3 ) of B. Given a point p on the mesh, the MDS embedding is calculated as MDS(p) = λ Given the geodesic distances, the MDS method has a complexity of O(N 2 ), where N is the number of vertices. However, the computation of geodesic distances has a time complexity of O(N 2 log N ). Due to this high computational expense the meshes are simplified to approximately 2,000 vertices before computing the canonical forms. The methods proposed in subsections 3.2-3.4 also need to compute geodesic distances, and likewise use simplified meshes.

Fast Multidimensional Scaling
The fast MDS canonical form method [13] uses a variant of the method in Section 3.1, in which the geodesic distances are projected into Euclidean space one dimension at a time. We first compute the geodesic distance between each pair of vertices i and j as d ij . For each of the three dimensions in turn we select the two most distant vertices O a and O b (in Euclidean distance). All other vertices are then projected onto the line O a O b : x i is used as the embedded vertex coordinates in the current dimension. The distances are then updated as This MDS method has a lower time complexity of O(N ), but overall this approach still requires the expensive O(N 2 log N ) computation of geodesic distances.

Least Squares Multidimensional Scaling
This is another variant on the method described in Section 3.1, also proposed by Elad and Kimmel [11]. We use the SMACOF (scaling by majorizing a convex function) algorithm to compute the MDS. SMACOF minimises the following functional: where N is the number of vertices, w ij are weighting coefficients, δ ij is the geodesic distance between vertices i and j of the original mesh, and d ij is the Euclidean distance between vertices i and j of the resulting canonical mesh X. The stress function is minimised iteratively using the code provided by Bronstein et al. [4]. This MDS algorithm has a complexity of O(N 2 ) and also depends linearly on the number of iterations, but again all pairs of geodesic distances must be found.

Non-metric Multidimensional Scaling
This method is very similar to the one in Section 3.3, but instead of matching the Euclidean distances to the exact geodesic distances, we only match the ordering of the distances. The stress function we minimise for this method is where N is the number of vertices, δ ij is the geodesic distance between vertices i and j of the original mesh, d ij is the Euclidean distance between vertices i and j of the resulting canonical mesh X, and f is an optimal monotonic function of the dissimilarities. This MDS method is less restrictive than metric MDS, and some researchers have found that it produces more desirable results [17], but again requires geodesic distances.

Procedure 1 Nyström-MDS
Input G = {D, V } with p = |V | vertices, parameters n, k. Output A matrix Z which contains the coordinates of the embedding into R k . 1: Choose n samples of the data using farthest point sampling and obtain the matrix F . 2: Compute R, by element-wise squaring F , R ij = F 2 ij . 3: Denote by S the intersection of R and R T in D (see Figure 2). 4: ComputeṼ andΛ, which contain the n 1 = round(n/2) largest eigenvalues and corresponding eigenvectors of S, using eigenvalue decomposition 5: Compute QW = JRṼ using QR factorization 6: ComputeṼ 2 andΛ 2 , which contain the k largest eigenvalues and corresponding eigenvectors of − 1 2 WΛ −1 W T , using eigenvalue decomposition. 7: Return the coordinate matrix Z = QṼ 2Λ

Accelerated MDS
Shamai et al. [40] proposed an acceleration framework for MDS, which accurately approximates the pairwise geodesic distance maps, which reduces the high complexities to quasi-linear.
Given a triangular mesh representing a 3D object, let Z p×k be the matrix holding the points {z i } p i=1 of the canonical form in its rows. Classical scaling finds the canonical form Z by minimization : where D ij holds the squared geodesic distance between points i and j in the mesh. The main drawback of classic MDS is its use of the matrix D, which is too large to be computed and stored. In this method, termed NMDS [40], the Nyström method [43] is improved by adding a regularization term, and using it to approximate the matrix D, by decomposing it into a product of smaller matrices. For this method, only n columns of the matrix D need to be computed (typically 10-20 columns), corresponding to n landmarks chosen from the mesh. Each column corresponds to the squared geodesic distances between a particular landmark and the rest of the mesh points. The landmarks are chosen using farthest point sampling [15]and the geodesic distances are computed using the fast marching method [18]. The NMDS method is summarized in Procedure 1.

Constrained MDS
Sahillioglu [38] presented a method that introduces a constraint on the original MDS formulation in order to preserve the initial geometric details in the output shape. The motivation for preserving details during the MDS embedding process is to better distinguish semantically similar objects with varying geometric details, which is potentially useful in a shape retrieval application. Specifically, this method exploits the perfectly accurate bijection between the original shape and its Landmark MDS embedding [9], where the former and the latter has no and significant geometric distortions, respectively. By moving the original vertices towards the corresponding embedding vertices in the presence of a deformation regularization energy, it is possible to obtain a detail-preserving MDS pose as depicted in Figure 3. While the method is accurate and fast, it requires a tetrahedralization preprocessing step [16] and relies on geodesic distances that alter drastically under topological noise, e.g., hands connected by noisy edges. The execution of this algorithm on a 3.40GHz PC with 8GB RAM is about 22 seconds for a mesh with 45K vertices, preceded by 44 seconds of tetrahedralization. The usage of the problematic geodesic distances is avoided in a recent work [39], which in turn alleviates the sensitivity to topological noise.

Detail-Preserving Mesh Unfolding (Yusuf 's method)
The constrained MDS [37] obtains a detail-preserving canonical form by treating the vertices of the Landmark MDS embedding [10] as handles to deform the original shape. However, their work employs the problematic geodesic distances and a simpler yet faster deformation regularization energy. Instead, they proposed a finite elements based method [39] which does not use geodesics and achieves regularization with a more sophisticated scheme based on springs and the finite element method, which in turn achieves more accurate results in terms of element inversions and retrieval performance.
Unlike classical approaches, such as least-squares multidimensional scaling, the method preserves the geometric details of the input shape in the resulting shape. The optimization framework, fed with a triangular or a tetrahedral mesh in 3D, tries to move each vertex as far away from each other as possible subject to finite element regularization constraints. Intuitively this effort minimizes the bending over the shape while preserving the details. Avoiding geodesic distances in computation renders the method robust to topological noise.

Global Point Signatures
We have implemented the method to compute the Global Point Signatures (GPS) embedding of mesh proposed by Rustamov [36]. Firstly we calculate the discrete Laplace-Beltrami operator with cotangent weights [14] on the mesh. We then compute the four smallest eigenvalues (λ 0 ≤ λ 1 ≤ λ 2 ≤ λ 3 ) and their associated eigenvectors Given a point p on the mesh, the GPS embedding is calculated as a combination of the first three non-constant eigenvectors φ 1−3 : The eigenspaces of the Laplace-Beltrami operator are invariant to metric preserving deformations, and therefore the GPS embedding is a pose invariant representation of the mesh. This method is fast to compute, as it avoids the computation of geodesic distances.

Skeleton driven canonical forms
A variant on the canonical forms presented by Elad and Kimmel [11] is used [33]. A canonical form is produced by extracting a curve skeleton from a mesh, using the method by Au at al. [1] ( Figure 4(a)). The SMACOF Multidimensional Scaling method used by [11] is then applied to the skeleton, to put the skeleton into a canonical pose (Figure 4(b)). The skeleton driven shape deformation method by Yan et al. [44] is then used to deform the mesh to the new pose defined by the canonical skeleton (Figure 4(c)). This produces a similar canonical form to [11], but with the local features better preserved.

Euclidean Distance Based Canonical Forms
The Euclidean distance based method by Pickup et al. [32] computes a canonical form of a mesh by stretching out its limbs so that its extremities are distant from one another. This is achieved more efficiently than using multidimensional scaling (MDS) on the geodesic distances [11]. It instead maximises the Euclidean distances between feature points on the extremities of the mesh, whilst preserving the original edge lengths to ensure isometric deformation.
We first scale the mesh so that the maximum distance of any point on its surface to the centroid of all vertices is one. We then use the method of Ben-Chen and Gotsman [3] to calculate the conformal factor of the mesh. The conformal factor increases along the length of mesh protrusions, resulting in high values at mesh extremities. To obtain a set of feature points for a mesh with N vertices, we sample the √ N vertices which have the largest conformal factors and also satisfy the requirement that they are local maxima (their conformal factors are greater than that of all neighbours in a 2-ring neighbourhood). Having √ N feature points means we are able to compute the Euclidean distances between all pairs of feature points in O(N ) time.
We compute the canonical form of the mesh by adapting the least-squares MDS formulation used by [11] (Equation 7), so that the value of δ ij for all connected vertices i and j is set equal to the length of the edge connecting them. This aims to preserve the edge lengths of the mesh, to ensure isometric deformation. In order to maximise the distance between feature points, the value of δ ij for each pair of the √ N sampled vertices is set to 10, as suggested by Pickup et al. [32]. As long as this value is large enough and the parameter β, discussed below, is optimised accordingly, any value can be chosen. If two vertices i and j are neither a pair of feature points nor connected by an edge, we do not enforce a target distance between them, so w ij is set to zero. Not having to compute and optimise the distances between these points is crucial in keeping the linear time complexity of the distance calculations.
The weights w ij in Equation 7 for all i and j that are connected by an edge are set to β/δ 2 ij , where β is a user defined parameter for preserving edge lengths. We divide by the square of the edge length δ 2 ij so that the distance in Equation 7 becomes a relative, rather than absolute, difference, making the weighting independent of the length of the edge. The conformal factor is normalised to lie in the interval [0, 1], and the entries in the weighting matrix w ij for each pair of feature points are set to the mean of their conformal factors. This results in vertices which are nearer the ends of long 'limbs' of the object having a higher impact on the resulting canonical form, and avoids stretching out inappropriate parts of the mesh. The SMACOF algorithm can then be used to minimise Equation 7 as described in [11].
In many cases there are fewer local maxima of the conformal factor than √ N . We want the number of feature points to be exactly √ N so that the number of edges connecting pairs of feature points grows at the same rate as the number of mesh vertices. This in turn ensures that we can use the same value for the parameter β regardless of mesh resolution. Pickup et al. [32] offer two different solutions to handling this issue. The first is to increase the number of feature points to √ N by adding extra randomly selected vertices as feature points. We refer to this solution as "Euclidean Random" in Section 5.
The second is to separately normalise the weightings w ij used for pairs of feature points, and for adjacent vertices. We normalise the weights for adjacent vertices by dividing by the total number of edges, and we normalise the feature point pair weights by dividing by the sum of all feature point pair weights. We refer to this solution as "Euclidean Normalised" in Section 5.
This method eliminates the need for expensive geodesic distance computation. Feature point selection and Euclidean distance calculations have time complexity of O(N ), and the distance computations for each iteration of the MDS stress minimisation algorithm also take O(N ) time.

Feature-Preserving Canonical Form (Lian's method)
Lian et al. [24] proposed a MDS-based method that yet much better preserves the detail features compared to the original algorithm. Their method first constructs the standard MDS canonical forms for each model. Then the models are segmented into individual components using random walk [20]. After the segmentations, each component is moved according to the guidance of the MDS canonical forms and finally these components are connected together by an as-rigid-aspossible method. The method uses the standard MDS canonical forms as guidance while preventing the distortions by component-wise movements. The performance of the method, however, largely depends on the segmentation's accuracy. An overview of the method is illustrated in Figure 5.

Mesh Unfolding using Semidefinite Programming (Liu's method)
Liu et al. [26] proposed an automatic mesh unfolding method by optimizing an objective function using semidefinite programming. Motivated by the Maximum Variance Unfolding (MVU) [42] algorithm, the method formulates the total vertex variance to the trace of the Gram matrix. Furthermore, the method integrates a locally linear preserving term into the objective function, which leads to an enhancement of the detail-preserving ability.
The basic idea is to maximize the total variance of the vertex set for a given 3D mesh, while preserving the details by minimizing locally linear reconstruction errors. By optimizing a specificallydesigned objective function, the vertices tend to move against each other as far as possible, which leads to the unfolding operation. Compared to other Multi-Dimensional Scaling (MDS) based unfolding approaches, their method achieves a stronger degree of unfolding and does not require geodesic distance calculation.
The overall variance of the vertex set can be formulated using the trace of the Gram matrix as: where K is the Gram matrix and hence belongs to the semidefinite matrix set. The length of each edge on the mesh is preserved by adding the constraints K k,k + K j,j − 2K k,j = S kj where S k,j is the squared length of edge < v k , v j >. The term G is the locality linear preserving term and the parameter α balances the unfolding term and detail-preserving term.

Retrieval Frameworks
We use two local feature based frameworks from the existing literature to produce retrieval results, one based on extracting features from rendered depth maps of the shape (see Section 4.1), and the other based on extracting features from the 3D mesh itself (see Section 4.2).

Clock Matching and Bag-of-Features
The Clock Matching and Bag-of-Features (CM-BOF) framework was proposed by Lian et al. [23], and has been shown to perform extremely well at non-rigid 3D shape retrieval when combined with a canonical form method [21]. To compute a descriptor of a 3D shape we first centre the mesh, normalise its scale, and use a combination of principal component analysis (PCA) and rectilinearity [25] to normalise its orientation. We then render 66 depth images of the mesh from viewpoints at the vertices of a soccer-ball. SIFT features [27] are then extracted from the depth images. A 1000 length histogram descriptor of each image of the shape is then created using the bag-of-words method. The similarity of two shapes is calculated as the sum of the similarities of their matching views.

3D Spatial Pyramids
We also use the 3D Spatial Pyramids (3DSP) framework proposed by Redondo-Cabrera et al. [35]. To describe a shape, we first extract a set of 3D SURF features [19] from the 3D mesh itself. The volume containing the mesh is recursively broken down into sub-cubes, forming a spatial pyramid. A bag-of-words is then computed for each part of the pyramid and then the histograms are concatenated to form the shape's final descriptor. The similarity of two shapes is computed as the similarity of their descriptors. The reason why we adopt this feature is that it is a shape-based descriptor. Namely, it directly extracts descriptors on 3D mesh while CM-BOF deals with the rendered views of 3D mesh and therefore a view-based feature. We use both types of feature to test the adaptability of different canonical forms.

Experiments
Here we detail the results of our experiments. We show a visual evaluation of the canonical forms in Section 5.1, the retrieval task used to evaluate the canonical forms is described in Section 5.2, the retrieval performance is presented in Section 5.3, and timings for each method are given in Section 5.4.

Visual Evaluation
Two example canonical forms produced by each method for each dataset are shown in Figures 6, 7 and 8. All canonical forms share the property that the limbs of the objects are straightened out. The different methods achieve this with varying degrees of success; there appears to be a trade-off between a fully straightened out pose, and preserving detailed shape features. Most MDS-based methods, and the GPS method, successfully straighten out the limbs of the models, but cause a large amount of distortion to the shape. The Constrained MDS and Skeletons methods are the two exceptions to this: while they both rely on MDS, they manage to preserve much of the shape detail. The GPS method causes so much distortion, it is extremely difficult, if not impossible, to visually recognize the shape of the object from its canonical form. Yusuf's method constructs canonical forms by solving a finite elements problem which requires the tetrahedralization of mesh. Their method significantly preserves the details. However, when handling non-watertight mesh, preprocessing is needed. Liu's method yields canonical forms by maximizing the vertex variance while preserving the edge length. Their method exhibits distortions at limb ends.
Some methods cause various other artifacts. The Constrained MDS method fails to completely transform some of the models, such as the man lying on his side. This method also exhibits curves and bends in the limbs of the objects. The Euclidean Random and Normalised methods do not produce as consistent a pose across different meshes, and also cause some "spiky" parts in the objects, where feature points do not lie at the mesh extremities. The Skeletons method can result in bulges at joints which were highly bent in the original mesh, and the faces of the Synthetic humans are highly distorted due to the complex skeleton formed in these areas because of the detailed geometry of the eyes, mouth and tongue.
Some of the meshes in the Non-Rigid dataset contain topological errors. Figures 9 and 10 show three examples of these, where parts of the meshes have been incorrectly fused together, and the resulting canonical forms. All the canonical forms fail to cope with such topological errors. The  method by Boscaini et al. [6] show they can handle some small topological errors, but we were unable to test their method as there is no publicly available implementation.

Retrieval Task and Evaluation Method
The canonical forms for each method were used in turn for shape retrieval. The retrieval task is defined as: Given a query model, return a list of all models, ordered by decreasing shape similarity to the query.   Every model in the database was used in turn as a separate query model. The evaluation procedure used to assess the results (see Section 5.3) is similar to that used by previous comparative studies [21,22]. We evaluate the results using various statistical measures: nearest neighbour (NN), first tier (FT), second tier (ST), discounted cumulative gain (DCG), and precision and recall curves. Definitions of these measures are given in [2] and [41].

Retrieval Results
Tables 1-5 show the retrieval performance for both retrieval frameworks applied to each set of canonical forms. In general the retrieval results were higher when using the view-based CM-BOF  retrieval framework. This shows that view-based frameworks may be more suited for use with canonical forms than frameworks based on local 3D features. All the canonical form methods achieved a significantly higher retrieval performance for the Non-Rigid datasets, performing poorly on both the Real and Synthetic human datasets. The human datasets were designed to be more challenging for retrieval than the more general Non-Rigid dataset. In general other methods not based on canonical forms achieve a much better result than ones based on canonical forms for such data [31]. As might be expected, for all datasets, using    canonical forms achieved a better result than simply running rigid retrieval on the original meshes, which supports the claim that canonical forms improve results by adding some pose-invariance.
The results on the Non-Rigid dataset show very high retrieval performance, with the best methods achieving close to perfection. There is a general issue with the low difficulty of this dataset, which is highlighted by the good results also achieved by performing retrieval directly on the original meshes, with no attempt at pose-invariance. This shows that the difference in shape between classes is likely larger than the difference due to within class pose variation, and indicates  the rather unsuitable nature of this benchmark. The GPS method achieved the best retrieval performance across all measures for the Real dataset, and the highest NN and DCG scores for the Synthetic dataset. The difference in NN performance for this method over the others on the Synthetic dataset is highly significant, with GPS achieving 0.4 for CM-BOF retrieval, whilst the next best score being only 0.14. This is surprising, as the GPS method appears to produce the largest distortions in the shapes, and achieves a significantly lower performance than all other methods on the Non-Rigid dataset. In fact, the GPS method is the only method to perform worse than performing retrieval directly with the original meshes on the Non-Rigid dataset.
The Least Squares MDS method achieved the best performance across all measures for the Non-Rigid dataset. The Non-metric MDS, Constrained MDS, and Skeletons methods all achieve the same NN performance, and only a slightly lower performance across the other measures.
We also evaluate the methods by using the TOSCA dataset and its noisy version. The noisy dataset is obtained by adding vertex-wise uniformly distributed random turbulence in an amplitude much smaller than the shape diameters (1% of the shape diameter). We conduct this experiment to test the robustness of each methods. The performance of TOSCA and TOSCA with noise are collected in Table 4 and 5 respectively. As expected the best performance in every measure suffers from an insignificant drop due to the added noise. And the methods appear to have non-uniform reactions to noise. On TOSCA, the Constrained MDS achieves the best results while on the noisy TOSCA it drops down. Methods such as Lian's feature-preserving canonical forms, Yusuf's detailpreserving mesh unfolding and CMDS are less sensitive to noise. That Yusuf's method survives on topological noise is due to the elimination of geodesics.
In general all methods achieve a very similar result to one-another on all the five datasets, with the exception of the GPS method, which performs significantly better on both human datasets and significantly worse on the Non-Rigid dataset.
To explore in detail why the methods perform poorly on the Real and Synthetic datasets, Table 6 shows the fraction of nearest-neighbour errors which exhibited the same pose as the query  Table 6: Fraction of incorrect nearest-neighbour retrieval results which are models with the same pose as the query.
model. This table helps to indicate how much the low retrieval performance was due to a lack of pose-invariance. We cannot produce the same table for the Non-Rigid dataset, as the same poses are not used across the different shape categories. These results show that for all methods, apart from GPS, at least 97% of the errors on the Real dataset, and 92% of the errors on the Synthetic dataset were caused by retrieving a model of the same pose, but incorrect shape, for the CM-BOF retrieval framework. Slightly lower errors are caused by pose for the 3DSP retrieval framework. This is evidence that the canonical forms are not completely pose invariant. The low retrieval results on both human datasets are therefore likely due to the variation in pose within each class, even after computing canonical forms, being greater than the variation in shape between classes.
The GPS method has a significantly lower error caused by lack of pose-invariance, especially for the Synthetic dataset. This indicates that the GPS method may be significantly more pose invariant, which probably accounts for the higher retrieval score on these two datasets. On the Non-Rigid dataset, where pose-invariance appears less important, it may be the high level of distortion which causes the lower retrieval performance.

Timings
The run-time of each canonical form method is shown in Table 7. The MDS methods which include the computation of a full geodesic distance matrix were run on meshes simplified to approximately 2,000 vertices using MeshLab [28]. This is because the run-time of these methods on the fullresolution meshes is impractical (up to several hours per mesh). Caution is therefore needed when comparing the timings between these methods and the others. The GPS method achieved the fastest run-time, but achieved mixed retrieval results. The next fastest method for the full-resolution meshes is Accelerated MDS. Both values for the number of distance matrix columns used produce faster timings than all the other methods, except GPS. When compared to the standard Classic MDS method, Accelerated MDS exhibits a small drop in performance, but still achieves decent results.
The next best timings are achieved by the Constrained MDS and Skeletons methods. The Skeletons method achieved better retrieval results for the Non-Rigid dataset, when using the better