A novel approach for partial shape matching and similarity based on data envelopment analysis

Due to the growing number of 3D objects in digital libraries, the task of searching and browsing models in an extensive 3D database has been the focus of considerable research in the area. In the last decade, several approaches to retrieve 3D models based on shape similarity have been proposed. The majority of the existing methods addresses the problem of similarity between objects as a global matching problem. Consequently, most of these techniques do not support a part of the object as a query, in addition to their poor performance for classes with globally non-similar shape models and also for articulated objects. The partial matching technique seems to be a suitable solu-tion to these problems. In this paper, we address the problem of shape matching and retrieval. We propose a new approach based on partial matching in which each 3D object is segmented into its constituent parts, and shape descriptors are computed from these elements to compare similarities. Several experiments investigated that our technique enables fast computing for content-based 3D shape retrieval and significantly improves the results of our method based on Data Envelopment Analysis descriptor for global matching.


Introduction
In last years, a considerable number of 3D models is growing in digital form on the World Wide Web, and this amount intends to increase in the future.As databases are in continuous expansion, indexing 3D models for comparing can be a complicated task, requiring considerable amounts of algorithms and tools to retrieve information.Thus, several techniques of 3D shape analysis and matching have been developed; among them shape similarity, indexation, shape retrieval, mesh segmentation, and many others.Computing the similarity between 3D shapes is a critical problem and a challenging task in computer vision, computer graphics, and a variety of other fields.The content-based approach has already attracted significant attention as a new accurate technique in various research areas.
The main idea of similarity search techniques for 3Dmodels is to find a suitable descriptor that can be extracted and compared quickly, and having a high discriminative power in distinguishing between similar and dissimilar 3D-models.The content-based retrieval process is done with offline and online steps, which are indexing, querying, matching and visualizing.Except for the first step that is processed offline, the rest three stages are performed online.We can distinguish two types of 3D indexation methods; the global ones that aim to compute a signature for the entire object and the partial methods which consist on matching subparts or regions by calculating the signature of each part of the object and require a segmentation preprocessing step.The literature provides a large number of signatures based on geometric and semantic criteria of the 3D-model, these works [2,3] present extensive surveys of the existing methods in the literature.Despite the variety of research toward indexation, most existing methods for shape analysis are based on global shape similarity functions.However, only a very few methods can handle efficiently 3D partial shape retrieval because it is much more difficult than the global similarity search problem, since it has to search and define the subparts before measuring similarities.
In this article, we propose to use a similarity search technique that compares the similarity between portions of 3D models rather than a global comparison.Using our approach, each 3D model is divided into its meaningful parts using significant segmentation methods presented recently in the literature, after segmenting the 3D objects we calculate for each part its descriptor by the method based on multi-criteria using DEA [4].Two models are considered as similar if their segments are similar.Using this partial similarity, we can perform a similarity search not only with the 3D models in their entirety but also with their different portions which are useful to find a model having parts that are close to parts of another model.
The present paper is organized as follows: In section 1, we review a brief state-of-the-art of the existing descriptors in the literature.In section 2, we describe our proposed approach.The experimental results are dis-cussed in section 3. Finally, in the last section, we present a conclusion and some perspectives.

Related works
During the last decade, several approaches to retrieve 3D models based on shape similarity have been proposed [2,3].Some of these content-based retrieval systems are based on 2D views for indexing 3D objects, the main idea of these techniques is to capture for each 3D object a set of 2D views, then, and a 3D model is represented by 2D shape descriptors associated to these pictures.Other approaches aim to calculate for each 3D model a global shape descriptor by capturing intrinsic characteristics of the object in its entire form.On the contrary of the partial 3D / 3D indexing approaches that consist of decomposing the 3D object into different parts and then the 3D model is characterized by the set of signatures computed for their constituent segments.For more details, readers are invited to refer to [3,5,8] which offers an extensive study and an experimental comparison of many of the existing methods in the literature.In this section, we present briefly an overview of the indexing methods proposed in the literature for each category.

View-based approaches
The principal idea of these methods is to capture for each 3D model a set of images taken by cameras distributed around the model from different angles.Then, a 3D model is described by 2D shape descriptors associated to these views, and the similarity is measured using 2D shape matching techniques.These approaches require a normalization step to overcome the problem of invariance at translation, rotation and scaling.The primary challenge of these methods is the selection of the characteristic pictures of the 3D object.The results of these types of techniques depend mainly on the choice of the characteristic views, a large number correctly represent the 3D model but it will generate a set of large descriptors, and therefore the retrieval process requires more running time, and vice versa.Several methods of indexing 3D objects by 2D views have been proposed in the literature.These approaches differentiate according to the number and position of views, the 2D shape descriptor used, and the way in which similarity is calculated between views.We can mention the work of [9] in which the authors propose a descriptor based on 2D views called Light Field Descriptor.First, a normalization step under translation and scaling is applied.Then 10 different silhouettes are extracted for each 3D model and both 35 coefficients for the Zernike moments descriptor, and 10 coefficients for the Fourier descriptor are used to characterize each silhouette.The authors in [10] included an approach for 3D mesh retrieval in which a set of 42 depth images is captured from different viewpoints, and then each capture is characterized using the 2D generic Fourier-descriptor.Finally, the distance between two models is deduced by computed all possible combinations of the two sets of descriptors.Papadakis et al. [11] proposed a descriptor based on a set of panoramic views of a 3D object, which describe the position, and orientation of the model's sur-face in 3D space.For each projection, the authors compute its corresponding 2D Discrete Fourier Transform and employ local relevance feedback (LRF) as a matching technique.
Global shape-based approaches Global shape-based approaches use global geometric or topological characteristics of the object in its entire form to process 3D models matching.Osada et al. [12] included five Shape distributions based on global geometric characteristics of the object such as distance, angle, area and volume.D2 and A3 are the most used which represent respectively the distribution computed as the distance between two random points on the surface and the distribution function that calculates the angle formed by three random points on the surface of the object.Zaharia et al. [13] introduced a descriptor called Shape Spectrum Descriptor (SSD).The proposed descriptor defined as the distribution of the shape index [14] over the entire mesh.The authors in [15] proposed a Density-based 3D shape descriptor defined as a sampled probability density function.This function is based on three types of local geometric multidimensional features of a point on the surface of the 3D object.The 3D Harmonics descriptor proposed by Funkhouser et al. [16] based on spherical harmonics to compute the similarity between objects.The authors firstly decompose the 3D model into a collection of functions defined on concentric spheres.Then, they use the spherical harmonics to discard orientation information.Our recent work has presented a new descriptor for 3D content-based shape retrieval [4].The proposed approach was based on Data Envelopment Analysis (DEA) [17] to combine multiple geometric and topologic criteria of the 3D object for a global matching.The chosen measures are the dihedral angle, the shape index, and the Shape Diameter function.In the same context, recently, our research team [18] introduced a new technique for retrieving and classifying 3D objects.This method is based on artificial neural network, which learns from several criteria that present information representative of the 3D mesh to extract a final descriptor.
Partial shape-based approaches The partial indexing approaches consist of decomposing the 3D object into its constituent parts and then the 3D model is indexed by the set of signatures computed for their segments.Despite the advantages that partial descriptors can provide, few content-based approaches support partial matching.Suzuki et al. [19] proposed an approach for partial matching using the information of the entire object.The authors firstly apply a simple decomposition technique by comparing angles created by normal vectors of each polygonal face to find sharp angles and cuts polygonal faces into parts based on a typical clustering approach.Then each part is initially normalized for scale and orientation by using principal component analysis pose normalization.Finally, the authors compute the descriptors for these segments and use a matching technique identical to the techniques used in typical 3D shape retrieval.Biasotti et al. [20] introduced an approach for partial shape matching based on the theory of Reeb graphs.They propose to encode the shape and all its constituent parts in a graph, which represents the structure of the object and its geometry.The graph-matching technique is used for sub-part correspondence.Chahhou et al. [21] proposed a hybrid approach for 3D object retrieval using a global-partial technique by combining the partial signatures of the segments of a 3D object and its global descriptor.Firstly, the authors decompose the 3D object into its constituent regions, and then they compute the D2 descriptor [12] for each part and use the shape spectrum descriptor [13] to provide global description of the shape of the objects.In this paper, we propose a new approach based on partial matching technique in which each 3D object is segmented into its different segments, and shape descriptors are computed from these elements to compare similarities.We also propose a new technique for calculating the similarity between the parts' descriptors.

Our proposed approach
In the last years, as a result of the extensive availability of 3D models, Content-based retrieval of 3D models has been considered as one of the major challenge in the research area.Until now, almost all the proposed approaches in the literature have been focused on global matching, where similarity is computed between entire models.In general, global 3D descriptors do not support a part of the object as a query, and they are poorly efficient for classes with globally non-similar shape models and also for articulated objects.The partial matching technique seems to be a suitable solution to these problems.The aim of this work is to introduce a new partial technique to resolve the problems encountered cited before using the global descriptor.
The global descriptor for 3D mesh retrieval using data envelopment analysis Our recent work has presented a new descriptor for 3D content-based shape retrieval [4].The proposed approach was based on Data Envelopment Analysis (DEA) [17] to combine multiple geometric and topologic criteria of the 3D object for a global matching.The chosen measures are the dihedral angle, the shape index and the Shape Diameter function.The dihedral angle represents a differential geometric property widely used in the 3D fields.While the shape index [14] is a measure describing the topology of the local surface using the principal curvatures.Finally, the Shape Diameter function is a scalar function representing the volume or thickness of a 3D mesh [22].These three characteristics are invariant under scaling, translation, and rotation.The main idea of this work is to merge these metrics using an optimization function to extract a final descriptor combining the advantages of each combined measures.To this end, we choose to use Data Envelopment Analysis (DEA) [17] that represent a linear programming based technique for measuring the relative performance multiple inputs and/or outputs.This method gives a significant performance in matching similar objects, and the resulted descriptor is invariant under scaling, translation, and rotation and does not need any pretreatment of normalization.For this reason, we opt to use this descriptor in our partial matching technique.
New partial technique for models matching Partial matching is the task of matching subparts or regions.In this paper, we describe a new technique based on partial matching in which each 3D object is segmented into its constituent parts.Then, shape descriptors are computed from these elements to compare similarities.It is clear that the most critical challenge in this type of approach is how to segment a 3D object into meaningful parts.To determine which segmentation method we will use, we have evaluated different segmentation algorithms by using some of the recent evaluation methods proposed in the literature, which give an efficient quantitative evaluation by comparing the automatic segmentation with a set of ground truth segmentations instead of one to one comparison, these evaluation methods are: NWLD [23], WKD [24], WDC [25], WOI [26], WSSD [27], Dj3D [28] and the AEI [29].Table 1 present for each segmentation method its average of the dissimilarity scores obtained for the entire database by using the cited evaluation methods.As can be deduced from this quantitative comparison, the dissimilarity scores of the learning-based algorithm (LB) [30] obtained by almost all the used evaluation methods are better than all the other methods, which highlight its performance.

Metrics /
Algorithms WDC WKD NWLD WOI WSSD Dj3D AEI LB [30] 0.16 0.11 0.18 0.18 0.28 0.13 0.22 RC [31] 0.21 0.17 0.28 0.23 0.38 0.21 0.29 NC [31] 0.27 0.18 0.31 0.25 0.40 0.26 0.43 FP [32] 0.27 0.23 0.34 0.25 0.54 0.28 0.56 KM [33] 0.30 0.25 0.39 0.28 0.41 0.31 0.64 SC [34] 0.22 0.19 0.24 0.22 0.39 0.22 0.30 After evaluating a set of segmentation methods by applying some of the most recent and efficient evaluation methods, we will continue by studying the behavior and the impact of the segmentation algorithms in the context of content-based research.To this end, we have applied our partial matching method using different segmentation algorithms, we calculate the recall-precision graph of each object in the database considered as a query, and then the recall-precision graph for the entire database is derived from the average of the curves obtained for each model.This process is repeated for each segmentation algorithm.The Fig. 1 shows the obtained results.As can be seen from the obtained results, the curve obtained using learning-based approach (LB) [30] gives once again a very good classification performance by being classified at the top of the automatic segmentation algorithms, followed by the Randomized Cut algorithm (RC) [31] which slightly superforms the approach based on spectral clustering (SC) [34].In addition, the learning-based ap-proach(LB) [30] offers a gain in operating time compared to the Randomized Cuts algorithm (RC) [31], which is costly in terms of response time of the different random segmentations it based.

Fig. 1. Precision-recall graph using our proposed approach with different segmentation algorithms
From all these results, we can easily deduce that the learning based method (LB) [30] is the better choice to be used in the partial indexation partial from all the tested segmentation methods.The Fig. 2 illustrates an example of segmentation obtained using this approach.Fig. 2.An example of segmentation obtained using learning based method (LB) [30] The Fig. 3 represents the global process of the proposed approach.
Firstly, in the offline step, each model is composed into several parts.This decomposition will reduce the complexity and highlight the topology of the 3D model.Then we calculate for each object, the different descriptors associated with each segment using our descriptor cited before [4].In the online step, we propose a new strategy for computing the similarity between the segments of the request object and objects in the database.When a query is presented, we firstly extract its constituent parts and for each segment, we calculate its descriptor.Then, to compute how similar two objects are, a mapping stage is done by searching the best corresponding descriptor of each segment in the query and an object form the database.Taking Q as a descriptor of a segment from the query object and {V 1 , V 2 , ..., V n } represent a set of descriptors associated to an object from the database; the best correspondence of Q is deduced by minimizing the following function: ( , ) min{ ( , ), } where t represent the index of the best corresponding descriptor of Q and D is the city block distance.

Fig. 3. The global process of the proposed approach
The Fig. 4

describes an example of correspondence between segments
When the number of segments in the query and an object from the database is different, calculating only the similarity between objects does not reflect the real correspondence between these models.For this reason, we propose to take into account the dissimilarity between objects in the global distance by adding the maximum deviation of unmapped segments computed as: ( , ) max{ ( , ), } where U is a set of unmapped segments.

Fig. 4. An example of correspondence between segments
The measure of similarity between objects is based on two criteria: -The distance between the best corresponding segments; -The maximum deviation of unmapped segments.These two metrics are combined to obtain the final matching result between two objects O 1 and O 2 using the following formula: Using this partial similarity approach, two models are considered similar if their segments are similar.Consequently, we can perform a similarity search not only with the 3D models in their integrality but also with their different parts, which improves the efficiency of the content-based retrieval systems.

Experimental results
The last section of the present paper is dedicated to evaluate and validate the proposed method.Through four sets of tests, we will try to show the effectiveness and the discriminative power of the proposed method, compared with other well-known which are PANORAMA [11], LightField [9], Harmonics [16], and our previously proposed method multi-criteria with DEA [4].
First, and before starting the experimental results, we should choose which database to use in our experimental tests.Many databases can be founded freely over the internet we can mention the Princeton shape benchmark (PSB) [35], the Shape Recognition Contest (SHREC), the National Taiwan University database, the Konstanze 3D Model Benchmark (CCCC) [36], or The NIST Generic Shape Benchmark (NSB) [37].Our choice went to use the Princeton's segmentation benchmark database [38], which is a modified version of the Watertight Track of the 2007 SHREC Shape-based Retrieval Contest [39].This choice has been made because of the number and the diversity of this database, it contains 380 objects divided into 19 categories (Human, Cup, Glasses, Airplane, Ant, Chair, Octopus, Table, Teddy, Hand, Plier, Fish, Bird, Armadillo, Bust, Mech, Bearing, Vase, and Fourleg).In addition, this database contains some noisy 3D models, and it will be a challenging task to retrieve them using our proposed method.
The first test is done to compare our previously published works for combining many features using DEA to generate a global descriptor for the 3D object [4] with the proposed one.In this test, we will compare the two methods on different classes; we choose (Human, Cup, Bust, and Fourleg).Fig. 5 shows the obtained results.From these results, we can easily notice that the proposed method outperform the global version of the DEA [4] in all the chosen classes, even if both methods are based on the same Multicriteria method, but the proposed one with the partial comparison add-on succeeded to ameliorate the performance of the proposed method.
The second test is a classic one and used to evaluate almost every information retrieval method; it is the precision-recall curves.The recall is the percentage of the relevant retrieved item to the total number of pertinent item in the database, and the precision is the percentage of relevant retrieved 3D objects over the whole number of the retrieved object.Mathematically they are represented as follows: relevant correctly retrieved Precision all retrieved = , (4) relevant correctly retrieved Recall all relevant = (5) This test was achieved by taking each 3D model as a query on the rest of the database, then compute the average precision-recall performance overall models.Fig. 6 represent the obtained result along with those obtained for the methods: PANORAMA [11], LightField [9], Harmonic [16], and our previously proposed method multicriteria with DEA [4].As we can see from the results, our method came in the first place followed closely by PANORAMA [11], while harmonics [16], LightField [9] and DEA perform almost the same and have been outperformed by the proposed method.We have achieved excellent performance especially for the categories including articulated objects compared to global approaches and view based approaches (Fig. 7).This superiority of the performance of the partial approach is due to the simplification of the shape into several simple shapes in order to highlight similar parts between objects of different poses.
In the third test, we will validate the performance of our proposed partial retrieval approach by measuring some evaluation metrics: -The nearest neighbor: this metric consists on calculating the percentage of the closest retrieved elements belonging to the same class of the query.-Discounted Cumulative Gain (DCG): a score that calculates the right results ahead of the list rather than the correct results at the end of the ranking list, knowing that a user is less likely to consider the items at the end of the list.-F-Measure: The F-Measure based on combining the recall and precision values to measure the overall performance of the retrieval system.We get the F-measure by the following formulas: Table 2 present the obtained results using our approach and the same methods used in the previous test.[11] 0.92 0.71 0.41 0.83 0.32 Fig. 5.A comparison between the obtained results using our approach and DEA Descriptor for some classes Fig. 6.Precision-Recall graph using our proposed approach and four different descriptors Fig. 7. Precision-Recall graph using our proposed for articulated objects The obtained results in this test are in concordance with those obtained in the precision-recall curves.Our proposed approach provides the best results in all the computed metrics, followed by PANORAMA [11], with a small difference.LightField [9] and DEA Descriptor [4] came next with average result, and Harmonics [16] in the last place.
In addition to the relevance and the effectiveness of a retrieval method, another important point needs to be verified is the response time.Indeed once the user submitted the query, the retrieval system should be very responsive and provide the user with the best results instantly.The last teste in our paper will compare the response time of our method compared to PANORAMA [11] method since they both performed very well and provided very good results.To do so, we use every model in the database as a query and compute the time needed for both methods to return the results, and then we compute this time per class, Fig. 8 shows the obtained results.Our method succeeded to provide results in a satisfactory amount of time outperforming the PANORAMA method [11] in almost all classes (like Bust, Plier, cup…), except for octopus, where PANORAMA [11] slightly outperform our method.From all the previous experiments, we can demonstrate that the proposed method is very responsive by providing an excellent result, in a reasonable time.

Conclusion
In this paper, we have proposed a novel approach for 3D mesh retrieval based on partial matching.Firstly, each 3D model is segmented into its constituent parts, and then shape descriptors are computed from these segments based on data envelopment analysis.Finally, a new technique has been applied to compare similarities between descriptors of segments.The experimental results illustrated the ability of the proposed approach in 3D shape retrieval system and its discriminative power in distinguishing between similar and dissimilar models in a reasonable time.

-
First Tier and Second Tier: these two measures calculate the percentage of objects belonging to the query class that appear in the top K matches, where K depends on the query class size.Let consider | C | as the size of the class, K = | C | -1 for the first tier, and K = 2*(| C | -1) for the second tier.

Fig. 8 .
Fig. 8.The response time of our method comparedto PANORAMA method

Table 2 .
The obtained result using our approach and the same methods