Multi-view registration of unordered range scans by fast correspondence propagation of multi-scale descriptors

Siyu Xu; Jihua Zhu; Zutao Jiang; Zhiyang Lin; Jian Lu; Zhongyu Li

doi:10.1371/journal.pone.0203139

Abstract

This paper proposes a global approach for the multi-view registration of unordered range scans. Our method starts with the pair-wise registration, where multi-scale descriptor is selected for feature point and the propagation of feature correspondence is accordingly accelerated. Subsequently, we design an effective rule to judge the reliability of these pair-wise registration results. According to the judgment of reliability, we propose a model fusion method, which can utilize reliable results of pair-wise registration to augment the model shape. Finally, multi-view registration can be achieved by operating the pair-wise registration, reliability judgment, and model fusion alternately. The proposed approach can be applied to scene reconstruction and robot mapping. Experimental results conducted on public datasets show that the proposed approach can automatically achieve multi-view registration of unordered range scans. Compared with other related approaches, the proposed approach has superior performances in accuracy and effectiveness.

Citation: Xu S, Zhu J, Jiang Z, Lin Z, Lu J, Li Z (2018) Multi-view registration of unordered range scans by fast correspondence propagation of multi-scale descriptors. PLoS ONE 13(9): e0203139. https://doi.org/10.1371/journal.pone.0203139

Editor: Eva O. L. Lantsoght, Technische Universiteit Delft, NETHERLANDS

Received: April 24, 2018; Accepted: August 15, 2018; Published: September 10, 2018

This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.

Data Availability: The stanford 3d scanning repository: http://graphics.stanford.edu/data/3Dscanrep/ Gazebo data set: https://vision.in.tum.de/data/datasets/rgbd-dataset/download EXBI data set: https://team.inria.fr/perception/research/jrmpc/.

Funding: This work was supported by the National Natural Science Foundation of China under Grant nos. 61573273 and 91648121. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Registration of range scans is a fundamental task, which has attracted great attention in many research fields, such as robot mapping [1–3], computer vision [4] and computer graphics [5,6]. Given a global reference frame, the goal of registration is to find optimal transformation(s) for one or more scans so as to transfer all scans into one unified frame. According to the number of scans to be registered, this problem can be divided into pair-wise registration and multi-view registration.

For pair-wise registration, Besl et al. [7] proposed the iterative closest point (ICP) algorithm, which is one of the most popular registration approaches. Although this approach can achieve the pair-wise registration with good efficiency, it is unable to get desired results for the registration of partially overlapping range scans. Besides, it is a local convergent approach. To obtain the desired results, good initial parameters should be provided to the ICP algorithm. For registration of partially overlapping scans, Chetverikov et al. [8] proposed the trimmed ICP (TrICP) algorithm, which introduces the overlap parameter to determine overlapping parts so as to estimate accurate rigid transformation. Further, Phillips et al. [9] improved the performance of original TrICP algorithm with high efficiency. To address the local convergence, Genetic algorithm (GA) [10,11] and particle filtering [12] were applied to global registration of scan pairs. To guarantee global optimality, reasonable initializations are still required; otherwise spaces of parameters are too large for heuristic search. What’s more, these global optimization approaches require high computation complexity. Subsequently, Yang et al. [13] proposed the first globally optimal algorithm, named Go-ICP. This approach is based on a branch-and-bound (BnB) scheme that searches the entire 3D motion space and able to produce reliable registration results regardless of the initialization. Besides, many feature matching based approaches have been proposed for the pair-wise registration [14–16]. Among these approaches, one of the most efficient and robust approaches is multi-scale descriptors with correspondence propagation [16], which can be further improved in efficiency.

Although these above-mentioned approaches may obtain good results for pair-wise registration, most of them cannot deal with multi-view registration. Given a set of range scans, the task of multi-view registration is to find optimal transformations of each scan to the reference scan. Compared with pair-wise registration, multi-view registration is somewhat more difficult due to many transformations required to be estimated. In recent years, many solutions have been proposed to solve this problem. Among these solutions, the sequential registration approach is the simplest one [17]. For multi-view registration, it alternately aligns and integrates two scans until all scans are integrated into one model. Although this approach is simple, it suffers from the error accumulation problem. To address this issue, Zhu et al. [18] proposed the coarse-to-fine registration approach, which sequentially aligns each scan to the model integrated by all other aligned scans. By traversing all scans, all transformations are sequentially refined by the pair-wise registration. Meanwhile, Govindu et al. [19] proposed a multi-view registration approach based on motion averaging algorithm, which takes a set of available relative motions (pair-wise registration results) as input to simultaneously estimate all transformations for multi-view registration. Based on this work, Guo et al. [20] proposed the weighted motion averaging algorithm for multi-view registration. This approach can improve the robustness and accuracy of multi-view registration by paying more attention to reliable pair-wise registration results. Recently, Arrigoni et al. [21] proposed a multi-view registration approach based on the low-rank and sparse (LRS) matrix decomposition. It stacks all available relative motions into a large matrix and replaces the unavailable ones with zeros, then utilizes the LRS decomposition to recover global motions for multi-view registration. Moreover, Georgios et al. [22] cast the multi-view registration problem into the framework of clustering. For clustering, it utilizes the Expectation-Maximization (EM) algorithm to simultaneously estimate Gaussian Mixture Model (GMM) and all transformations for multi-view registration.

Although these above discussed approaches may obtain desired results for multi-view registration, they should be provided with fine initial registration parameters. Otherwise, it is difficult to obtain desired registration results. For the global multi-view registration, Daniel et al. [23] proposed the approach for fully automatic registration of multiple range scans. It applied the pair-wise registration on all scan pairs involved in multi-view registration and then tested these results for surface consistency. Accordingly, reliable results of pair-wise registration can be checked and selected for the estimation of multi-view registration. Since there are many scan pairs involved in multi-view registration, the computation complexity of this approach is very high. Subsequently, Guo et al. [24] proposed a feature match based approach for global multi-view registration. This approach is also based on the pair-wise registration, which is more efficient than previous approaches. However, it sometimes cannot find the minimum number of good feature matches for pair-wise registration, which leads to the failure of multi-view registration. Recently, Zhu et al. [25] proposed the global multi-view registration approach based on the construction of spanning tree. It designs the dual criterion to judge the reliability of pair-wise registration results. By viewing the first scan as the root node, all other scans can be added by the breadth-first search with the reliable results of the pair-wise registration. As most scan pairs contain low overlap percentages, it is also required to align and check many scan pairs to construct a full spanning tree. Therefore, its computation complexity is also high.

Taking these above problems into account, this paper proposes a global approach for the multi-view registration of unordered range scans. The contribution of this paper can be delivered as follows: 1) It accelerates correspondence propagation of multi-scale descriptor for pair-wise registration; 2) It designs an effective rule to judge the reliability of pair-wise registration; 3) It proposes the model fusion to merge scan pairs, which can be successfully aligned. To demonstrate its effectiveness, the proposed approach is tested on public available data sets and compared with two other related registration approaches.

The reminder of this paper is organized as follows. We briefly review related work of pair-wise registration approach in Section 2. In Section 3, we present details of the proposed multi-view registration approach. All experimental results are displayed in Section 4. Section 5 demonstrates applications of the proposed approach in scene reconstruction and robot mapping. Finally, conclusion and future works are discussed in Section 6.

Pair-wise registration

This section briefly reviews TrICP algorithm and the correspondence propagation of multi-scale descriptors for pair-wise registration.

TrICP algorithm

Suppose there are two partially overlapping scans in , the data shape and the model shape , where ξ denotes the overlap percentage and P_ξ indicates the overlapping part of P to Q. Accordingly, the overlap percentage can be calculated as , where denotes the number of point included in the subset P_ξ. Subsequently, it is convenient to define trimmed mean square error (TMSE) as: (1) where R is the 3×3 rotation matrix, t is the 3×1 translation vector and (p_i,q_c(i)) denotes a pair of point correspondence. According to [8], the task of pair-wise registration is to find an optimal rigid transformation (R,t), which can be obtained by minimizing the following objective function: (2) where λ is a preset parameter (λ = 2 in this paper).

Currently, Eq (2) has been solved by the TrICP algorithm, which achieves pair-wise registration by iterations. Given initial parameters , three steps are included in each iteration:

Establish point correspondences between two range scans: (3)
Update the overlapping percentage and its corresponding subset: (4)
Calculate the new transformation: (5)

A rigid transformation will be obtained by repeating steps (1)-(3) until the iteration k reaches the maximum number K or |ψ_k-ψ_k-1|<ε, where ε denotes a given small positive number. Similar to the original ICP algorithm, the TrICP algorithm is also local convergent. To obtain desired results, fine initial parameters should be provided.

Correspondence propagation of multiscale descriptors

For the pair-wise registration, multi-scale descriptors have been proposed in [16]. Given a point, L different support radii are considered. Within each support radius, all included range points are used to compute the variance matrix C_l. Based on these L circles and variance matrix, it is convenient to compute multi-scale descriptors (N,D), where N denotes the normal vector and D indicates the eigenvalue based vector. Accordingly, the pair-wise registration can be solved by the effective approach displayed in Fig 1.

Download:

Fig 1. Flow chart of the pair-wise registration approach based on correspondence propagation of multi-scale descriptors.

https://doi.org/10.1371/journal.pone.0203139.g001

As shown in Fig 1, after the calculation of multi-scale descriptors, the correspondence of each point from y₁ to y_n is established by performing the nearest neighbor (NN) search on the eigenvalue based vector D. For correspondence propagation, each correspondence of these points can be viewed as a seed match. Then, the propagation session exploits both D and N to propagate each seed match (e.g., (y₁,x_s)) into a set of matches (e.g., M₁ = {(y₁,x_s),…}). For each match set, the RANSAC algorithm should be used to find the maximum consensus set so as to calculate one optimal rigid transformation. Based on down sampled scans Q_d and P_d, a quality function formulated from distances error is defined to identify the best one from all obtained rigid transformations. Finally, the TrICP algorithm refines the results of pair-wise registration by taking the best transformation (R,t)_best as its input.

According to [16], the time complexity of each propagation operation is about O((m+n)logn), where n and m denotes the number of points included in the down sampled scans Q_d and P_d, respectively. As the required number of propagation is n, the total complexity of correspondence propagation is O(n(m+n)logn). Although this approach is robust for global pair-wise registration, the operation of correspondence propagation is too time-consuming.

Global registration of multi-view scans

For multi-view registration of unordered scans, a global approach is presented in Fig 2. Given a set of unordered scans, the frame of reference, without loss of generality, can be attached to the first scan. As shown in Fig 2, the proposed approach includes three main operations; there are pair-wise registration, reliability judgment of pair-wise registration and model fusion. By alternately implementing these three operations, the proposed approach can achieve multi-view registration with good performance. Subsequently, we will present more details of these three operations and the implementation of this approach.

Download:

Fig 2. The flowchart of the proposed approach for multi-view registration, which operates pair-wise registration, the judgment of pair-wise registration result and model fusion alternately until all range scans are integrated into one model.

https://doi.org/10.1371/journal.pone.0203139.g002

Pair-wise registration by fast correspondence propagation

For the unordered range scans, there is no prior information about initial parameters in the stage of global registration. Therefore, Lei et al. [16] proposed the multi-scale descriptors with correspondence propagation for global pair-wise registration. As shown in Fig 1, this approach requires to do several operations to achieve pair-wise registration. Among these operations, correspondence propagation is the most important and time-consuming one.

According to [16], multi-scale descriptors consist of the normal vector N and the eigenvalue based vector D, where D is rotation invariant. Hence, seed matches are established by performing the NN search on D. But these seed matches may be unreliable due to the low dimension of D. To obtain reliable point correspondences, it is necessary to apply correspondence propagation. However, it is not required to apply correspondence propagation on each seed match. Because once the propagation starts from one reliable seed match, the exploitation of N and D will certainly propagate it into a set of good matches. Accordingly, some reliable seed matches are enough for the correspondence propagation of multi-scale descriptors. The problem is how to find reliable seed matches. Fig 3 illustrates both reliable and unreliable seed matches. Actually, denote three reliable matches. But (p₁,q₃) and (p₃,q₁) may be mismatched due to noises or other reasons. Therefore, one seed match (p₁,q₃) or (p₃,q₁) with small distance of the eigenvalue based vector may not be reliable. While, the reliable seed match (p₂,q₂) must contain small distance. To this end, it is better to sort all established seed matches in ascending order by their distances of the eigenvalue based vector, then the correspondence propagation is only applied to top δ × 100% sorted seed matches. In this case, the total time complexity of correspondence propagation can be seriously reduced so as to result in more efficient pair-wise registration. Based on the above discussions, an efficient pair-wise registration algorithm is summarized in Algorithm 1.

Download:

Fig 3. Demonstration of different seed matches, one solid line with bi-directional arrow denotes a reliable seed match, each dotted line with bi-directional arrow denotes an unreliable seed match, the direction and length of red line with arrow denote the normal vector and the eigenvalue based vector, respectively.

https://doi.org/10.1371/journal.pone.0203139.g003

Algorithm 1. Efficient pair-wise registration algorithm

Input: Scan pair P and Q

Get P_d and Q_d by down sample;

Calculate descriptors (N,D) for each point in P_d and Q_d;

Establish seed matches by performing the NN search on D;

Sort all seed matches in the ascending order by their distances of D;

For i = 1:δn

Propagate the ith seed match into a set of matches M_i ={(y_i,x_c(i)),…};

Find maximum consensus set in M_i by RANSAC;

If (|M_i|>10)

Use M_i to calculate (R,t)_i;

End

Find the best alignment (R,t)_best by the quality function;

Utilize TrICP to refine (R,t)_best and obtain (R,t).

Output: Pair-wise registration results (R,t)

In this algorithm, the required number of correspondence propagation is δn. As the propagation of each seed match is same to that of the method proposed in [16], the total complexity of correspondence propagation is reasonably reduced to O(δn(m+n)logn). Obviously, small value of δ will lead to low time complexity. Since the seed match with small distance may not be reliable, the value of δ should not be set too small. Otherwise, there may be no way to apply correspondence propagation to reliable seed match, which will lead to unexpected pair-wise registration. The choosing of parameter δ will be discussed in the experiment part.

Reliability judgment of pair-wise registration

For these scan pairs involved in multi-view registration, some of them may contain high overlapping percentages, other may have low percentages or even no overlapping areas. Given a pair of scans, one rigid transformation will be estimated by the proposed pair-wise registration algorithm. For the scan pair with high overlapping percentage, the estimated transformation is reliable. However, it is unreliable for these scan pairs without high overlap percentages. Generally, it is difficult to predict or estimate the overlapping percentage for each scan pair, so the pair-wise registration may be applied to scan pair without high overlapping percentage and obtain unreliable transformation. As shown in Fig 2, only reliable results can be utilized for the model fusion. Therefore, it is required to judge the reliability of pair-wise registration.

As the TrICP algorithm is finally utilized to refine the result of pair-wise registration, it also determines overlapping areas and provides TMSE for each scan pair. Fig 4 displays pair-wise registration results of two scan pairs, where one scan pair gets reliable registration due to its high overlap percentage and the other one gets unreliable registration due to its low overlap percentage or other reasons. As shown in Fig 4, the TrICP algorithm determines suitable overlapping areas with small TMSE for reliable registration and finds false overlapping areas with very large TMSE for the unreliable registration. Therefore, TMSE is related to the reliability of pair-wise registration.

Download:

Fig 4. The illustration of TMSE for pair-wise registration of two scan pairs.

In the 2nd row, each black line denotes the distance of one point pair located in the overlapping area. (a) Reliable registration. (b) Unreliable registration.

https://doi.org/10.1371/journal.pone.0203139.g004

For the same registration result, the TMSE is affected by the point resolution of model shape. Fig 5 demonstrates the same pair-wise registrations under different point resolutions of model shape. As shown in Fig 5, high resolution reduces the TMSE and low resolution increases the TMSE. Therefore, we can judge the pair-wise registration is reliable if the TMSE satisfies the following condition: (6) where d_Q denotes the point resolution of model shape. Otherwise, the pair-wise registration is judged to be unreliable. To achieve multi-view registration, the pair-wise registration should be implemented many times in the proposed approach. During multi-view registration, it is easy to collect TMSEs from reliable registrations, which can be further utilized to judge the reliability of posterior registration results. Therefore, it is better to revise Eq (6) as follows: (7) where m_TMSE indicates the mean TMSE of reliable registrations. Before the first reliable registration has been checked, m_TMSE is set as m_TMSE = 0. After the check of one new reliable registration, it can be further updated. Here, a and β are two free parameters, which will also be discussed and determined in the Section of experiment.

Download:

Fig 5. TMSEs under different point resolutions of model shape.

(a) Low resolution results in large trimmed MSE. (b) High resolution reduces the trimmed MSE.

https://doi.org/10.1371/journal.pone.0203139.g005

Model fusion

By the proposed judgment, it is easy to judge whether the pair-wise registration is reliable or not. For reliable registration, data shape should be transformed and merged into the model shape. The simplest method is to directly add points of transformed data shape into the model shape. However, this rude fusion will result in redundant range points with multi-scale descriptors, which can reduce the efficiency of subsequent registration. Hence, model fusion should be carefully designed. Accordingly, we propose the model fusion illustrated in Fig 6.

Download:

Fig 6. Illustration of model fusion.

(a) The point pairs located in the overlapping area, where each pair of points is connected by one line with arrow. (2) Model and data shape is merged into one shape, where each point pair is replaced by its intermediate point in the overlapping area.

https://doi.org/10.1371/journal.pone.0203139.g006

Based on the estimated rigid transformation (R,t), the data shape is first transformed into the reference frame of model shape as follows: (8) Then both model shape and transformed data shape are divided into two independent parts: (9) where Q_ξ and P_ξ denotes overlapping parts of these two shapes, A and B indicates non-overlapping parts of them. Accordingly, the fused model shape is composed of raw model shape and transformed data shape as follows: (10) where , N_f = ξN_p, and (11)

Similar to these two raw shapes, the down sampled data shape P_d should be correspondingly transformed and merged into the down sampled model shape Q_d. And they can be merged in the same way as these raw shapes. Besides, multi-scale descriptors should also be merged. As eigenvalue based vectors D are rotation-invariant, they can be merged as follows: (12) where D_P,i and D_Q,c(i) denote eigenvalue based vectors of two matched points. While, normal vectors N are rotation-invariant, it can be merged as follows: (13) where N_P,i and N_Q,c(i) indicate normal vectors of two matched points. In this way, the model shape is properly fused with corresponding multi-scale descriptors, so there is no need to recalculate multi-scale descriptors for the fused model shape.

Compared with rude fusion, the proposed fusion method can delete redundant range points with multi-scale descriptors.

Implementation

The overall process of the proposed approach can be summarized in Algorithm 2.

Algorithm 2: Multi-view registration of unordered range scans

Input: A scan set with N unordered scans View the 1st scan as Q and delete it from scan set;

Set m_TMSE = 0, N' = 0

Do

N' = 0;

For i = 1:N

View P_i as P, estimate (R_i,t_i) and TMSE_i by Algorithm 1;

Use Eq () to judge the reliability of (R_i,t_i);

If ((R_i,t_i) is reliable)

N' = N' + 1;

Update Q by the Sec. 3.3;

Delete P_i from scan set;

Recalculate m_TMSE and d_Q;

Record (R_i,t_i);

End

N = (N − N');;

While (N > 0)

Output: Multi-view registration results

Given a set of unordered range scans, the proposed approach can automatically achieve global multi-view registration without any prior information.

Results and discussion

To validate the performance of our proposed method, quantitative experiments were conducted on four data sets taken from the Stanford 3D Scanning Repository [26], which provides multi-view scans with ground truth of rigid transformations . For calculating multi-scale descriptors, raw range points of each scan were down sampled with the sampling frequency set to be about 100. Besides, raw range points of each scan were down sampled with the sampling frequency set to be 10 for the pair-wise registration. For quantitative comparison, rotation error and translation error are defined as and , where denotes rigid transformations estimated by multi-view registration approach. In all multi-view registration approaches, the NN search method based on k-d tree [27] was utilized to establish point correspondences. All multi-view registration approaches were implemented in Matlab on a desktop with four-core 3.6GHz processor and 8GB of memory.

Parameter initialization

In the proposed approach, three free parameters require to be chosen. To choose α and β, we can fix the setting of δ = 1.0 and then test the proposed approach on Stanford Bunny and Armadillo. For each reliable pair-wise registration of multi-view registration, the value of trimmed mean square error TMSE, the point resolution of model shape d_Q, and the mean TMSE of reliable registrations m_TMSE are recorded in Fig 7. As shown in Fig 7, the values of TMSE show significant change in each reliable pair-wise registration. Usually, TMSE and m_TMSE are slightly larger than d_Q. Therefore, α and β should be assigned with large values so as to pick out all reliable pair-wise registration. However, large α and β will introduce unreliable pair-wise registration, which will lead to the failure of multi-view registration. Actually, unreliable pair-wise registration is caused by the low overlap percentage of data and model shapes. With model fusion, low overlap percentage will become high. That means unreliable pair-wise registration will turn to be reliable after model fusion. Considering these factors, α and β can be assigned with small values so as to avoid the involvement of unreliable pair-wise registrations. In following experiments, we set α = 2 and β = 1.5, which can result in robust multi-view registration.

Download:

Fig 7. Illustration of TMSE, d_Q,and m_TMSE in each reliable pair-wise registration.

(a) Bunny. (b) Armadillo.

https://doi.org/10.1371/journal.pone.0203139.g007

After choosing α and β, we assign different values to δ and then test the proposed approach on Stanford Bunny and Armadillo. For each value of δ, the required number of pair-wise registration, run time and registration error of the proposed approach are recorded in Table 1. As shown in Table 1, within reasonable range, small value of δ will result in efficient registration without loss of registration accuracy. However, too small value of δ will lead to the failure of multi-view registration. For some scan pairs without high overlap percentages, there are small numbers of reliable seed matches. Under small value of δ, these reliable seed matches are all outside the top δ × 100%, which can lead to the failure of correspondence propagation and further result in registration failure. Accordingly, it is unreasonable to assign δ with too small value. By considering both robustness and efficiency, we set δ = 0.3 with enough tolerance. During experiments, we find this setting can lead to robust and efficient registration of multi-view scans.

Download:

Table 1. Results of the proposed approach with different values of δ on the Bunny and Armadillo datasets.

https://doi.org/10.1371/journal.pone.0203139.t001

Validation

To validate the proposed approach, it is compared with two versions of multi-view registration approaches based on multi-scale descriptors: multi-view registration approach based on original correspondence propagation with rude fusion and multi-view registration approach based on original correspondence propagation with model fusion, which are abbreviated as OCPRF and OCPMF, respectively. As the proposed approach is based on fast correspondence propagation with model fusion, it is abbreviated as FCPMF. The performances of multi-view registration are measured by runtime and registration error. Fig 8 displays the number of pair-wise registrations and runtime required to achieve multi-view registration for each approach. Table 2 illustrates registration error of each approach tested on four datasets.

Download:

Fig 8. Comparison of efficiency for different approaches based on multi-scale descriptors.

(a) Required number of pair-wise registrations. (b) Run time.

https://doi.org/10.1371/journal.pone.0203139.g008

Download:

Table 2. Comparison of accuracy for different approaches based on multi-scale descriptors, where small value indicates accurate registration and bold number denotes the best results.

https://doi.org/10.1371/journal.pone.0203139.t002

As shown in Fig 8, these three approaches require almost the same number of pair-wise registrations, which is a little more than the number of range scans to be aligned. As all these three approaches utilize the proposed reliability judgment, it is reasonable to conclude that this judgment is very effective for pair-wise registration. Sometimes, FCPMF may require a little more number of pair-wise registrations than other two approaches. In some scan pairs with low overlap percentages, there are a small number of reliable seed matches, which are all outside the top 30%. For these scan pair, the application of OCPRF and OCPMF may obtain good pair-wise registration results due to the application of correspondence propagation to all seed matches. Since FCPMF only applies the correspondence propagation to the top 30% sorted seed matches, it is unable to get good registration results. Besides, OCPRF may also require a little more number of pair-wise registrations than other two approaches. This is because the rude fusion results in many redundant points with multi-scale descriptors, which may reduce the robustness of pair-wise registration. Although FCPMF may require a little more number of pair-wise registrations, it is more efficient than other two approaches. By selecting the top 30% sorted seed matches, it can accelerate the correspondence propagation, which will reduce the runtime of pair-wise registration. With the model fusion, it deletes repeated range point with multi-scale descriptors so as to further improve the efficiency of subsequent registration. Therefore, both fast correspondence propagation and model fusion are helpful for the efficiency of multi-view registration.

As displayed in Table 2, all these three approaches can obtain accurate registration results, where the results of OCPRF are slightly more accurate than that of other two approaches. This is because the rude model fusion keeps many redundant range points, which is good for accurate registration. However, it leads to high computation complexity. Since many exiting approaches can refine initial registration parameters, efficiency is more important than accuracy in global multi-view registration. Accordingly, OCPRF is not a good choice. For efficient registration, the proposed model fusion is necessary to delete redundant range point with multi-scale descriptors. As most reliable seed matches contain small distances, it is easy to find good match set by the application of correspondence propagation to the top 30% sorted seed matches. Compared with raw correspondence propagation, fast correspondence propagation not only improves efficiency, but also obtains more accurate multi-view registration results.

In one word, the proposed approach is a reasonable and effective solution for the global registration of unordered range scans.

Comparsion

To demonstrate its performance, the proposed approach was compared with two related approaches; there are automatic multi-view registration approach based on surface consistency (SurfC) [23] and automatic multi-view registration approach based on spanning tree (SpanT) [25]. Multi-view registration results are also reported in the form of runtime and registration error.

Accuacry and efficiency.

For comparison of efficiency and accuracy, experiments were carried on four data sets. Fig 8 illustrates the number of pair-wise registrations and runtime required to achieve multi-view registration for all competed approaches. What’s more, Table 3 displays registration error of all competed approaches tested on four datasets. To demonstrate comparison in a more intuitive manner, Fig 9 displays the registration results of four data sets for all competed approaches in the form of cross-sections.

Download:

Fig 9. Efficiency comparison of different approaches tested on four data sets.

(a) Required number of pair-wise registrations. (b) Run time.

https://doi.org/10.1371/journal.pone.0203139.g009

Download:

Table 3. Comparison of accuracy for different approaches, where small value indicates good performance and bold number denotes the best result.

https://doi.org/10.1371/journal.pone.0203139.t003

As shown in Fig 9, SurfC requires more pair-wise registration results than other two approaches and the proposed approach requires the least number of pair-wise registrations among all these competed approaches. Actually, SurfC should apply pair-wise registration to all scan pairs involved in multi-view registration. Given N range scans, SurfC requires N(N − 1)/2 pair-wise registrations. Therefore, this approach is inefficient. For multi-view registration, SpanT should find at least (N − 1) reliable pair-wise registration of raw scan pairs to construct completed spanning tree. As most of raw scan pairs contain low overlap percentages, it also requires a large number of pair-wise registrations. While, the proposed approach only requires a small number of pair-wise registrations to achieve multi-view registration. During multi-view registration, the proposed approach utilizes the reliable pair-wise registration to augment the model shape, which can increase the overlap percentage between the model shape and data shape. With the increased overlap percentages, it becomes easily in computing reliable pair-wise registration. Hence, the number of required pair-wise registrations is reasonably reduced. Since the proposed approach utilizes the pair-wise registration by fast correspondence propagation of multi-scale descriptors, it is more efficient than other two approaches.

As shown in Table 3 and Fig 10, the proposed approach can obtain the most accurate results for multi-view registration among all competed approaches. Since these competed approaches utilize the pair-wise registration to achieve multi-view registration, they all suffer from the problem of error accumulation. In SurfC and SpanT, the pair-wise registration algorithm is directly applied to raw scan pairs. Since few raw scan pairs have high overlap percentages, few reliable results of pair-wise registration are very accurate, which will lead to large accumulated error in multi-view registration. While, the proposed approach uses reliable pair-wise registration results to augment model shape, which can increase overlap percentages of scan pairs to be aligned. With increased overlap percentages, results of pair-wise registration become more reliable and accurate, which will reduce accumulated error in multi-view registration. Therefore, the proposed approach can always obtain the most accurate results for multi-view registration. As all completed approaches utilize results of pair-wise registration to achieve multi-view registration, the defined multi-view registration can be viewed as the mean error of pair-wise registration. As shown in Table 3, pair-wise registration error of our approach is also small than that of other approaches.

Download:

Fig 10. Registration results of different approaches in the form of cross-section.

(a) Unordered scans (b) Reconstructed 3D models. (c) Results of SurfC. (d) Results of SpanT. (e) Our results.

https://doi.org/10.1371/journal.pone.0203139.g010

Robustness.

For the comparison of robustness, all considered approaches were tested on Stanford Bunny with five groups of different order. Multi-view registration results are also reported in runtime and registration errors. Table 4 illustrates registration results of different approaches.

Download:

Table 4. Comparison of robustness for different approaches, where small value indicates good performance and bold number denotes the best result.

https://doi.org/10.1371/journal.pone.0203139.t004

As shown in Table 4, all considered approaches are robust to the order of multiple range scans and the proposed approach can obtain the most accurate and efficient registration results for Stanford Bunny under different orders. Although the order of input range scans may be changed, all these three approaches utilize reliable pair-wise registration results to estimate rigid transformations for multi-view registration. Therefore, they can always achieve fine results for multi-view registration. As SpanT and the proposed approach only apply the pair-wise registration on some scan pairs, the change of scan order will cause their required number of pair-wise registration to be varied. Accordingly, the run time of these two approaches varies under different scan orders. However, SurfC applies pair-wise registration on all scans pairs involved in multi-view registration and its run time is relatively stable under different scan orders. By only considering the accuracy, the proposed approach is very robust.

Application

In fact, many kinds of range scans can be viewed as the input of our approach. In this section, the proposed approach is tested on two data sets so as to demonstrate its potential applications on scene reconstruction and robot mapping.

For scene reconstruction, the proposed approach is tested on EXBI data set [22]. This data set contains 10 RGB-D scans recorded by the Kinect sensor, which provides range point clouds with associated color information. During the record of this data set, the Kinect sensor was manually moving in front of a indoor scene. It should be noted that only range points are utilized for scene reconstruction and color information can be used to help the final assessment. Fig 11 displays the scene reconstruction of the proposed approach. Since RGB-D contains noises, they should be filtered for good scene reconstruction. As shown in Fig 11, the proposed approach has the potential for fusing RGB-D data and can be applied to scene reconstruction.

Download:

Fig 11. Scene reconstruction of EXBI data set.

(a) Scene image of each rang scans. (b) Raw range scans. (c) Scene reconstruction with noises. (d) Scene reconstruction without noises.

https://doi.org/10.1371/journal.pone.0203139.g011

For robot mapping, the proposed approach is tested on Gazebo data set [28] with 32 range scans, which was obtained from a small-scale outdoor environment (72x70x19 m). During the record of this data sets, robot was equipped with a laser range finder and followed the path to form a closed loop (4x5x0.09 m). As the raw data contains range points of ground surface, they should be filtered for accurate mapping. Fig 12 displays the mapping results of the proposed approach. As shown in Fig 12, the proposed approach can achieve robot mapping with good accuracy. For small-scale environment, the proposed approach only requires a little number of reliable pair-wise registrations to achieve mapping, where the accumulation of registration error is small, results in accurate mapping. Therefore, the proposed approach can be potentially applied to robot mapping of small-scale environment. For mapping of large-scale environments, the accumulation of registration error will become very large, which may lead to the failure of mapping.

Download:

Fig 12. Mapping result of Gazebo data set.

(a) Scene image (b) Raw range scans. (c) Mapping result with noises and ground surface. (d) Mapping result without noises and ground surface.

https://doi.org/10.1371/journal.pone.0203139.g012

Conclusions

This paper proposes the global approach for multi-view registration of unordered range scans. This approach is based on the pair-wise registration, which is achieved by the fast correspondence propagation of multi-scale descriptors. For multi-view registration, the reliability judgment is designed to choose reliable pair-wise registrations, which are utilized to augment the model shape. By model fusion, the overlap percentage of scan pair will be reasonably increased. With the increased overlap percentage, the pair-wise registration is able to obtain reliable accurate results, which will reduce the required number of pair-wise registration and accumulated error for multi-view registration. Experimental results conducted on public available data sets illustrate that the proposed approach can robustly achieve multi-view registration of unordered range scans with good accuracy and efficiency.

Although the proposed approach has good performance, it does not mean that this approach can solve all multi-view registration problems. Given a set of unordered scans, there may be no other scan that contains high overlap percentage to one scan. If this special scan is viewed as the reference scan, it is difficult to obtain reliable pair-wise registration for this scan, which will lead to the failure of multi-view registration. However, it should be noted that all existing approaches for multi-view registration proposed so far share this limitation as well. In the future, we will extend this approach to robot mapping of large-scale environments.

Acknowledgments

Supported by the National Natural Science Foundation of China under Grant nos. 61573273 and 91648121.

References

1. Henry P, Krainin M, Herbst E, Ren X, Fox D. RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments. International Journal of Robotics Research. 2012 Feb; 31(5): 647–663.
- View Article
- Google Scholar
2. Ma L, Zhu JH, Zhu L, Du SY, Cui JR. Merging grid maps of different resolutions by scaling registration. Robtica, 2016 Nov; 34(11): 2516–2531.
- View Article
- Google Scholar
3. Endres F, Hess J, Sturm J, Cremers D, Burgard W. 3-D mapping with an RGB-D camera. IEEE Transactions on Robotics. 2014 Feb; 30(1): 177–187.
- View Article
- Google Scholar
4. Zhang X, Jian L, Xu M. Robust 3D point cloud registration based on bidirectional Maximum Correntropy Criterion. PLoS ONE, 2018 May; 13(5): e0197542. pmid:29799864
- View Article
- PubMed/NCBI
- Google Scholar
5. Zhou QY, Koltun V. Dense scene reconstruction with points of interest. ACM Transactions on Graphics, 2013 Jul; 32(4): 1–8.
- View Article
- Google Scholar
6. Zollhofer M, Nießner M, Izadi S, Rehmann C, Zach C, Fisher M, et al. Real-time non-rigid reconstruction using an RGB-D camera. ACM Transactions on Graphics. 2014 Jul; 33(4): 1756–1766.
- View Article
- Google Scholar
7. Besl PJ, McKay HD. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1992 Feb; 14(2): 239–256.
- View Article
- Google Scholar
8. Chetverikov D, Stepanov D, Krsek P. Robust Euclidean alignment of 3D point sets: the trimmed iterative closest point algorithm. Image and Vision Computing. 2015 Mar; 23(3): 299–309.
- View Article
- Google Scholar
9. Phillips M, Ran L, Tomasi C. Outlier robust ICP for minimizing fractional RMSD. Int. Conf. on 3-D Digital Imaging and Modeling (3DIM), 2007: 427–434.
- View Article
- Google Scholar
10. Lomonosov E, Chetverikov D, Anikó E. Pre-registration of arbitrarily oriented 3D surfaces using a genetic algorithm. Pattern Recognition Letters. 2006 Aug; 27(11): 1201–1208.
- View Article
- Google Scholar
11. Zhu JH, Meng DY, Li ZY, Du SY. Yuan ZJ. Robust registration of partially overlapping point sets via genetic algorithm with growth operator. IET Image Processing. 2014 Oct; 8(10): 582–590.
- View Article
- Google Scholar
12. Sandhu R, Dambreville S, Tannenbaum A. Point set registration via particle filtering and stochastic dynamics. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2010 Aug; 32(8): 1459–1473. pmid:20558877
- View Article
- PubMed/NCBI
- Google Scholar
13. Yang JL, Li HD, Campbell D, Jia YD. Go-ICP: A globally optimal solution to 3D ICP point-set registration. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2016 Nov; 38(1): 2241–2254.
- View Article
- Google Scholar
14. Radu BR, Nico B, Michael B. Fast point feature histograms (FPFH) for 3D registration. IEEE Int. Conf. on Robotics and Automation, 2009:1848–1853.
- View Article
- Google Scholar
15. Guo YL, Sohel F, Bennamoun M. Lu M, Wan JW. Rotational projection statistics for 3D local surface description and object recognition. International Journal of Computer Vision. 2013 Oct; 105(1): 63–86.
- View Article
- Google Scholar
16. Lei H, Jiang G, Quan L. Fast descriptors and correspondence propagation for robust global point cloud registration. IEEE Transaction on Image Processing. 2017 Aug; 26(8): 3614–3623.
- View Article
- Google Scholar
17. Chen Y, Medioni G. Object modeling by registration of multiple range images. Image and Vision Computing. 1992 Apr; 10(3): 145–155.
- View Article
- Google Scholar
18. Zhu JH, Li ZY, Du SY. Ma L, Zhang T. Surface reconstruction via efficient and accurate registration of multi-view range scans. Optical Engineering. 2014 Apr; 53, 102–104. 53(10): 102104.
- View Article
- Google Scholar
19. Govindu VM, Pooja A. On averaging multi-view relations for 3D scan registration. IEEE Transactions on Image Processing. 2014 Mar; 23(3): 1289–1302. pmid:23412615
- View Article
- PubMed/NCBI
- Google Scholar
20. Guo R, Zhu JH, Li YC, Chen DP, Li ZY, Zhang YQ. Weighted motion averaging for the registration of multi-view range scans. Multimedia Tools and Applications. 2018 May; 77(9): 10651–10668.
- View Article
- Google Scholar
21. Arrigoni F, Rossi B, Fusiello A. Global registration of 3d point sets via LRS decomposition. European Conference on Computer Vision (ECCV), 2016: 489–504.
- View Article
- Google Scholar
22. Evangelidis GD, Horaud R. Joint alignment of multiple point sets with batch and incremental Expectation-Maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017 Jun; 40(6): 1397–1410. pmid:28644798
- View Article
- PubMed/NCBI
- Google Scholar
23. Huber DF, Hebert M. Fully automatic registration of multiple 3D datasets. Image and Vision Computing. 2003 Jul; 21(7): 637–650.
- View Article
- Google Scholar
24. Guo YL, Sohel F, Bennamoun M, Wan JW, Lu M. An accurate and robust range image registration algorithm for 3D object modeling. IEEE Transactions on Multimedia, 2014 May; 16(5): 1377–1390.
- View Article
- Google Scholar
25. Zhu JH, Zhu L, ZY , Li C, Cui JR. Automatic multi-view registration of unordered range scans without feature extraction. Neurocomputing. 2016 Jan; 171: 1444–1453.
- View Article
- Google Scholar
26. Marc L. The stanford 3d scanning repository: http://graphics.stanford.edu/data/3Dscanrep/.
27. Nuchter A. Lingemann K. Hertzberg J. Cached k-d tree search for ICP algorithms. Int. Con. on 3-D Digital Imaging and Modeling(3DIM), 2006: 419–426.
- View Article
- Google Scholar
28. Pomerleau F, Liu M, Colas F, R. Siegwart R. Challenging data sets for point cloud registration algorithms, International Journal of Robotic Research. 2012 Sep; 31(14): 1705–1711,
- View Article
- Google Scholar

[ref1] 1. Henry P, Krainin M, Herbst E, Ren X, Fox D. RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments. International Journal of Robotics Research. 2012 Feb; 31(5): 647–663.
View Article
Google Scholar

[2] View Article

[3] Google Scholar

[ref2] 2. Ma L, Zhu JH, Zhu L, Du SY, Cui JR. Merging grid maps of different resolutions by scaling registration. Robtica, 2016 Nov; 34(11): 2516–2531.
View Article
Google Scholar

[5] View Article

[6] Google Scholar

[ref3] 3. Endres F, Hess J, Sturm J, Cremers D, Burgard W. 3-D mapping with an RGB-D camera. IEEE Transactions on Robotics. 2014 Feb; 30(1): 177–187.
View Article
Google Scholar

[8] View Article

[9] Google Scholar

[ref4] 4. Zhang X, Jian L, Xu M. Robust 3D point cloud registration based on bidirectional Maximum Correntropy Criterion. PLoS ONE, 2018 May; 13(5): e0197542. pmid:29799864
View Article
PubMed/NCBI
Google Scholar

[11] View Article

[12] PubMed/NCBI

[13] Google Scholar

[ref5] 5. Zhou QY, Koltun V. Dense scene reconstruction with points of interest. ACM Transactions on Graphics, 2013 Jul; 32(4): 1–8.
View Article
Google Scholar

[15] View Article

[16] Google Scholar

[ref6] 6. Zollhofer M, Nießner M, Izadi S, Rehmann C, Zach C, Fisher M, et al. Real-time non-rigid reconstruction using an RGB-D camera. ACM Transactions on Graphics. 2014 Jul; 33(4): 1756–1766.
View Article
Google Scholar

[18] View Article

[19] Google Scholar

[ref7] 7. Besl PJ, McKay HD. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1992 Feb; 14(2): 239–256.
View Article
Google Scholar

[21] View Article

[22] Google Scholar

[ref8] 8. Chetverikov D, Stepanov D, Krsek P. Robust Euclidean alignment of 3D point sets: the trimmed iterative closest point algorithm. Image and Vision Computing. 2015 Mar; 23(3): 299–309.
View Article
Google Scholar

[24] View Article

[25] Google Scholar

[ref9] 9. Phillips M, Ran L, Tomasi C. Outlier robust ICP for minimizing fractional RMSD. Int. Conf. on 3-D Digital Imaging and Modeling (3DIM), 2007: 427–434.
View Article
Google Scholar

[27] View Article

[28] Google Scholar

[ref10] 10. Lomonosov E, Chetverikov D, Anikó E. Pre-registration of arbitrarily oriented 3D surfaces using a genetic algorithm. Pattern Recognition Letters. 2006 Aug; 27(11): 1201–1208.
View Article
Google Scholar

[30] View Article

[31] Google Scholar

[ref11] 11. Zhu JH, Meng DY, Li ZY, Du SY. Yuan ZJ. Robust registration of partially overlapping point sets via genetic algorithm with growth operator. IET Image Processing. 2014 Oct; 8(10): 582–590.
View Article
Google Scholar

[33] View Article

[34] Google Scholar

[ref12] 12. Sandhu R, Dambreville S, Tannenbaum A. Point set registration via particle filtering and stochastic dynamics. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2010 Aug; 32(8): 1459–1473. pmid:20558877
View Article
PubMed/NCBI
Google Scholar

[36] View Article

[37] PubMed/NCBI

[38] Google Scholar

[ref13] 13. Yang JL, Li HD, Campbell D, Jia YD. Go-ICP: A globally optimal solution to 3D ICP point-set registration. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2016 Nov; 38(1): 2241–2254.
View Article
Google Scholar

[40] View Article

[41] Google Scholar

[ref14] 14. Radu BR, Nico B, Michael B. Fast point feature histograms (FPFH) for 3D registration. IEEE Int. Conf. on Robotics and Automation, 2009:1848–1853.
View Article
Google Scholar

[43] View Article

[44] Google Scholar

[ref15] 15. Guo YL, Sohel F, Bennamoun M. Lu M, Wan JW. Rotational projection statistics for 3D local surface description and object recognition. International Journal of Computer Vision. 2013 Oct; 105(1): 63–86.
View Article
Google Scholar

[46] View Article

[47] Google Scholar

[ref16] 16. Lei H, Jiang G, Quan L. Fast descriptors and correspondence propagation for robust global point cloud registration. IEEE Transaction on Image Processing. 2017 Aug; 26(8): 3614–3623.
View Article
Google Scholar

[49] View Article

[50] Google Scholar

[ref17] 17. Chen Y, Medioni G. Object modeling by registration of multiple range images. Image and Vision Computing. 1992 Apr; 10(3): 145–155.
View Article
Google Scholar

[52] View Article

[53] Google Scholar

[ref18] 18. Zhu JH, Li ZY, Du SY. Ma L, Zhang T. Surface reconstruction via efficient and accurate registration of multi-view range scans. Optical Engineering. 2014 Apr; 53, 102–104. 53(10): 102104.
View Article
Google Scholar

[55] View Article

[56] Google Scholar

[ref19] 19. Govindu VM, Pooja A. On averaging multi-view relations for 3D scan registration. IEEE Transactions on Image Processing. 2014 Mar; 23(3): 1289–1302. pmid:23412615
View Article
PubMed/NCBI
Google Scholar

[58] View Article

[59] PubMed/NCBI

[60] Google Scholar

[ref20] 20. Guo R, Zhu JH, Li YC, Chen DP, Li ZY, Zhang YQ. Weighted motion averaging for the registration of multi-view range scans. Multimedia Tools and Applications. 2018 May; 77(9): 10651–10668.
View Article
Google Scholar

[62] View Article

[63] Google Scholar

[ref21] 21. Arrigoni F, Rossi B, Fusiello A. Global registration of 3d point sets via LRS decomposition. European Conference on Computer Vision (ECCV), 2016: 489–504.
View Article
Google Scholar

[65] View Article

[66] Google Scholar

[ref22] 22. Evangelidis GD, Horaud R. Joint alignment of multiple point sets with batch and incremental Expectation-Maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017 Jun; 40(6): 1397–1410. pmid:28644798
View Article
PubMed/NCBI
Google Scholar

[68] View Article

[69] PubMed/NCBI

[70] Google Scholar

[ref23] 23. Huber DF, Hebert M. Fully automatic registration of multiple 3D datasets. Image and Vision Computing. 2003 Jul; 21(7): 637–650.
View Article
Google Scholar

[72] View Article

[73] Google Scholar

[ref24] 24. Guo YL, Sohel F, Bennamoun M, Wan JW, Lu M. An accurate and robust range image registration algorithm for 3D object modeling. IEEE Transactions on Multimedia, 2014 May; 16(5): 1377–1390.
View Article
Google Scholar

[75] View Article

[76] Google Scholar

[ref25] 25. Zhu JH, Zhu L, ZY , Li C, Cui JR. Automatic multi-view registration of unordered range scans without feature extraction. Neurocomputing. 2016 Jan; 171: 1444–1453.
View Article
Google Scholar

[78] View Article

[79] Google Scholar

[ref26] 26. Marc L. The stanford 3d scanning repository: http://graphics.stanford.edu/data/3Dscanrep/.

[ref27] 27. Nuchter A. Lingemann K. Hertzberg J. Cached k-d tree search for ICP algorithms. Int. Con. on 3-D Digital Imaging and Modeling(3DIM), 2006: 419–426.
View Article
Google Scholar

[82] View Article

[83] Google Scholar

[ref28] 28. Pomerleau F, Liu M, Colas F, R. Siegwart R. Challenging data sets for point cloud registration algorithms, International Journal of Robotic Research. 2012 Sep; 31(14): 1705–1711,
View Article
Google Scholar

[85] View Article

[86] Google Scholar

Figures

Abstract

Introduction

Pair-wise registration

TrICP algorithm

Correspondence propagation of multiscale descriptors

Global registration of multi-view scans

Pair-wise registration by fast correspondence propagation

Algorithm 1. Efficient pair-wise registration algorithm

Reliability judgment of pair-wise registration

Model fusion

Implementation

Algorithm 2: Multi-view registration of unordered range scans

Results and discussion

Parameter initialization

Validation

Comparsion

Accuacry and efficiency.

Robustness.

Application

Conclusions

Acknowledgments

References