AUTOMATIC RECOGNITION OF PIPING SYSTEM FROM LASER SCANNED POINT CLOUDS USING NORMAL-BASED REGION GROWING

In recent years, renovations of plant equipment have been more frequent, and constructing 3D as-built models of existing plants from large-scale laser scanned data is expected to make rebuilding processes more efficient. However, laser scanned data consists of enormous number of points, captures tangled objects and includes a high noise level, so that the manual reconstruction of a 3D model is very time-consuming. Among plant equipment, piping systems especially account for the greatest proportion. Therefore, the purpose of this research was to propose an algorithm which can automatically recognize a piping system from large-scale laser scanned data of plants. The straight portion of pipes, connecting parts and connection relationship of the piping system can be automatically recognized. Normal-based region growing enables the extraction of points on the piping system. Eigen analysis of the normal tensor and cylinder surface fitting allows the algorithm to recognize portions of straight pipes. Tracing the axes of the piping system and interpolation of the axes can derive connecting parts and connection relationships between elements of the piping system. The algorithm was applied to large-scale scanned data of an oil rig and a chemical plant. The recognition rate of straight pipes, elbows, junctions achieved 93%, 88% and 87% respectively. * Corresponding author. This is useful to know for communication with the appropriate person in cases with more than one author.


INTRODUCTION
During recent years, renovations in plant equipment have been frequent because of the short lifetimes of plant products.However, the renovation histories were not necessarily recorded in the plant drawings.Thus, unintended collisions between existing equipment and new designed ones often take place in the construction stage.
On the other hand, because of advances of laser scanner performance, shape of objects in real environments can be easily captured as 3D point clouds.3D as-built model can also be reconstructed from the laser scanned data.Using the as-built model, the unintended collisions can be avoided in the planning stage.However, the 3D point clouds of plants have massive points, capture tangled objects and include a high noise level.Therefore, recognizing each plant component from the point clouds and building as-built models of the plants are very time consuming in manual way.
Thus, automatic recognition and construction of 3D models from point clouds needs to be strongly encouraged in the plant engineering field.Plants consist of various types of components.Among them, the piping systems especially account for the greatest proportion and are renovated frequently.A piping system consists of various elements and connection relationships; straight pipes, connecting parts such as junctions and elbows, and attached parts such as indicators, etc.Also, the connection relationship defines the logical connectivity between these piping elements.
Several studies have been proposed to recognize a piping system from laser scanned point clouds.However, these algorithms could not be applied to the point clouds which have been already registered or lacked the intensity image, or could only recognize very limited elements of piping system.Masuda et al. proposed a method which could recognize planes and cylinders from scanned data of plants (Masuda et.al 2012).But it required a pair of scanned points and a reflected intensity image obtained from a scanner.Rabbaini et al. proposed a method, which reconstructed 3D primitive model from the combination of a point cloud and a photograph taken from the same location (Rabbani et. al 2004).However, the method needed a pair of point clouds and a photo shot from the scanner position.Piping system occupies a broad area of plants, and multiple scans and their registrations are essential to capture point clouds covering the whole shape of the piping system.Therefore, it is quite probable that the algorithms could not be applied to registered point clouds which are generated from multiple scans and lack the image data.On the other hand, Andrew et al. proposed a method where a 3D CAD model in database was matched to a point cloud using spin image (Andrew et.al 1997).However, it is difficult to prepare an exact 3D model of a straight pipe because their length is not fixed.Recently, Lee et al. proposed a method which recognized straight pipes, elbows and junctions from points on piping system using the Voronoi diagram (Lee et.al 2012).However, in their algorithm, the input point clouds were assumed to include only pipes themselves, and did not include any unwanted parts such as flanges or valves, or supporting members.But the point clouds other than the piping system must still be removed for the point clouds manually.In the methods of Belton et al. (Belton et.al 2006) and Rabbani et al. (Rabbani et.al 2006), scanned points of plants are classified and partitioned into those on planar surfaces and on cylindrical surfaces using covariance analysis and using a combination of normal-based region growing and plane fitting.However, points on elbows and junctions, and the connectivity of the piping systems were not identified.In the method of Vosselman et al. (Vosselman et.al 2004), planes and cylinders are recognized using 3D Hough transform.However, only straight pipes are ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-5/W2, 2013ISPRS Workshop Laser Scanning 2013, 11 -13 November 2013, Antalya, Turkey recognized in the piping system.In the method of El-Halawany et al. (El-Halawany et al. 2011), cylinders are identified from scanned points of pole-like objects on a road-side using a combination of eigen-based segmentation, linear feature extraction and cylinder fitting.However, if applied to scanned points of plants, only straight pipes could be extracted from them.Marshall et al. proposed a non-linear least square-based method where scanned points of general objects are segmented into spheres, cylinders, cones and tori (Marshall et.al 2001).However, the method dealt only with the point clouds of relatively simple-shaped objects, so it could not work well when applied to the tangled objects in plants.
Therefore, the purpose of our research is to propose a new algorithm that can automatically recognize piping elements and their connection relationships only from a registered laser scanned point cloud of a plant.The algorithm is also designed to recognize the defining parameters of straight pipes, elbows and junctions with high accuracy.The algorithm was tested for large-scale point clouds of real plants, and the accuracies of recognition of the straight pipes and connecting parts were verified quantitatively.
We have already proposed a piping system recognition algorithm (Kawashima et.al 2011).However, it had some problems; scanned points on the plant components other than the piping system must be removed in advance, the recognition rates of the connecting parts were less than 50%, and the computational performance of the recognition was low due to the heavy-loaded normal vector and the normal tensor estimation (2 hours only for 4.5 million points).Therefore, in this paper, we solve these problems by; 1) introducing normalbased region growing which automates the removal of the points other than the piping system, 2) tracing axes of the piping system and fitting line-arc segments to the axes to improve the recognition rates, and 3) interlacing the normal calculation only at the decimated points to reduce their calculation time.Thanks to these methods, we achieved significant improvement in recognition rates of connecting parts and calculation time compared to (Kawashima et.al 2011).

Overview
As shown in Fig. 1, our algorithm consists of two pre-processes and three main processes.
In the pre-processes, at first, grid cells are positioned in whole scanned environment.All scanned points are assigned to one of the cells.Then, for each cell, a quadratic polynomial surface is fitted to the points in the cell, and normal vectors and principal curvatures of the points are calculated.
In the main process, at first, only the points on the piping system are extracted using normal-based region growing.Next, points on straight pipes are further extracted by normal tensor and region growing, and cylinders are finally fitted to the points.The radii and positions of the cylinders are also determined as the defining parameters of the straight pipes.Finally, tracing axes of the piping system and interpolation of the axes enables us to estimate the axes of connecting parts and remaining straight pipes.The defining parameters of the connecting parts are also estimated from the axes of the piping system.

Creating grid cells
In order to increase the computational efficiency of calculating normal vectors and normal tensors, a grid whose cell has a width g l is placed to cover the whole scanned space.Each grid cell keeps up scanned points inside the grid cell.g l was set as one third of the smallest pipe radius min r expected in the environment in the algorithm.

Estimating normal vectors
In order to estimate the normal vector of all scanned points, for each grid cell, covariance where a set of points included in its assigned grid cell and in the 26 one-neighbour grid cells Then, the eigenvalues ) 0 ( , , normal at the point j p .Thus, 3 e is determined to be an initial normal vector

Extracting points of a piping system
After calculating the normal vectors, only the points on the piping system are extracted from all scanned points P , by applying normal based region growing.First, a seed point is a set of neighbouring points contained in the sphere of the radius 1 r , centered at s p are added to the region.Each of the added points is then chosen as new seed points, and the other points satisfying the same condition are iteratively added to the region.The above steps are iterated until any point satisfying the condition resides in the neighbourhood of the seed points.
Then, as shown in Fig. 2, a covariance matrix is calculated using all points in one region, and the eigenvectors 2 1 , ˆe e and 3 ê are obtained in a similar way to Section 2.3.An oriented bounding box(OBB) of the points aligned to the eigenvectors is calculated.If the scanned points exist on a straight pipe, the second longest edge of the bounding box 2 l will become very small.On contrast, the points exist on planar surfaces such as floors or walls, 2 l will become larger than that of the former case.Thus, if the 2 l satisfies l l   2 where l  is a threshold to 2 l , the points in the region are classified as points on the piping system.Otherwise, they are classified as points on the other components.For distinguishing points on the pipe surface from those of planar surfaces in a stable way, n  was set to be 0.97 based on our experiments.Also, for enabling the system to recognize the largest pipe, l  was set as from simple geometric relation where max r is the largest pipe radius expected in the measured environment.

Recognizing straight pipes
The point clouds extracted from the process of 2.4 could include those of straight pipes, elbows and junctions.Therefore, based on the normal tensor analysis, only the points on straight pipes are distinguished from those of the elbows and junctions.
To extract the points on straight pipes, first, if grid cell i contains any scanned points on the piping system, the normal tensor i T is evaluated for a set of point ) , ( i i r N g using the equation ( 3) and (4).
where  becomes much smaller than 1  and 2  .When i g is close to an elbow or a junction, the normal vectors are distributed three-dimensionally, and 3  takes larger value than that of the former case.Thus, limiting 3  enables the points to be classified into ones on straight pipes and those on other elements such as elbows and junctions.This threshold value for 3  is automatically determined by the discriminant analysis method.
After classifying points into the ones on the straight pipes, the points on one straight pipe are furthermore integrated into a single region expressing the straight pipe using the region growing method.First, a seed point After grouping points on the straight pipes, cylinders are fitted to the groups and the axes of the straight pipes and their radii and length are estimated.First, a normal Gauss image is created from all points in a region.Then, a plane is fitted to the Gauss image using RANSAC, and an initial axis vector init v is obtained as the normal vector of the plane.In the RANSAC, the sampling number is 100, and the outlier threshold to the inner product of normal vectors is set as 0.01 in this paper.Then, all points in the region are projected to the plane, and a circle is fitted to the projected points by the least-square method.The center point and the radius of the fitted circle are respectively taken as an initial point on an axis init p and an initial radius init r of the straight pipe.Then, a cylinder is precisely fitted to the points of the group using the Levenberg-Marquardt Method (Shakarji et.al 1998).In this method, the parameters of a cylinder are expressed as ) , , , , , , ( the components of a position vector p of a point on the axis, and r the radius of the cylinder.Then an objective function of the fitting to be minimized can be expressed in equations ( 5), ( 6) and ( 7 Finally, the points adopted to the straight pipe region are projected to the cylinder axis, and two endpoints of the projected points on the axis are taken as the edge points, e and e .The line segment e e  is further used for recognizing connection relationships among pipes.

Recognizing connection relationship
In order to further discover unrecognized straight pipes and connecting parts, the connection relationships between the elements of the piping system are estimated by tracing the axis segments of straight pipes generated in section 2.5 and by the scanned points.

2.6.1
Tracing points on axes of a piping system: As shown in Fig. 4(a), first, a sphere is centered at one of the endpoints i e , and scanned points } { k p nearly existing on the sphere are collected from all scanned points.The radius of the sphere p r is set as , where k n is a normal vector towards the inside of the fitted cylinder.Third, as shown in Fig. 4(c), a cube space c V with a length p r 2 , where the center point is at i e is placed.Forth, c V is uniformly partitioned into a set of small voting spaces v v .Fifth, each offset point k p is voted for v v .If the number of the votes in the most voted space and its-one neighbouring 26 spaces exceeds half the number of points k p , an axis point i d is estimated as an average position of the points included in its 27 voting spaces.
However, in the scanned points near any attached parts, such as flanges, k p will not concentrate in any grid space as shown in Fig. 4(d).In this case, i d is determined as The above steps finish if either of the following conditions is satisfied; 1) any scanned point exists near a segment

2.6.2
Continuous arc-line fitting After recognizing the axis points of a piping system, a line or an arc segment is alternately fitted to these points using the method of (Bauer et.al 2009) in order to recognize the connection relationship.As shown in Fig. 5, first, a plane Q passing through three axis points ) 1 ( , , is obtained.Then, the subsequence of the axis points ] , , [ and their intersection point L are calculated.For each projected axis point } ,..., { 1 l and passes on k d is found by the following method.The tangent point 1 M between C and 1 l satisfies equation ( 8). 1 where t is a directional vector of the bisector line of 1 l and 2 l , and v a directional vector of 1 l , the equation ( 8) can be rewritten to equations ( 9) and (10).
The largest solution of the quadratic equation ( 9 However, arc segments which have very large radii are sometimes fit even for axis points of straight pipes.To correct these irrelevant fitting results, if consecutive line-arc-line segments are nearly collinear, the three segments are replaced with one line segment.

2.6.3
Complementing line segments on the occluded part First, pairs of nearly collinear axis segment .Moreover, if any arc segment is connected i e or j e , a junction may be falsely recognized as an arc segment.In this case, the arc segment is removed.

2.6.4
Complementing line segments on the junction After the complement of line segments on the same straight pipe, junctions which connect three pipes are identified.As shown in Fig. 6, first, a pair of line segments i q , where r was set as 1.1 times of i r .The new divided segments which connected i q are determined as a junction segment.

2.6.5
Complementing arc segments on the elbow After the complement of junction parts, elbows which satisfy the following conditions and could connect two mutuallyperpendicular pipes are identified.First, a pair of line segments , and both of two distances between Next, as shown in Fig. 7, radius of the tangential circle n C is changed step-by-step between 0 to M R , and for each step, a line-arc-line segments , and j j j n l e a   , are calculated.Then, two cylinder surfaces and a part of torus surface whose axes are coincident to the three segments are defined, and distance error between scanned points and these surfaces are evaluated.Finally, a combination of the segments which gives the least distance error is adopted.
After the recognition of axes of a piping system, all segments are classified to one of three classes; "straight pipe," "elbow," or "junction."These classes have defining parameters of the elements in the piping system from recognized axis segments."Straight pipe" class has the pipe radius, the pipe axis points, and the pipe direction."Elbow" class has the pipe radius, bending radius and bending angle."Junction" class has the pipe radii and angles between connecting line segments.

RESULTS
As experiments of the recognition, as shown in Fig. 8 The recognition accuracies of the straight pipes and the connecting parts are also summarized in Table 1 and Table 2, respectively.The recognition rate of the straight pipe was 87%~93%, that of elbows 76%~88% and that of junctions 73%~87%.When being compared them with our former research (Kawashima et.al 2011), the recognition rate of elbows was improved from 59.6% to 88.5%, and that of junctions from 46.1% to 87.0%.The calculation time was

Figure. 1
Figure.1 The overview of the proposed piping system recognition algorithm assigned to all scanned points in the grid cell.These eigenvalues and eigenvectors show spatial distributions of a Gauss image where the end point distribution of the unit normal vectors are drawn, as shown in Fig.3.For example, at a point i g is close to a straight portion of pipe as shown, the normal

spp
Figure.2 Calculating an OBB to the segmented points by normal-based region growing Points on straight pipe Points on T junction of a point on a straight pipe  N number of the grouped points If the radius r and average fitting error  of the scanned point to the fitted cylinder become  corresponding grouped points are dismissed as outliers.
r is the radius of the fitted cylinder.Second, as shown in Fig.4(b), a set of offset points } { k p located near the axis is generated from } point i d is considered to lie outside of pipes, 2) the consecutive fail estimation of i d repeat over the specified times.Finally, an axis point sequence ] region of pipe.

Figure. 5
Figure.4 Tracing axes of piping system ) gives the center point c and the radius R of C .From the solution, are edge points of the axis segment, u a unit direction vector of the line segment, and ij e a unit vector of a line connecting the two edge points i

q
are position vectors of the intersection points of the common perpendicular line of two lines , as shown in Fig.6, if the segment pair gives the distance is less than the pipe radius of the line segment divided into two segments at a place of distance r from defined, and j e is projected to the plane.A circle i C which is tangent to two lines larger value is adopted as maximum value of radius M R .
(a) and Fig.9(a), a large point cloud was scanned from an actual oil rig and a chemical plant by laser scanners.The point clouds had 4,524,324 points and 21,880,374 points, respectively.The thresholds of the oil rig recognition were  .The total recognition process took 413 sec and 2845 sec, respectively, using a standard PC (Core-i7 memory).The final recognition results of the cylinders fitting to the straight pipes and those of the connecting parts are shown in Fig.8(b) and Fig.9(b).
Figure.6 Complementing axes of the junction