Mathematical morphology directly applied to point cloud data

Many of the point cloud processing techniques have their origin in image processing. But mathematical morphology, despite being one of the most used image processing techniques, has not yet been clearly adapted to point clouds. The aim of this work is to design the basic operations of mathematical morphology applicable to 3D point cloud data, without the need to transform point clouds to 2D or 3D images and avoiding the associated problems of resolution loss and orientation restrictions. The object shapes in images, based on pixel values, are assumed to be the existence or absence of points, therefore, morphological dilation and erosion operations are focused on the addition and removal of points according to the structuring element. The structuring element, in turn, is defined as a point cloud with characteristics of shape, size, orientation, point density, and one reference point. The designed method has been tested on point clouds artificially generated, acquired from real case studies, and the Stanford bunny model. The results show a robust behaviour against point density variations and consistent with image processing equivalent. The proposed method is easy and fast to implement, although the selection of a correct structuring element requires previous knowledge about the problem and the input point cloud. Besides, the proposed method solves well-known point cloud processing problems such as object detection, segmentation, and gap filling.


Introduction
Mathematical morphology (MM) is one of the main techniques of image processing. It is based on object shape and set theory and was proposed and developed by Matheron &Serra in 1964 (Matheron andSerra, 2002). The basic and most widespread MM operations are morphological erosion and dilation, which combined give rise to morphological closing (dilatation followed by erosion) and opening (erosion followed by dilation). Its application in image processing is very useful for segmentation, refinement, feature extraction, and individualization (Alegre et al., 2016;Serra and Soille, 2012).
Unlike other techniques, such as Hough Transform (Borrmann et al., 2011), Region Growing (Vo et al., 2015), Connected Components (Trevor et al., 2013), Histogram of Oriented Gradients (HOG) (Rahmani et al., 2014;Ren and Sudderth, 2016), Bag of Visual Words (BOVW) (Yu et al., 2016), Scale-Invariant Feature Transform (SIFT) (Jiang et al., 2018) or Convolutional Neural Networks (CNN) (Balado et al., 2020;Griffiths and Boehm, 2019), MM has not made a clear transition to the 3D point cloud environment. Nevertheless, MM has shown its usefulness in solving problems with point cloud data through the transformation into 2D or 3D raster data. Morphological erosion has been used to measure the width of urban ground elements, such as sidewalks and threads (Balado et al., 2018). Morphological operators have been also used to segment curbs in point clouds acquired with Mobil Laser Scanning (Rodríguez-Cuenca et al., 2016). Building façades have been segmented based on the iterative application of basic operators such as erosion and dilation (Serna et al., 2016). The transformed top-hat coupled with a morphological opening has been implemented to detect urban objects (Serna and Marcotegui, 2014). Sidewalk and floor occlusions have been regenerated through an iterative morphological closure (J Balado et al., 2019;Jesús Balado et al., 2019;Raid et al., 2014). In Aerial Laser Scanning point clouds, morphological gradients (difference between erosion and dilation) are widely used for ground segmentation in Digital Terrain Models generation (Dong et al., 2012;Li et al., 2017Li et al., , 2015. In (Chen et al., 2007), point clouds are automatically registered by recognizing targets by MM. Skeleton of trees are obtained by implementing MM, through point cloud structured in voxels (Gorte and Pfeifer, 2004) or octrees (Bucksch and Wageningen, 2012).
Recently, point clouds are considered as an information model in itself, and not only as a data source to be processed to extract information for other models. The smart point cloud concept implies using and enriching point clouds with user-centered information rather than raw data (Poux et al., 2016). Continuing with the adaptation of image processing techniques for point clouds is fundamental to process the clouds without the need to transform other data types. Especially, mathematical morphology presents a set of operations very useful to process spatial data.
The structuring of point clouds to the 2D grid involves information loss, usually the Z component. Sometimes, it is possible to recover the information, saving the indexation of the points belonging to each pixel, as most of the works above mentioned do. But MM application is always done on 2D, or 3D, grid data that implies a resolution loss and the generation of "empty" pixels or voxels. The main problem that accompanies the MM application from images to point clouds is the adaptation of a 2D raster technique to 3D vector data. This implies a need to structure the vector data through neighbourhoods between points. Some authors have addressed the problem by converting the point cloud into a mesh or surface defined as point set surfaces (Alexa et al., 2001;Amenta and Kil, 2004;Guennebaud and Gross, 2007).
In (Calderon and Boubekeur, 2014), point clouds are structured as a closed volumetric element defined by its surface, and MM operations based on a structuring element are applied. Since it is a volumetric object defined by the surface, the object centre is considered as interior and the erosion/dilation is equivalent to expansion/contraction of the volume. This approximation of MM application to point clouds is not always possible since not all point clouds can be approximated as volumetric objects. Point clouds in remote sensing applications cannot be always approximated as closed volumetric objects. Point clouds resemble more a set of open surfaces. Other authors opt for a MM that only affects point cloud attributes, not varying the position and the number of points in space [30]. In this way, erosion and dilation operations copy attribute values (intensity, colour, label, etc.) between neighbouring points. These approaches do not provide a solution to problems of detection, segmentation, individualization, or gap filling typically solved by using MM in images.
The aim of this work is to design morphological operations directly applicable to 3D vector data, without point cloud transformation to point surfaces, volumetric objects, 2D or 3D grids. The main challenges are: • Adapt a method designed for images (2-dimensional structured data) to point clouds (3-dimensional unstructured data). • Adapt dilation and erosion operations from binary data (object and background) to the existence of points (object) or absence. • Redefining the structuring element (SE) based on point cloud features. • Adapt the overlap between SE and input data.
• Ensure complementarity between expansion and erosion to combine them in morphological opening and closing operations.
This paper is structured as follows. In Section 2, the proposed method is explained. Experiments are showed in Section 3. Section 4 is devoted to conclude the work.

Method
In image processing, Mathematical Morphology applied to binary images I(I m , I n ) ∈ {0, 1} is based on object shapes and their comparison with a Structuring Element U(U m , U n ) ∈ {0, 1} by means of set algebra. In order to preserve the shape evaluation, this process must be conducted on point existence, which produces the object shape. In the image, object and background are differentiated by the pixel value. In the point cloud, points are the object and the empty space is considered the background. Therefore, changes in binary pixel values are equivalent to the existence or non-existence of points. The two main input elements for applying MM to point clouds are the SE S = ( S x , S y , S z ) ∈ R 3 and the point cloud P = ( P x , P y , P z ) ∈ R 3 as a continuous threedimensional geometric space on which to apply the SE. In images, an object is defined by a set of pixels delimited by its border, both the object and the border have a different pixel value from the background. Point clouds are substantially different: the object is defined by the surface of points that forms it, the interior is empty, so, according to the previous definition, if there are no points inside, this empty space can be considered as background. This consideration is different from that of other authors (Calderon and Boubekeur, 2014) who consider data acquired in point clouds as full objects. But not all data acquired with point clouds can be assumed as closed volumetric models: (e.g. point clouds acquired with Aerial Laser Scanning, streets, or partially acquired objects). The authors have adapted mathematical morphology from image processing to work with point clouds as sets of points, not only closed objects. In addition, since MM is an exclusively geometric process, other point cloud attributes are not required, such as intensity, timestamp, RGB colour, etc. In this section, SE features, morphological dilation, and morphological erosion in point clouds are defined.

Structuring element (SE)
Likewise, in image processing the SE is an image, in point cloud processing the SE should be a point cloud S. For point cloud processing, the SE defined for images U(U m , U n ) ∈ {0, 1}must be adapted for conversion to S = (S x , S y , S z ) ∈ R 3 . Several characteristics from 2D or 3D vector data do not exist in U, and they should be defined. These characteristics are invariant throughout the iterative process of morphological dilation and erosion explained in the following sections. The SE is used in point clouds to analyse the presence or absence of points in P, therefore, since only geometry is relevant, the only attributes of the SE are 3D coordinates. The geometric characteristics that define the SE are: • Shape and size: These features also exist in image processing, but shape and size should be adapted to 3D environment. Both size and shape parameter values are dependent on the problem to be treated. In point clouds, the shape is not structured into a grid, the points can be distributed freely in 3D space. The hull of the SE points can be a linear element (1D), planar element (2D), or volumetric element (3D). Planar and volumetric elements can contain points only on their border or also on the inside (dense elements). • Orientation: In image processing, MM operations are performed on the image plane according to orientation on 2D axes and, in point cloud processing, on 3D axes. Both in image and in point cloud processing, the SE orientation depends on the problem to be treated. S and P must have the same 3D reference axes. • Density d S : In image processing, the pixel size between SE U and the image I are equal. In point cloud processing, it is very difficult for SE points to coincide exactly with the cloud P, because the density of a point cloud is typically variable. So a relation between P and S is established based on thresholds of spherical proximity between points in the local environment, not as an exact coincidence. The S density must be minor than P density or similar between S and P, d S ≲d P .
In image processing, the SE is iteratively displaced through the image I by analysing each pixel of the image with respect to the central pixel U. To obtain the same behaviour in point cloud processing, a reference point R in S is defined as the point that translates S to each Pi. R may not coincide with the geometric centre point of S.
The choice of the SE is a manual task based on the user experience. The same principle is followed in image processing. Since the SE as a point cloud can have infinite shapes, densities and orientations; the selection of the above mentioned characteristics is given by the problem to be solved and the geometry. Some guidelines for choosing the right SE are listed below: • The reference point R is the point on which the SE is built. To make the selection of the other characteristics easier, it is recommended to place R at the origin of coordinates of the SE. • The shape of the SE is given by the shape of the objects to be detected, segmented or the space to be filled (examples in Section 3.4). The shape should be as simple as possible, because complex shapes imply more points in S, increasing the processing time, and the probability of adding points in the morphological dilation or removing points in the morphological erosion. The shape can define a linear, planar, or volumetric SE. • The size of the SE must be the same order of magnitude as the size of objects to be detected, removed or gaps to be filled. • The orientation must be the same as the orientation of the objects to be detected, segmented or areas to be filled in the cloud.  • The point density of the SE must be as accurate as possible to the point cloud P to be processed, in order to maintain a correct relation.

Morphological dilation
Morphological dilation is defined as the process in which the object is expanded according to the SE. In raster space, dilation moves over background pixels and turns these into object pixels at places where the SE shape partially covers an object. Since in point clouds there is no "background points", dilation moves over object points and create new object points (as defined in the SE) around existing ones. Given a point cloud P and a structuring element S with a reference point R, the process of morphological dilation consists of the translation of R (with S) to each point P i of the cloud P and the addition of the points of S if there are no existing nearby points. Proximity threshold is defined as half the average distance dpp between nearest neighbors in P. The threshold value corresponds to an equitable division of the space between two adjacent points. The distance dpp is calculated as the mean of distances between each P i and its nearest five points, which can be considered as their adjacent points. As the addition of points is iterative, while checking the existence of points, S translated is compared with P and dilated points are added until the last iteration. The iterative dilation process is shown in Fig. 2, and the pseudocode is collected in Algorithm 1. The points in the cloud P are numbered to improve understandability. The order of processing points has few impact on the final result. The exact position of new point varies with the order of visiting the input points and addition of new points, but new points with different order of visiting are located within dpp distance.

Morphological erosion
Morphological erosion is defined as the process in which the object is reduced according to the SE. Erosion is the opposite operation of dilation. In raster space, it is an iterative process where the SE is moved successively to each pixel of the object, values of pixels corresponding to SE are checked and, in the case of not matching, the pixel value is changed. In the same way, given a point cloud P and a structuring element S with a reference point R, the process of morphological erosion consists of the translation of S to each point P i of the cloud P, checking the existence of points in P within a proximity threshold of S translated and eliminating P i in case there are no points P. The proximity threshold is defined as half the average distance dpp between nearest neighbors points in P. As the erosion of points is iterative, in checking the existence of points, S translated is compared with input P, no with iterated eroded point cloud E until the last iteration. The iterative erosion process is shown in Fig. 3, and the pseudocode is collected in Algorithm 2. In the same way as previous section, the points in the cloud P are numbered in order to improve understanding. The order of processing points has no impact on the final result, it only affects the order in which points are removed.

Experiments
This section compiles the results of applying MM to different artificial 2D and 3D point clouds, the Stanford Bunny model, and applications with real acquired data. The artificial clouds were designed to show a variety of cases in combination with different SE. Artificial clouds and their dimensions are shown in Fig. 4. The point position in the SE in Fig. 4.b-d is indicative. The number of points depends on the parameters l (length of the SE) and dse (distance between points); and varies according to both parameters.

Application of MM on planar data
The application of MM on a planar point cloud is intended to show the behaviour of the proposed method in comparison with 2D raster data. The artificial point cloud generated has a hole for concave and convex areas. Fig. 5 shows the morphological dilation on planar data of six different SE aligned with the input point cloud and defined by shape, size, point density, and orientation. In Fig. 5.a, a dilation was applied with an SE in the form of a line in X direction, with l = 1 m and dse = 0.1 m, according to dpp = 0.1 m. As can be observed, the cloud grew in the direction of the SE correctly with an equal density of original data. In Fig. 5.b, an SE line X was used increasing l = 2 m with the same dse = 0.1 m, so the number of points added was doubled in corresponding geometric space in comparison with Fig. 5a. Fig. 5c and d show results with  SE in 2D on the Z plane, both growths are according to the shape of the SE. In Fig. 5e and f, a SE line Z was applied with l = 1 m and with dse = 0.5 m and dse = 0.1 m respectively. In Fig. 5e, two separate copies of the original data were generated while, in Fig. 5f, it was generated more compact growth.
In Fig. 6, morphological erosion was applied with different SE on planar data. In Fig. 6a, the point cloud was reduced with SE line X with l = 1 m and dse = 0.1 m. In Fig. 6b and Fig. 6c, SE in two dimensions were applied on plane Z with l = 1 m and dse = 0.1 m, the difference in the result (point line existing in Fig. 6b) was due to the adjustment of the SE cross XY over the data gap. An erosion being SE a line Z would eliminate all points of the original cloud. Fig. 7 shows the morphological opening and closing operations. The opening is an erosion followed by dilation with the same SE. The closing is the opposite process, dilation followed by erosion with the same SE. In the morphological opening, being SE line X with l = 1 m and dse = 0.1 m, cloud was segmented by size in X direction and narrow areas were eliminated (Fig. 7a). With a cross XY, corners, or narrow areas were segmented (Fig. 7b). This can be explained by the fact that the first erosion eliminates points of narrow areas, and the subsequent expansion only adds points to the remaining points. In the Z-plane morphological closing being SE a square XY (Fig. 7c), the gap was filled. The first process of dilation filled the gap, and the erosion did not affect the inner edges since they did not exist after the previous dilation.

Application of MM on volumetric data
The application of MM in volumetric point clouds aims to show the behaviour of the proposed method on 3D data representing volumetric elements not necessarily being enclosed. The generated artificial point cloud had a continuous gap on three of its surfaces, different sizes, and two open sides. Fig. 8 shows the application of morphological dilation on the cloud based on SE aligned with the input point cloud and defined by shape (1D (lines) and 2D (planes)), size, point density, and orientation. The cloud grows according to the size and direction of the SE. The application of SE with a dse = dpp (0.1 m) generates more compact clouds. In Fig. 8a, the SE line X closed the gap and extended the cloud in the X direction. In Fig. 8b, the gap is closed and the surfaces grow in a compact way in the XY axes. In a similar way, it also happened in Fig. 8c, although due to dse = 0.5 m, there were discontinuities in vertical surfaces. In Fig. 8d, horizontal surfaces were duplicated in the Z direction, generating two copies. Fig. 9 shows the application of morphological erosion on the cloud based on SE of 1D (lines) and 2D (planes). The cloud was reduced according to the size and direction of the SE. The choice of dse values was not as relevant as in the dilation operation. The results in Fig. 9 were obtained with a dse of 0.1 m, but the same result was obtained with dse of 0.5 m. In Fig. 9a, the cloud was reduced following the X direction. In Fig. 9b, horizontal surfaces in the XY axes were reduced and vertical were eliminated. In Fig. 9c verticals were reduced in the Z direction and the horizontals were eliminated. In Fig. 9d the upper corner was detected from an SE with the shape and orientation of that corner. Fig. 10 shows the application of morphological opening and closing on the cloud based on SE of 1D (lines). In these processes, it is preferable to use a dse ≈ dpp, so that the expansion process generates data with a density equal to that of the existing data. By means of a morphological opening, parts of the objects can be segmented. In Fig. 10a, the narrowest area of the original data was eliminated and, in Fig. 10b, vertical surfaces were segmented. By means of a morphological closing, gaps were filled even in 3D. In Fig. 10c, the gap on the outer faces was filled and, in Fig. 9d, the inner gap was filled, forming a compact volume.

Application of MM on Stanford bunny point cloud
The Stanford Bunny (Turk and Levoy, 2005) is one of the most widespread models in the 3D data processing. In this case, the surface of the 3D model has been transformed into a point cloud. The bunny point cloud had 30 thousand points uniformly distributed on the surface. Morphological dilation and erosion have been applied to see the behaviour of the algorithm with respect to other works. Fig. 11 shows how the point cloud was dilated through the addition of points based on various SE. As can be seen, the bunny shape became more elongated when using a line-shaped SE and, in the case of using cubes or spheres as SE, the point cloud had a squarer or rounded shape respectively. The exterior result was similar to that shown in other works that propose the treatment of clouds as closed volumetric surfaces (Calderon and Boubekeur, 2014). Although only the surface points are shown in Fig. 11, the growth of the points in the present method also occurs inside the bunny point cloud. In the dilation operation, the growth was defined by the SE, therefore, by translation a sphere or a cube to each point and copying the points of the SE in the point cloud ( Fig. 11d and Fig. 11e), points were added both towards the outside and towards the inside of the bunny. If only the surface point cloud of the bunny is desired, it can be estimated by applying a convex hull on a dilated point cloud. Fig. 12 changes the erosion result significantly concerning other works. Three lines were applied as SE, each in the direction of one of the main axes. The resulting points coincided with SE in shape, density and orientation. The result from other works is a contracted the 3D object (Calderon and Boubekeur, 2014). As the proposed method did not work with closed volumetric surfaces, the algorithm could not distinguish between inside and outside of the bunny point clouds and, therefore, a contracted bunny point cloud could not be generated. The equivalent of contracted objects in point clouds can be obtained from a convex hull of the empty space inside dilated point clouds, since the dilation in this work not only occurs towards the exterior, but also towards the interior of the object. A cube and sphere were not applied as SE since the bunny has no parts that match these two shapes. The result of applying a morphological erosion with a cube or a sphere as SE eliminated the entire point cloud.

Application of MM on real point clouds
Although the main objective of this work is to provide the conceptual basis for the direct use of MM in point clouds, this subsection aims to show the value and potentiality of the method by applying it to real point cloud and to solve specific problems such as shape detection, segmentation, and gap filling.
Morphological erosion, since it is based on the comparison between an SE cloud and each point Pi, allows the detection of those points in P that coincide with the shape of the SE in a single operation. Greater complexity of the SE allows the detection of more complex shapes in P. To design the SE for morphological erosion, the shape and orientation of the object to be detected should be taken as a reference. Fig. 13.a shows a point cloud of stairs and ramp acquired with a terrestrial laser scanner Faro Focus 3D X330 (Khoshelham et al., 2017) with dpp = 0.022 m. In this point cloud, a morphological erosion was applied to detect the risers of the stairs, being the SE a 3D step of the same dimensions and orientation of the existing stairs: thread (Line Y with two points) -riser (Cross 2D in XZ with five points and central point as reference) -thread (Line Y with two points). The result and the simplified SE are shown in Fig. 13b. As can be observed, a step-shaped SE detected the existing steps in the cloud in that orientation.
The morphological opening is based on erosion, which detects the object, followed by dilatation, which reconstructs it. In the morphological opening, point densities between SE and P must be similar, since point cloud P is replaced by SE in each form detected. Fig. 14.a shows a traffic sign acquired with a LYNX Mobile Mapper of Optech (Puente et al., 2013) with dpp = 0.0078 m. In this cloud, a morphological opening was applied to segment the sign from the pole, being SE a Line Y and with l the width of the pole. The result is shown in Fig. 14b, the traffic sign was correctly segmented.
The morphological closing allows to close gaps in point clouds. It is very useful for detecting and regenerating occlusions. In morphological closing, the dilation first expands the cloud according to the SE. The morphological dilation is the process that fills the gap. The SE must be large enough to fill the entire gap. Afterwards, erosion only acts on the existing outside contours of the point cloud, not being the already filled gap because it has no contour since it was closed in the dilation. The aim of morphological erosion is to return the remaining cloud contours to their original locations. Fig. 15.a shows an urban point cloud with an occlusion produced by a parked car. This cloud belongs to IQmulus & TerraMobilita Contest dataset (Vallet et al., 2015) with an average dpp = 0.026 m with strong variations, in the area near the MLS trajectory dpp = 0.021 m while sidewalk area dpp = 0.062 m. In the point cloud, morphological closing was applied to fill the gap left in the road, curb, sidewalk and façade at the same time, being the SE a line Y with an l of the occlusion length in the Y axis and a point density similar to the cloud. The result is shown in Fig. 15b, the gap has been regenerated correctly according to the geometry of each urban element with only a single operation.

Discussion
In the design of the method (morphological dilation, erosion, and SE), different aspects have been prioritized. Image processing has been taken as a reference to adapt MM to the point cloud structure, without losing the applicability of MM and the capacity to solve the same problems as in image processing and new ones based on 3D environments. However, there are certain aspects of the method that give rise to discussion. Comparison with image processing and other 3D methods, the relation of the SE points to the input point cloud, processing time, point density, and orientation are discussed in this section.
Point clouds are a type of data with distinctive characteristics. The point cloud conversion to a point surface (Calderon and Boubekeur, 2014) implies a complete transformation of the same way that if a point cloud is transformed into an image. As mentioned above, the results on a point surface are more typical of closed volumetric surfaces and are completely different from those obtained from the direct MM application to point clouds. In this work, priority has been given to work directly on the points, without estimating other attributes, such as normals or dimensionality, and without considering the points as surfaces, 3D images.
Erosion and dilation for direct application on point clouds have been designed to be complementary to each other as in image processing. From an input image, different images can be generated by changing SE and/or morphological operations. These images of partial results can later be operated between them to obtain the final result. The combination of outputs is key to detect complex shapes that cannot be detected with only one operation. Typical applications with the combination are corner or contour detection. In corner detection, SE with different orientations are applied to input data do detect the different oriented corners. The contour can be obtained from the difference between the input and the dilated data, or the eroded data. The coordinate reference system remains unchanged after applied any morphological operation. From an initial point cloud, the clouds obtained after applying different SE and/or different morphological operations, can be combined with each other to detect complex shapes or with different orientations.
Priority was given also to a computationally efficient. In image processing, the MM application is instantaneous and it is made by applying masks or sliding windows. The point cloud equivalent is to run each point, which corresponds to an iteration. The most relevant operation in terms of computational cost per iteration is distance estimation to check the existence of points near to SE points. The solution provided has been centred on the calculation of point-to-point KNN distances between clouds S and P based on a threshold according to the point density dpp. An alternative could be the conversion of the SE into a polygon since point search within polygons is a more optimized process. But the polygon, in turn, must be distributed because the match condition between SE points and P must be given uniformly in the whole polygon.
Processing times depend on the number of points in P and S, as well as the MM operation. The code was run on an Intel Core i7 CPU 2.8 GHz with 16 GB RAM using MATLAB. The artificial point clouds, with about 2000 points were processed in less than 2 s. The Stanford bunny point cloud with 30,000 points took 14 s to be dilated with a line, 40 s to be dilated with a sphere of 500 points, and 13 s to be eroded with a line. With the acquired point clouds, the morphological erosion of the steps in a cloud with 70,000 points took 120 s, the morphological opening of the traffic sign in a cloud with 17,000 points took 10 s (5 s in morphological dilation and 5 s in morphological erosion), and the morphological closing of the occlusion left by a car in a cloud with 20,000 points took 2 h (only 9 s corresponds to the morphological dilation, erosion occupied most of the time). Most of the processes took little time to process, with the exception of the morphological closing for urban gap filling. The high cost of time is due to the number of points added in the dilation. The gap produced by the occlusion occupies most of the elements. The cloud went from 20,000 points to 300,000 points after dilation. The subsequent erosion took place over these 300,000 points and took 2 h. From this result it can be deduced that to apply a morphological closing to an entire street point cloud in a reasonable time, it may be preferable to reduce the point density of the input data before applying the morphological closing. In terms of complexity algorithm, the dilation and erosion algorithms have to visit all the points of the cloud (size n) and calculate neighborhoods with respect to a structuring element (size m) to add or remove points. Since the neighborhood search not focused on performance, the complexity of the algorithm can be expressed in terms of Big O notation as O(m⋅n 2 ), but the efficient search would take O (m⋅n⋅log(n)), which is the case in real applications.
In the design of the SE and given point cloud characteristics, adaptive orientation, and adaptive point density were considered, even if there is no equivalent image processing behavior. With the adaptive orientation, the SE could be self-orienting to each point P i . It would be necessary to implement a normal estimation, which would increase the processing time. In addition, the adaptive orientation could reduce the field of application by not maintaining a constant orientation of the SE as in image processing. Although it is a good option for the application of MM in environments whose orientation is unknown and will be analysed in future work, in most environments the orientation can be obtained based on previous knowledge, Principal Component Analysis or auxiliary data; or SE with different orientations can be applied (combination of outputs). In addition, the use of thresholds for the proximity calculation between S and P points also implies a threshold in the orientation of S with respect to P.
With regard to an adaptive point density of the SE to local point density of P, no clear need has been seen to adapt the SE in each iteration to the local density of P i . The adaptation would suppose in each iteration to calculate the local density around each point P i and to generate the SE according to this density, with the consequent increase in processing time. With an SE point density dse≳dpp, the algorithm offers consistent results. When the point cloud has strong point density variations, the condition dse≳dpp local cannot met and causes undesired behaviour of the dilation and erosion algorithms. Fig. 16 shows both behaviours on a 2D point cloud distributed on a logarithmic grid, with dpp = 0.8 m and dpp local between 0.3 m and 2.1 m. Being dse = dpp, the morphological dilation added points at point cloud borders but also re-densified areas where dse≲dpp local . Erosion removed points at point cloud borders and all points where dse≳dpp local . Re-densification in dilation and total elimination in erosion occurs because, in the check between S and P overlap, the point search based on dpp threshold finds no matching points, and considers the point as a point on the border (Fig. 17). In the case study of gap filling (morphological closing), the point density varied from dpp local = 0.02 m to dpp local = 0.06 m. With the application of an SE with dse = dpp = 0.02 m, low point density areas were re-densified in the morphological dilation and then, morphological erosion was applied in a constant dpp local = 0.02 m, so only border points were removed.
The result of applying a SE with dse≳dpp produces different results when the growth is in a direction perpendicular to the surface. In cases where the growth is in the direction of the surface, the application of erosion and dilation is compensated by the existence of points successively behind the border of the cloud, so it only changes the order in which points are added and eroded (dilation is exemplified in Fig. 18 and erosion in Fig. 19). The addition of points depends on the input point cloud shape and the number of points in the SE. The number of final points np D of the cloud D is greater than the number of initial points np P and less than np P by the number of points of the SE np SE : np P < np D < np PÂ ⋅np SE . Therefore, in view of the tests carried out, a dse≳ dpp is recommended in case of low point density variations and a dse≳ min(dpp local ) in case of strong point density variations.

Conclusion
This work presents a method that adapts mathematical morphological operations to direct point cloud processing. The main operations correspond with the basic binary mathematical morphological operations of dilation and erosion, that can be combined in opening and closing. This set of operations, implemented directly on a point cloud without an intermediate transformation to 2D or 3D images avoiding dimensional losses and orientation restrictions, contribute to consolidate the smart point cloud concept and consider the point cloud as a data model in itself and not as raw data. Also, a structuring element based on point cloud characteristics were defined. Although the operations are easy and fast to implement, the correct selection of the SE features (shape, size, orientation, point density, and reference point) requires prior knowledge of the problem and the input point cloud. The performance of mathematical morphological operations in image processing has been adopted as a reference and redesigned for direct use in 3D point cloud data. In analogy with a binary image pixel values, the existence or absence of points in the cloud was considered. Therefore, the processes of dilation and erosion correspond to addition and removing points according to iterative SE translation. A comparison between SE points and input point cloud was conducted on point-to-point proximity criteria with a point density threshold.
The proposed method is easy and fast to implement, as it is based on translational operations and point-to-point distance estimation. It was tested in four types of case studies: point clouds artificially generated based on a 2D and 3D grid distribution, a point cloud generated from 3D Stanford bunny model, and point clouds acquired with different laser scanning equipment. The method showed a behavior according to the existence of points, and it was clearly different from other methods that transform the cloud into point surfaces or that work on closed volumetric surfaces. The addition and erosion of points were in accordance with the SE and robust against point density variations provided that dse≳dpp local . Although priority was given to the design of an efficient computational method, the application to large point cloud datasets involves a significant processing time, but it works quickly with local case studies. The direct application on the point cloud provided solutions to detection, segmentation, and gap-filling problems without the need for conversion of the point cloud to images, not achieved by other point surface based methods.
Future work will focus on improvement and optimize the algorithm. Parallel computing techniques will be implemented to reduce processing time. Also, the code will be vectorised and the iterations will be on the points of the SE and not on the input point cloud, which would reduce the number of iterations. Another alternative to study is the application to only relevant points in the cloud, reducing the number of iterations. Independence of the method from the point density will also be investigated. More complex MM operations, such as skeletonization and tophat transform, will be introduced to solve the same problems presented in image processing.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.