POTENTIAL OF MOBILE MAPPING TO CREATE DIGITAL TWINS OF FORESTS

: Forests are irreplaceable and are being studied extensively. Better forest inventory and understanding necessitate effective mapping, modeling, and automatic analysis. As a result, considerable research effort is being devoted to digitizing forest environments. Recently, digital twins have come to the attention of the geospatial community as a virtual representation of the Earth’s surface linked to its corresponding physical asset. This concept is applicable to forests and has been studied in the literature. This requires initial input data obtained through reality capture. Among mapping techniques, laser scanning has emerged as a state-of-the-art technology for vegetation modeling. In this paper, we look into the potential of mobile laser scanning for forest digital twinning. While most studies concentrate on single tree detection, modeling, and estimation of dendrometric parameters, we also include lower vegetation in our investigations. To accomplish this, we first detect single trees and then investigate different vegetation densities and levels using geometric metrics. We also demonstrate how to model the underlying layers of vegetation in a digital twin. We perform the tests on data from mobile laser scanning (MLS) and compare the results to those from airborne laser scanning (ALS). We show that single tree detection based on crown separation using MLS data works similarly to or slightly better than ALS data. Furthermore, we demonstrate that MLS data allows for more detailed analysis of understory vegetation taking into account different height levels and a multi-level representation, whereas ALS data only allows for rough analysis of the lower parts of forest vegetation.


INTRODUCTION
Forests are a large and important part of the natural environment, essential for clean air and water and a place of recreation.As carbon sinks, they play an important role in the global carbon cycle, which is crucial for the self-regulation of the Earth's climate (Pan et al., 2011).At the same time, forests might be seen as an important source of renewable resources, because they provide lumber and are used for energy and material applications.As a result of the climate crisis, forests face changing water availability and temperature extremes.Understanding resilience and changes in forests due to this stress is part of current research.
In Germany, numerous federal states have produced an annual report on the condition of their forests.For this purpose, forestry experts go into the forest and collect data by visually interpreting and manually measuring the trees (Nordwestdeutsche Forstliche Versuchsanstalt and Hessisches Ministerium für Umwelt, Klimaschutz, Landwirtschaft und Verbraucherschutz, 2022).They concluded that many tree species have difficulties coping with the changing climate over the last few years, even species that have been native for hundreds of years.
In order to better map and analyse forests as well as understand the processes involving forests, there are attempts to digitize forest environments.One of the elements of this digitization is the reality capture part which is supposed to (1) create a model of this environment; (2) monitor the changes in forests.Among remote sensing sensors, LiDAR-based sensor systems turned out to be one of the mostly used technologies to map vegetation including forests.

Related work
LiDAR point clouds obtained by TLS (terrestrial laser scanning), ULS (UAV laser scanning) and ALS (airborne laser scanning) are a common source of information related to the geometry of trees.In order to retrieve this information on the single tree level, tree segmentation can be conducted (Weiser et al., 2022).In such segmented point clouds, some geometric parameters, such as height, crown area or trunk perimeter can be determined.In order to model the forest holistically, not only trees, but also understory vegetation should be captured and included in the model.The potential of TLS and ALS for extracting lower vegetation in forests has been already presented in the literature.(Huo et al., 2022) showed that TLS improves significantly detection of shrubs when symmetrical structure detection is applied instead of local maximum detection.(Ferrara et al., 2023) combine TLS and ALS to map understory vegetation density.
Despite high quality of the data and its high potential for the forestry, TLS can be applied only locally as a time-consuming acquisition technique.In recent years, we observe rapid development of terrestrial mobile mapping platforms, such as vehicles, but also backpacks or hand-held devices.Their application for reality-capture tasks in forest environments can open new possible views on forest vegetation including trees and bushes.Extending the regular inspections by forestry experts with such mobile mapping systems could improve the efficiency of the survey and bring new possibilities to model the forests.
Instead of visual interpretation and specific manual measurements by human effort, technological advancements meanwhile allow the use of backpack mobile mapping systems for acquiring a dense 3D point cloud by simply walking through the scene.Such backpack systems typically involve one or more laser scanners (Nüchter et al., 2015, Blaser et al., 2018) and have also been proposed for the acquisition of a 3D counterpart of a forest area (Blaser et al., 2020, Blaser et al., 2021, Hartley et al., 2022).The acquired 3D data can in turn be interpreted by means of automated processes and deliver relevant information for creating a digital twin of a forest.For this purpose, standard geometric 3D features may be used (Weinmann et al., 2017a, Weinmann et al., 2017b).These may contain metrics derived from the eigenvalues of the 3D structure tensor, e.g. in terms of linearity, planarity, sphericity, omnivariance, eigenentropy, anisotropy, sum of eigenvalues or surface variation.Furthermore, other geometric properties like height differences or standard deviations of height values may be taken into account.Features specifically dedicated to forest environments have been addressed with descriptors related to height, cover, and structural complexity of an ecosystem (Valbuena et al., 2020, Coops et al., 2016), a combination of radiometric information as well as descriptors of vegetation cover, height and vertical variability (Koma et al., 2021, Moudrý et al., 2023), structural metrics derived from LiDAR returns from canopy and terrain features (Carrasco et al., 2019), or a combination of landscape composition, fragmentation and configuration metrics (Zieba-Kulawik and Wezyk, 2022).Such features may in turn be used to enrich acquired 3D data with semantics up to tree level, to reason about essential biodiversity variables, or to conduct systematic and regular monitoring for assessing forest condition and vulnerability.

Contribution
In this study, we investigate the advantages of LiDAR-based mobile mapping systems (MMS) in forest inventory tasks compared to airborne laser (ALS) scanning as well as discuss their complementary properties.For this, we investigate state-of-theart methods to extract vegetation levels as well as individual trees and vegetation geometric information, which are used as processing steps for digital twinning of forests.In contrast to other studies (Liang et al., 2016, Nit , ȃ, 2021), we apply those methods to mobile laser scanning (MLS) and ALS data and compare the results, particularly regarding the detectability of bushes and shrubs.For this vegetation layer, we also propose and analyse a data preparation approach for digital twins.Finally, we investigate and discuss the potential of fusing the ALS data with MLS data to combine their advantages.

METHODS
In this work, we outline and discuss digital twinning of forests based on a literature review and implement an exemplary pipeline.In particular, we investigate two approaches to segment the forest and park areas.First, we investigate the common segmentation approaches for extracting individual trees and derive the information such as tree height and width Then, we investigate different geometric metrics and divide point clouds into different vegetation densities and levels.The goal of this investigation is to show the potential of this approach to detect bushes and shrubs.Finally, we register airborne data with mobile mapping data in order to investigate the complimentarity between both data modalities.

Digital twins of forests
A digital twin can be considered as the digital counterpart of a physical asset.According to (Grieves, 2014), the digital twin concept model comprises a) the physical asset, b) the corresponding virtual asset, and c) the connections that link the virtual and real assets to each other.Thus, a digital twin extends a digital model by its connection to the corresponding physical counterpart.
Regarding the creation of a digital twin of a forest, a digitization of the given ecosystem including different entities and underlying processes is required.In this regard, there are different opportunities that depend on the given application and thus may vary with respect to numerous factors such as the used input data, the defined representation, the level of detail, the considered entities or the considered processes.For instance, a 2D representation of large forest areas may be used for addressing large-scale applications related to the benefits of a forest for the local climate.On the other hand, for instance, a detailed 3D representation of a forest area allows for considerations on both tree level and forest level.On tree level, relevant characteristics are given by tree position, tree species, tree height, tree structure, stem curve, diameter at any height, or microhabitats.On forest level, relevant characteristics could be represented by the number of trees, tree cadaster, relative occurrence of tree species, biomass distribution or health condition, but also the composition of different layers could be meaningful.
In the scope of this work, we are mainly interested in the middle layer of forest vegetation which comprises shrubs and bushes.The latter are important for many aspects such as flooding (Leyer et al., 2012) and wildfire simulations or observation of rejuvenation of forest trees.Hence, we need to focus on the acquisition of detailed 3D data which is achieved by the use of a backpack-based mobile mapping system.The acquired data in turn needs to be enriched by semantic information, e.g.addressing single trees and their characteristics.Accordingly, a segmentation of single trees as well as the assessment of tree top and tree canopy could be important, but also the analysis of different layers of the forest.

Segmentation and tree top detection
In order to detect single trees in point clouds, typically methods based on segmentation are utilized.We select an approach which separates the ground points, detects tree tops and segments the point cloud into single trees.
For this, we select the forest area from ALS point cloud, then use the simple morphological filter (SMRF) algorithm (Pingel et al., 2013) to segment ground and non-ground points and extend it with Delaunay triangulation to estimate the ground level at each point in the ALS point cloud data.Then, we normalize the elevation of each point by subtracting the interpolated ground elevation from the original elevation.

Subsequently, we generate the Canopy Height Model (CHM).
For this, we use the previously normalized elevation values and exclude lower and medium vegetation.Then, we use a local binning algorithm which divides the point cloud into a xy-grid and we create a binary marker image with tree top locations and filter the CHM complement by minimal imposition to remove minima that are not treetops (Pitkünen et al., 2004).Finally, we use marker-controlled watershed segmentation (Chen et al., 2006) to segment individual trees.

Forest layer analysis
The classical approaches for detecting single trees, such as the method presented in Sec.2.2, exclude low and medium vegetation (typically points below 2 m) in order to improve the Simply cutting the point cloud into slices of those levels might not be the optimal approach, since the tree stems and parts of the shrubs can also be represented by points below 2 m.
In this study, we define the height level classes dynamically by observing the border between the vegetation levels in the given data rather than choosing a fixed height.As shown in (Goebel et al., 2023), geometric metrics calculated on the 3D point cloud, such as normal change rate, planarity, sphericity, linearity and surface variation, differ depending on height levels of forest vegetation.
All metrics depend on a radius r which describes the included spherical neighborhood and has to be selected according to the point density.If a point has less than two neighbors in the defined neighborhood, the metric cannot be calculated and this point is not included in the results.This way, outliers are automatically excluded.If the radius is too small, the parameters for many points cannot be calculated, because there are not always 3 points within the radius.In addition, the informative value about large-scale structures is reduced.If the radius is too large, subtleties of the point cloud will not be represented, such as distinctions between trail and grass areas.A uniform radius over all metrics favors good comparability.
In this research, we select the metrics roughness, sphericity, volume density and normal change rate for our investigations (Tab.1).Roughness describes local curvature behaviour of the curves and surfaces, or deviation from the plane.The normal change rate represents angular change between the normals of local surfaces.The degree to which an object's shape resembles that of a perfect sphere is determined by the sphericity.
In order to create a model for understory vegetation, we slice the point cloud and rasterize our data in each slice.We conduct this only for slices representing the medium vegetation.In some sense, we voxelize the space with understory vegetation, whereas the voxels are not restricted to be cubes.Each voxel gets assigned the metric value which is calculated as average value of all points being within this voxel.

Extraction of lower vegetation
As the result of the layer analysis, we observed, that the geometric metrics are useful for separation of different parts of vegetation.Sphericity, for instance, is helpful to separate parts of the point cloud with very low vegetation, such as meadows as well as tree stems, from those not belonging to vegetation, such as ground.The value of sphericity for those surfaces is lower than those for shrubs, bushes and tree crowns.Volume density, instead, helps to separate tree crowns from the understory vegetation by excluding points with a low volume density.
We propose a method for extracting medium vegetation that is based on sequential sphericity and volume density thresholding with an adaptive threshold.At this stage of the research, the adaptive threshold is empirically extracted from the graph that displays the metric change over height, but this step has a high potential for automation.

Coregistration of MLS and ALS data
In order to compare or use MLS and ALS data together, they must be represented in the same coordinate system.Direct georeferencing of the MLS data is often not sufficient for direct fusion of ALS and MLS data.Thus, we perform a coregistration procedure.Having approximate georeferencing of the point cloud from GNSS data, a procedure for the refinement is needed.For this, we use the iterative closest point (ICP) approach (Besl and McKay, 1992).The coregistered data is used to investigate the complementarity of the ALS and MLS data.

Data acquisition
For the experiments, two data types were used: ALS and MLS.
The ALS data was collected and processed by order of the State of Hesse and published by this institution.ALS data is available as digital surface model (DSM) and seven-responses point cloud (ALS-7R).For our investigations in this study, we selected ALS-7R data, because it penetrates the vegetation and points related to bushes and shrubs are included.ALS data was collected in the winter months of 2016.
The MLS data was collected using a multi-sensor backpack, including two Velodyne VLP16 LiDAR and an Xsens Mti 700 IMU (Internatial Measurement Unit).With this mobile mapping system we collected data beneath the canopy.The MLS data was collected in 2022 in multiple measurements between summer and winter.In this paper, we selected subset of the data from August 2022 as representation leaf-on data in summer months and from December 2022 as representation of leafoff data.More specifically, we use three following subsets of the data: (A) point cloud collected in August 2022 in a park with low density vegetation; (B) point cloud collected in August 2022 in a forest with high density vegetation; (C) point cloud collected in December 2022 in a park with medium density.All data is situated in Darmstadt, Germany.Forest layer analysis is conducted with all three subsets.Comparison with ALS data is conducted using only winter data in order to ensure comparability with ALS data,which was collected in winter.

Data selection
For the presentation of the results and the evaluation we selected an area, for which MLS and ALS were available.After coregistration, exactly the same area was cut out from MLS and ALS data (Fig. 1).The number of points in the MLS data is about 13 million and for the ALS data about 100 thousand points.

Implementation
For single tree detection, we used in Matlab the functions from the LiDAR Toolbox.Points below 2 m in height were excluded before generating the model.To implement the forest layer analysis and coregistration, we used the software CloudCompare.The radius for calculating the geometric metrics was set to r = 0.5 m for MLS and r = 1 m for ALS with regard to the point density and expected size of object details.For the extraction of lower vegetation, we used sphericity and excluded values below 0.2.Hereafter, we calculate the volume density with a radius r = 2 m and excluded points with volume density values ≤ 300.

Quality investigations of MLS point cloud
In order to investigate the quality of the MLS data, we perform precision investigation in the measured point cloud.For this purpose, we select an object, which is easy to identify in the point cloud.We select a fence from our data set and measure it five times in the point cloud.The reference height was measured with a ruler in the field during the data acquisition.
From those measurements, we calculate mean height, RMSE and standard deviation (Tab.2).Calculated RMSE and standard deviation define the measurement precision of about 14−16 cm.We can see, that the difference in the measured mean height measured in the point cloud and the reference height (10 cm) is within the measurement precision.

Results on single tree detection
First, we perform tree top detection and single tree segmentation on both selected test data sets: MLS and ALS.Results of this segmentation are presented in Fig. 2.
In the ALS point cloud 31 trees were detected, compared to MLS point cloud with 27 tree tops.However, when comparing the results in detail, we can see that the algorithm groups the tree differently.This is highlighted in Fig. 2 with white and orange circles.As shown from the side in the highlighted part of Fig. 2 (bottom, white circle), the three individual trees in MLS point cloud are recognized and segmented well, while in ALS point cloud, there is one tree recognized as two.To verify the results of the automatic tree detection, we counted the trees twice manually with the result of 29 and 28 trees respectively (Tab.3).
The difficulty with the manual counting of the trees and, consequently, differences in the two independent attempts is related to the fact, that in the test area, many trunks are split very close to ground and it is not easy to decide visually, whether it is one or more trees.Besides, in this area, many tiny bushes are present, which can be also confused with trees.
Furthermore, we observed that large trees are correctly segmented in both data sets.Most problems with segmentation occur when trees are close to each other.Additionally, the MLS data contains more information about the vegetation beneath the canopy, as it was collected from the ground.This supports the decisions made during the segmentation computation.However, in this process, lower vegetation is not regarded as the focus lies in the segmentation of trees.

Results on forest layer analysis
To analyse the understory vegetation, we calculated simple geometric features.We have cut point clouds into 0.5 m thick slices, to see changes of the metric values based on height.Fig. 3 shows results on normal change rate calculation using MLS in park and in forest environment and its distribution over height.In the first one meter, the normal change rate value increases drastically.In both data sets, a drop of this value can be observed between 2 and 4 m height.Above 6 m the value stays stable with exception of few values close to maximum height.We observed in our investigations, that normal change rate and sphericity behave the same way, as they both describe curvature, and thus can be utilized in a similar way.
We calculate sphericity and roughness on the same point cloud as used in the tree segmentation.The results are shown in Fig. 4. The outcome is similar to the one in (Goebel et al., 2023).Focusing on the lower vegetation, we cut out the point cloud from 0.5 m to 2 m, which is the middle vegetation defined above.Fig. 5 shows that sphericity can be well used to distinguish ground and low vegetation.On the contrary, the metric roughness does not have a strong difference between ground and vegetation.
Based on the calculated metrics using MLS, we generate an understory vegetation model of the vegetation from 0.5 m to 2 m.We divide the data in three 0.5 m slices and rasterize each slice with 0.5 m pixels size.For comparison, we generate the same results for metrics calculated using ALS data.Here, however, due to the much lower point density we rasterize the slices with a 1.0 m raster.The results of this process are shown in Fig. 6 on the example of sphericity.

Results on extraction of lower vegetation
Finally, Fig. 7 depicts the extraction of the lower vegetation.The point cloud is reduced from 13 million to 5 million points

Results on coregistration
For the evaluation of the coregistration, we use Hausdorff Distance (HD) and Mean Hausdorff Distance (MHD), which describes the distance between two point clouds.After coregistration using the ICP approach, HD = 7.5 m and M HD = 1.2 m.

DISCUSSION
Single tree detection works well with both data sets: MLS and ALS-7R.The methodology used in this paper was developed to cope with airborne data.Therefore, such advantages of MLS data, as dense point cloud and high level of detail in the bottom part, remain unused.The algorithm focuses on the tree crowns and the borders between them.In addition, all points below  2 m are removed.Yet the segmentation of MLS point cloud works comparable, or even better than segmentation of ALS point cloud.
In Fig. 8, we can see the errors in the segmentation of MLS point cloud, which are related to the reasons mentioned above.
In the first example in Fig. 8 (top), a single tree was recognized and segmented as two trees.Considering points below 2 m would help to understand the geometry of the tree and segment this tree as one object.In the second example (bottom) a single tree is well segmented, but some branches of other trees were added to this tree.Here, algorithms which trace the branch structure would be helpful for better segmentation of the trees.
This shows the need for further development of the methods for single tree detection for MLS data, which focuses on stem detection and modelling rather than on crown separation.Stem diameter is an important tree parameter to be included in a digital model of the forest, which can be obviously better measured in MLS data.Although our MLS is a ground based approach and the highest point density is about the ground, still the crowns can be entirely captured and their shape can be modelled.We can observe in our examples that this applies for winter and summer data sets.In our case, the average point density of MLS is more than 100 times higher than of the ALS-7R point cloud, which enables this high level of detail also at the tree tops.This shows that path planning is important for using mobile mapping systems in the forest, since it determines the point cloud density, its level of detail and it can help to avoid occlusions.
The main advantage of MLS for forest modelling, however, is its much higher level of detail regarding the lower and medium vegetation.Using MLS, bushes and shrubs can be better captured and also appropriate vegetation models can be developed.
A first analysis shows that the metrics planarity, normal change rate and sphericity are useful for a separation between ground and vegetation.A separation between medium and high forest vegetation can be done well in a low density forest.
To further use the results of the forest layer analysis, the representation shown in Fig. 6 can be very useful.Here, the vegetation properties, in this case sphericity of the point cloud, can be represented in a handy way and can be used as input for simulation of different processes (e.g.flood simulation).In these results, we can see that MLS delivers much more information regarding the underlying vegetation levels, as compared to ALS data.The differentiation in 0.5 m slices can be reasonably done only in the MLS point cloud, as such refined slicing causes gaps in ALS data.
Because the ALS data was collected from above the canopy, the lower vegetation is not included in detail.However, ALS can cover a much larger area in one campaign, than a mobile mapping platform in the same time.Fusion of those two data sets has potential to improve large-area ALS-based acquisition and extend them with MLS data for selected areas of interest.
The achieved results for coregistration are acceptable for the test area selection, as described Sec.3.2, but for a fusion of both data sets, an accuracy improvement would be needed.
Moreover, a further investigation on the reason for the still high HD after coregistration could be helpful.It should be considered that the ALS data originate from 2016 and the MLS data from 2022.Already this time discrepancy would cause differences in the point cloud which can explain the remaining error.

CONCLUSION AND OUTLOOK
In this paper, we investigated the potential of MLS to create digital twins of forests.In particular, we focused on the medium vegetation, such as shrubs and bushes.We observed that the main advantages of MLS is its very high point density compared to ALS-7R, which was about 100 times higher in our case.Although this data acquisition type is groundbased, the crowns and tree tops are mapped completely enough to use tree segmentation approaches based on crown segmentation designed for ALS data.At the same time, ground-based acquisition allows for detailed mapping of tree stems, bushes and shrubs.We showed that using MLS allows for a slice-wise analysis of forest vegetation, while ALS-7R data enables only rough analysis of the lower parts of a forest's vegetation.In case of MLS, we observed that using geometric metrics can contribute to dynamically separate different vegetation levels, instead of using predefined height thresholds.
Much more research is needed to find out, which geometric metrics represent the data the best, according to a particular application.For flood simulation, for instance, the force of vegetation to water flux is of high interest (Wunder et al., 2011).
In the future, it should be investigated which of the metrics or which combination would deliver the most reliable simulation results.Furthermore, for digital twinning, more flexible data storage and access are needed.Newest developments in this field include data spaces (Gaia-X, 2023), which should also be considered in future work.

Figure 1 .
Figure 1.Selected study area.MLS (test area C) colored in height (blue to red) and ALS point cloud, uncut, colored in height (black to white).

Figure 2 .
Figure 2. Single tree segmentation results, test area C

Figure 3 .
Figure 3. Normal change rate (radius = 0.5) based on the MLS point cloud in the Park (MLS subset A) and Forest (MLS subset B): Results based on height (top), visual results in the park (middle) and visual results in the forest (bottom).Histograms of the value distributions are depicted on the righthand side of the legends.

Figure 4 .
Figure 4. Point clouds (MLS subset C) colored by the metrics sphericity and roughness, with the radius in brackets.(first row)The point clouds are colored by height.

Figure 5 .
Figure 5. Lower vegetation colored in sphericity and roughness, with the radius in brackets.The point clouds (MLS subset C) include heights from 0.5 m to 2 m.

Figure 7 .
Figure 7. Low vegetation extraction result on MLS point cloud C. Background is the whole point cloud (MLS subset C) colored by sphericity as grey scale.

Figure 8 .
Figure 8. Two cases of segmentation results for the MLS point cloud (subset C).Point cloud coloured by intensity (left) and segmented into single trees (right).

Table 1 .
Overview of selected metrics

Table 2 .
Result on quality investigation of MLS point cloud.

Table 3 .
Number of detected trees