THE ARCHITECTURE OF A STEREO IMAGE BASED SYSTEM TO MEASURE TREE GEOMETRIC PARAMETERS

: Stereo photogrammetry enables collecting precise and detailed three-dimensional data of terrestrial objects. The estimation of qualitative and quantitative tree attributes, in particular those related to geometric measures, is crucial for forest management. In this study, a stereo imaging system is designed in order to measure a set of geometric attributes of urban trees such as crown dimensions, height and diameter at multiple height levels. The system consists of two hardware and software components. The hardware comprises two cameras with a specified baseline, two raspberry pi 3 model B+ boards, a GPS, an IMU and a power bank, all embedded in a box. The software includes a connection between the camera and the raspberry pi 3 in each side as well as data transfer to a laptop. The calibration is conducted in laboratory prior to applying the system and leads to achieve a disparity image from a pair of stereo imagery, which is then processed to extract dense point clouds. The system enables measuring basic, yet crucial tree attributes such as height and diameter in near real-time basis. The entire process is conducted by means of drastic libraries in Robot Operating System (ROS). Apart from being convenient and real-time, the system is associated with the potential for timely and precise measurements, which enable comparative analysis against other existing remote measurement systems as well as reference field data.


INTRODUCTION
Urban vegetation has many benefits including energy saving, water management improvement, air pollution decrement, and citizens' connection to nature (McPherson, 2006).To maximize these benefits, urban vegetation recovery and monitoring are continuously required.
Basic information for monitoring urban vegetation can be subdivided into number of single trees, tree species, spatial distribution of trees and trees health conditions.In fact, the tree geometric parameters provide researchers precise information about basic vegetation information, which calls for monitoring single trees and their structural status.(Ferrini, 2017) indicated without accurate information on the location, structure and condition of urban trees, it is impossible to effectively manage and evaluate urban forest ecosystem services.Previously, (Novak et.al. 2008) concluded that the question of how to measure in an urban forest has multiple facets depending on the field or purpose of measurement.A set of comprehensive variables is needed to improve urban forest management.This information may require more detailed structures to evaluate ecosystem services.(Ferrini, 2017) also demonstrated that the measured variables can fall under the categories of location (latitude, longitude), 2D structure such as diameter at breast height (DBH), height, crown spread, 2D structure (e.g.crown area), 3D structure (e.g.volume), form (e.g.crown shape, lean), condition (e.g.defoliation, leaf chlorosis), and risk (e.g.cracks, weak attachments).
Multiple references stated that the existing methods for measuring the tree geometric parameters can be classified manifold into destructive or non-destructive, direct or indirect, contact or non-contact, precise or general, expensive or low cost and time consuming or fast.The choice of referring to either of these categories are largely based on the user´s interest as well as on management requirements and goals.
Although destructive techniques are direct and precise, they are prohibitive when it comes to repeated measurements.The increasing demand for precise information on individual trees and stands calls for development of more accurate terrestrial systems for reference and tree inventory data collection.(Clark, 2000) reviewed multiple types of forest inventory instruments that exhibit varying properties, for example in terms of their measurement precision, accuracy, operational simplicity, and cost.The hitherto applied methods can be divided into two categories of contact (tape, caliper, electronic tree measuring fork) and non-contact (optical calipers, rangefinder dendrometers, optical forks) methods.As noted by (Mikita et.al. 2016) the contact methods allow for the highest accuracy in individual tree measurement.However, owing to issues like time constraints and safety, they are limited to measurements conducted on the lower tree boles.Meanwhile, noncontact methods are considered as inaccurate or expensive in terms of both time and costs.This calls for development of complementary or alternative, geo-located methods for basic as well as simultaneous, multi-attribute tree measurements.
There are several three dimensional platforms and tools that result in data used for tree and forest measurements, as reviewed by (Latifi, 2012) in a review.The existing data, including those from satellite, aerial and terrestrial platforms to measure forest characteristics, are associated with several additional methods, fusions and algorithmic procedures for specific applications.In particular, terrestrial remote sensing technologies like those based on photogrammetry and laser scanning are being used more frequently to accurately measure various attributes of urban trees and forests.The TLS 1 is accurate enough, but due to the cost, time and heavy equipment, is limited to specific applications.Thus, there are alternative techniques like terrestrial photogrammetry (with its different variations) to estimate tree geometric parameters.
For example, (Javernick et al., 2014;Westoby et al., 2012) pointed out that advances in computer vision technology have led to the availability of a common photogrammetric methods like Structure-from-Motion (SfM) that, when combined with Multi-View Stereo-Photogrammetry (MVS), is a semi-automatic and Provides low cost for the production of high-resolution three-dimensional point cloud of two-dimensional image collections.In addition, (Miller et al., 2015a;Morgenroth, 2014) have used SFM-MVS for research applications to evaluate urban tree geometric parameters.(Miller et al., 2015a) calculated the ability of the SFM-MVS to reconstruct accurate 3D models from which twodimensional (height, canopy width, crown depth, trunk diameter) and three-dimensional tree (volume) estimates.To evaluate the estimated SFM-MVS of trees, 30 small greenhouse planets were photographed and measured using traditional dendrometry and the results were compared.The results show that SFM-MVS is capable of producing two-dimensional and three-dimensional estimations with comparable accuracy to laser scanning.Factors such as the position of the tree relative to the environment and Light changes appear to influence the model's success.The study found that the technique is cheaper than TLS and equipment needed to be used by people with less expertise or training.
According to (Hosseininaveh Ahmadabadian et al., 2013b) Photogrammetric methods for dense 3D surface reconstruction are increasingly available to both professional and amateur users who have requirements that span a wide variety of applications.They also have intimated that Multi-View Stereo (MVS) can be a comparable with laser scanners as a low cost technique for precise 3D reconstruction if the scale of the model is resolved (Ahmadabadian et al., 2014).A fusion of stereo imaging equipment with photogrammetric bundle adjustment and MVS methods, known as photogrammetric MVS, can generate correctly scaled 3D models without using any known object distances (Hess et al., 2014).Although a huge number of stereo images (e.g.200 high resolution images from a small object) captured of the object contains redundant data that allow detailed and accurate 3D reconstruction, the capturing and processing time is increased when many of high resolution 1 Terrestrial laser scanner images are employed.Moreover, some parts of the object are often missing due to the lack of coverage of all areas.These problems demand a logical selection of the most suitable stereo camera views from the large image dataset (Hosseininaveh et al., 2014, Hosseininaveh et al., 2012, Hosseininaveh et al., 2016).Recently, many software packages have been developed working based on photogrammetric multi-view stereo strategies with reliable and fast performance.These packages have been reviewed and compared by researchers based on accuracy and completeness criteria (Ahmadabadian et al., 2013, Hess et al., 2014, Mousavi et al., 2018) (Kamil et.al.2019) compared the TLS and single image photogrammetry methods to estimate the properties of single trees and concluded that the single image photogrammetric method could be successfully used to extract the architectural features of trees in mature forests.Furthermore, (Shimizu et.al, 2014) has estimated tree diameter using digital camera and image processing software.Since this method requires measuring the distance and height of the target point in the image, it uses a range finder to measure the angle between the camera and the tree.This study may be considered a more advanced version of the study by (Takahashi et.al., 1997).(Mokroš et.al. 2018) investigated the effect of camera lens, tree species, diameter and height on the accuracy of tree diameter by close range photogrammetry.That study was conducted using two camera lenses and included measuring three tree species with their heights.Data were collected from a tree in a 3 m circle around the tree.The best accuracy was obtained when the height was 1.3 m and the fish eye lens used.(Roberts et.al.2018) has focused their study on the extraction of the geometric properties of a single urban tree by means of a ground photogrammetric method.The trunks of the larger modeled DBH trees were not well estimated, which can be attributed to problems of estimating the diameter of non-circular trunks of speciesspecific urban species (e.g.Quercus virginiana).Last ut not least, (Berveglieri et.al. 2016) applied fisheye lenses to extract the three-dimensional tree model.The tree stems were reconstructed using the SFM to allow measurement and extraction of several features.The results showed that 3D reconstruction using vertical camera is as good as the SFM method with the vertical camera, but the advantage of this method was the use of fewer images as well as the use of one shooting station rather than the SFM.
It should be noted, however, those studies used photogrammetric methods, but are still considered as contact methods because of their dependence of the target on the tree location.Moreover, the need for further postprocess these methods is obvious.
A stereovision system enables to provide 3D field imagery by combining two monocular field images taken simultaneously using a binocular camera.The main advantage of stereoscopic over conventional monocular vision is its ability to detect ranges, i.e. distances between scene objects and the camera.(Kise et al., 2005) previously demonstrated a computer stereo vision implying the extraction of 3D information from digital images obtained by a Charge-Coupled Device (CCD) digital cameraa.
Therefore and by means of stereo photogrammetric techniques, this paper mainly focused on scheming a stereo image based system designed for real-time, lowcost tree measurements, with a simple architecture and nominal accuracies comparable to the existing contact measurement methods (Figures 1 and 3).The aim of this study is to provide a stereo-based photogrammetric system for estimating tree geometrical parameters.The system instantly captures the image and estimates the parameters.In fact, due to the existence of raspberry pi as the processor, this system can be considered as a robotic photogrammetric system.
The following section describes the architecture of the proposed system in both hardware and software aspects.Considering the importance of the subject, an attempt was made in addition to develop a system to estimate the geometrical properties of a single urban tree, obtaining the exact position of the tree in the global positioning system using GPS and IMU which is presented in section 3. Finally, a conclusion with suggestions for future work is given in section 4.

THE SYSTEM
The current set-up comprising two raspberry pi and a number of packages in Robot Operating System (ROS) as open source middle ware.The system is leveraged to create dense point clouds used for geometric tree measurements such as crown and trunk height and diameter.The following flow chart, also demonstrates the typical workflow so that the process is clarified (Figure 2).The system consists of two hardware and software components.Prior to the above-mentioned steps, a calibration is needed, which was implemented in a photogrammetric laboratory.More details of each steps are provided in the following sections.

Hardware components
The hardware includes a raspberry pie Microcontroller model 2 and a raspberry pie Microcontroller model 3, both serially connected to 5-megapixel cameras designed for this type of microcontroller.The Raspberry Pi is an internal Wi-Fi driver that eliminates the need for this model to have a Wi-Fi dongle.It also has a GPS-IMU module on board, which is designed for this type of microcontroller connected to the Model 3 Raspberry.This module is connected to a GPS antenna.A Wi-Fi dongle has been attached to the Raspberry pie Model 2 to create a wireless network.On both raspberry pies (referred to as nodes here), a set of software is applied that will be further mentioned in the software part.Furthermore, a power bank of 20,000 MAh per hour is used to electric supply of the system.All components are optimally embedded in the box for acquisition data easiness.It is possible to control the provided system using a laptop which is known here as the server.Server plays the role of data processing and data display in the system.

Server Preparation
A laptop was used to create the server and was installed in a Windows, virtual box environment, with Ubuntu 14.04 LTS installed.To connect the server to the nodes, a wireless network was created and a set of software was installed on Ubuntu.Each of these sections will be discussed below.

Installing Ubuntu Software
1. Pre requirements include a set of software that are not directly used but need to be installed on the system to work with the main software.These include: FFMPEG ،Libuvc ، Pangolin، Opencv، Eigen, BLAS and LAPACK ، Image Libraries (libjpeg libpng12 libtiff5 libopenexr) 2. The main softwares are those that are directly used.These include ROS (indigo), Image_Pipline, which itself contains other packages including Camera Calibration and stereo_image_proc.
ROS is a very powerful tool for connecting to networked hardware, obtaining data from sensors and processing data in the form of an advanced architecture according to (Garage, 2017).It has been developed in several versions known as distribution.One of the most widely used distributions is indigo, which was used for both nodes and servers in this study.

2.1.3 Node preparation
1. Network settings 2. Installing software on raspberry pies

Connection settings between server and nodes under ROS
The connection between different parts of this system is handled with ROS.Before capturing data, the connection between each camera, its own raspberry pi and laptop should be tested

2.1.5 Stereo Camera Calibration
The internal parameters of each camera and their relative parameters are determined according to the Pinhole model.This step was accomplished using a code written in OpenCV and under ROS known as stereo_calib.This package exploits a chess board.The 8 × 6 corner points are identified using feature detector algorithms and a homographic camera calibration algorithm is used to estimate the relative and intrinsic parameters of the cameras (Figure 4).To calibrate the camera using the stereo_calib, several images were captured from the chess board from different poses and a bundle adjustment procedure was applied on the 2D extracted points on the board, its 3D coordinates, and the exterior and interior parameters of the cameras in pinhole camera equation with considering homographic constraints.
Following data acquisition, drastic packages like Stereo_image_proc are applied to create the needed disparity image.In order to simultaneously capture the image of the crown top and the stump of the tree, system needs to be vertically set up and the distance to target tree should be (currently) set in an ad-hoc process.Further processing was done using stereo_image_proc node, which can be sited between the stereo camera drivers and vision processing nodes.This will also compute disparity images from incoming stereo pairs using OpenCV's block matching algorithm.
The image_pipeline stack is designed to process raw camera images into useful inputs to vision algorithms like rectified mono/color images, stereo disparity images, and stereo point clouds and it is consist of several packages like camera_calibration package to calibrate camera, image_proc node for Monocular processing, stereo_image_proc node for Stereo processing, Depth_image_proc for Depth processing and image_view package to visualize the final results.For the intended purpose and to get disparity stereo_image_proc is used, the following flowchart shows the input and out output, out of this node.(Figure 5) It Performs rectification and de-mosaicing of raw stereo camera image pairs.May also perform stereo processing to generate disparity images and point clouds.Point clouds are generated in the optical frame of the left imager (X Right, Y Down, and Z out).
In general, to achieve a disparity map out a pair of stereo images in a computer vision system, several preprocessing steps are required and they are needed to take respectively.The image must first be undistorted.underimage rectification, the image must be projected back to a common plane to allow comparison of the image pair.At All of this steps and more are in this package and must just be run and then disparity image is shown, by just running a simple code.Data acquisition, image rectification and disparity image can be found on figure 6 to 8.

RESULTS AND CONCLUSIONS
The obtained results showed that, like most photogrammetric methods, this system does not perform well in high light and shade conditions.Moreover, the camera is not allowed to be farther at least two meters from the tree.This system successfully can estimate the crown dimension and tree height in almost every light condition, yet poor performances were achieved when measuring DBH.
To better understand the accuracy of this system, tree height was measured by other methods and ground truth data were obtained.The following table compares the obtained height.
The mentioned system provides the user with detailed real-time, 3D information on single tree and crown attributes, which are essential basics for reference and inventory data collection on urban trees.However, and due to the availability of multiple other terrestrial equipment for field-based measurement systems, this study sets its following goal on comparative analyses on geometric accuracy and performance of the measured trees compared against conventional reference data collection methods such as those triggered by cellphone stereo camera and laser range finders.The potential application of stereo based system can possibly go beyond the current estimation of basic tree attributes, since the resulted point cloud enables extraction of further information on tree architecture and structure.
The innovation of this design is the existence of the ROS operating system.It has the potential to easy-touse applications.The proposed system is a preliminary and experimental step to achieve an ideal system based on the factors discussed earlier.The next goal is to increase the accuracy of the system by using maybe some parts of hardware or changing the capturing method.Actually this system has the ability to record and showing the disparity in video mode.About ongoing plans and to make it easier to use, a monitor can be replaced instead of the laptop.

Figure
Figure 3. the hardware architecture

Figure 4 .
Figure 4. Calibration in laboratory Figure 7. Left: left_image_rectified right: right_image_rectified: the rectified images from each camera