INNOVATIVE AND EFFICIENT DATA CENSUS ON A CLOUD BASED DIGITAL TWIN PROVIDED BY MOBILE MAPPING

: The article introduces the use of modern indoor mobile mapping systems (iMMS) equipped with a high resolution photographic camera, for the realization of asset acquisition campaigns of large building sites and for the generation of Digital Twins. The mapping project of a large part of the social housing real estate of the Municipality of Milan is taken as an example. The technical specifications and management methods of the 3D measurement and photographic acquisition processes in the field and the processing workflow, quality testing/certification and way to share on the cloud the Digital Twins are described.


INTRODUCTION
In several engineering sectors, engineers and technicians are looking closely at innovations that could increase the efficiency of the processes, reducing the operating costs.The challenges of intelligent information management and utilization are therefore rapidly increasing.At the same time, the technologies of Mobile Mapping Systems (MMS) are developing very fast, and in particular the quality of the RGB photographic information growing in quality and usability.The mobile instruments are rapidly moving from tools mainly used to get the 3D geometric characteristics of the environments, to devices capable to get high resolution and visualized as 360° panoramic cameras mapped on the 3D point cloud or as images used to colorized the points of the point cloud model.This characteristics expand the application of MMSs to the creation Digital Twins (DT) of real estate and industrial environments Lu, Qiuchen, et al.(2020).The DT model can be used to support and improve the efficiency of census activities of the assets of the mapped site.Actually, the census activities are usually carried on by an operator moving inside the site and tagging on a PDA (personal Digital Assistant), using as reference PDF draft plant of site, the assets recognized moving along the site.The software platforms can me less or more advanced, but the approach remain the same.An operator link to draft map of the site the assets directly recognized on the field.This data are then moved on a Data Base.Geometric measurements are not generally supported; robust data validation processes are not implemented and the information update processes are realized with physical on the spot investigations.The success case introduced in this paper, present an innovative use of MMSs devices, used to create cloud based 3D virtual tours of large real estate assets.The project starts from the need of Metropolitana Milanese (MM), an Italian Engineering Company, to realize the census of the assets of almost 500 buildings.This real estate is part of the social housing buildings of the municipality of Milan, a town located in North Italy and MM has the task to manage all these structures.The paper describes in details the specifications and procedures to run the 3D mapping process _____________________ *corrisponding author and to create a detailed Digital Twin to extract all the geospatial information needed to manage the real estate complexes.The procedures required to implement a 3D mobile mapping procedure, having a local accuracy of no less than 3 cm accuracy, global maximum drifts of 20 cm and the possibility to acquire 360° High Resolution images to support a Virtual Tour of the mapped sites.The company that won the tender decided to use the Gexcel's 3D wearable Mobile Mapping system Heron Twin Color.Gexcel official website (2023).The capture head of this instrument contains two Velodyne Lidar sensor for 3D acquisitions, a RGB hi-res panoramic camera for images collection.The 360° panoramic images can be acquired on demand at the resolution of 8K.The raw data acquired by the mobile devicehave to be processed in a post processing software.The results consists in a 3D point cloud model and by calibrated panoramic images mapped on the tridimensional model.The results are finally stored in a E57 format and memorized in the web.These data become a company value that can be considered as a commercial asset.

Paper objectives
In this context, this paper presents an investigation into the characterization of historic gardens by proposing a comparison between two three-dimensional survey methodologies: the use of a wearable indoor mobile mapping system and the use of a multi-camera photogrammetric instrument.The comparison aims to assess the applicability of the two techniques in this field by evaluating their advantages and disadvantages.The comparison focuses on two different aspects: (i) on obtaining the data, i.e., the characteristics of the in situ geometric survey phase as well as the data processing phase; and (ii) on the quality of the data, i.e., the characteristics of the point clouds and the information extracted.With regard to the first aspect, the practicality and speed of the operations and the problems encountered are highlighted; while with regard to the second aspect, a qualitative comparison of the point clouds obtained is detailed by investigating the completeness of the different geometries acquired, whether architectural structures or trees.The qualitative investigation was carried out by visual comparison of extracted portions of the point clouds and horizontal and cross sections of the trees and terrain.

FROM BIM TO DIGITAL TWIN
In the most recent years we have observed the fast introduction of BIM throughout the design, construction and management cycle of buildings and costrcution.Numerous international regulations provide for the widespread use of digital technology in the construction sector.EU Commission (April 2021).International regulations define as mandatory the use of BIM in large public construction activities and this requirement is gradually be extended to small-scale constructions as well.On the other hand, construction companies are also gradually becoming aware that the digital management of construction sites and real estates guarantees advantages in all the construction and site management phases.The laser scanner surveying of buildings has therefore spread considerably in order to support the "scan to BIM" processes.The generation of a parametric model of the building in order to be able to proceed with the BIM design of buildings in renovation operations (which represent a large part of the interventions in European countries), has often created numerous problems due to the high costs to create a BIM model of an existing building.It is for this reason that there are numerous studies that envisage an increasing use of segmentation for the object classification of a laser scanner survey rather than the modeling process as described by Romero-Jarén, R.. et al.. (2021).In particular, in the context of Cultural Heritage and in the management of the MEP of buildings, Bosché, F., et al. (2013,) the use of the segmentation approach rather than the three-dimensional modeling of reality is increasingly seen.In this rapid and tumultuous process of technological development, on the other hand, it has been observed how modern digital survey technologies, if coupled with a BIM model, can facilitate and speed up, thanks to appropriate software, the stages of monitoring the progress of the works, completely avoiding the Scan To BIM process.Sgrenzaroli, M., et al. (2022), Vassena, G., (2021).Always in order both to monitor the progress of the construction but also to manage the different management needs of a construction site, the need to manage the notations taken in real time on the field has grown exponentially through the use of solutions that are based on platforms that use tablets that display the PDF of site plans and support annotations.The use of cameras in order to document what is visible on site has become usual.This approach has rapidly seen the introduction of technologies that manage the taken of RGB spherical photographic images of the surrounding reality, and through an edges recognition process that find the correspondence of the same details in the photographs and in the BIM model allow to determine the trajectory of the surveyor and the approximate position and orientation of the photograph.Dalux (2023), Cupix (2023).This correspondence allows an operator to automatically get his position on the field and, thanks to an appropriate software to see in the PDA or tablet the spherical images and the BIM model aligned, in real time.These systems have a considerable success, even if their major limitation consists on the need of a BIM model to orient the images in order to estimate the trajectory of the operator.Furthermore, these systems garantee a very approximate estimation of the position and of the 3D geometries-They mainly support a fast association of information and notes to objects and elements observed in the photographs acquired in the field.The geolocation and geometric measurement quality remain poor.At the same time, laser scanner detection systems have evolved considerably, making it possible to speed up the acquisition phases in the field (thanks to the introduction of static laser scanners capable of carrying out an accurate and high-resolution scan of reality in less than a minute).Faro (2023), Leica Geosystem (2023), Zoller&Frohlich (2023).The evolution of indoor mobile mapping systems, based on the SLAM algorithms, have further made it possible to make the processes of extensive acquisition of building heritage models and contextual photographic mapping highly productive and economically sustainable.Cantoni., S. et al. (2019).The two approaches described, i.e. simple photographic mapping of buildings or construction sites, and photographic / 3D mapping with professional type mobile surveying systems and professional accuracy are now finding the perfect synthesis in software platforms that integrate, on the same platform, the two approaches described.

CASE STUDY
The case study here described is a large mapping and documentation project realized by Metropolitana Milanese (MM) that is an engineering company that manages the social housing buildings of the Municipality of Milano (North Italy).The real estate consists of 498 buildings organized in 124 compexes of structures (Figure 1).-To be able to recognize all the required assets present and to associates them to several categories as rooms, stairs, public area, external path, technical rooms; -To be able to make cm level local measurements; -To public the results of the survey on the cloud in the form of a virtual tour, through which MM technical staff could recognize the assests and to fill up the DataBase; -To obtain the 1:200 orthophoto images of the "visible front views" of all the buildings -To store the Point Clouds and the RGB data on a server no proprietary formats as E57, JPG, TIFF, LAS; -A quality check procedure has to implemented so to run a real time quality check of the survey

Instrumentation used
To satisfy the tasks detailed in 3.1, MM defined the following instruments specifications.

Instrument for the surveyof the Facades
For every visible façade of the buildings a resolution of at least 1 point every 4 cm was required.The Terrestrial Laser Scanner had to satisfy the following characteristics.Minimum level of accuracy of the points of the point cloud, 3 mm at 20 meters.Maximum measurement range more then 120 meters.The laser scanner used should have a dual axis compensator to assure the verticaly of the model, without using total stations.The orthophotos should allow MM to recognize the following items: -Downspouts; -Type of parapet; --Type of coverage; -Elevation height of the ridge plane; -Flues.

The 3D capture of the buildings
To run the 3D Capture activity, it was required the use of an indoor mapping system, based on SLAM technology and multibeam Lidar sensors, capable to garantuee a 3D model of the environment with synchronized and calibrated high resolution images.The system should garantuee to provide the following results and characteristics: -The system must be able to evaluate the surveyor trajectory without the obligation to start and finish from the same position, so to speed up the mapping process in particular along the mapping of the high rise buildings stairs.-A picture should have been garantueed at least every 5 meters -To garantuee a resolution of the final point cloud of at least 1 point every 3 cm.-The global accuracy should not be less then 20 cm.
-In case of mapping trajectory intersection, a maximum discrepancy of 4 cm will be accepted -The data elaboration process have to be managed locally by the surveyor not using cloud processes that do not garantuee the level of privacy required by MM -The acquired images have to follow a almost real time processof anonimization -The final data have to be produced in a no-propietary format (it means open format as JPG -TIFF -LAS -E57).
It is important to specify that the level of accuracy required is defined to avoid any error on items recognition and any mistake to assign an item to a wrong area of the building.The 3D model is required to allow local measurements and to have a 3D approximate model to support the software that need a model to support 3D navigation.A request of an higher level of accuracy of the 3D point cloud model, would have increased drammaticaly the costs because a control points measurement should have been carried on with classical surveying approach (Total station and GNSS).
MM have developed a large and detailed DB of the assets, but a census process has to be carried on.To be able to use in the future advanced tools as the ones described in 3., based on virtual thanks to the acquisition of 360° panoramic RGB pictures, as previously introduced a 3D point cloud model if a BIM model is not available, is needed.For this reason, MM decided to opt for an indoor mobile mapping approach, where the 3D point cloud model was to be acquired just for basic local measurements and as the 3D background to make possible to use the software tools previously introduced

Mobile mapping system -Heron Backpack
The Heron, Maset, E., et al. ( 2021), MS Twin Color used as SLAM mapping device, in the 2023 version, consists of a smart controller that contains the intelligence of the device and a small battery pack.The capture head consists of two 16 lines multibeam lidar sensors, an IMU and a 4 lenses 360° panoramic camera capable to acquire a RGB video at 24 Hz with 4K resolution (for point cloud coloring) or high resolution 8K RGB images on demand (to support 360° virtual tours).The controller is managed wireless by a PDA (Personal Digital Assistant).
The following characteristics of the mapping systems make the use of Heron particulary fitted to the MM requirements and to the characteristics of the sites to be mapped: -The system is wearable and not on wheels based -The device can work both in indoor and outdoor environment without using GNSS.The sites to be mapped are characterized by strong geometry and by urban canyons -The instrument support a lighting system to support images acquisition in low lighting conditions After mapping the data are processed to extract the trajectory traveled by the instrument, thanks to the use of the SLAM (Simultaneous Localization and Mapping) algorithm implemented on a proprietary software platform named Heron Desktop.The nominal local accuracy of the sensors is 3cm and is due to the nominal accuracy of the Velodyne VLP 16 sensors used.A better local accuracy could be reached using a Hesai 32 lines sensors, with a nominal accuracy of 1cm.Hesai (2023).

MAPPING ORGANIZATION
To realize the mapping process, a detailed organization of the mapping workflow have been defined and in real time, during the mapping and data elaboration activities, verified.

Data structure organization and and on the field tagging
First of all every site have been organized in a structure MM need is to link the assets to their location identified not as coordinate but by the site description, so to be later easily identified on the field.
Figure 3: Data structure for every site Every complex of buildings have been identified, first of all, by its official code.Every complex have been structured in building, technical rooms and outdoor sites, mainly parking lots, gardens, access paths.For building of the complex, the different layers have been identified as stairs, Facades and technical rooms.Every stairs has been structured in levels.It means that in the final virtual tours published in the cloud, the MM technicians can easily associate the recognized assets to their location.Practically every image have to be connected to one of the layer.For example the images and point cloud associated data, acquired at the level 3, of the stair number 2, of the building B of the Complex N, have to be stored in the specific directory.So that can be later easily published in the Vitual Tour Cloud platform in the correct data structure (figure 4).This association can be realized in two ways.One is in post processing.This means that during the data post processing procedure, an operator, that usually is different from the surveyor that has carried on the field work, have to associate to every image the right location.This time consuming procedure, could be avoided with an innovative procedure tested by authors that consists on an innovative interface (figure 5) that allows the operator, while surveying, to insert structured tag so associate information linked to the sensor position.The first information to be acquired is its real-time position (third floor, room one…), and then associate more info to this spot.Obviously this tagging process, if developed entirely just during the survey, can dramatically reduce the post processing phase.
Figure 4: The data moved in the virtual tour are organized in location directories already tagged in the field directly by the surveyors Figure 5: Smart interface for the PDA that control the indoor mapping system that associate data to the mapping trajectory

Pilot test
To organize and test all the mapping activities before starting the mapping, data elaboration and web publishing of the complex of buildings, a pilot test on a site have been organized.This phase have to be consider to the success of the project, to the high level of innovation of the technologies and workflow required.The mapping have been started only after the conclusion of the pilot project, where both the surveyors, MM and the quality check team (QCT) have been working to verify both the technical and timing procedures.

Planning the field work
The field work require that the surveyor move along the site using a very expensive device and the large amount of acquired data have to be organized in details so to be managed with efficiency.Secondly the real time quality check, requires both from MM and the QCT a continuos check of the mapping activities, before and after the field work phases.For this a reason a detailed protocol has been defined, where before mapping a supervisor team visit the site to be mapped for a pre access check and to collect various information neede to the surveyors team.In particular a full inspection report with specific notes (Figure 6) have to be realized, containing access information and informing MM if any problem to carry on the surveying phases are present in the complex of buildings.For every prospect a full map (Figure 6) of the site with the name given to each prospect have to be compiled and a report about the possibility to full survey the façade has to be compiled (Figure 8).All the path to be followed during the mobile mapping activity with Heron have to be tested and verified in advanced, so to speed up and to be sure of the complete mapping of all the areas requestes.Any possible problem to access the areas or to map them is previously shared with MM.In Figure 9 an example of the trajectory studied in advanced that the surveyor has tofollow map the external area of the site.This last temporary delivery is made to make possible a QCT.Figura 8: Exemple of notes associated to every façade Figura 9: Example of report of the planned trajectory to be mapped during the survey activies by iMMS

DELIVERABLES AND QUALITY CHECK PROCEDURES
As final deliverables it is requested to the surveyor to provide the following deliverables.After the mapping activity on the field, a survey report have to be shared with MM and the QCT.The survey report is compared with the inspection report to check is the inspection specifications, and previously approved have been satifield, and if not for which reason (Figure 10).The final result of the facades mapping consists in an orthophoto, at the required resolution, provided both in JPG format and in a already scaled Autodesk Autocad® format.
Following an example of a façade characterized by an error of resolution (Figure 11).Using the post processing software, this case 3D Reconstructor, Stonex (2023), the QC Team visit the virtual tour entirely to check the quality of the images.Essential for the following phase of assets recognition.During the Pilot Project test phase, it was also agreed that in some cases, an very high resolution picture have to be taken with a dedicated camera.This cases occur, for example, when the documentation of the names on the intercoms or the text of small technical sheet placed on machines as the ones on the central heating room, is needed.In this case these images organized in a dedicated directory inside the main directory of area mapped.
Figure 13: Example of the image validation phase.The operator moves along the directory selecting the blue dote that represent the position in the trajectory where e high resolution image have been take

Mobile mapping survey deliverables and verification
The Cintoo platform is a collaborative platform where the image, 3D model and annotation data can be easily shared.The survey company has provided to MM the access to this platform where MM technicians can load the structured data provided by the company, to manually recognize the assets and to fill the assets DataBase.Theplatform allows to associate to object recognized in the image an external file, as a PDF data sheet, an image or a link (Figure 14).The data are loaded on the cloud platform only temporary, for the time needed to compile the MM DataBase.The E57 and JPG file are archived on a "cold" cloud server, so to be loaded on a final interactive interface if so decided in the future by MM.The huge amount of information and data that will be obtained by this project, represent an important value and can represent a company valuable economic asset for MM.
Figure 13: Example of the Cintoo interface where files and links can be added to a tag placed on a recognized object.In this case an Fire extinguisher.

CONCLUSIONS AND FUTURE WORK
The project described is an example how assets management has a key role on mobile mapping activities.3D capture reality thanks to innovative cloud platforms see a growing integration between digital photogrammetry, SLAM based mobile mapping solutions.Innovative collaborative cloud platforms allow to share 3D model and information.Machine learning procedures are coming to support the final users to ricognize and map the assets.
It is required from the geomatics community to support this process and these procedures, hardware and software integration, with the aim to underline the importance of clearly define the way to face this process in particular on how to manage the various resolution, accuracy and reference system that every technology introduce.

Figure 1 .
Figure 1.An example of a complex of buildings 3.1 Project objectives MM desire was to obtain several goals from the mapping procedure:-To be able to recognize all the required assets present and to associates them to several categories as rooms, stairs, public area, external path, technical rooms; -To be able to make cm level local measurements; -To public the results of the survey on the cloud in the form of a virtual tour, through which MM technical staff could recognize the assests and to fill up the DataBase; -To obtain the 1:200 orthophoto images of the "visible front views" of all the buildings -To store the Point Clouds and the RGB data on a server no proprietary formats as E57, JPG, TIFF, LAS; -A quality check procedure has to implemented so to run a real time quality check of the survey

Figura 2 :
Figura 2: Heron MS Twin Color during the field tests

Figure 6 :
Figure 6: Example of note sheet compiled by the mapping activities supervisor

Figure 7 :
Figure 7: Draft map of the Facades to be surveyed by TLS

Figura 10 :
Figura 10: Example of comparison between survey report (left) with the inspection report (right)