UAV in the advent of the twenties: Where we stand and what is next

has surged in the last two decades, making them popular instruments for a wide range of applications, and leading to a remarkable number of scientific contributions in geoscience, remote sensing and engineering. However, the development of best practices for high quality of UAV mapping are often overlooked representing a drawback for their wider adoption. UAV solutions then require an inter-disciplinary research, integrating different expertise and combining several hardware and software components on the same platform. Despite the high number of peer-reviewed papers on UAVs, little attention has been given to the interaction between research topics from different domains (such as robotics and computer vision) that impact the use of UAV in remote sensing. The aim of this paper is to (i) review best practices for the use of UAVs for remote sensing and mapping applications and (ii) report on current trends - including adjacent domains - for UAV use and discuss their future impact in photogrammetry and remote sensing. Hardware developments, navigation and acquisition strategies, and emerging solutions for data processing in innovative applications are considered in this analysis. As the number and the heterogeneity of debated topics are large, the paper is organized according to very specific questions considered most relevant by the authors.


Introduction
Unmanned Aerial Vehicles (UAVs) represent one of the most relevant emerging technologies in the geoscience and remote sensing fields of the last two decades.They have become a popular instrument for a wide range of applications and replaced other platforms thanks to their flexibility and (relatively) moderate costs.This is also reinforced by the number of scientific papers about UAVs published in different research communities in the last two decades.According to Scopus 1 , >80,000 papers have been published using the term "UAV", "drone" (or, less frequently, "UAS" and "RPAS") in the title or the keywords since 2001 (Fig. 1): most of these works belong to the engineering and computer science domains, while the majority of contributions come from remote sensing research (Chabot, 2018).This trend is confirmed by other citation indexing databases, showing an increased interest in UAVs from the scientific community over the last few decades.UAV business has also been valued at several billion dollars a year 2 , and the trend looks promising for the future, despite the fact that the biggest share of the market is still in military applications.This economic interest, the technological development and the growing miniaturization of onboard sensors, as well as the development of new algorithms and software have been recent pushes for the conception of new applications that will further boost the use of UAVs.The initial UAV surveying applications have been flanked by more advanced applications, creating new needs and (therefore) new businesses opportunities 3 .In this regard, UAV systems for rapid, automated and autonomous collectionof geospatial data are contributing to the fourth industrial revolution (Xu et al., 2018).Construction and infrastructure monitoring, precision farming and search and rescue in indoor spaces are just some examples of new and emerging applications of UAVs.Behind all these applications, mapping and remote sensing research (in the broad sense) play a fundamental role in the development of successful solutions.
The growing interest in UAVs has led to the publication of several peer-reviewed papers in the last years.Among the first works, (Colomina and Molina, 2014) and (Nex and Remondino, 2014) gave a broad overview on the use of UAVs for photogrammetric and mapping applications.These contributions were part of the early growth of UAV applications and considered challenges related to hardware and data processing workflows.Since then, the increasing number of publications and the growing specialization of UAV solutions have triggered new review papers focused on specific applications, onboard hardware or data processing strategies exploiting UAV data.Examples of reviews on specific applications are given in (Adão et al., 2017;Tsouros et al., 2019) for agriculture, (Torresan et al., 2017) for forestry, (Kerle et al., 2019a) for structural damage assessment, (Giordan et al., 2018) for natural hazards, (Ren et al., 2019) for mining areas, (Themistocleous, 2020) for cultural heritage, and (Rakha, 2018) for building inspection.In addition, reviews of UAV data processing or specific hardware development are presented in (Aasen et al., 2018) for UAV spectroscopy, (Aggarwal and Kumar, 2020) for path planning, and (Boukoberine et al., 2019) for power supply.The aim of these publications is mainly to give an overview of the state of the art on specific uses.Only (Yao et al., 2019) attempt to give a broader overview of the different remote sensing applications.The UAV domain is, however, characterized by the integration of very different expertise, combining different hardware and software components on each platform depending on the intended use.These reviews, although very detailed, do not focus on cross-disciplinary perspective: research in adjacent domains (such as robotics and computer vision) already impact remote sensing, but too little attention has been given to these aspects so far.As a result, most of the aforementioned works limit their investigations to their own topics and provide few hints about future trends.Lastly, despite the incredible improvement of both hardware and software solutions, best practices for specific problems, such as how to improve accuracy in UAV photogrammetry, are often ignored, although they often represent a bottleneck for wider UAV adoption.
This paper aims to fill the remaining gaps from previous works by looking at UAV domain with two different (often interconnected) goals.The first goal is to tackle the existing unsolved or overlooked issues in the daily use of UAVs with currently available technologies.Issues include the selection of the best sensors for specific tasks and the most reliable way to acquire data for UAV mapping and remote sensing applications.The aim is to review papers dealing with these issues to promote best practices for the use of UAVs.The second goal is to investigate current developments in UAV technology (often from adjacent domains) to observe their potential and anticipate their future influence on photogrammetry and remote sensing.These two elements have been analysed following a logical workflow from the selection of sensors to the new trends in the data processing.Given the heterogeneity of the topics being addressed by these goals, sections have been organized according to "burning questions" considered relevant by authors.References are concise and point to the most relevant works.In Section 2, the developments for onboard passive and active acquisition sensors are reviewed with the aim of discussing the specific properties of available sensors, the importance of their technical specifications according to the application and their differences compared to their counterpart on manned platforms.In Section 3, best practices on how to calibrate and integrate onboard sensors and fully exploit their potential in daily practice are given.Current endurance limits and the strategies to extend flight time are also debated.Section 4 has a double goal: it summarizes best practices for improving the quality of mapping products and reports on emerging acquisition scenarios that promise to widen the extension of UAV surveys.Section 5 reports on trends in UAV data processing and analysis, focusing on real-time processing methods and on the surge of artificial intelligence onboard UAVs: the peculiarities of these algorithms are discussed in detail.Section 6 reports on lessons learned, and the future challenges for UAV technology are given.

Data acquisition sensors
A growing number of miniaturized active and passive sensors specifically conceived to capture data from UAVs are available on the market.In this section, we review the quality of these instruments, analyse their pros and cons, compare them with their "full size versions" 3 https://www.mckinsey.com/business-functions/operations/our-insights/imagining-constructions-digital-future accessed on 12 September 2021 F. Nex et al. on manned airborne platforms and give best practices for their operational use.

RGB cameras: UAV sensors state of the art and comparison with airborne sensors
Compared to the conventional airborne case, UAVs normally operate closer to the ground and have hard constraints given by the payload and the speed of the platform (particularly for fixed-wing aircrafts).The first UAVs had cameras adapted from terrestrial applications (i.e., compact cameras) while a new generation of dedicated sensors has been conceived over the last few years.Although 3D photogrammetric reconstruction should be possible with any kind of overlapping images, photogrammetric cameras typically follow special design rules to obtain high levels of performance and efficiency rules that are highly constrained by the weight and size limitations, making compromises inevitable.

What are the specifications that determine the quality of a UAV camera?
General design.Traditional mapping cameras are very sophisticated and complex.As a result, they are not only expensive but also bulkier and heavier systems that cannot be integrated on standard drone platforms.The most limiting factors for any kind of sensor equipment on UAVs are maximum-take-off-mass (MTOM) and size.This is also the reason why UAV cameras do not simply copy the concepts of large format mapping cameras.The image frame is not a limitation as most of the current UAV scenarios are much more restricted in mapping area size, and more compact cameras with smaller image formats have fewer negative impact on smaller projects.If a UAV has a Maximum Take-Off Weight (MTOW) of around 25 kg, 150 MPix camera is currently the best that can be integrated onto the platform.For a more standard UAV (with MTOW < 5 kg) the image formats range between 20 and 60 MPix, nowadays.In addition, as the UAV operates closer to ground and aims at a Ground Sampling Distance (GSD) of only few centimetres, the cameras placed on fixed-wing aircraft need to have fast shutter to avoid motionblur due to the absence of forward-motion compensation.
Optics.In comparison to larger imaging sensors, UAV cameras are mainly mono cone (i.e., have single sensitive element) and often use a lens with a shorter focal distance (28 -50 equivalent).All the (typical) UAV cameras designed for mapping are built as single cone systems, which automatically excludes the use of multiple channels for separate colour bands.Single frame cameras often use special colour filter arrays to arrange RGB colour filters on the photosensor pixel matrix, called the Bayer-filter.Such Bayer-like colour filter arrangements need demosaicking which reduces the original spatial resolution.In the first years of UAV-based mapping almost all cameras were standard off-theshelf cameras.These cameras were not specifically designed for photogrammetric mapping and quite often used focusable, sometimes collapsible lenses or even zoom lenses.Such non defined and non-stable lens geometry negatively affects the quality of the photogrammetric processing.As the use of UAVs in mapping increases and users gain more experience, drone suppliers changed their approach and tried to transfer some of the main design features of traditional mapping cameras onto UAV cameras.The use of fixed focus lenses with no moveable parts, and with optical image stabilization and a stable camera body with rigid lens mount is now part of many systems to address mapping applications.
Shutter design.Traditional photography has central and focal plane mechanical shutters.The central shutter usually comprises several shutter blades or leaves that open and close in the same way as a classical lens aperture.Opening and closing can be regarded as instantaneous, i.e., the whole image focal plane is illuminated at one distinct point of time, with one perspective centre position and attitude.Focal plane shutters, contrary to this, comprise two curtains moving one after the other to form a slit that captures the whole image in the focal plane.The width of this slit defines the exposure time.Unlike the central shutter, it takes some time to capture the whole image.Consequently, moving objects are distorted on the image plane, which is always the case when images are taken from airborne moving platforms.In digital imaging the global shutter and rolling shutter can be seen equivalently to the mechanical central and focal plane shutters.Similarly, the rolling shutter will introduce the same distortion effects as the focal plane shutter.This effect can be modelled mathematically as the six exterior orientation parameters of each image will not be seen as constant but variable over the exposure interval (Vautherin et al., 2016).This concept is adapted from the processing of line scanning sensors (Hinsken et al., 2002) and ideally uses measured orientation information from Inertial Measurement Unit (IMU) data (Colomina and Blázquez, 2014).If the true sensor movement is not observed by an inertial platform (especially the angular rate), the mathematical model only approximates the real change in sensor orientation over time and the full sensor performance might not be fully exploited here.When UAV mapping first began, most of the metal-oxide semiconductor sensors (CMOS) were using rolling shutters due to their simple electrical design.Today most of the cameras designed for mapping feature electronical global shutters, and some of the high-end systems even use a mechanical central shutter or a combination of both.
Reduction of image blur / motion compensation: In airborne mapping, images are captured during the movement of the platform.This movement can be divided into translational and rotational components.Both will affect the sharpness and thus the quality of the image data.The problem is fairly well known, and so-called forward-motioncompensation (FMC) was introduced in the mid-1980s for analogue mapping cameras.FMC moved the film during exposure to overcome the blur caused by the forward motion of the plane (Schöler, 1987).This linear correction was adopted for digital cameras too.The electronic design of charged-coupled-device (CCD) sensors used in the first generation of airborne large-format mapping mimics the analogue FMC by moving the charges on pixel during integration (exposure) time (socalled time-delayed-integration, or TDI) (Hinz, 1999).For cameras based on CMOS imaging sensors, the sensor is physically moved as was done with film (Mueller and Neumann, 2016).In all these cases only the main part of the image blur can be compensated and deviations from mean flying height above ground or mean speed leave uncompensated effects.Additionally, any movements that are not aligned with the direction of the transport are not corrected.As a result, FMC only reduces image blur.As changes in rotation during exposure also generate significant blur, the FMC technique must be combined with fully stabilized platforms to actively compensate for the rotations of the carrier platform.Additional vibration dampeners compensate for the highfrequency angular effects present in multi-rotor UAVs.FMC combined with fully stabilized mount is called full-motion-compensation.For practical reasons, small UAVs do not have true-FMC with moving components, which limits the ground resolutions for fixed-wing drones as a function of ground velocity and shutter speed.Thus, the reduction of forward motion is accomplished by minimizing exposure times, sometimes called the (radiometric) blur control technique.It is based on the radiometric performance of the sensor in combination with fast shutter speeds.This is not a specific modification in the sensor, it just relies on very-fast shutters and the extended radiometric performance of sensors based on larger pixels or higher sensitivity.This approach in principle can be applied for any kind of sensor but is often used for RGB frame cameras working with colour mosaic filters.
From technical point, micro filter arrays (one filter evaporated onto every single pixel, so-called Bayer-pattern mentioned above) are used to capture colour information.Missing colours then will be interpolated from the neighbouring pixels, the so-called debayering or demosaicking (Meißner et al., 2018).This pixel-wise changing colours prevents charge moving technologies like the TDI (Time Delay Integration) approach, and it is often preferred because it does not need any specific sensor modification, and the camera can be offered at lower cost using standard (off-the-shelf) sensors.Just recently methods known from medical F. Nex et al. external beam radiotherapy were adapted for the blur control in airborne mapping cameras 4 .Even though details on this adaption are not yet available, this purely software based adaptive motion compensation corrects for both forward and angular motions within different image scales, i.e., covering nadir and oblique views, and it might be transferable into future UAV mapping camera systems.

What to consider when we choose a sensor?
UAV mapping started with traditional off-the-shelf cameras but the need for more dedicated systems became obvious soon.Today many of the mapping drones use specifically designed or modified cameras.Practically, all of them implement CMOS sensors with single lens only and many of them use electronic global or mechanical central shutter.RGB information is derived from colour mosaic filters.Image blur is normally minimized by selecting shortest exposure times in combination with angularly stabilized mounts, which are standard for multi-rotor UAVs today.In principle those cameras can be classified according to their image sensor format and their overall weight, which is most critical for drone installations.Heavy systems are in the range or above 1 kg and may even reach close to 2 kg for large cameras.Very light systems are below 200 g, all inclusive (i.e., battery, optics, etc.) (Table 1).Furthermore, other aspects can be considered, such as how well cameras fulfil photogrammetric requirements like geometric stability or how flexibly the systems can be integrated into different platforms.Accessibility of raw image data plays a role, as this is quite often linked with internal pre-distortion corrections (James et al., 2020).
In Table 1 some cameras commonly used in today's UAV mapping are listed and classified according to their sensor format.This parameter is often correlated with the camera weight as larger sensors need heavier lenses.There are generally proprietary systems, developed by big UAV system providers, optimized for their specific platforms and often not exchangeable to other drones (such as DJI sensors 5 ) while other solutions (e.g., SenseFly 6 cameras) offer open interface, which allows integration in other platforms.Important to note, that some of these sensors come with so-called direct in-flight georeferencing: the full exterior orientation parameters from Global Navigation Satellite System (GNSS) and IMU position and exact orientation are recorded for each image capture and become part of the image header information.No additional GNSS/IMU units are needed here.Smaller UAV providers still rely on off-the-shelf Digital Single Lens Mirrorless (DSLM)-type cameras with full format sensors, like the Sony a7 series.These cameras are fully selfcontained with internal power supply and storage, offer a variety of lens options and undergo continuous improvements as triggered by a large consumer market.On the other hand, they are not primarily designed for UAV applications, thus concessions have to be made, for example in terms of deep integration into the UAV controller or onboard data preprocessing.
Another element refers to the overall weight, which normally is in 1kg range including lens, shutter design (focal plane versus global) and additional features like image stabilization which may influence the camera intrinsic parameters.Different to such off-the-shelf cameras, specific drone mapping systems try to combine larger formats and good quality with high compactness and less weight.Their design tries to fulfil the photogrammetric demands as mentioned for the proprietary systems.Here the Camlight designed by the IGN France research labs and now commercialized by Delair, one provider of UAV solutions, should be spotted (Martin et al., 2017).This camera offers full-format 50 MPix images with below 500 g overall weight for many body-lens combinations.Analysis of data flown with the IGN engineering model showed extremely stable sensor geometry (Roth, 2019), very close to traditional photogrammetric cameras.No hidden internal pre-distortion corrections are applied.The mid-format up to 150 MPix cameras as offered by Phase One today mark the upper end of UAV-based cameras and already link to the manned airborne mapping platforms, where some of the large format camera systems are based on such camera components.These lens-camera combinations are close to 2 kg and therefore can only be used in larger UAVs.
Precise 3D reconstruction needs high resolution imagery to accurately identify and measure corresponding image points.The quality of images is measured by the geometric resolving power, which is dependent on the camera-lens-system and the environmental conditions during image acquisition (including the motion blur compensation).Geometric resolution is visually obtained from defined bar pattern, like the USAF1951 resolution target.More objective measurements are given by the quantitative analysis of modulation transfer and point spread functions, that mathematically define the image resolution (Meißner et al., 2020).These values are obtained from resolution targets like Siemens star patterns or clearly defined edges (slanted-edge method).Another quality measure is the signal-to-noise ratio (SNR) which defines the radiometric quality.The SNR is determined from the noise at a specified average signal typically derived from homogenous surfaces (Reulke and Eckardt, 2013).This measure is not only important for remote sensing, but it also affects the matching between overlapping images.The size of individual pixels on the digital sensor is then an important factor to get low-noise imagery.For many manufacturers, a clear trend is to decrease size of individual pixel to bring more pixel on a given sensor frame, which reduces the individual area to collect photons per pixel and thus reduces the SNR.However, in the CMOS sensors significant portion of each pixel is non-sensitive to light as occupied by the electronics, at least if the pixel related electronics addressing each individual photo diode is left unmodified.Due to permanent innovation in CMOS design, even with reduced individual pixel sizes, high fill factors (ratio between light sensitive part to total pixel area) and high quantum efficiency (ratio of incoming photons to collected electrons on pixel) (Blanc 2001) are possible guaranteeing a good SNR as well.Introduction of back-side-illumination (BSI) is one of these innovations, where the sensor is turned and now illuminated from the back-side.This increases the fill-factor (i.e., the light sensitive area per pixel with respect to its total area) as the control electronics do not limit the lightsensitive area and may outperform traditional front-side illuminated (FSI) sensors especially for smaller pixel sizes below 2 μm.In the BSI design the light sensitive elements are separated from the remaining electronical elements, optimizing the optical path independently from the electronics by inverting the light incidence (Paiva Gouveia and Choubey, 2020).Differently, the light has to further penetrate the silicon which limits miniaturization of pixel sizes on above 1 and 2 μm, where a BSI sensor still provides better signals than a FSI 7 .
Although image distortions do not influence the camera resolution, they are relevant to determine the choice of a UAV camera.The correction of distortions is an essential part in the photogrammetric reconstruction process.Large distortions, as known from wider-angle lenses, affect the generation of an image, even in consumer photography.Therefore, some (especially small and lightweight) cameras on current UAVs internally pre-correct lens distortions by pre-defined models to generate "higher image qualities".The images look distortion free, even though the original raw images underline strong distortion effects.Such approach might be nice for visual impression, but these pre-corrected images may not fit well the standard photogrammetric calibration models.As shown in (Hastedt et al., 2021) when the geometry of the image is pre-corrected in a way, the traditional models of insitu calibration are not completely able to cope with this new camera geometry.This requests for extended and modified models to describe the new geometry.This is the main reason why images with their original image geometry should be preferred in 3D reconstruction, or the original correction functions and parameters be made available.

Multi-and hyper-spectral sensors: Evolution and trends
Multi-and hyperspectral cameras are used to capture spectral information of objects of interest (Aasen et al., 2015;Adão et al., 2017).Hyperspectral (HS) cameras provide contiguous spectral signatures by acquiring tens to hundreds of densely placed narrow spectral bands, e.g., with a spacing of 1 nm and<10 nm full width of half maximum (FWHM).Multispectral (MS) cameras have a few (typically 3-10) spectral bands optimized on specific spectral regions, such as blue, green, red, red-edge and near-infrared (NIR).Looking at the UAV domain, regular color (RGB) cameras and modified color-infrared (CIR) cameras equipped with Bayer matrix based mosaic filters (see also previous section) capture spectral data, but they have less optimized spectral bands than the MS and HS cameras (Aasen et al., 2018).On the other hand, MS cameras, such as MicaSense Altum 8 and RedEdge and SAL Engineering Maia 9 , have entered widely on markets because of their affordability and efficient processing chains in the commercial software (such as Agisoft Metashape 10 and Pix4D 11 ).UAV HS cameras are still used predominantly in scientific and research purposes, because they are relatively expensive, heavy, capture vast volumes of data and are more challenging to operate and process than the optimized MS cameras.Furthermore, understanding on how the more precise spectral characterization can be utilized in practical applications is still inadequate.As MS technology is already widely applied in practice, the rest of this section will focus on HS technology which represents our vision of the future, though many aspects are also applicable to MS systems.

What are the differences between hyperspectral sensors?
HS technologies differ based on the approach used to achieve the spatial and spectral discrimination capabilities and thus in the arrangement, range, number, and widths of bands that they feature (Aasen et al., 2018).They can be classified as: (i) pushbroom scanners (ii) 2D cameras using HS imaging techniques and (iii) point spectrometers that integrate spectral signatures over the projected footprint of the sensor, thus not providing continuous images.HS sensors can be further categorized according to their spectral sensitivities to visible and NIR (VNIR: 400-1000 nm), NIR (1000-1700 nm), shortwave-infrared (SWIR: 1000-2500 nm), mid-wave infrared (MWIR: 3-5 µm) and long wave infrared (LWIR: 7-12 µm) (Adão et al., 2017).State-of-the-art MWIR and LWIR cameras are still heavy and therefore seldomly used in UAV applications.
A visualization of selected commercial miniaturized HS cameras is presented in Fig. 2 and in Table 2.The weights are indicative as they are based mostly on the information from the manufacturer webpages thus there can be differences due to the considered components such as lenses, covers, GNSS/IMU, onboard computers and batteries.In the VNIR spectral range, 2D frame format cameras weight 0.5-1 kg while pushbroom scanners weigh 1-3 kg; the NIR-SWIR range cameras are heavier, weighing 2-6 kg.
Pushbroom scanning is the most common technical implementation of HS cameras.The object is scanned line by line by projecting the scene to the sensor through a slit and dispersing the slit image with a prism or a grating.Miniaturized pushbroom cameras for drones are commercially available for wide spectral ranges of 400-2500 nm.The entire 400-2500 nm range is covered by the Corning VIS-SWIR camera that provides a single detector solution based on a Mercury Cadmium Tellurium (MCT) detector (Corning 12 ), while Hyspex Mjolnir and Headwall are based on coaligned VNIR and SWIR sensors, with silicon and indium gallium arsenide detectors, respectively (Hyspex, Headwall 13 ).
In the cases of 2D-frame format cameras, the spectral discrimination is provided using techniques such as tuneable Fabry-Pérot  (Aasen et al., 2015)).Current commercially available 2D HS sensors operate in the VNIR-range (400-900 nm) but also SWIR range cameras are becoming available (Honkavaara et al., 2016).
Point spectrometers have been used onboard UAVs in a variety of studies (Becker et al., 2019;Burkart et al., 2014;Burkhart et al., 2017) and have also been implemented in whiskbroom scanning mode (Uto et al., 2016).An emerging application of point spectrometers is monitoring of vegetation photosynthesis using sun-induced chlorophyll fluorescence (SIF); a narrow spectral range of 100-200 nm between the red and NIR regions is observed with FWHM ≤ 0.39 nm, a resolution < 0.2 nm, and a high SNR on the level of ~ 1000:1 (Pacheco- Labrador et al., 2019).An advantage of the point spectrometers in comparison to pushbroom and 2D cameras is their superior spectral sensitivity and resolution; their disadvantage is the inferior spatial performance.Pushbroom scanners provide better spectral resolution than 2D-frame sensors, but the advantage of the latter is their superior spatial quality due to the rigid imaging geometry and capability to capture stereoscopic images, enabling a single-sensor solution for hyperspectral 3D reconstruction of objects (Aasen et al., 2015;Honkavaara et al., 2013;Oliveira et al., 2019).However, the quality of point cloud extraction is not in general as good as with good quality RGB cameras due to the lower pixel resolution and SNR ratio of the HS images (Honkavaara et al., 2016(Honkavaara et al., , 2013)).
In recent years, rapid development has taken place in miniaturized HS sensor techniques although the situation has not stabilized, and new solutions are expected in the near future.In comparison to the previous reviews (Aasen et al., 2018;Adão et al., 2017), the numbers of alternatives have increased, the weight of the sensors has decreased, and the availability of sensors in the extended NIR and SWIR ranges has improved.Increasingly, the manufacturers are offering fully integrated turn-key systems, which will lower the threshold to start using HS systems while during earlier years the user him/herself often had to assemble the system from separate components.There is an increasing trend to integrate additional sensors to HS systems, particularly GNSS/ IMU or additional cameras for georeferencing, irradiance sensors, and LiDAR.
Comparing the modern miniaturized MS and HS cameras to the mature cameras operated with manned aircraft, several differences can be observed (Fig. 3).The conventional spectral remote sensing sensors are predominantly pushbroom scanners, while a variety of technical implementations are offered on UAVs.The miniaturization of the detectors and lenses in UAV systems leads generally to poorer SNRs considering the radiometric performance.The aircraft-based systems offer typically>50 cm GSDs while the modern UAV systems capture  5-10 cm GSDs, therefore providing improved levels of detail and precision.An comparison of hyperspectral images used for forest health study captured with a 2D-frame format camera is given in Fig. 3: in this case, UAV images (GSD 10 cm) show greater details and better spectral separation of different health classes as compared to airborne images (GSD 50 cm).When miniaturized devices are installed onboard agile platforms, it is possible to perform object monitoring with high time resolution, in varying conditions below clouds, and year-round.The diversity of UAV-based devices and operating scenarios is significantly greater than that of traditional devices; this increases the challenges of data processing.In general, it can be considered that the aircraft and UAV-based technologies are complementary: mature and heavy aircraft solutions provide high-quality and stable spectral data and are feasible for infrequent capture over wide areas, while UAV solutions provide higher spatial and temporal resolutions, and their technologies and applications are in a rapid development stage.

How do we need to process hyperspectral images?
Processing of HS datasets has some differences in comparison to ordinary camera images.These differences are due to the geometric configuration of imaging systems as well to the requirement for quantitative radiometric processing.
The 2D-format spectral images can be georeferenced utilizing well established Structure from Motion (SfM) techniques (Section 4.1) with some sensor specific modifications; e.g., carrying out georeferencing and Digital Surface Model (DSM) extraction using high spatial resolution panchromatic images and combine the lower spatial resolution HS images with them as in the case of the Cubert camera (Aasen et al., 2015) or to model (Berveglieri et al., 2019;Honkavaara et al., 2017;Tommaselli et al., 2019) or correct for the impacts of the time-sequential spectral scanning (Honkavaara et al., 2017).Pushbroom scanning results in a different position and orientation for each image line requiring accurate trajectory information to be captured utilizing a precise GNSS/IMUsystems (Lucieer et al., 2014) or integrating lower precision GNSS/ IMU, 2D camera images and SfM (Büttner and Röser, 2014;Jaud et al., 2018;Suomalainen et al., 2014).The point spectrometers are the most challenging due to their dynamic nature (Natesan et al., 2018), furthermore, the location and size/shape of spectral footprint of each measured spectra is different and determined by the position, orientation, Field of View (FoV), and integration time of the spectroradiometer, flying height and speed of the UAV, and the surface topography (Gautam et al., 2019).HS sensors are also often integrated with LiDAR (Lin et al., 2019) or dense photogrammetric surface reconstruction (Aasen et al., 2015;Honkavaara et al., 2013;Oliveira et al., 2019;Suomalainen et al., 2014) to enable accurate georeferencing and characterization of 3Dobjects.
Quantitative radiometric processing is required in order to obtain unbiased spectral data (Schott, 2007).The image grey values (Digital numbers; DN) are subject to a number of factors that need to be compensated for.The essential radiometric processing steps include the correction based on sensor laboratory calibration, compensation of illumination related effects, atmospheric correction, object anisotropy correction i.e. so called bidirectional reflectance distribution correction (BRDF correction), and topography correction (Aasen et al., 2018).The following briefly describes four popular methods for producing reflectance outputs of drone datasets: (i) Empirical line methods (ELM) have been used in majority of studies (Aasen et al., 2018).ELM requires at least two reflectance targets to determine a linear model to transform the DNs to reflectance factors.(ii) Direct reflectance transformation transforms image radiances to reflectances utilizing the onboard irradiance observations.Such techniques can be found in research systems (Burkhart et al., 2017;Hakala et al., 2018;Suomalainen et al., 2018) as well as increasingly in commercial products, e.g.MicaSense incident light sensor (Micasense) and the Headwall's fiber optic downwelling irradiance sensors (FODIS) (Headwall).Independence from in situ reflectance targets makes the direct approach efficient, being particularly useful in environments where suitable places for installing reflectance panels are not available and in Beyond Visual Line Of Sight (BVLOS) operations.Irradiance sensors also provide valuable information about changes in illumination that can be used in the correction process (Honkavaara et al., 2013).(iii) Software based solutions estimate the irradiance using atmospheric radiative transfer models (Aasen et al., 2018;Zarco-Tejada et al., 2012).(iv) When utilizing blocks of images, it is also possible to use radiometric block adjustment based approaches that use radiometric tie points to model various factors causing differences in DNs in overlapping images, and can also use reflectance targets and irradiance observations to carry out the reflectance transformation (e.g.(Honkavaara et al., 2013)).
Radiometric sensor calibration and characterization is a crucial manufacturing step of spectral cameras in order to enable quantitative spectral analyses.The fundamental calibration parameters concern the lens-falloff, absolute radiometric calibration coefficients, and spectral response functions.Furthermore, spectral distortions (keystone, smile, and temperature effects) (e.g., (Aasen et al., 2018)) as well as noise levels (Barreto et al., 2019) need to be characterized.There are situations in which precise absolute radiometric and spectral sensor calibration is particularly important.Firstly, when reflectance calibration is performed using the direct method without any in-situ calibration targets, the calibration of each sensor separately and together is crucial (e. g. (Burkart et al., 2014;Burkhart et al., 2017;Hakala et al., 2018;Suomalainen et al., 2018).Secondly, precise characterization is needed when analysing very narrow spectral responses, e.g. to measure SIF ( Pacheco-Labrador et al., 2019) and in general in high precision tasks, such as reflectance spectra measurements to reflectance libraries.In these situations, regular sensor calibration and in-situ validation procedures are required.It is typically recommended to repeat the radiometric calibration in 1-3 years intervals.On the other hand, requirements set for the absolute radiometric calibration can be relaxed in some cases: when reflectance panels are installed in the area, the analysers are calibrated using in-situ data (e.g., biomass or nitrogen content), if sensors are used in relative mode, e.g., using different indices, or if radiometric calibration is not critical for the analysis task.
The previous processing steps are in most cases carried out as postprocessing.For 2D frame format cameras the mainstream photogrammetric software, such as Pix4D 16 or Agisoft Metashape 17 can be used; for pushbroom cameras, for example, the Atcor4 and Parge 18 are used.(Horstrand et al., 2019b(Horstrand et al., , 2019a) demonstrated a real-time implementation of the processing steps for the Specim FX100 pushbroom scanner.Their onboard processing chain included control of the flight trajectory, management of the data acquisition, georeferencing, image calibration and calculation of different vegetation indices as well as anomaly detection.As spectral remote sensing becomes more common, processing technologies must evolve to be automatic, efficient, reliable, and agile.

What are the bands to use according to the application?
HS cameras produce huge volumes of data while capturing tens to hundreds of spectral bands.To avoid capturing unnecessary data, a relevant question to pose is which bands are ideal for specific analysis tasks.The HS signatures are particularly relevant in scientific research to understand the behaviour of phenomenon that are not yet clearly defined.Precise spectral signature information enables detailed scientific studies of object characteristics, modelling of its function and composition, as well as studies of light matter interaction modelling (Forestier et al., 2013;Schaepman et al., 2009).HS signatures are, for example, needed in complex analysis tasks, such as mineral exploration, urban material detection, classification of species rich forests, and biodiversity assessment.Selection of optimal spectral bands is dependent on application: vegetation health and vigour assessment, mineral exploration, hazardous material detection etc. all have different requirements for the spectral bands combinations necessary to provide the desired information (e.g.(Alves et al., 2019;Askari et al., 2019;Forestier et al., 2013;Li et al., 2014;Moghimi et al., 2018;Näsi et al., 2015)).Information about optimal spectral bands can be used to build sensors with optimized spectral bands, to select optimal spectral bands during data acquisition or to support analytics.
Researchers have compared the performance of HS and MS sensors in different analysis tasks.HS imaging outperform MS imaging in complex analysis such as tree species classification in species rich scene (Tuominen et al., 2018) and crop quality parameter estimation (Askari et al., 2019;Lu et al., 2019;Näsi et al., 2015;Oliveira et al., 2019), while similar performance can be obtained if the selected bands are in coincidence with MS-bands for a considered task.

UAV LiDAR: evolution, present state and challenges
The first works using laser scanning from a UAV were presented a decade ago by (Jaakkola et al., 2010;Lin et al., 2013;Wallace et al., 2012).The flight times of these systems were typically only few minutes, and the scanners available had few kHz frequencies, much slower than sensors today.Since then, the positioning and LiDAR sensor technologies as well as the UAV platforms and autopilots have advanced significantly.UAV laser scanning has become a common tool for a wide variety of mapping and modelling applications, and the field is being rapidly developed by a diverse range of actors.

Are laser scanners installed on UAVs comparable to airborne sensors?
UAV laser scanning fills the gap between traditional airborne laser scanners (ALS) and terrestrial mobile laser scanners (MLS).Compared with ALS, UAV systems allow for denser point clouds and easier operation, lower cost in small area projects, and short response times than those typical for high altitude aircraft or helicopter-based system.Compared with terrestrial and ground mobile systems, UAV laser scanning provides a multi-direction perspective that frees the platform from moving on the ground.The wider field of view of some UAV sensors, that are often installed on instruments with 360-degree capacity, provides along track views instead of a single nadir looking scan plane.This enables the improved capture of vertical features such as building walls (see Fig. 4).The relatively short (usually < 200 m) measurement ranges for UAVs keeps the beam spot size within centimetres on the target surfaces, instead of a few decimetres for ALS, and allows thus for the reconstruction of centimetre scale objects, supported by the reflectance or intensity information.This allows, for instance, to perform branchlevel analyses of tree structure and use reflectivity for species classification (Hyyppä et al., 2020), the detection of road paintings, or vegetation in proximity of cables, delivering results comparable to terrestrial instruments.
There is a clear division in the market towards high-end and low-cost UAV platforms and instrumentation.High-end laser scanning systems provide faster data rates with more accurate and dense point clouds through high-end UAV platforms, positioning units and laser sensors, but they request a significant capital investment.Low-cost platforms use consumer sensors and are more affordable for many entry level users.These systems can reach similar point densities as compared to high-end systems, but the accuracy and precision of the acquired data is typically lower (e.g., point wise geometric accuracy).High-performance scanners provide longer ranging capability (up to 1 km) that requires high performance Inertial Measurement Units (IMUs) to be exploited, as the point wise spatial accuracy degrades with increasing range.Residual vibrations of the platform, not "reconstructed" by the IMU, propagate into the point cloud, and are difficult to compensate as there is no overlap between the adjacent scan lines.ALS systems are vibrationdampened, and often mounted on "2-axes-stabilised" platforms to maintain the sensor orientation towards the target regardless of turbulence, while UAV payloads are only suspended by wire or rubber dampers.
LiDAR provides discrete points in 3D space, typically determined by the detected first, intermediate, last and strongest echoes solved from the backscattered ToF (Time-of-Flight) signal, though alternatives exist, e.g., phase-shift and frequency modulation ranging.LiDAR sensing does not suffer from sunlight shadowing which can hamper 3D reconstruction, and has better penetration through vegetation as compared to passive sensors (Vastaranta et al., 2018;Wittke et al., 2019;Yu et al., 2015).Early airborne scanner systems returned echo signal parameters such as echo length and/or amplitude of the strongest peak, while current high-end systems also provide reflectance data at large bit depth for multiple echoes solved from the signal of each transmitted laser pulse, enabling surface property characterization.These systems seem to provide promising opportunities (Kukko et al., 2020), but their widespread adoption as well as the wide use of miniaturized multispectral LiDAR sensors will take several years to become the standard on UAVs.On the opposite, the reflectance properties are currently not considered in many low-cost instrument designs, due to their cost and complexity.In this regard, typical commercially available wavelengths implemented in UAV operable sensors are 532, 865, 905, 1064 and 1550 nm, though others may exist, while the spectral bandwidths are typically a few nm. 16 These bandwidths, integrated in a unique sensor, would allow for the implementation of a multi-spectral laser scanning system.
Beam divergence, which defines the illuminated area by a single laser pulse, is around 0.3-0.5 mrad in high-end UAV laser scanning systems (such as Riegl miniVUX19 series, Amuse Oneself TDOT 20 , Optech 21 CL-90), providing 1.5-2.5 cm and 3-5 cm beam footprints at 50 m and 100 m distance from the scanner, respectively.On the contrary, low-cost solutions (such as Velodyne Puck VLP-16 22 , Ouster OS2-128 23 , Livox AVIA 24 and RoboSense RS-Lidar-32 25 ), have a larger beam divergence (between 1.5 and 6 mrad) that provides 8-30 cm and 15-60 cm footprints at 50 m and 100 m distance, respectively and introduces point localization uncertainty resulting in blurred point clouds independently from the beam direction and distance measurement accuracies.Low-cost instruments provide only the last and the strongest echoes of the returning signal, without guaranteeing that the highest canopy elevations are accurately captured.However, these types of sensors offer good detection thanks to designed mutual overlap of the adjacent beam spots, while small beam size of the high-end sensors brings forth a challenge for penetration as the whole beam gets easily blocked by branches or leaves of a few centimetres in size.
The laser scanning sensors can be further categorized as single and multi-layer beamed instruments.Single-beam 2D scanners typically direct the laser beam perpendicular to the mirror rotation axis, as Riegl miniVUX-1UAV 26 and Optech CL-90 27 do, virtually spanning a single scan plane in 3D space.However, movement of the sensor along the mirror rotation axis direction generates a helix in the 3D space when projected on a cylinder coaxial with the direction of the movement.On the opposite, the multi-layer rotating scanners create a series of lines simultaneously.These systems often acquire along a set of parabolas when projected onto a planar surface as each beam draws a virtual cone with obtuse angle corresponding to its nominal off-nadir angle.The curvature of the parabola increases with the growing off-nadir angle of a particular laser line, which greatly affects the point pattern and density on the ground further away from the scanner nadir.When such a scanner is mounted in a tilted position, this effect is further amplified.Other solutions adopt more complicated scan patterns (e.g., Livox Horizon 28 ) although their use is still limited in UAV literature.It is also worth noticing that low-cost sensors often suffer from severe inconsistencies in the geometry of the measurement layout, even up to a couple of degrees (Putkiranta, 2019).Interesting additions to sensors hosted on UAVs are given by arrayed LiDAR sensors initially conceived for the autonomous driving and traffic surveillance markets.One example is Cepton SORA200 29 , but also Neuvition Titan M1 30 fits in the weight range for large UAV applications, and the launch of Velodyne Velarray H800 is anticipated to breach into the UAV use at some point as well.

What are the technological challenges?
Beside the issues discussed above, UAV LiDAR has still some technological challenges to face.
Point cloud georeferencing.The GNSS/IMU integration allows for precise positioning and attitude determination of the sensors to be carried out during the flight.In the last years, many companies (such as Applanix 31 , iMAR 32 , NovAtel 33 , Advanced Navigation and SBG 34 ) have presented GNSS/IMU systems sufficiently light and efficient to be adapted to UAV.A centimetric level of accuracy can typically be ensured on the global positioning using multi-constellation satellite observations and DGNSS post-processing.Though rapidly developing, the current UAV LiDAR sensors do not acquire data with high-resolution at wide longitudinal (along track) angles: this limits the adoption of scan-to-scan matching algorithms to orient the acquired data in real-time solutions, making the use of GNSS/IMU systems mandatory.The attitude accuracy (i.e., roll, pitch and yaw/heading) depends on the type of adopted inertial sensors.In this regard, Microelectromechanical systems (MEMS) technology is mostly used on UAVs because of the reduced weight, and low price (Sahawneh and Jarrah, 2008;Strohmeier and Montenegro, 2017), although it is still unable to deliver precise enough orientation data for long range scanning in terms of angular accuracy and data rate, which limits the achievable accuracy in high dynamic conditions.Relatively short measurement ranges and altitudes typical of UAV laser scanning serve to maintain the spatial accuracy achievable to within reasonable limits, and processing of the point cloud can improve the internal data accuracy through subsequent trajectory optimizations.Long range / high altitude applications, however, necessitate higher grade IMU units.As an example, a measurement accuracy of roll/pitch and heading of 0.015 and 0.035 degrees respectively, which is typical of many instruments installed on a UAV, translates to 2.6 and 6.1 cm spatial errors at 100 m range.Points at the nadir and at the far end of the field-of-view at 50 m above the ground are measured at an accuracy better than 3.6 cm and 7.1 cm, respectively.This value needs to be summed to the positioning and ranging errors, which further increases the cumulative errors of a single point (although these errors are often temporally correlated and can be reduced by data adjustment in postprocessing).The typical low flight altitudes forced by the drone aviation regulations along with the development of data post-processing and matching methods compensate for these performance deficits.The final point cloud accuracy can be considered below 10 cm in most of the cases.Narrow FoV sensors are very well suited for the airborne perspective for area and corridor mapping, as they typically provide higher single-pass data density, while scanners with 360-degree FoV can collect 3D data for building and infrastructure asset modelling, detection of vertical objects and improved ability to reach beneath forest canopy and structures with lower angular resolution (see Section 4.2).
Weight.LiDAR instruments are still relatively heavy (payload with GNSS/IMU and battery easily about 1,5-3 kg) and their use is thus limited to larger UAV platforms, though smaller sensors have just recently become available 35 .Heavy payload, comprising of LiDAR and positioning sensors, control and data storage unit and necessary batteries, has negative implications on flight times, operability in high winds, different weather conditions, and operations in populated areas (Stöcker et al., 2017).
Flight time.On multi-rotor UAV systems, point density can be thousands of points per square metre, as such platforms could fly very slow, and even hover stationary, although this is avoided because of the lower heading accuracy achievable under such conditions.The typical flight speeds are around 5-10 m/s to allow for adequate along track sampling (5-10 cm for a sensor with 100 Hz scan frequency), and flight duration is typically around 15-30 min on battery operated UAVs.Fixed-wing UAVs typically need air speeds from 80 km/h and above with the current LiDAR payloads, so their performance is similar to those of manned aircrafts.ALS data acquisition usually exhibits parallel flight lines with a limited number of crossing lines, and narrow side overlap for efficiency, while implementation of UAV laser scanning is often carried out in a grid pattern, or some other form that ensures data coverage over the desired objects or area with a different density.With gasoline hybrid fixed wing aircrafts, copters or multi-copters, the flight time can be increased significantly, up to 2-3 h.In these cases, the flight operations require more careful planning in terms of air space reservation and weather forecasting.
Calibration.LiDAR sensors usually constitute fast moving parts in the form of rotating mirrors and laser heads, depending on the implementation.This has currently two main consequences in their use on UAVs: (i) sensors are consume much more power than cameras, and (ii) these instruments necessitate maintenance/calibration during the life span, which typically are not under user's capacity, in contrast to the widely adopted calibration schemes for image data.In this regard, the developments of solid state LiDARs (Lemmetti et al., 2021) with reduced costs could represent a turning point for this technology.

System calibrations and flight endurance
The last decade witnessed the improvement of the hardware installed on UAVs.In this section we focus on how these components can be better exploited, with specific regards to onboard sensor integration and battery life.An overview on the strategies to fuse data acquisition sensors with the onboard navigation instruments is first presented with the aim of exploiting these instruments at their full potential.Additionally, we describe the performance of modern batteries, traditionally a bottleneck for UAV applications, and we give some hints on promising strategies that could change our use of UAVs paving the road for new applications.

System lever-arms and bore-sight: What you need to know
The observations from optical and navigation sensors principally refer to different origins and axes orientation.As the latter may significantly affect even the lever-arm of the GNSS antenna(s) and IMU, all offsets need to be resolved in precise (cm-level) mapping.

How to determine a lever-arm?
Although the spatial separation (lever-arm) between devices on a micro-UAV is relatively small, employing Real-Time Kinematic/ Post Processed Kinematic (RTK/PPK) technology generally requires its consideration.In such a case the first spatial offset to examine is the distance between the IMU navigation center and the phase center of the GNSS antenna, r GPS IMU , the second one is between the navigation solution and an optical device r c IMU, (Fig. 5).If the bundle adjustment does not use directly raw inertial data (Cucci et al., 2017), the former lever-arm needs to be expressed in IMU-body axes, so the GNSS/IMU software can account for the differences in relative velocities between those two sensors.This fact faces two practical challenges: (i) the IMU-body axes are not directly observable; (ii) in contrast to larger airborne devices, the accuracy of small IMUs prevents static self-alignment with respect to geographical North.
Although an INS/GNSS software may allow "estimating" the leverarm, the obtained precision is sufficient only for larger carriers with more precise IMUs.Different methods are then needed whenever the errors in level-arm measurements (e.g., performed by a caliper via a 3D UAV model) exceeds 1 cm once expressed in the camera reference frame (Daakir et al., 2017).It should be noted that this limit is already reached for a 30 cm long lever-arm if the error in the assumed camera bore-sight exceeds 2 degrees.In these cases, the surveying method exploits the concurrent determination of the lever-arms among IMU, GNSS-antenna and an optical device, e.g., camera, together with the orientation of the camera while considering the bore-sight between camera and IMU (determined separately).This method is based on taking pictures of a field equipped with many optical targets while simultaneously measuring the position of the GNSS antenna with a theodolite (Fig. 6 and Rehak and Skaloud, 2015).For all small UAVs this method can be performed on ground.This approach is also applicable without IMU, i.e. for determining the GNSS antenna-camera lever-arm in the camera axes, and generally more precise than its determination via bundle adjustment (BBA) (Rehak and Skaloud, 2017a) where it is often (strongly) correlated with other parameters.
Larger uncertainties of the lever-arm between the navigation and optical devices can, however, be tolerated when BBA uses relative aerial positions within the (straight) flight-line, e.g.(Blazquez, 2008) for a general airborne case and (Rehak andSkaloud, 2016, 2017a) for UAVs.The absorption effect using relative positioning with unknown lever-arm is similar to that of GNSS-position biases, e.g., due to a wrongdetermined carrier-phase ambiguity within RTK/PPK (Skaloud et al., 2014) because of existing correlations among these parameters within regular flight-lines.The presented approaches refer to fixed-wing platforms, while the estimation of the lever-arm on rotor platforms is further complicated by the rotations of the gimbal and the uncertainties in their measurement (Ekaso et al., 2020).
3.1.2.When is the bore-sight needed?.The determination of an orientation offset, so called bore-sight, between an IMU and an optical device on a micro-UAV needs to be performed in-flight.Methods based on the static alignment of the IMU (thanks to gravity and Earth rotation), e.g., (Bäumker and Heimes, 2001), cannot be used due to the previously mentioned incapacity of small IMUs to complete its "alignment" without certain velocity and orientation dynamic while using GNSS observations.The dynamical aspects of UAV trajectory are needed for determining not only the azimuth but also mitigating the influence of (large) accelerometer biases on roll and pitch angles.Once the IMU is aligned (i.e., the uncertainty of its attitude reaches its stated capacity) the determination of bore-sight follows.In aerial-triangulation the need for determining bore-sight depends on the chosen method of orientation (Blazquez and Colomina, 2012a;Colomina, 1999).The integrated approach, i.e., the concurrent use of image observations together with INS/GNSS trajectory, allows for the determination of bore-sight within the BBA.In such a case even inaccurate IMU alignment can be tolerated because it will be "absorbed" via bore-sight parameters due to similar influences.However, such absorption is valid only for the current flight and not for subsequent projects.
The correct "calibration" of bore-sight requires the de-correlation between the IMU-alignment, bore-sight and other parameters, such as interior orientation (lens calibration parameters).For that purpose the strategy follows that which has been user for larger platform (Lichti et al., 2008): the block geometry should be strong in terms of sufficient overlap, possibly some converging images, existence of cross-strips with variations (>15%) in AGL and depth of field, the use of precise observation of poses through INS/GNSS (e.g. also called exterior orientation or aerial position and attitude control) and possibly some Ground Control Points (GCPs).Obviously, the choice of interior models also affects the bore-sight (Cledat et al., 2020a).Achieving or maintaining sufficient accuracy of IMU alignment is generally challenging for some small UAVs, especially in (frequent) hovering operations.In such a case the desirable approach of dealing with UAV bore-sight is either using relative orientation within the BBA that removes its need analytically (Rehak and Skaloud, 2016;Skaloud et al., 2014), or the introduction of raw inertial observations (Cucci et al., 2017).Nevertheless, a correct knowledge of bore-sight is required for direct orientation, which is the chosen orientation method for UAVs with LiDAR.There, the bore-sight determination technique may, for instance, follow (Skaloud and Lichti, 2006).The obtained accuracy is likely to be driven by the quality of IMU attitude.For a given inertial hardware the attitude performance can be considerably increased when using redundant configuration sensors (for reducing random noise), or by applying on-site calibration before takeoff (for improving absolute attitude precision) as discussed in (Clausen and Skaloud, 2020).

How to check time-stamping?.
A key prerequisite of the correct usage of navigation sensors for orientation purposes is the correct time stamping of acquired imagery in the global time frame in which the navigation sensors operate.This issue is especially critical when utilising UAVs with consumer-grade cameras for which the precise time registration with navigation components is not trivial to realize and its performance not easy to assess.Even professional drones with factory payloads may experience some issues with correct time-stamping such as limited resolution of a system (autopilot) time, some residual offset   (Rehak and Skaloud, 2015).
between system and GNSS time, registration of start or end time of exposure rather than mid-exposure, etc.The influence or tolerance of time-stamping errors will depend on the ground sampling resolution and trajectory accuracy w.r.t.UAV operational speed and angular rate (e.g., see graphics in (Rehak and Skaloud, 2017b)).There are several techniques for determining whether registration delay is present in the optical data.The methodology depends on the available observations and on the a-prior knowledge of system and sensor calibration parameters.
The classical models for using absolute (w.r.t. to mapping frame) or relative (between successive images) observations of pose through INS/ GNSS (aerial position and attitude control) can be extended to spatiotemporal aerial control.This, under certain conditions, may allow determining time-stamping delays within the BBA as investigated by (Blazquez, 2008) for the case of manned aircraft and by (Rehak and Skaloud, 2017b) for UAVs.The latter work confirmed that a correct recovery of a constant time-stamping offset is possible within the BBA using velocity observations when the UAV ground speed varies sufficiently.At the same time, it showed that its final impact on mapping accuracy is relatively small in block configurations where it gets absorbed/compensated by other self-calibrated parameters, e.g., the principal point coordinates.However, in scenarios of lower redundancy such as the single strip operation, the induced errors cannot be absorbed and therefore, the impact on ground accuracy is significant.

When to use an IMU?
. As the answer to this question is linked to other navigation-orientation topics presented in Section 4.2, we provide only a general response herein.The use of IMU is required for direct orientation (implicit for LiDAR) of the platform w.r.t. the local mapping frame (e.g.E, N, h system).It is also needed for mount stabilization (frame or line cameras) and very useful for aiding the final orientation of line and even frame cameras in corridors, or other geometrically less favourable scenarios (Rehak and Skaloud, 2015).
It should be considered that a multi-rotor UAV without an IMU cannot fly at all, and generally, all UAV autopilots need IMU for drone stabilisation, control and guidance.The fact that an IMU is practically always onboard, but not always available or exploited for orientation purposes, has more historical than pragmatic reasons, because its benefits have been known for more than two decades (Colomina, 1999).The evolution of the professional drone market started with frame-cameras and there it caught up with academic propositions of IMU usage (e.g., (Bäumker et al., 2013;Rehak et al., 2013)) by implementing separate IMU/GNSS boards (initially almost exclusively from Trimble-Applanix 31 ) on high-end hovering drones, e.g., (Mian et al., 2015).Manufactures of small fixed-wing drones identified its benefits only later and those producing its own payload, employed a conceptually better design where only the IMU is rigidly mounted to the camera and the small payload is isolated from vibrations.The hardware quality of such newer small IMUs is potentially sufficient for performing good relativeorientation (Rehak and Skaloud, 2016) and with advanced INS/GNSS processing (Clausen and Skaloud, 2020) or tight integration (Cucci et al., 2017) even absolute orientation applicable to lasers (Cledat et al., 2020a;Vallet et al., 2020).Nevertheless, the IMU potential remains underexploited in the popular UAV photogrammetric packages, e.g., Pix4D 11 , AgiSoft Metashape 10 .This is not only in the absence of possibility of using raw inertial observations in both programs but also the missing input for relative orientation or even user-definition of weights for absolute orientation.Especially in difficult scenarios (geometry, texture) using the IMU data correctly according to its capacities (Blazquez and Colomina, 2012a;Cucci et al., 2017;Rehak and Skaloud, 2016) can improve not only the final orientation and calibration of framecameras but also the automated process of tie-feature identification and matching.

Energy consumption and flight time
UAV flight time is one of the main limiting factors compared to airborne platforms, limiting the extension of the surveyed area and the temporal frequency of data collection.However, battery limits and the introduction of recharging stations promise to drastically mitigate this drawback in many applications.

3.2.1.
Where is the battery limit?.Energy consumption and battery life are two limiting factors affecting the productivity of UAVs in all their applications.For this reason, many studies on the optimization of energy consumption (Dietrich et al., 2017;Hwang et al., 2018) have been published in the last years.The first prototypes of UAVs developed >10 years ago were often powered by internal combustion engines (Eisenbeiss et al., 2005).However, the introduction of smaller and lighter models has pushed the widespread adoption of electrical engines powered by Lithium batteries.Batteries are usually characterized by their capacity (indicated in mAh) that measures how much energy the battery can store.The number of cells in each battery determines the voltage of the battery, while another important element is given by the number of cycles each battery is able to perform.These devices are lighter and with higher power to weight ratios compared to combustion engines, representing most propulsions of UAVs used in geoscience and remote sensing.Lithium batteries have improved incredibly over the last few years enabling UAV flight times to double (sometimes triple).The energy density of these devices has been constantly improved in the last decade, allowing their extensive uses in several fields, while the balance between energy and size/weight of the battery has strongly influenced the design of platforms and sensors to host onboard.Among the different typologies of batteries, Lithium Polymer (i.e.LiPo) batteries are most commonly used on UAVs because of their reduced weight compared to their alternatives such as Lithium-Ion.LiPo batteries have shown an increase in their energy density of about 3% per year (Zu and Li, 2011) and could reach their performance ceiling in 2025 (Galkin and DaSilva, 2018).Other types of batteries such the Lithium-sulphur are showing extremely promising results and could lead to better solutions in the near future (Service, 2018).Hybrid-powered systems (i.e.solar coupled with hydrogen) have replaced internal combustion engines only for larger and long-endurance flight drones (Lei et al., 2019).
Off the shelf UAVs have different flight endurances according to the type of platform considered.Large differences (a factor 1:3) is usually noticeable between rotor and fixed-wing drones: quadcopter can fly for 30-35 min max in optimal conditions while several fixed-wing models can stay in action for almost 2 h.On rotor UAVs the number of sensors installed in the payload has a large influence on the flight time: in this regard, many manufacturers report detailed analysis on this (e.g., DJI Matrice 60036 ).Live streaming of images, lights and other accessories can further reduce the battery level.The way to conduct a flight (number of turns, speed variations and average speed) can also drain batteries (Dietrich et al., 2017).Environmental conditions, such as higher wind speed (Tseng et al., 2017) and changes in altitude (Paredes et al., 2017), can negatively influence the flight time up to 20%.

3.2.2.
Are 24/7 UAV applications possible?.The use of UAVs in monitoring applications, with repeated flights and 24/7 acquisitions has recently pushed the development of autonomous procedures to recharge the platforms without human intervention.All these approaches are mainly conceived for rotor UAVs, as the ones more limited by flight time (Galkin and DaSilva, 2018).Repeated flights can be tackled according to three different approaches.(i) using multiple drones that alternate their flight over the interest area.(ii) automated battery swapping where robotic actuators can change autonomously the UAV batteries: several prototypes have been developed in the last years (Barrett et al., 2018;Herath et al., 2017) that have the capacity to change a battery within 60 s.(iii) wireless transfer power approaches that can be divided in electromagnetic field (EMF) chargers and non-EMF chargers (Lu et al., 2018).In this last category, magnetic induction or similar techniques are used to transfer energy to the drone, adopting specific recharging stations where the UAV lands to be recharged (Junaid et al., 2017;Rohan et al., 2018).The recharge is performed using short distances (i.e., few centimetres) and it is often unable to refuel the battery in very short times.Non-EMF systems refer to photo-voltaic cells installed on the UAVs (Jung et al., 2019) to keep the battery level high: these solutions are usually adopted for large drones, as they are not feasible for small solutions.Other solutions adopted directional laser beams (Achtelik et al., 2011) directed on a modified photo-voltaic panel installed on the drone.In this case, the visibility between recharging station and UAV is a requirement of the method.
In the last few years, the number of commercial solutions embedding UAV, recharging station (often called "drone in a box", see Fig. 7) and data processing, has largely increased, providing solutions for continuous monitoring of entire areas (Percepto37 , Skydio38 , AirRobotics39 , Mapture40 , etc.).The search for more efficient and long-lasting flights have also pushed to improve the aerodynamics and the configuration of the platform (Dai et al., 2019).Dedicated platforms have designed propellers specifically optimized for specific tasks such as hovering.Other asymmetric configurations of UAV propellers have been conceived to optimize the energy consumption for specific tasks (Verbeke et al., 2014).

Flight planning, acquisition and adjustment
UAV data acquisitions are devoted to a growing number of applications requiring different accuracies and facing different challenges according to the environmental conditions where UAVs operate.In this section, we give an overview on the best practices for "conventional" mapping and we present the main challenges to be faced in GNSS-denied environments.We then describe two new emerging operational scenarios (i.e., collaborative UAVs and BVLOS) and we discuss their current limitations and the possible solutions for their wider adoption in our domain.

Mapping tolerances and orientation requirements
The mapping tasks and therefore the specifications for individual UAV-projects might differ considerably in daily practice or research.For projects where geometric properties of target objects need to be determined, the achievable 3D-point position accuracy in object space is the most important parameter.Depending on the discipline in which UAVprojects are embedded, we need to distinguish between tolerances and standard deviation.In mechanical or civil engineering, a tolerance is a maximum allowable difference to an optimal (or pre-defined) value, while in measuring disciplines accuracy or error very often refers to a standard deviation.In order to convert between tolerance T and standard deviation σ, we assume that besides the random error inherent in any measurement process no systematic or gross errors remain and that the random error is normally distributed.According to (Kuhlmann et al.,   2017), σ = T/4, applies for a probability of error of 5% for the measuring accuracy to be maintained.In the following, the standard deviation will be considered as reference parameter.

RTK and PPK: How accurate can they be?
With the development of UAV photogrammetry, where very highresolution images are acquired, a "dilemma" becomes obvious: the inner or relative accuracy within the image block is usually very high (rule of thumb: ½GSD to 1xGSD in X, Y, 1xGSD to 2xGSD in Z).To ensure this accuracy also within the reference frame (datum) and to prevent socalled block deformations, it is necessary that the standard deviation of control points is better by at least a factor of 3 compared to the required standard deviation.According to the application, required standard deviations can be 2-3 cm in planimetry and height (e.g., topographic surveys) or up to 2 mm in planimetry and 3 mm in height (e.g., engineering surveys).This means that the inner or relative accuracy can only be assumed over the whole block under certain conditions.
If, for example, only GNSS RTK is available for the determination of control points (optimistic assumption: σ X,Y,Z = 2 cm), a thorough accuracy analysis can only be performed for a GSD of 5-6 cm.If, on the other hand, a (local) total station network is created (realistic assumption: σ X, Y,Z <=3mm), a GSD of approximately 1 cm can be checked and realized in the entire network, given that the control point arrangement is good enough.An example for a possible solution to combine very high accuracy terrestrial network survey with UAV-based image acquisition is shown in Fig. 8: A reflector-based GCP signalisation for tachymetric survey is integrated with a coded target for the airborne image data, so it can be used as a 3D point for image orientation.This system has been developed and tested within a rail inspection project (Ghassoun et al., 2021).
Research has shown (Benassi et al., 2017;Gerke and Przybilla, 2016;Varbla et al., 2021) that a direct observation of the sensor position on the UAV leads to better overall block accuracy compared to only using indirect sensor orientation through GCPs.When it is concluded that a differential GNSS (DGNSS) on the UAV for direct position estimation might be good enough for a particular project, there might still be a choice between RTK and PPK as shown in Table 3. From this table, it is clear that many factors can influence this choice.In addition, the expected accuracy is better for PPK, especially under higher dynamic or sub-optimal signal reception (Cledat et al., 2020b) because more historic and forward correction data can be used.Whenhowever -DGNSS is needed for accurate navigation and not only for photogrammetric applications, RTK is indispensable.A general problem using DGNSS onboard is that of sensor synchronisation and lever-arm alignment, as discussed in Section 3.1.

UAV mapping:
What are the strategies to improve the accuracy?.It is known that point measurement accuracy increases with multiple image overlaps.For example, (Förstner, 1998)  side-lap usually depends on the available flight time and thus might be more difficult to be realized.
Ground control points basically have two functions: they are needed to transform the image block into the mapping datum and to increase the overall block accuracy, that is, to minimize block deformations.For the first task, the datum definition, three well distributed 3D-GCPs are theoretically sufficient to perform the similarity transformation between the image block and the datum, defined through the GCPs.If onboard GNSS is used, no additional GCPs are theoretically needed, but research (Benassi et al., 2017;Gerke and Przybilla, 2016) showed that a remaining absolute shift parameter might be needed to be solved through one GCP.However, if a minimal quality control and verification is needed it is advisable to follow the usual rules of distributing GCPs in a regular pattern in the site (Stöcker et al., 2020).The density of this pattern is also depending on terrain complexity.
With the exception of large photogrammetric camera installed in professional, manned aircrafts, it cannot be assumed that the internal camera geometry is stable over a longer period of time (see Section 2.1).This means that the parameters of the interior orientation, i.e., focal length, the position of the image principal point and the parameters that determine the lens distortion should be estimated in the BBA.There is a strong correlation between focal length and flight altitude, or distance of the camera to the object space, in the case of vertical images.For this reason, it is advisable to realize different flying altitudes during the mission.If the target area does not show large natural differences in altitude (20-30% difference), it is advisable to "artificially" create this scale difference by realizing two flights at different heights (Gerke and Przybilla, 2016).Furthermore, it is shown that oblique images (nick angle 20-45 degrees) contribute to the increase in accuracy (James and Robson, 2014;Nesbit and Hugenholtz, 2019;Rupnik et al., 2015;Verykokou and Ioannidis, 2018).This means that in addition to vertical images, flight patterns with oblique images aligned in all four cardinal directions are introduced into the BBA.The reason for the increase in accuracy is that the correlation focal length -distance is thereby reduced.Experiments have shown that in the ideal case all three coordinate components can be estimated with equal accuracy (Rupnik et al., 2015).
4.1.3.How to plan flights for accuracy in complex environments?.So far, we assumed that flight planning is done in simple terrain and above three-dimensional structures, that is, GNSS reception is guaranteed and reliably available.However, the mapping quality is not intuitively predictable, particularly in complex environments (e.g., in urban and industrial surveying, or in cluttered mountainous environments), where the accuracy of the RTK or PPK positioning varies, and it might be difficult or time intensive for operators to conceive a flight plan that would cover the entire area of interest with uniform resolution and without leaving any uncovered part.In such a case, the conventional loop of data acquisition, post-flight processing, and derivation of the quality control at the end of the survey process is not ideal, as aerial position control quality is likely to depend on the time of a flight.While conventional aerial surveys can be easily planned in two dimensions, cluttered environments call for flight planning software that allows for the definition of complex three-dimensional trajectories (Gandor et al., 2015;Gómez-López et al., 2020): such tools provide the user with fine control of the flight trajectory and allow them to check for the GSD variation, the image footprints, and ensure that the area of interest is fully covered and with sufficient overlap, according to the digital models available for the area.Some features of those are now implemented in Fig. 8. High-precision reflector for tachymetric network survey integrated with a coded target for photogrammetric processing: left terrestrial view, right: image from UAV-based acquisition @ GSD = 0,9mm (Ghassoun et al., 2021).advanced commercial flight planning software 41 .In addition, planned trajectory at different times can be considered already in the planning phase to obtain a probabilistic measure of expected GNSS precision.
As demonstrated practically on a number of flights in (Cledat et al., 2020b) this information can be further combined with other elements depicted in Fig. 9, as the requirement of calibration of interior orientation, prevailing type of texture or foreseen placement of GCPs, to simulate the acquisition process realistically and derive the expected mapping quality before the mission.(Cledat et al., 2020b) demonstrated practically that the PPK approach is preferable over the RTK technology in an environment where frequent occlusion of satellite signals occurs owing to either the drone motion or its surroundings.For the sake of navigation safety, the method of predicting the satellite positioning quality is thus worth considering not only at camera stations, but also (possibly using a different criteria) over the entire drone trajectory, including the take-off and landing zones.

Poor or deniable GNSS environments and other operational scenarios
The use of UAVs in unconventional spaces such as indoor or GNSS denied environments (where no GNSS signals can be received or they are unreliable, e.g., because of multi-path or strong electromagnetic interferences) are getting more popular, despite the additional challenges for the safety and the quality of the collected information.

Can UAVs navigate without GNSS?
Open-sky navigation can nowadays be considered as solved employing low-cost GNSS and inertial sensors available on the consumer electronics market.These are commonly employed, for example, in mature open-source and open-hardware ecosystems, e.g., (Meier et al., 2012).On the contrary, when navigating without GNSS (e.g., because of obstructions, multi-path or jamming) no absolute position/velocity measurements are available with respect to a well-defined Earth ellipsoidal reference frame (e.g., WGS-84).Inertial sensors could fill the gaps between GNSS fixes, however, the ones currently employed in autopilots do not allow to cope with outage times greater than a few to dozens of seconds: after that, the position drift will become too severe (it can be hundreds of meters in a matter of minutes) even to ensure safe landing.Many ideas have been proposed to tackle this problem, which can be essentially divided in the following two approaches.The first is to obtain position/velocity measurements from other sources or sensors.Replacing or augmenting GNSS with other positioning methods requires us to distinguish whether we need absolute positioning with respect to geographic coordinates (e.g., in mapping applications) or if relative positioning with respect to the environment is sufficient.The second approach consists of attempting to reduce the drift in the position and attitude estimates obtained with inertial sensors by means of more sophisticated models of such sensor and/or the platform.The main source of relative positioning observations is nowadays led by the visual and visual/inertial methods, such as visual and laser Simultaneous Localization and Mapping (SLAM) (Artieda et al., 2009;Caballero et al., 2009;Cadena et al., 2016;Stachniss et al., 2016).In visual SLAM, monocular or stereo images are used in real-time to simultaneously construct a map of the environment and localize the camera (and thus the platform) with respect to it (Armenakis and Patias, 2019;Durrant-Whyte and Bailey, 2006;Thrun and Liu, 2005) thus solving the guidance problem of UAVs without the need of a GNSS sensor (Cioffi and Scaramuzza, 2020;Scaramuzza et al., 2014).Visual SLAM algorithms can be divided in feature-based methods and direct methods.Feature-based methods rely on the tracking of distinctive features in the images (Campos et al., 2020;Mur-Artal et al., 2015) while direct methods rely on the actual pixel intensities, attempting to employ the entire image frame (Cremers, 2017;Engel et al., 2014;Yang et al., 2020).Direct methods can generate denser representation of the environment (useful for collision avoidance) than feature-based algorithms, although their reconstruction can have lower geometric qualities (Gaoussou and Dewei, 2018).Both feature-based and direct methods are unable to detect tiny objects, making necessary the use of ultrasonic, infrared or time-of-flight sensors to further improve the collision avoidance in proximity of objects (typically <2 m).
Typically, the maps generated with visual SLAM methods are not geo-referenced and their scale is not well determined (except for visual/ inertial systems, where the metric scale is determined to some extent, depending on the quality of the inertial sensors and their real-time bias determination).The limitation of these systems is that they depend on the presence of a sufficient unambiguous information in the images.SLAM systems can be deceived by repeating or self-similar patterns and cannot be used during night or in high dynamic range lighting conditions.However, promising research based on event cameras has been shown to largely mitigate lighting conditions issues (Gallego et al., 2020;Vidal et al., 2018).Another limitation is that cameras need to be carefully calibrated beforehand.If an a priori, geo-referenced map is available, visual based methods can localize with respect to it and thus produce absolute position estimates.In the last years, laser SLAM, or LOAM (Lidar Odometry and Mapping) has been implemented using lowcost multi-layer scanners (e.g., Velodyne 24 , Ouster 25 and RoboSense 42 ) and low-cost inertial sensors.While the sensor directly outputs a point cloud of the surroundings, successive scans need to overlap to recover the platform motion.Laser SLAM is applicable only with 3D scanners, 2D scanners mounted on rotating supports (such as CSIRO/HoverMap, Zeb Horizon, GeoSLAM), or solid-state lidars (Nam and Gon-Woo, 2021).Typically, laser SLAM algorithms rely on the successive registration of locally consistent 3D scans by means of the Iterative Closest Point (ICP) algorithm (Chetverikov et al., 2002).However, fast platform motion, as is typical in UAVs, introduces non-negligible distortions in single scans which need to be corrected, e.g., by employing inertial sensors (Bosse et al., 2012;Ceriani et al., 2015).Modern laser SLAM algorithms are based on factor-graph formulation of the localization and mapping problem and are also capable of incorporating measurements from cameras (Lowe et al., 2018).The advantage of laser SLAM compared to visual SLAM is mainly given by the possibility to work in dark environments too (Sofonia et al., 2019).
Alternative ways, to replace GNSS sensors or augment them with similar concepts are given, for instance, by Ultra-Wide Band (UWB) ranging sensors (Adams et al., 2001), or pseudolites (Amt and Raquet, 2007;Rizos, 2013).These technologies mimic the GNSS principle of operation, where the receiver measures the distance with respect to multiple beacons for which the location is known a priori.These are typically placed on the ground and their position must be surveyed separately.Thus, operations are only possible in environments that have been structured beforehand.UWB signals are also affected by the environment (reflections, multi-path, etc.) and by obstructions.Nevertheless, UWB beacons allow UAV to navigate even indoor (Queralta et al., 2020;Tiemann et al., 2015) and enable mapping in GNSS denied environments in the decimeter-level accuracy (Masiero et al., 2017).
Other approaches attempt to reduce the drift caused by the integration of noisy inertial readings without the need of position updates, as in the case of vision-based systems using Visual Odometry (VO) (Leutenegger et al., 2015;Wang et al., 2017), where the change of the camera position and orientation is determined by tracking local visual features and can be fused with inertial readings to control the position and orientation drift.While VO is substantially simpler than SLAM, the distinction between SLAM and VO is fading, as loop closure and global trajectory optimization are being added to VO systems to reduce the 41 https://www.sensefly.com/whitepaper/generating-highly-accurate-3d-data-using-sensefly-albris-drone/ accessed on 12 September 2021 42 https://www.robosense.ai/en/RS-LiDAR-M1accessed on 12 September 2021 F. Nex et al. long-term drift.In this regard, the first hardware implementations are also appearing (Suleiman et al., 2019).
The knowledge of the physical properties of the UAV can be used to obtain extra observations useful for navigation without adding any further sensor: the control inputs to the motors and to the control surfaces, as issued by the autopilot, can be used together with a physical model of the platform to obtain angular and linear acceleration pseudoobservations.These can then be fused with inertial ones in Extended Kalman Filters (EKFs) (Khaghani and Skaloud, 2016) and dynamic networks (Nisar et al., 2019).This approach is still in its infancy but has been shown to have the potential to mitigate position drift in GNSS denied environments by orders of magnitudes (Khaghani and Skaloud, 2018).
While nowadays the technology is ready for guiding a UAV in controlled environments without the need of a GNSS sensor, the application of academic research to the commercial sector is still young, namely because of the additional costs of the setup or because of difficulties of guaranteeing the performances, e.g., of vision-based systems in arbitrary conditions.On the other hand, no system is currently able to match the accuracy of RTK and PPK GNSS observations if not replacing those with special pseudolites (Rizos and Yang, 2019).In summary, all this makes it difficult today to perform accurate metric surveys of unknown environments without the stable and reliable reception of GNSS signals.
More accurate navigation sensors, especially inertial ones, would be greatly beneficial in high-accuracy mapping applications: every time an accurate orientation estimate is needed, e.g., in LiDAR, aerial triangulation for corridor mapping and direct geo-referencing, the current generation of inertial sensors that fits the weight, size and cost constraints of UAVs don't provide sufficient quality, in terms of noise and bias instability.Advanced processing methods have been proposed to cope with lower quality inertial sensors in such applications, as described, for example, in (Cucci and Skaloud, 2019;Cucci et al., 2017) for rigorous integration of MEMS inertial measurements into BBA, and (Cledat and Skaloud, 2020) for photogrammetry to LiDAR integration.At the same time, simpler methods that were introduced long ago such as the use of absolute or relative position and/or orientation information obtained for INS/GNSS systems (Blazquez andColomina, 2012b, 2012a) and whose potential for UAV orientation was practically demonstrated (e.g., (Rehak andSkaloud, 2017a, 2016;Skaloud et al., 2014)) are not available in commercial photogrammetric software yet (c.f., discussion in Section 3.1).

What are the emerging operational scenarios?
Beside the "classical" use of UAVs for mapping applications, new and more challenging operational scenarios have been emerging in the last years.Among them (i) the use of collaborative drones (swarms and integration with other systems, such as unmanned ground vehicles, UGV) and (ii) new solutions considering BVLOS flights promise to bring unmanned platforms closer to manned performances in many applications.
Collaborative vehicles.Small unmanned aerial mapping platforms have limited payload capacity, and therefore may be limited to a single sensor, and limited flying time.This has pushed the use of two or more unmanned vehicle platforms in synergistic mode to perform cooperative exploration and mapping tasks, such as executing complimentary tasks for data collection, information sharing, path planning, co-mapping generation and validation, and change detection between the 3D map databases and the actual 3D geometry captured by the sensors of the platforms.Such multi-UAV collaborative operations offer higher scalability of the operations, more limited mission execution times, reduction in risk due to redundancy, possibility of multi-modal data capturing and overall improvement in the performance for example in time-critical operations such as search and rescue (Arnold et al., 2018;Ruetten et al., 2020).According to the implementation, UAVs could carry different sensors, could have partial / complete overlap between the surveyed areas and be operated in local or global reference systems.With UAV platforms operating in tandem over a large area and collecting data in sync (i.e., swarm type missions), tasks and actions can be divided amongst the platforms to operate in fully collaborative mode.The operating architectures vary from centralized control and communication base-station of each UAV to distributed schemas of operations and information sharing among the UAVs.In this regard, more efficient communications (Chen et al., 2020) as well as Edge Computing (Ai et al., 2018) are enabling the efficient exchange of tasks and data sharing information.
In multi-UAV missions, path coordination and synergistic navigation (Albani et al., 2019;Jospin et al., 2019), environment exploration, task identification and sharing, operation synchronization and data fusion are areas to be addressed (Cledat and Cucci, 2017;Madridano et al., 2021;Merino et al., 2012;Tosato et al., 2019).Many algorithms have been developed to guarantee the efficient coverage, complimentary sensing and mapping due to environmental complexity (e.g., occlusions), and mapping operations with data sharing (e.g., pose, area coverage, complimentary mapping from two sensors, rendezvous locations).However, 3D navigation and mapping in large scale, natural, complex, and dynamic environments still remain an open research Fig. 9. Workflow for predicting mapping precision as in (Cledat et al., 2020b).
F. Nex et al. problem despite significant progress in this area (Cadena et al., 2016).The interest for multi-UAV SLAM approaches has grown in the last years, in many cases considering GNSS denied or low signal quality environments (Trujillo et al., 2018).Most of the works on this topic consider centralized architectures for collaborative UAVs that allow for the sharing and fusion of information to efficiently produced fused maps, extending the features (e.g., relocalization, loop closure) of the classical (single platform) SLAM approaches to the multi-platform case (Mahdoui et al., 2020;Schmuck and Chli, 2017;Yang et al., 2018).Although most of these approaches are still in the simulation phase, the improvement of onboard sensors and communication technologies is spurring, wider adoption of AI approaches (e.g., reinforcement learning-based navigation), as well as their clear potential in many applications are expected to further boost their development in the future.
Synergistic operations are mostly between UAV-UAV but a growing number of examples of integrations between UAV and a ground moving vehicle manned or unmanned (UGV), where the cooperative mapping extends the capability of a single robot by sharing and merging data between group members (Butzke et al., 2015;Lin et al., 2013;Olson, n. d.;Zhang and Singh, 2018), is available in literature as well.Teams of mobile mapping platforms (UAV/UGV) can effectively explore and map the environment where they can ensure consistency by combining their data in shared maps (Ropero et al., 2019).Collaboration of aerial and ground unmanned mapping platforms improves the planning of a global path.Aerial views or a-priori terrain data can provide information for global route planning while the platform's sensors can focus on resolving the surrounding local environment (Molina et al., 2017).UAV and UGV collaboration could involve real-time operations where UAV provide map data to the ground vehicle for route planning, ground mapping and synergistic mapping of occlusion based on multi-view data collection (Qin et al., 2019).
Beyond Visual Line-of-Sight operations (BVLOS).The flying field operation of UAV is subject to jurisdiction of the regions where they operate and needs to comply to numerous restrictions.While the regulations may vary from country to country, the common denominator is to ensure the elimination and mitigation of risks from potential in-flight accidents with manned aircrafts operating in the same air space as well as accidents with people and damage to properties, and security concerns.The most common accepted rule is the operation of UAVs under Line of Sight (LOS) (Stöcker et al., 2017).That is, the platform is operated under the unaided visual contact of the operator thus enabling the operator to maintain operational control of the UAV, scan the air space, scan for objects and know its location.An extension of LOS is the Extended Visual of Sight operation (EVLOS) where an extension of the beyond visual line of sight is obtained by using observers at different locations who keep the UAV in their line of sight and communicate their visual observations to the pilot.However, progress with the UAV capabilities and the recognition of the benefits to cover far greater distances for maximum efficiency (e.g., to handle extreme emergencies, fewer deployments to complete an air survey mission) has led to increase demand for Beyond Visual Line of Sight, where the UAV flies beyond the visual range of the operator while the operator pilots the UAV via a virtual cockpit (Davies et al., 2018).This allows UAVs to navigate without the direct supervision of human operator and conduct more complex asks (Dąbski et al., 2020;Wood et al., 2020).BVLOS is no longer dependent on the pilot to avoid any obstacles and the controlling of the UAV highly depends on data provided by onboard instruments transmitted via telemetry links as well as the ability of the UAV for situation awareness, that is the ability to sense and avoid obstacles.Certain countries are already considering permission for BVLOS operations under various conditions such as in isolated areas, atypical airspace, and uncontrolled airspace (Alamouri et al., 2021).In general, safe UAV BVLOS operations require systems able to perform different tasks (Fang et al., 2018): (i) stream real-time UAV trajectories using telemetry transmitters and Automatic Dependent Surveillance-Broadcast (ADS-B) systems (Ropero et al., 2019); (ii) employ extended and reliable communication links; (iii) detect and track non-cooperative aircraft using Traffic Collision Avoidance System (TCAS) or a Unmanned Aerial System (UAS) Traffic Management system (UTM) (Hsieh et al., 2020;McCarthy et al., 2020); (iv) be equipped with evasive maneuvering algorithms for collision mitigation actions; (v) give the pilot visual and audible alerts, including in case of any reduced functionality, such as latency and failure (vi) provide First Person View (FPV) video system, a flight termination system, and a Geo-fence and return-tolaunch point functionalities.Additionally, pilots must be trained accordingly, including awareness of existing airspace classes, temporary flight restrictions, and the necessary mitigation actions in the event of an in-flight failure.

Trends in UAV data processing and analysis
UAV technology is at the cross-road of many domains.Research undertaken in adjacent fields (such as robotics and computer vision) influences how data are processed and exploited in geoscience and remote sensing too.This section only reports on the emerging trend in the development of autonomous UAVs for real-time mapping and deep learning methods for semantic analysis of UAV-collected data.These two elements look most promising to innovate the use of UAVs in remote sensing.Other topics such as offline image orientation or dense reconstruction are largely covered in other review papers and are intentionally overlooked in this section.

Towards autonomous UAVs
Automation in UAV flights has been often perceived as a strategy to perform safer and more complete data acquisitions.Many developments in robotics show that autonomous flights could strongly boost data collection in remote sensing applications, paving the road for new research and applications.In Section 4.2.1, the potential of SLAM algorithms for real-time localization and mapping has been already discussed.In the following, we report the state of the art and trends of autonomous navigation algorithms for UAVs and describe the current solutions to process the collected data in real-time.

Is autonomous navigation getting real?
Until a few years ago, UAV flights were mainly maneuvered by a pilot with a remote control: take-off, landing as well as the execution of a given flight were under the control of a human operator, though simple way point trajectories were controlled by an autopilot.In the last 5-10 years, we have seen a growing number of commercial platforms (fixed-wing and rotor) able to reliably take-off, land and execute flight acquisitions in an autonomous way according to more complex different patterns (Cabreira et al., 2019;Murtiyoso and Grussenmeyer, 2017).These solutions are performed according to pre-planned flights (as no autonomous decision is taken) and rely on GNSS-based navigation (as discussed in Section 4.1).Additional sensors (such cameras and distance sensors) have been added on newer commercial platforms to detect big obstacles and avoid collisions preventing their flight too close to obstacles (or at least stopping them before the collision) but only very few recent models have the capability of detecting and circumventing obstacles (e.g., DJI Mavic Air 2 5 ).
Autonomous navigation in unknown environments requires continuous spatio-temporal perception of environmental elements, understanding of the scene situation through data and information, decisions on next stages to take, and ability to make quick knowledge-based decisions based on all these previous elements, with minimal or no intervention from any human operator.All this has been an open research topic in robotics for the last two decades, as witnessed by the huge number of scientific works published, most of them devoted to indoor environments (De Croon and De Wagter, 2018).Approaches to create maps from mobile platforms are either passive where one perceives the environment to simply build a map or active ones where additionally the trajectory of the platform is planned to travel through the environment (Stachniss, 2009).These last approaches can be often divided in three different iterative steps: (i) sensing, (ii) localization and mapping (Section 4.2.1) and (iii) path planning.Different sensors are normally adopted in the autonomous navigation of a drone: mono-or stereocameras as well as lasers are used for mapping the surrounding environment.Inertial units and GNSS can be combined to localize the platform while ultrasound and IR cameras have been more recently added to detect obstacles in proximity of the UAV.Despite the development of miniaturized active sensors, the current trend in literature is the exploitation of RGB cameras that, beside their reduced weight, cost and energy consumption (Carrio et al., 2017), can be efficiently processed with onboard units delivering more complete and flexible information in real-time.
Autonomous path planning refers to the sequence of decisions that the autonomous vehicle takes to create a set of collision free waypoints and reach the destination point.Each autonomous flight needs an overall objective to be accomplished, being it (i) to reach a target point or (ii) explore an unknown environment, with the aim of maximizing its information coverage (Zhou et al., 2020).The path is then constrained by different elements such as completeness of the acquisition, flight time, energy consumption, shortest path to fly or maneuverability limits of the used platform (Cabreira et al., 2019;Wang et al., 2015).
If the goal of the approach is to reach a destination point, both offline and online algorithms can be found in literature.In the offline approaches, trajectories are planned in advance given an already rough geometric knowledge of the environment that allows to plan a safe flight (Li et al., 2018).Different strategies can be used for this task (Aggarwal and Kumar, 2020;Bircher et al., 2016): in most cases the 3D environment is discretized in sets of nodes and then the flight is optimized according to algorithms such as Rapid-exploring Random Trees (RRT) (Yang et al., 2013), RRT-star (RRT*), A-star (A*), probabilistic roadmaps (PRM), or particle swarm optimization (PSO) aiming at the generation of collision free and shortest paths (Roberge et al., 2013).According to the implementation, these algorithms can be exclusively run offline or can update the plan considering the new information collected during the flight: in this case, the next move of the platform is iteratively recomputed to reach the final target.These online methods consider a dynamic and partially unknown environment where the exploration of a new part of the space progressively builds the surrounding 3D environment in a map.Most of the approaches use octomaps (Wurm et al., 2010) to quickly represent and update the 3D environment.Obstacles can be typically divided in static and dynamic and in many approaches the merging of two different types of path (global and local) are combined together (Oleynikova et al., 2016).Global flight path defines the best possible path according to the prior knowledge of the path, while local planners recalculate the path to avoid possible dynamic obstacles (Marin-Plaza et al., 2018).The interaction among these two planners can vary according to the implementation changing the way the drone avoids obstacles (Roberge et al., 2013;Tordesillas et al., 2019).
The absence of a target destination allows the exploration of completely unexplored environments: the general idea is to maximize the information gain achieved at each movement of the drone, determining the next point of view on the boundary between the known and the unexplored space.These frontier-based approaches are generally conceived to generate 3D reconstruction, detect objects or classify the captured regions.Algorithms devoted to 3D reconstruction are often called next-best-view and optimize a function considering parameters such as 3D uncertainty of the 3D reconstructed points (Bai et al., 2016;Palazzolo and Stachniss, 2018) or completeness of the generated point cloud (Mostegel et al., 2016).Other methods aim at maximizing the coverage of a certain area according to a utility function that considers a certain information gain or to increase the knowledge of an area according to some rationale such as the presence of an object of interests in the neighborhood.Most of the early implementations of autonomous navigation were implemented for the exploration of confined (indoor) unknown environments, while a growing trend of autonomous approaches are nowadays developed for outdoor spaces and a larger variety of tasks (Popović et al., 2020).
The development of autonomous driving strategies coupled with the surge in deep learning have outsourced several solutions (such collision avoidance strategies) that influenced the development of drone algorithms (Fraga-Lamas et al., 2019;Zhao et al., 2018).Traditional SLAM algorithms (see Section 4.2.1) are frequently supported by deep learning networks (Tateno et al., 2017;Yang et al., 2020) and the 3D reconstruction is fused with semantic segmentation, to reduce the effect of dynamic objects in the scene (Yu et al., 2018) or combining geometric and semantic information mutually improving the scene understanding of the 3D environment (Bavle et al., 2020;Dang et al., 2019).The generation of quick depth maps thanks to single (Luo et al., 2019;Madhuanand, 2021;Marcu et al., 2019) and stereo image (Cigla et al., 2018) depth estimations as well as the improvement of miniaturized GPU devices installed on drones (Cetin and Yilmaz, 2016) had boosted the large adoption of these algorithms to detect obstacles (Fraga-Lamas et al., 2019), and scale the scene only relying on images (using Single Image Depth Estimation algorithms, see Section 5.2).On the other hand, collision avoidance can be embedded in end-to-end data driven solutions where Convolutional Neural Networks (CNNs) output commands such as speed and change in direction from a pre-planned path instead of outputting simple depth maps (Chakravarty et al., 2017;Dai et al., 2020) or to avoid obstacles.Networks trained with dedicated datasets have also shown to follow path, rails or roads, dynamically avoiding obstacles (Loquercio et al., 2018;Smolyanskiy et al., 2017).Autonomous navigation along specific trajectories can also be trained using reinforcement learning, where the UAV receives rewards for actions taken in the environment that work towards achieving its objectives (Szeremeta and Armenakis, 2021).
Most of the approaches have been conceived for indoor (confined) environments, or to follow structured features like roads or rivers (Nuske et al., 2015), however, some fully autonomous navigation solutions in cluttered environments are also available (Maciel-Pearson et al., 2019).Different frameworks such as reinforcement learning from simulated or real datasets (Gandhi et al., 2017;Ramezani Dooraki and Lee, 2021;Tai et al., 2017) or networks learning the temporal dependencies (such as Recurrent Neural Networks) (Kelchtermans and Tuytelaars, 2017) for image sequences are further increasing the adoption of these methods in new and more challenging applications.Despite the great interest for all these solutions, most of them are still "research" solutions while their market adoption will need the implementation of further rigorous tests to assess and certify their safety in unattended operational conditions.5.1.2.How to process UAV data in real-time?.Both real-time mapping and autonomous navigation entail the capacity to quickly process the acquired images and reliably deliver the needed information.In this regard, onboard computing and communication technologies are enablers for intelligent small UAV mapping systems.Communications between mobile platforms or between platforms and infrastructure through Wi-Fi, radio as well as edge computing serve for exchange of tasks and data sharing information (Zeng et al., 2019).Fast (near realtime) processing can be achieved through onboard dedicated devices or streaming the data to remote computers (Fraga-Lamas et al., 2019).
In the aerial photogrammetric domain, some first examples of realtime 3D mapping have been implemented on UAV platforms with the aim of delivering real-time image orientation and rough orthophoto generation (Bu et al., 2016;P. Chen et al., 2018b;Hein et al., 2019;Hinzmann et al., 2018;Kern et al., 2020;Wang et al., 2019;Zhao et al., 2021;Zhou, 2009).These methods have mainly adapted robotics approaches (such as SLAM algorithms) to photogrammetric needs with the aim of delivering geometrically consistent maps looking at the processing speed as a priority more than the geometric accuracy of the F. Nex et al. delivered results.The main bottleneck/challenge lies in the reduced processing capabilities that has limited the number of tasks that can be performed onboard.This is often mitigated by using approximations in the image orientation, adopted in robotics, or simplified intermediate products (such as the use of sparse point clouds or rough point cloud densification strategies) in the orthophoto generation.
The surge in deep learning has been recently pushed by the development of efficient GPUs capable of processing complex CNNs in reduced time.Beside the adoption of GPU installed on powerful workstations, miniaturized solutions have been developed to process data on the edge (i.e., close to the sensor) to prevent problems due to limited or absent connection in the field.The commitment of many leader companies (such as NVIDIA 43 , Google 44 and Intel 45 ) to develop relatively cheap (few hundreds euro) devices, easy to program using conventional libraries (e.g., PyTorch 46 , TensorFlow 47 ) is boosting their large adoption on UAVs deployed in the field (Wang et al., 2018).The development of deep learning algorithms for collision avoidance and dense reconstructions is further extending the potential of these boards to traditional mapping purposes (Bloesch et al., 2018;Tateno et al., 2017).The processing speed of these devices is not comparable to high-end desktop boards (almost one order slower) but several applications with relatively shallow networks can be performed in real-time (Loquercio et al., 2018).These devices have relatively little energy consumptions compared to their full-size versions and they can be still embedded on many UAVs (Qasaimeh et al., 2019).Examples of these methods can be currently found in road monitoring (Fan et al., 2019), emergency management (Tijtgat et al., 2017), vehicle detection (Azimi, 2019;Balamuralidhar et al., 2021;Meng et al., 2020;Wu et al., 2019) and multiple object tracking (Hossain and Lee, 2019), among others.The effort in the miniaturization of these devices has led to the development of autonomous platforms of few grams in weight (Palossi et al., 2019) designed for exploration purposes.These solutions are permeating the market, with the introduction of commercial drones able to track objects, perform 3D reconstructions and semantic understanding for recreational purposes 48 .
Although miniaturized solutions are growing, onboard computing still has some limitations given by the processing speed, power consumption and the level of customization of the drone.When small latencies are acceptable, streaming the data on a remote PC for processing can still be an option.Several examples of these approaches have been developed using local networks (such as wi-fi) or radio connections and cloud engines to allow the stream of images, videos and geolocalization of the drone (Li-Chee-Ming and Armenakis, 2014).Traditional GSM connections (i.e., 4G) can be also used for relatively limited amounts of data.The development of Software Development Kit (SDK) for many commercial platforms has allowed for the implementation of these solutions for different applications too (Meng et al., 2020;Nex et al., 2019).The advent of 5G communications could further increase these typologies of architecture in the near future (Ullah et al., 2019;Zeng et al., 2019).Further edge computing and IoT (Internet of Things) solutions will allow the computing to be done at or near the source of data.
Dedicated 24/7 solutions aiming at surveillance and monitoring of large industrial plants using UAVs are currently offered by a growing number of companies in the World (Percepto 37 , Mapture 40 and Skydio 38 ) coupling UAVs with dedicated docking stations to recharge/ replace the batteries and exchange data (See Section 3.2.2).In many cases, UAVs are considered flying sensors, already embedded in larger IoT networks (Saha et al., 2018;Sterbenz, 2016): the acquired images become part of a larger on-line processing of heterogeneous data in realtime.In these solutions, the development of hybrid strategies with onboard computing and remote processing on a cloud seems the most effective solution, at the moment.The coming development given by faster communication and smaller latencies (i.e.5G) will probably further revolutionize the way to process the data, pushing the development of real-time solutions and new services adopting UAVs.

What are the peculiarities of deep learning for UAV image semantic analysis?
Despite the great interest in the last years, the number of published works adopting deep learning methods for UAV images for semantic scene analysis is still relatively limited (Osco et al., 2021).UAVs offer high potentials, and additional challenges for the extraction of semantic information, due to the very high spatial resolution and the 3D data acquisition capability.Papers using deep learning mainly focused on (i) image classification, (ii) semantic segmentation (or pixel-wise classification) and (iii) object detection.Examples of image classification are presented by (Natesan et al., 2019) to classify individual tree species using high-resolution RGB images, or by (Kerle et al., 2019b) to detect structural building damages after earthquakes.Segmentation algorithms are used to process UAV data in a wide range of applications, such as weed mapping over rice fields (Huang et al., 2018), semantic segmentation of plant species from high-resolution UAV imagery (Kattenborn et al., 2019), monitoring of mining activities (Giang et al., 2020) and delineation of visible cadastral boundaries (Crommelinck et al., 2019;Xia et al., 2019).(Gevaert et al., 2020) propose a deep learning framework to detect changes in an informal settlement using UAV imagery and a detailed elevation model (Fig. 10) while (Ferreira et al., 2020) incorporate different architectures to map individual palm trees in the Amazon forest using RGB UAV images.At last, example of object detection are discussed by (Tang et al., 2017) for vehicle detection, while (Han et al., 2019) customized a network to detect tiny objects in UAV images.Environmental applications such as mammal detection in the African savannah (Kellenberger et al., 2018), muck pile characterization (Schenk et al., 2019), diseases on crops such as the detect flavescence dorée on grapevine (Musci et al., 2020) or detection of Antarctic seals and flying birds (Mustafa et al., 2019) can be also found in the literature.
The used networks vary in-depth, number of parameters, complexity, memory and computational cost (Bianco et al., 2018).Most of these works start from popular computer vision networks that often lead to satisfactory results thanks to fine-tuning or small modifications to process UAV data.Resnet-like networks (He et al., 2016) are often adopted in classification, while encoder-decoder networks (Badrinarayanan et al., 2017;Noh et al., 2015;Ronneberger et al., 2015), dilated (or atrous) convolutions (Persello and Stein, 2017;Sherrah, 2016;Yu and Koltun, 2016) or their combinations such as DeepLab efficiency (Chen et al., 2018a) are frequently used in semantic segmentation.State of the art Yolo networks (Liu et al., 2020) are commonly used as a starting point in object detection.This can be explained considering that UAV data have higher resolution compared to airborne and satellite images.In this regard, traditional machine learning techniques, based on feature handcrafting, easily stagnates in the analysis of sub-meter and subdecimetre resolution nadir imagery, where spatial-contextual features carry most of the discriminative information (Gevaert et al., 2016).Deep learning has shown the ability to improve accuracies by learning highlevel, deeper spatial features.This has often led to consider UAV acquisitions comparable to common computer vision applications, overlooking the additional variability of used sensors, typologies of acquisitions and applications of interest.Successful off-the-shelf computer vision models are not always the best solution for the task at hand as domain knowledge can often help design or adapt networks to the specific characteristics of the UAV data and the semantic problem.Computer vision networks are designed for RGB images, but they need to be adequately modified to fully exploit the spectral information acquired by multispectral and hyperspectral sensors.This may require more than naively incrementing the number of filters in the first network layer (Li et al., 2019) that are often not considered.Compared to airborne and satellite data, the UAV spatial resolution influences the expected spatial auto-correlation and, therefore, the network's required receptive field.It is therefore expected that higher-resolution imagery demands for networks with larger filters' FoV.Another information that significantly influences the analysis task (and the design of deep learning networks) is the acquisition geometry of UAV imagery.In contrast to large computer vision datasets (often used for pre-training), where the same object (e.g., a person) could be at different distances from the camera, in UAV imagery, the distance to the object of interest is generally fixed, although oblique and nadir views  (Gevaert et al., 2020).make a large difference in terms of appearance of the objects.This can often reduce the variability in the appearance of objects and can provide an important prior knowledge of the average object size (in pixels).In these cases, lighter networks tailored to the characteristics of the data and problem at hand could be preferred to facilitate training, reducing computational cost, and improving generalization ability.
A significant challenge that often precents the use of deep learning for UAV is the lack of freely available datasets for training.Some recent introduced benchmarks are trying to mitigate this problem (Lyu et al., 2020), but collecting field data or manual digitization is the only solution for generating a representative training set.The lack of dedicated datasets also limits the use of multi-task learning techniques, despite their higher performance achieved taking advantage of the relationship between tasks.
In most of the acquisitions, UAV data provide the additional opportunity to extract 3D information in the form of digital models or point clouds.This additional information contributes to characterize spatial features of target objects in the 3D environment.Promising results have been shown in Gevaert et al. (2018) where 2.5D topographic and 3D geometric features are integrated into a classical machine learning workflow to improve accuracy in complex classification problems.This integration of 2D and 3D information could lead to improved performance in many other applications in the future.Another interesting trend is given real-time applications (discussed Section 5.1).These developments also boost the need for optimized and lighter architectures, which can run on small processor and achieve performances similar to deeper networks (Balamuralidhar et al., 2021;Yang et al., 2021).

Conclusions
This paper has critically reviewed the most prominent developments on UAVs for geoscience and remote sensing applications, paying attention to the elements coming from adjacent domains that could influence their use in the upcoming years.Onboard sensors development and their optimal use, flight planning and orientation/navigation issues as well as some relevant issues on efficient processing and analysis of the collected data have been detailed.The best practises to follow using the available technologies as well as their still existing limitations have been reported.The adoption of new solution and technologies that will influence future trends of UAVs have also been presented.From this analysis, several technological and scientific challenges as well as many opportunities can be observed.

Current challenges
Both passive and active sensors installed on UAVs have largely improved in the last few years.If past sensors were mainly "adaptations" of terrestrial applications, customized solutions for UAVs are nowadays on the market.Low-cost solutions enable the acquisition of decent quality data in many applications, while high-end solutions are often very close, in terms of performance, to airborne solutions.In this case, costs, energy consumption and (often size and weight) still represent a limit to make UAV technology more competitive with traditional solutions in a wider range of applications.
While multispectral cameras have already reached mature levels and are used in practical applications, hyperspectral is evolving rapidly and is largely used by the research community.Further development is required to reach the consumer or wider professional/commercial markets.Hyperspectral sensors must be further miniaturized preferably to weight well below one kilogram.The wide variety of technical implementations makes the development of efficient solutions for processing and analytics challenging.The radiometric correction of hyperspectral datasets is challenging as UAVs are operated in varying conditions, while efficient and established chains for data capture and processing as well as solutions to manage large spectral data are still missing.In the same way, LiDAR instruments are still relatively expensive and heavy to cover large areas.However, the implementation of low-cost and smaller sensors, the tighter integration with navigation and/or image sensors, and the higher point cloud density compared to airborne systems promise to make these instruments a valid alternative to traditional terrestrial surveys too.
UAVs install miniaturized (and often high-quality) inertial and positioning sensors: these instruments need to be carefully integrated together when assembling a UAV or often re-calibrated to verify the specifications provided by the manufacturer.The steps to accomplish this process have been presented, showing how the airborne procedures can be adapted to UAV platforms.These procedures could allow to periodically calibrate the platforms with relatively easy and reproducible steps.If the quality of the data collected has improved, the UAV productivity has also increased thanks to the improvement of the battery life, although surveyed areas are still at least one order of magnitude lower than using a manned airplane.
Nowadays best practices on flight planning, data acquisition and adjustment allow for the effectiveness of UAV flights to be maximized, reaching accuracies compatible with the requirements of many mapping applications.The introduction of more automated and intuitive software to plan flights and acquire data has enabled many (often unexperienced) practitioners to adopt UAVs in their everyday surveying activities.As a countereffect, this has often generated false expectations on the quality of the used platforms (installing low-cost sensors) and in the accuracy of the performed surveys.Many problems such as best practices on the use of RTK/PPK sensors, synchronization and calibration of multiple sensors are often overlooked.
Although regulations provide clarity on the use of flying platforms, several restrictions are still in place, limiting their use.BVLOS applications could represent a big game-changer for UAV technology, but they are still largely unexplored, not only due to lack of platforms capable of long duration flights with reasonable payloads.A challenge of the research community will be, therefore, to implement safer and more reliable BVLOS flight strategies to expedite the acceptance of these types of flights.One possible solution could be given by the integration of GNSS/IMU, SLAM and communication technologies for robust failsafe navigation and collision avoidance in automated operations.In addition to the mentioned technical challenges, still the ongoing development of currently quite complex regulation procedures may delay the use of BVLOS in practice (Alamouri et al., 2021).
Traditional data processing has reached its maturity, as confirmed by the flourishing of commercial and open-source photogrammetry workflow solutions.However, users' needs are evolving, pushing the implementation of new algorithms and solutions to enable UAV technology with new features.Faster and miniaturized hardware components for real-time processing as well as the surge of deep learning algorithms are influencing all the steps of data processing from the autonomous navigation of the UAV to the 3D reconstruction and the semantic understanding of the scenes.Many algorithms on UAVs require the complex interaction and integration of sensing, planning and "understanding" in interpreting and reacting to the external environment.Early semi-autonomous path planning algorithms are currently conceived for infrastructure monitoring, patrolling, 3D reconstruction as well as simple classification or detection tasks, but their extended use on more general applications has still to come.Although some solutions appear in commercial platforms, their wider adoption seems to be still quite far as no rigorous ways to assess (and in the future to certify) the reliability and safety of such systems, particularly during unattended operations outside classical test scenarios, already exist.
Considering scene semantic understanding tasks, UAV acquisitions have specific characteristics in terms of acquisition conditions (illumination, viewing angle), phenological state of vegetation, and set of semantic classes or target objects, where customized networks should be coupled with large, dedicated training datasets.Although available annotated UAV datasets are progressively increasing, their number is still insufficient resulting in the request of collecting extra field data for each F. Nex et al. acquisition campaign.

Future opportunities
Despite the massive growth seen in the last decade, the demand for unmanned vehicles capable of replacing traditional surveying and mapping techniques in several activities is still high.The market for UAVs had been expanding in unexpected domains until a few years ago.UAVs are at the crossroad between computer science, robotics and remote sensing, as well as other application domains and promise to be an active investigation area in the upcoming years too.The general demand from the market is for even more accurate, autonomous, reliable (and possibly) cost-limited platforms.In particular, the increase in the level of automation of UAV operations will open new opportunities for this technology: precision farming, infrastructure and natural hazard monitoring or, still, smart cities pledge to be among the most popular application areas in geoscience and remote sensing in the near future.
On the hardware domain, the improvement in the sensor miniaturization and surge of new solutions, quality and accuracy as well as in the battery-life have so far only partially made UAV an alternative to airborne systems and traditional surveying instruments.However, the yearly release of new and more efficient sensors, platforms and power systems are progressively closing this gap.Compared to manned platforms, UAVs could represent a valuable solution thanks to higher resolutions and flexibility in data collection (allowing for customized acquisitions) and every day closer productivities.The development of more reliable frameworks to increase the use of emerging scenarios such as collaborative (air-air and ground-air) platforms and the growing automation of flights will further close the gap between UAV and manned aerial and terrestrial surveys.The extensive use of recharging stations for commercial UAVs and their easier connection to larger sensor networks (making UAVs part of IoT networks) will give an additional degree of freedom to conceive new applications with short revisiting periods and ubiquitous data collections.In this regard, UAVbased data capture and processing could replace traditional surveying methods in many activities requiring repeated data collection (e.g., progress monitoring in construction).
Autonomous UAVs will become a reality thanks to more advanced algorithms enabling safer navigations, understanding of the environment and autonomous decision-making capacity.Collision avoidance and autonomous navigation in unknown environments, semantic analysis of the collected and even SLAM algorithms are already taking advantage of deep learning methods: this process promises to lead to a major improvement in the development of safe and autonomous UAVs and will be the main driver in the improvement of multiple applications.Given the concurrent work of different domains and the optimization of dedicated hardware to run them onboard and in real-time, this trend will grow in the future.This process will be facilitated by the availability of more complete training datasets (including synthetic data as well) or by emerging approaches such as self-supervised learning, reinforcement learning or the automatic generation of pseudo-labels exploiting already existing data (Gevaert et al., 2018).Questions regarding the transferability of these models to heterogenous and unexpected operational conditions will need to be addressed with rigorous and innovative strategies.
The technological and scientific improvements highlighted in this paper will pave the path for the adoption of UAVs in new domains and applications.However, an important role will be played in all these scenarios by the establishment of trusted (and regulation compliant) BVLOS frameworks that will enable UAVs to overcome current constraints facing the technology in terms of extension and human supervision.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Fig. 1 .
Fig. 1.Trend line showing the number of scientific papers published in the last 20 years using UAV, UAS, drone or RPAS as a keyword or in the title.

Fig. 2 .
Fig. 2. Illustration of commercial hyperspectral cameras.The blue circles indicate 2D-frame format cameras, and the orange circles indicate pushbroom scanners.They are grouped according to their spectral range, approximately VNIR: 400-1000 nm, NIR: 900-1700 nm, SWIR: 900-2500 nm; VNIR + SWIR: 400-2500 nm; (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 3 .
Fig. 3. Hyperspectral images from a forest health study site.(a) Drone (GSD 10 cm) and aircraft (GSD 50 cm) based hyperspectral images; (b) spectral data sampled from the drone and aircraft images, respectively.Source Näsi et al., 2018

Fig. 4 .
Fig. 4. (a) Urban UAV sensor data from 100 m above ground level (AGL) captures the building, street, vegetation and terrain characteristics for detailed urban and traffic environment mapping and planning purposes.(b) National open ALS data (0.5pt/m^2) of the same location for comparison shows significantly less detail, lack of building walls and traffic infrastructure due to low point density and limited field of view in connection to longer ranges/higher altitude used primarily for ground topography, and larger laser beam size on the surfaces.(ALS data courtesy to NLS, 2021).

Fig. 10 .
Fig. 10.Results of the semantic change detection between the UAV imagery at t1 (a) and t2 (b) using RGB + DSM data (c) to determine the change mask(Gevaert et al., 2020).

Table 1
Current cameras used in drone mapping applications (selection only, as of 2021).M and E refer to mechanical and electronic shutters, respectively.EFL stands for Equivalent Focal Length.

Table 2
Examples of commercially available hyperspectral cameras operating in 2D and 1D (pushbroom) range.

Table 3
Comparison of properties of PPK and RTK-based DGNSS solution onboard.VRS: virtual reference station, CORS: Continuously Operating Reference Station, BVLOS: Beyond visual line of sight (see Section 4.3).F.Nex et al.