UAV & satellite synergies for optical remote sensing applications: A literature review

Unmanned aerial vehicles (UAVs) and satellite constellations are both essential Earth Observation (EO) systems for monitoring land surface dynamics. The former is frequently used for its acquisition flexibility and its ability to supply imagery with very high spatial resolution (VHSR); the latter is interesting for supplying time-series data over large areas. However, each of these data sources is generally used separately even though they are complementary and have strong and promising potential synergies. Data fusion is a well-known technique to exploit this multi-source synergy, but in practice, UAV and satellite synergies are more specific, less well known and need to be formalized. In this article, we review remote sensing studies that addressed both data sources. Current approaches were categorized to distinguish four strategies: “data comparison”, “multiscale explanation”, “model calibration” and “data fusion”. Analysis of the literature revealed emerging trends, the supply of these distinct strategies for several applications and allowed to identify key contributions of UAV data. Finally, the high potential of this synergy seems currently under-exploited; therefore a discussion is proposed about the related implications for data interoperability, machine learning and data sharing to reinforce synergies between UAVs


Introduction
The synergy between UAV and satellite data (UAV/Satellite synergy) is essential for understanding the dynamics of the Earth's surfaces (Kuenzer et al., 2015). On the one hand, these two data sources produce significant volumes of data contributing to the Big Earth Observation (EO) Data (Liu et al., 2018). Indeed, this exponential production of EO data since the beginning of the 2000s can be partly explained by the increase in the number of satellites in orbit (Ghamisi et al., 2019) and the democratization of UAVs due to the decreasing cost of exploitation (Sun and Scanlon, 2019). On the other hand, each EO system has specific acquisition features that result from a trade-off between resolutions (spatial, spectral and temporal), swath ( Fig. 1) and signal-to-noise ratio (Alavipanah et al., 2010). Pending a new technology with all required features, it is necessary to combine data from different sources to enhance observations. Satellite constellations and UAV for EO provide complementary data interesting for a synergy approach.
Current satellite constellations are numerous and offer different trade-offs that can be grouped into four categories: global monitoring (GM-), environmental monitoring (EM-), nano-and civilian-satellites.
GM-satellites in high orbit (e.g. MODIS Terra or NOAA AVHRR) or geostationary satellites have high temporal frequency, which makes it possible to provide daily to infra-hourly data at coarse spatial resolution (>100 m). EM-satellites (e.g. Landsat or Sentinel-2) are balanced in their resolutions, providing high temporal data (few days) with high spatial (10-100 m) and spectral (ca. ten bands) resolutions. These first two categories of satellites provide long-term data time-series, for example, up to 50 years for the Landsat legacy. Next, nano-satellites (e.g. Planet), due to the large number of them placed in low orbit, provide daily global coverage at high spatial resolution (1-10 m) but with a lower data quality that does not satisfy all applications. Finally, civilian-satellites (e.g. Pleiades or Ikonos) are low-orbit spaceborne satellites with sensors that provide data at very high spatial resolution (VHSR; <1 m) but with low spectral resolution (5 bands), and whose cost limits the achievement of global coverage and dense time series. Globaly, optical remote sensing (RS) satellites have little flexibility in their acquisition features, for example they are constrained by cloud cover, view angle or acquisition time.
Remote sensing by UAV (RS UAV) has emerged due to the development of robotics, computer vision and sensor miniaturization (Colomina and Molina, 2014). The openness of data-acquisition skills (Milas et al., 2018), due to the development of micro-UAVs (i.e. weight less than a few kg), changes the paradigm of RS by giving end-users the ability to control acquisition features. Ultraspatial resolution (centimetric to millimetric) and acquisition flexibility are the strengths of RS UAV. The ability of UAVs to acquire data close to the surface allows such spatial resolution but also the ability to be quasi-independent or less-affected by clouds. UAV flexibility allows acquisition conditions to be chosen: the type of sensor, angles of view, spatial resolution, time and frequency of acquisition. The choice of acquisition dates and times is an essential characteristic for certain applications, such as monitoring biological, hydrological or geomorphological dynamics (Abdullah et al., 2018;Müllerov a et al., 2017;Fytsilis et al., 2016). However, RS UAV has the disadvantage of having a smaller swath (a few km 2 ), often because of its low energy reserves and the legislation needed to protect air traffic and people safety and privacy (Cracknell, 2017).
Finally, UAVs offer a different and complementary profile to satellites. There are complementarities in resolution between UAV and satellite acquisitions, particularly in spatial and temporal resolution and swath ( Fig. 1), as well as advantages and disadvantages of using each vector (Table 1). For example, although RS UAV has high acquisition flexibility, it requires a ground operator, management of large volumes of data and pre-processing, while satellite data are easily available on webbased platforms and are generally ready to analyze. Furthermore, UAV provides data at a resolution unreachable by satellite but cannot rival the latter's observed extent and remains constrained in particular territories by national and/or international legislation. Between these two sources of RS data, advantages of the former appear to compensate for disadvantages of the latter, and vice versa, revealing a strong potential for synergies. Moreover, it is considered necessary to use this synergy (Zhu et al., 2018;Vihervaara et al., 2017) to bridge the gap between the abilities of EO systems and the data needs of different application fields (Fig. 1).
To our knowledge, there is no review of the literature that analyzes UAV and satellite synergies. The synergy between multi-resolution RS data has already focused on the complementarity of satellite and airborne data (Liu et al., 2018;Zhang, 2010), however the complementarities between UAV and satellite are more specific (Kakooei and Baleghi, 2017). We therefore attempt to fill this gap in the scientific literature by reviewing these synergies. This review categorizes four types of strategy, identify emerging trends and key contributions of UAV data. Then a discussion is proposed to overpass current limits and to fully exploit the potential of this synergy.

Literature review process
This review aimed to collect, in the most exhaustive way, peerreviewed articles that dealt simultaneously with UAV and satellite data. To this end, the Google Scholar, Science Direct and Web of Science databases were queried for the period 2000-2019. Articles were first selected using the following query: "UAV syn " AND "Satellite syn "  (Transon et al., 2018), and Zhu et al. (2018) (Zhu et al., 2018). EO: Earth Observation; EM: Environmental Monitoring; GM: Global Monitoring; sat.: satellite; LC: Land Cover; LU: Land Use. with UAV syn and Satellite syn corresponding to a list of synonymous terms or well-known sensor names (Appendix A). A total of 495 articles were found in this search. By analyzing the entire corpus, articles that mentioned the two data sources but did not deal with them were eliminated. The remaining articles were then filtered to retain only those that dealt with optical data acquired by a passive sensor (panchromatic, RGB, multispectral and hyperspectral) and related to the study of the Earth's surface (excluding studies of the atmosphere and oceans, which have other specific characteristics). Ultimately, the corpus of our study was based on 137 articles published in peer-reviewed journals (Appendix A).

Categorization in strategy types
To guide future studies in the exploitation of UAV/Satellite synergies, we categorized articles of our corpus. Our categorization approach distinguished different strategies according to a three-criteria hierarchical decision tree (Fig. 2).
The first criterion distinguishes weak and strong synergies (Fig. 2) based on the degree of data integration. The synergy will be strong if combining the data provides more information than using each data source separately ("1 þ 1 ¼ 3 00 ) (Pohl and Genderen, 1998). Conversely, weak synergy, called "data comparison", compares only advantages and disadvantages to determine which data source is the most suitable ("UAV or Satellite") for the application under study (Fig. 3A). Data combination in this category is for visual or quantitative comparison but does not help in the final interpretation.
Among the strong synergies (Fig. 2), the second criterion distinguishes studies whose observation object is the same for both UAV and satellite acquisitions from those whose object is not the same. The latter refers to the "multiscale explanation" strategy, in which the objects observed by UAVs and satellites are at different spatio-temporal scales, generally with a different spatial extent. Information extracted at a finer resolution for a small site is used to explain the information for a larger extent that contains the former. Generally, these studies observe an object precisely by UAV and use satellite data to obtain the more global context in which this object is located. The data are therefore processed separately, and the extracted information is then used together to improve the scientific interpretation (Fig. 3B).
The last criterion distinguishes studies that develop models (e.g. classification, regression or detection) based on only one data source ("UAV for Satellite") from those based on both sources ("UAV and Satellite"). The former defines "model calibration" strategy, i.e. one data source is used to calibrateingion by inference a model based on the other source (Fig. 3C). Two sub-categories are distinguished: qualitative (e.g. hard classification) and quantitative (e.g. regression or soft classification). The first generally refers to labeling satellite pixels by interpreting UAV data. While the second refers mainly to the use of raw (e.g. reflectance) or derived (e.g. biophysical variables) numerical values from UAV data to calibrate a satellite-based model. In a few rare cases, the roles of UAV and satellite data are reversed.
The last type of strategy, "data fusion", concerns studies in which new data are produced from a model based on both UAV and satellite data (Fig. 3D). These methods make it possible to improve the resolution of each data source. There are several ways to fuse data by coupling or complementing spatial, spectral and temporal resolutions. In general, fusion techniques in RS can be classified into three categories according to the degree of fusion (Zhang, 2010): pixel-level, feature-level and decision-level.
The bibliographical analysis on which this review is based was performed with the help of a reading grid (Appendix A). Using this grid, a variety of information was extracted (e.g. application field, strategy type, data type, methods).

Emerging trends
The recent emergence and growing interest in UAV/Satellite synergies was evident from the bibliographical analysis (Fig. 4A). The first articles addressing this synergy emerged in 2008, which logically coincided with the democratization of UAVs. Interest in this practice remained modest for several years, accumulating barely 19 articles by 2013. From 2014-2019, the number of studies increased considerably, reaching 37 articles published in 2019.
Reviewed studies were unevenly distributed among the strategy categories (Fig. 4B). Of the 137 articles, 29% compared only UAV and  satellite data and thus exhibited weak synergy. However, when the synergy was strong, "model calibration" strategy was most common, at 48% of the articles (qualitative and quantitative pooled). "Multiscale explanation" and "data fusion" strategies were under-represented (9% and 10% of the studies reviewed, respectively). The undefined category represented only 0,7%, one methodological article about multiscale data fusion. Fig. 5 shows that the UAV/Satellite synergy emerged with the "calibration model" strategy and evolved with it. The "data comparison" appeared later, in 2012, and seemed to evolve slightly. The "multiscale explanation" appeared relatively early but remained punctual. And lately, the "data fusion" appeared in 2014, and although present every year until 2019 it seemed to evolve poorly. Application fields were also unevenly distributed among the strategy categories (Fig. 4C). Ecology and precision agriculture were the main fields of application of these synergies, representing respectively 48% and 26% of the reviewed articles. While ecology was based mainly on "model calibration" strategy, precision agriculture used all strategies, with a slight preference for "data fusion". Although ecology was the field with the most UAV/Satellite synergies, it had no "multiscale explanation" strategy study for the period studied. Less common fields of applications like geosciences and disaster were frequent over time, unlike applications in archaeology, urban and water resources that were rare and punctual. It can be seen that these less common fields of application were mainly strategies through "data comparison" or "multiscale explanation".
Optical data used for these synergies came mainly from EM-and civilian-satellites, while nano-and GM-satellites were rarely used (Fig. 7A). This may be due to the recent release of nano-satellites and to the coarser spatial resolution of GM-satellites than that of UAVs (i.e. the scale factor is too high). Overall, 58% of the studies reviewed used opensource satellite data such as Sentinel-2, Landsat, Gaofen or MODIS, thus demonstrating the contribution of open-source data to this synergy. On the UAV side, multi-rotor UAVs seemed to be used more than fixed-wing UAVs, although a large percentage (30,7%) of the articles did not mention the characteristics of the UAV used (Fig. 7B). Although, 10% of  articles did not mention the types of UAV optical data collected (RGB, multiscpectral or hyperspectral) it seems that RGB data was more used for strong synergies, and multispectral data for "data comparison" (Fig. 6). On average, UAVs provided a resolution of 26 cm in a range of 0.2-500 cm depending on the application, with a median of 8 cm. The scale factor between UAV and satellite data therefore varied from 2-20,000, with a median of 100.

Strategy types
"Data comparison" strategy "Data comparison" is a weak synergy in which data are not combined to improve interpretation. In comparativet studies the specific features of UAVs and satellites are identified, and thus their complementary nature is highlighted (Fig. 1).
Satellites provide data with a larger extent that UAVs cannot achieve (Jacobsen, 2012;Rau et al., 2014). They cover areas that are difficult for UAVs to access, such as urban areas (Müllerov a et al., 2016) or conflict zones (Matou skov a et al., 2016). Moreover, standardization of the pre-processing of satellite data (analysis-ready data) facilitates and accelerates the processing of these data (Müllerov a et al., 2016). In addition, UAVs provide low-cost VHSR data that reveal fine patterns such as inter-row and intra-plot variability in vineyards (Aleem et al., 2019;Matese et al., 2015), the fine morphology of glaciers (Fugazza et al., 2015), and even small water bodies (Cândido et al., 2016). This VHSR also makes it possible to isolate the study object from areas that could interfere, such as adjacent bare soils (Aleem et al., 2019) and shadows (Rupasinghe et al., 2019), which can bias the spectral signatures of vegetation. The low cost of the technology makes this vector particularly attractive; however, according to Ruwaimana et al. (2018) (Ruwaimana et al., 2018) UAVs are cost-effective for acquisitions only for long-term monitoring. Finally, the flexibility of UAVs is a major asset when acquisitions must be made at a specific date or time, as may be the case for precision agriculture to determine the addition of inputs (Brinkhoff et al., 2018;Jure cka et al., 2018), or to perform rapid assessment during climate events and natural disasters, such as landslides (Casagli et al., 2017).
To conclude the comparison of UAVs and satellites, the choice of data source depends mainly on the scale (extent, resolution) of analysis and the objective of the study. The features required for the study (e.g. ground resolution, temporal frequency, types of acquisition, extent) will guide the choice of data source the most, but it may also depend on the study's organizational and financial conditions (Jacobsen, 2012). Among these comparative studies, 19% of them considered strong UAV/Satellite synergies the most in their perspectives.

"Multiscale explanation" strategy
The "multiscale explanation" strategy combines the observation scales of each data source to interpret the data better. This complementarity was used mainly for natural risk management (33%), geosciences (25%) and archaeology (25%). In natural risk management, this approach enables, for example, relations between the presence of gullies and agro-industrial development to be established (Aït Hssaïne et al., 2011). The satellite is thus used to obtain information about the dynamics of changes in land occupation and use, while the UAV provides fine-scale analysis of the gullies via photo-interpretation and analysis of digital surface models (DSM) generated by photogrammetry (Aït Hssaïne et al., 2011;Wang et al., 2016). In geosciences, a study of yardangs in a desert environment showed that UAVs can be used to extract morphological features, while satellites provided information about their spatial distribution (Zhao et al., 2018). In archaeology, satellite imagery is used either for prospection studies (Ding et al., 2016) or to contextualize an archeological site in its environment (Gruen et al., 2012;Lin et al., 2011), while UAVs reveal details of archeological sites that are invisible even from the ground (Lin et al., 2011) and provide spatial description in three dimensions (Ding et al., 2016;Gruen et al., 2012). And, an original use of this approach made it possible to study coastal dynamics both physically and socially (Papakonstantinou et al., 2019). Satellite data provided information on the evolution of the coastline and geomorphological changes of available beaches, while UAVs counted and characterized tourist infrastructure on these beaches. Finally, through this strategy, satellites make it possible to analyze spatial or temporal patterns, contextualize or locate at a regional scale, while UAVs provide additional information, often in the DSMs produced by photogrammetry, or refine the characterization of the study objects.
"Model calibration" strategy "Model calibration" uses one data source to calibrate a model (qualitative or quantitative) based on the other data source. Among the articles classified as strong synergies, it was the most frequent strategy, with 70% of the articles ("data comparison" excluded) and was widely used by applications in ecology and precision agriculture (Fig. 4B).
Typically, the qualitative approach allowed satellite-based classification (or post-classification (Nhamo et al., 2018)) to be made using labelled samples from UAV data. Labels can be determined by: 1) photo-interpretation by an expert, 2) thresholding of metrics (e.g. spectral indices or biophysical parameters) or 3) automated classification (pixel or object oriented) that can be supervised or not. These labels are then used to calibrate supervised classification models. Those produce land-cover or land-use maps and thus provide information about    landscape structure like the spatial distribution of habitats, and can help forto change detection like deforestation in wooded systems (Marx et al., 2017). For example, Szantoi et al. (2017) (Szantoi et al., 2017) used this strategy to map land cover and land use and quantify loss of primary forest to assess their impacts on orangutan habitat in Indonesia. The quantitative approach uses numerical values directly from UAV data to calibrate a satellite-based model. Usually, the numerical values extracted are (bio)physical parameters of the surface (e.g. chlorophyll content or aboveground biomass) derived from spectral measurement  or from photogrammetrically estimated height models (e.g. tree height (St-Onge and Grandin, 2019)). But UAV data may also have other natures, such as land-cover rates for sub-pixel validation of soft classification models to estimate the fraction of vegetation cover (Riihim€ aki et al., 2019;Melville et al., 2019), for example to improve estimation of flooded areas (Xia et al., 2017) or detection of invasive species (Kattenborn et al., 2019). Lastly, raw radiometric data can be used directly to calibrate spectral unmixing models. For example, Alvarez-Vanhard et al. (2020) (Alvarez-Vanhard et al., 2020) used pixels considered to be "pure" (not mixed) extracted from UAV data to calibrate an unmixing model of wet grassland habitats based on Sentinel-2 data.
A specific case of the "model calibration" strategy is the data intercalibration where the spectral signature is modeled (Fig. 8). The intercalibration needs a reference dataset to calibrate the rest of the dataset. This reference can be the satellite data to calibrate a finer resolution (Houborg and McCabe, 2018) or the UAV data, itself calibrated by in-situ spectroradiometer measurements, to calibrate the satellite data (Padr o et al., 2018).
In most cases, UAV data were used in addition to in-situ data. Generally, these validation data were integrated in a nested model. For example, Xia et al. (2017) (Xia et al., 2017) used in-situ surveys to validate an object-oriented UAV classification that then facilitated selection of "pure" pixels for a satellite-based spectral unmixing model, as well as to obtain independent training and validation data. However, 32% of the studies used UAVs to replace in-situ surveys, and thus depended only on interpretation of VHSR images. Some studies compared the influence of the use of in-situ data to that of UAV data. For example, Forster et al. (2018) (Forster et al., 2018) demonstrated that UAVs can be used with confidence to provide ground truth for producing thematic maps. Spence and Mengistu (2016) (Spence and Mengistu, 2016) showed that an intermittent-stream detection model calibrated with in-situ data was more accurate than a UAV-calibrated model, but the latter was more robust. Finally, Melville et al. (2019) (Melville et al., 2019) and Liang et al. (2017) (Liang et al., 2017) were specific cases who calibrated a UAV-based model with satellite data. "Data fusion" strategy "Data fusion" can be considered as the strongest synergy because it tries to use the features of each data source fully to create new data. This strategy, although little used, was nonetheless explored by the precision agriculture field (Fig. 4), which aims to extract the land cover and biophysical features of vegetation cover at a fine resolution.
Two-thirds of "data fusion" strategy articles were pixel-level studies with the objective of creating an enhanced-feature dataset by combining one or more resolutions of each source. The most basic method is the densification or completion of time series. For example, using a multisource time series Nikolakopoulos et al.  (Hassan-Esfahani et al., 2017) improved the spatial resolution of Landsat imagery by a factor of four through supervised learning using UAV image patches. Thus, high-frequency details in each band and their derivatives can be recovered from Landsat imagery. Spatial-temporal (Zhu et al., 2018): Pioneering studies demonstrated the contribution of spatial-temporal fusion, especially the Spatial Temporal Adaptive Reflectance Fusion Model algorithm (Gao et al., 2006). Gevaert et al. (2014) (Gevaert et al., 2014) combined UAV hyperspectral data with a Formosat-2 time series to derive biophysical parameters (leaf area index and chlorophyll content) of potato crops and make consistent predictions with fine spatial patterns. In the same vein, Liu et al. (2019) (Liu et al., 2019a) fused a sparse UAV time series with a denser time series from the Planet nano-satellite to estimate grassland forage production finely. Spectral-temporal: Gevaert et al. (2015) (Gevaert et al., 2014) used Bayesian theory to fill in missing spectral information in multispectral satellite data with hyperspectral data from a UAV. The Spectral-Temporal Response Surface model designed in their study provided continuous spectral reflectance at high temporal frequency. The results generated correlated well with spectral measurements in the field (r ¼ 0.953) and allowed for derivation of biophysical variables (leaf area index and chlorophyll content) that were consistent with the observations. The remaining one-third of the studies used a feature-level fusion approach with a less classic approach. For example, Kakooei and Baleghi (2017) (Kakooei and Baleghi, 2017) used different UAV and satellite view angles to assess post-disaster damage on buildings. Using oblique images obtained by the UAV, they extracted features on building facades, while satellite imagery provided features on roofs. These features were fused to estimate damage levels. Another application in change detection used a method for near real-time detection (Fytsilis et al., 2016) that compared features of UAV data to those of historical satellite data in order to detect potential changes in a territory. The method developed was presented as being robust to differences in spatial and spectral resolution and misregistration issues. Finally, decision-level fusion, although having interesting potential (Atkinson, 2013), was not used at all with UAV and satellite data.

Key contributions of UAV data
In this synergy, UAVs played a "bridging role" by complementing and magnifying the potentials of in-situ and satellite data (Fig. 9). UAVs provided new data that can be acquired only by this vector and thus provided a single intermediate observation scale. This characteristic allowed UAVs to play three types of roles according to the strategies: (1) explanation, (2) validation and (3) completion of satellite data.
Explanation was a major use of RS UAV within the UAV/Satellite synergy. It consisted of providing complementary data (e.g. VHSR or DSM) to reveal inaccessible details or unseen from space or on the ground. In-situ observations made by experts or a network of sensors are generally precise but punctual (Gamon, 2015), while satellite observations cover large areas but have a resolution that remains too low to be interpreted properly without in-situ information. Although these two observation scales complement each other, pairing them remains however uncommon because of the difference in surface of the areas they observe (Alvarez-Vanhard et al., 2020). Between the two scales, UAVs provide spatially explicit local data that reveal spatial patterns of prime importance to study processes using satellite measurements (Gamon, 2015;Fawcett et al., 2020). This role of UAVs was particularly highlighted in the "multiscale explanation" strategy, but it also forms the basis of all strong UAV/Satellite synergies.
The validation role consisted of using UAV data as "ground truth" (or "drone truth") and relied on UAVs' ability to explain satellite data. This was the main purpose of UAV data in this synergy in particular via the "model calibration" strategy. UAV data can be used in synergy with insitu data in a nested model or can replace them. As mentioned, UAV data can replace in-situ observations for applications in which the object to be detected was clearly identifiable, such as water bodies (Tschudi et al., 2008;G oraj et al., 2019), the fraction of plant cover (Bian et al., 2016) or certain invasive species (Martínez-S anchez et al., 2019;Elkind et al., 2019). Indeed, UAVs provided spatially explicit data fine enough for experts to identify elements of the landscape directly. Moreover, UAVs had the advantage of covering modest to large areas quickly and can easily reach the least accessible environments. In the current era of data-driven models developed by non-parametric supervised learning methods (e.g. random forest, support vector machine or neural networks), UAVs can play a key role in providing "ground truth". Indeed, validation data are critical in these methods, but the heavy logistical involvement required for large-scale acquisitions reduces the potential of such approaches. UAVs provide an affordable solution for acquiring ground-truth data to analyze or calibrate satellite data. Carbonneau et al. (2020) (Carbonneau et al., 1002) showed that this approach is valid, robust and allows for hydrogeomorphological analysis to be extended to the regional scale by applying locally trained models to new satellite data. The transferability of the models needs to be assessed more widely in terms of seasonal and interannual variability and also between different geographical areas. Nonetheless, this approach should be tested on other applications.
Lastly, the completion role of UAVs lied in their ability to fill gaps in satellite data. Spontaneous UAV acquisitions made it possible to fill temporal gaps in sparse satellite time series (Mengmeng et al., 2017) and to produce change-detection maps in near-real time (Fytsilis et al., 2016). It can also fill spatial gaps, e.g. in which optical satellite data were degraded by cloud cover. UAVs can provide data under clouds, which is particularly useful in tropical areas where cloud cover is frequent for long periods of the year. Through fusion methods, UAVs can improve spatial, temporal or spectral resolutions of satellite data by providing a complementary dimension. These methods were still rarely used at the scales of observation provided by UAVs, but the first applications (Gevaert et al., 2014(Gevaert et al., , 2015Liu et al., 2019a) show high potential that can open the field toward new applications that require very high resolutions, such as monitoring hydrological regimes, phenological traits of plants or degradation states of environments.

A still under-exploited synergy
The analyses made in this paper have highlighted interesting synergies between UAV and satellite data. However, to our opinion, current synergies are not fully exploited and one can go a step forward to go beyond the validation purpose of UAV data and exploit together UAV and satellite data. Today this mutual exploitation is limited to some studies mainly for the two following reasons: interoperability is not obvious and UAV/Satellite synergies often answer to a specific use case without the aim of fully exploring its potential. These two points are discussed below.

Data interoperability remains challenging
The quality and interoperability of satellite data are guaranteed by the Committee on Earth Observation Satellites (CEOS), whose original function was to standardize data formats and ensure the validation, intercalibration and inter-comparison of satellite products. For UAV data, however, there is no guarantee of data quality since the data-acquisition skills are transferred to users. Acquisition and pre-processing protocols can vary among users and sensors, which does not guarantee consistent data, thus making multi-source interoperability difficult. Ensuring this interoperability is an important challenge because models based on the synergy between UAV and satellite data are sensitive to the quality of the input data (Carbonneau et al., 1002;Mengmeng et al., 2017;Belgiu and Stein, 2019). Intercalibrating data (geometric and radiometric) and estimating uncertainty in multisource models are therefore essential steps to ensure the quality of the results from this synergy.
Misregistration between data from one or more sources causes errors in analysis at different spatial and temporal scales. To ensure the quality of analyses, sub-pixel geometric intercalibration is necessary (Fytsilis  Pohl and Genderen, 1998). Although pre-processing UAV and satellite data provides sub-pixel inter-band and inter-date co-registration, their different resolutions imply calibrations of different orders (centimetric and metric, respectively). Methods for automatically georeferencing multisource data can ensure this consistency. For example, the multi-scale SIFT (Scale-Invariant Feature Transform)-RANSACT (RANdom SAmple Consensus) methods (Oh et al., 2011) automatically generate tie points to locate the data correctly using an affine transformation, or optical flow methods (Brigot et al., 2016), which do not use tie points and are effective for multimodal datasets.
Using the full optical spectrum (multispectral or hyperspectral) through different sensors requires ensuring consistency between these radiometric measurements, which remains a challenge in itself. Indeed, reflectance values may vary depending on the type of sensor (wavelength, vignetting), acquisition protocol (angle of view, ground resolution), environmental conditions (atmosphere, topography) and corrections made to the data (Schaepman-Strub et al., 2006). For example, the bi-directional reflectance distribution function of the surface can be ignored and considered as Lambertian for observations at low spatial resolution (satellite), which is not the case for UAV observations, whose high variability in reflectance values is due to the heterogeneity in the optical properties and 3D structure of the surfaces, which leads to heterogeneous observation geometries (Stow et al., 2019;Stark et al., 2016). Moreover, low-cost UAV sensors may have limitations or defaults that are not yet well known and that make it difficult to combine them with satellite data (Fawcett et al., 2020). These differences in reflectance measurements between UAV and satellite observations must therefore be considered because they influence multi-scale models. In the absence of absolute reflectance measurements, radiometric intercalibration is an effective solution for multi-sensor interoperability as seen in the section "Model calibration" strategy.
Despite data correction and intercalibration, measurement uncertainty can persist and must be considered as it propagates into the scales of fusion or nested inference models. For example, Solazzo et al. (2018) (Solazzo et al., 2018) used Monte-Carlo simulations to estimate uncertainty in a dune-volume prediction model based on UAV data and express it in a satellite-based model.
Up to now, these questions of interoperability and intercalibration have strongly limited the nature of exploitation of the synergies between UAV and satellite images.

An unexplored potential
From our review, it seems that the potential of the UAV/Satellite synergy is currently not fully exploited: (1) several scientific fields do not take enough advantage of this synergy, (2) the capacities of the different EO systems are under-exploited, and (3) stronger synergies are less used.
Ecology and agriculture scientific fields have been mainly involved in the exploitation of UAV/Satellite complementarities, unlike other earth observation fields (geosciences, disaster, archaeology, water resource or city monitoring) that generally require less important resolutions (Fig. 1). However, a breakthrough can be anticipated in these less common fields. For example, Antoine et al. (2020) (Antoine et al., 2020) highlight the challenge of combining different observation scales for geosciences and disaster purposes, and identify combined workflow between space-based and UAV data as a proper solution.
EO systems used in this synergy are under-exploited, in particular for UAV capacities. First, EM-satellites are most used, certainly because these data are open (e.g. Sentinel-2, Landsat and Gaofen) and their observation scales provide good complementaries with UAV data. Yet, nano-and civilian-satellites are interesting in particular for the "data fusion" strategy in view of similarities and the low scale factor with UAV data, but their cost remains a barrier. Then, the use of UAV data is too limited to RGB, leaving the high potential of optical RS still unexploited. Specific characteristics of the observation geometry remain little used but have an interesting potential to improve characterization of surface reflectance properties, such as anisotropy (Roosjen et al., 2017;Su et al., 2007;Martonchik, 1994). Similarly, optical properties of the surface observable by multispectral sensors vary and are related to acquisition conditions (e.g. wavelengths, viewing angle, ground resolution); thus, the flexibility of UAVs does not yet appear to be fully exploited and new applications that use both UAVs and satellites that may lead to new types of synergies remain to be discovered and developed. Lastly, technological advances related to the installation of LiDAR and hyperspectral sensors on satellites or UAVs will also increase the potential of this synergy.
Currently, this synergy remains in the "UAV or Satellite" and the "UAV for Satellite" paradigms. The former corresponds to the "data comparison" use case, whose purpose is well summarized by the question of Abdullah et al. (2018) (Abdullah et al., 2018): "Can UAVs replace Satellites?". This purpose can finally be seen as opposed to a synergy. The latter paradigm refers to the "model calibration" strategy using UAV data for a validation purpose. Although this strategy is efficient to answer scientific questions, we think this way is not optimal and doesn't keep information from the multiscale observations. Actually, UAV data is often reduced to a label or up-sampled to satellite spatial resolution losing fine spatial patterns. The strongest synergies "multiscale explanation" and "data fusion" are strategies preserving information from each source, and should be more explored to move towards a new paradigm: "UAV and Satellite".

Towards a stronger exploitation of synergies
From the previous section one observes that UAVs have completed satellite images for validation, completion or explanation purposes but these two sources of data have not that much been used together for fusion issues. This is to our opinion due to the fact that: -the questions of interoperability are tricky; -usually UAV are acquired for a specific application and no more exploited then; -fusion requires advanced methodologies not easily accessible.
From this latter point, it is important to outline that since several years, spectacular advances in machine learning and especially with deep neural networks have enabled a breakthrough in the processing of massive and heterogeneous spatial data.

Machine learning for UAV/Satellite fusion
The idea behind deep learning is to construct a neural network composed of a large number of layers, enabling to model very complex relations between inputs (multi-source data) and outputs (estimation of parameters, labels, etc). Though the idea is old, the progresses in the recent years come from the fact that we have now enough data and associated computational resources to train such complex networks (and then to condition the optimization processes associated with the large number of calibration parameters to estimate). In addition, some theoretical progress on the definition and optimization of such networks have opened a wide range of applications. The reader can find in (Yan et al., 2015) a general introduction to deep learning.
As for the processing of spatial data, the state of the art network for assigning a label to an image is the well-known CNN (Convolutional Neural Network). Since then, many architectures, either adapted to assign a label to each pixel (Fully Convolutional Networks and variants (Long et al., 2015)), to deal with unstructured data (Qi et al., 2017) or to time series for example (Fazle et al., 2017) have been proposed. More recently, the community is also focusing on the fusion of complex data, as for example (Zhenfeng and Cai, 2018) in the context of satellite images. The great quantitative improvements have encouraged researchers to explore the fusion of heterogeneous sources and some studies using UAV have already been proposed (Liu et al., 2019b;Barrero and Perdomo, 2018). Obviously we encourage the community to explore this point and to fully exploit the synergies between UAV and satellites.
Associated with these new methodological insights, the open access to data is also a trend the community has to follow.

Facilities for multisource workflow
The UAV/Satellite synergy has the potential to overcome current limitations of EO systems. However, the complementarities of the data sources are not sufficient for this potential to be fully exploited. The combined workflow between UAV and satellite data must be facilitated and adapted to the Big EO Data. To achieve this, facilities are needed to ensure the Findability, Accessibility, Interoperability, and Reusability (FAIR principles (Wilkinson et al., 2016)) of data and methods. In this section, we discuss current initiatives that contribute to this vision of an open and reproducible EO science.
There is a trend to the openness of satellite data which are now easier to find due to the multiplication of downloading platforms based on the model of ESA's Copernicus program. Conversely, sharing of UAV data is not following this trend to the same degree. The difficulty in sharing UAV data is understandable given the large number of suppliers who acquire data with specific objectives. The lifespan of UAV data is therefore currently limited to a single use, although it could be used more widely for global applications if they would become open-access. Nonetheless, initiatives such as Open Aerial Map (OpenAerialMap. OpenAerial, 2020) or the GEOSS platform (GEOSS portal, 2020) allow UAV data to be shared freely. At European level, the INSPIRE directive (European Parliament and t, 2007) requires public authorities to make their environmental spatial data open and accessible on the internet. This directive also concerns public bodies producing ortho-imagery by UAV. For example in the West of France, the spatial data infrastructure Indigeo contributes to this effort by sharing UAV data among other geodata (Indigeo, 2021). However, the free sharing of data raises the issue of personal data and privacy that are accessible through VHSR UAV imagery, i.e. information allowing the direct identification of natural persons (e.g. identifiable faces), or indirect (e.g. number plates) and private property. For example, in the European Union, the diffusion of data must be done in conformity to the General Data Protection Regulation (GDPR) (European Parliament and o, 2016) which prohibits the diffusion and processing of personal data without the consent of the person, owner or manager of a property. Lastly, data sharing must be accompanied by explicit description of the acquisition conditions based on the model of the Spatio Temporal Asset Catalog initiative (STAC (SpatioTemporal asset cata, 2020)), which facilitates queries of spatial data. Indeed, as mentioned, interoperability must also be achieved for the radiometric measures, which depend on the acquisition conditions. These metadata and their meanings are widely transmitted for satellite products but, again, it is not yet standard for UAV products.
Finally, infrastructure such as EO Data Cubes (EODCs) facilitate and formalize the integration, processing and analysis of Big EO Data, thus contributing to the reproducibility of EO science (Giuliani et al., 2019). EODCs provide an architecture that brings together different spatial databases to facilitate data storage and manipulation. Many initiatives have emerged this decade, such as Open Data Cube (Lewis et al., 2017), Google Earth Engine (Gorelick et al., 2017), JEODPP (Soille et al., 2018) or Rasdaman (Baumann et al., 2019). However, each initiative has its own architecture, which is not compatible with the others. OpenEO (openEO, 2020) addresses this problem by providing an API (Application Programming Interface) that generalizes connections between users and EODCs. Recently, in addition to these initiatives, code sharing on platforms such as GitHub has contributed greatly to strengthening this synergy by contributing to the development of new algorithms and making it possible to test them on new datasets. Although this practice remains rare, some applicable contributions to this synergy exist (Carbonneau et al., 1002;Tan et al., 2019).

Conclusion
The scientific literature was reviewed to guide future studies that wish to use the strong complementarities between UAVs and satellites. Overall, 137 articles published in peer-reviewed journals were collected. This corpus of literature made it possible to identify four main strategies: "data comparison", "multiscale explanation", "model calibration" and "data fusion". These four types of synergy helped in filling the gap between sensor capacities and the data needs in EO applications. However, this categorization was specific to the corpus of this review and will aim to evolve with future applications. In just a few years, this synergy emerged with various strategies and applications. Through this bibliographical study highlighted the following trends: 1. Ecology was the main application area that used this synergy and used mostly the "model calibration" strategy. 2. A significant part (29%) of UAV/Satellite synergies were weaks ("data comparison"). 3. "Multiscale explanation" and "data fusion" strong synergies were under-exploited. 4. RS UAV can replace in-situ surveys for basic applications. 5. UAV capacities offered greater potential than what was currently used.
We concluded that the UAV/Satellite synergy evolved quickly and provided proper solutions to answer scientific questions in need of multiscale observations. However, this potential was under-exploited using mainly UAV data for validation purposes. We suggest to the scientific community to explore "multiscale explanation" and "data fusion" strategies to fully exploit these multiscale data. Advances in multisource interoperability, data sharing and machine learning will help move towards these stronger synergies.

Funding
This study was supported by public funds (R egion Bretagne) received in the framework of the emerging research project (PER) coordinated by "Groupement Bretagne T el ed etection" (BreTel), the University of Rennes 2, the ANR project MATS (ANR-18-CE23-0006) and from the European Regional Development Fund (ERDF) under the umbrella of INTERREG Atlantic Area (EAPA_261/2016) "Improving the management of Atlantic Landscapes: accounting for biodiversity and ecosystem services (ALICE)".

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.