Renewable and Sustainable Infrared thermography in the built environment: A multi-scale review

The paper presents a review on major contributions in infrared thermography to study the built environment at multiple scales. To elaborate the review, hundreds of studies conducted between the 1980s and 2020s were first selected based on their relevance to the scope. Afterward, the most relevant contributions were classified and chronologically sorted. From the classification, it is observed that most reviewed studies were conducted to evaluate the thermal performance of buildings or detect their defects using images collected by an infrared camera. At the same time, a considerable number of studies used thermal images obtained by a satellite to observe the urban heat island effect. Despite the important number of contributions in infrared thermography at multiple scales of the built environment, three main research gaps or opportunities can be identified in the literature. First, it would be possible to perform a more detailed analysis of urban heat fluxes using thermal images collected at multiple scales. Then, thermal images collected by a mounted or handheld infrared camera could be used to create building energy models. Finally, better visualization tools would be developed to monitor a city’s energy use and improve its sustainability if thermal images were integrated into Internet-of-Things and digital twin platforms.


Introduction
Since the Industrial Revolution in the 19th century, a significant portion of the world population has moved from rural to urban areas. Urban areas are expected to accommodate more than 65% of the world population in 2050, and more than 85% in the most developed regions [1]. This rapid growth of the built environment has caused an augmentation of CO 2 emissions due to the building energy consumption. According to the International Energy Agency [2], 28% of the world's CO 2 emissions in 2019 are due to the energy consumed in buildings. Given this observation, considerable efforts have been made by the scientific community to better understand the built environment using different sensing technologies.
One of the most common methods to observe outdoor conditions in the built environment is to use a network of automatic weather stations [3]. Weather stations typically measure the temperature, relative humidity, pressure, wind speed and direction, solar irradiance, and rainfall, which is an important number of variables in comparison with other sensing technologies. However, one weather station only provides this set of information at a single point. As a consequence, no matter how large a network of automatic weather stations is, it will Urban Surface Temperature using infrared thermography. Infrared thermography can also be applied in different manner when thermal images are collected using an Unmanned Aerial Vehicles (UAVs) as explained in the review of Rakha and Gorodetsky [7]. Instead of describing possible applications of infrared thermography at multiple scales of the built environment, most reviews explained the various methods that were used to assess the energy efficiency of a building from thermal images. These methods were classified by Fox et al. [8] concerning their measurement method, which can be qualitative or quantitative, their experimental environment, which can be indoor or outdoor, and their analysis scheme, which can be active or passive. While active infrared thermography is usually used to detect internal defects of a material or building layer using an internal or external source of excitation, passive methods aim at observing the heat emitted by a surface. The review conducted by Kylili et al. [9] elaborates more on active and passive analysis schemes in infrared thermography. In addition to classification criteria of Fox et al. [8] and Kirimtat and Krejcar [10] considered the analysis type, the envelope components, the surface material, and the testing location in their review. Other reviews focused on methods used for the energy audit of buildings as Lucchi [11] or for the detection of heat losses as Nardi et al. [12].
Regardless of whether reviews considered applications or methods of infrared thermography in the built environment, their exploration of the literature is limited to a specific scale. In particular, the majority of reviews described applications or methods of infrared thermography at the building-scale, the lowest possible scale of the built environment. Few of them reported how thermal images were used to observe the built-environment at higher scales, and none at multiple scales. Consequently, a considerable knowledge gap remains on applications of infrared thermography that were performed at one specific scale of the built environment or at multiple ones. By bridging this gap, it can be shown what kind of applications can be developed in the future to better understand the built environment at multiple scales.
For this reason, a comprehensive review is introduced on infrared thermography in the built environment at multiple scales. The objectives of the review are to: (1) show applications of infrared thermography at each scale of the built environment; (2) describe these that were performed at multiple scales; and (3) indicate research opportunities. From the review, the scientific community would gather what contribution can be made using infrared thermography at a single scale or multiple ones of the built environment. Practitioners would also learn how thermal images were collected from different infrared systems, and how they might be integrated in the future to develop support tools in urban planning.
The review is organized as follows. The early history and fundamentals of infrared thermography are described in Section 2. Section 3 describes the methodology. Results are shown and discussed in Section 4. In Section 5, several opportunities are suggested for future research on infrared thermography in the built environment. Finally, conclusions and future work are explained in Section 6.

Origins and principles of infrared thermography
Research in infrared thermography has a long history of more than two centuries. Fig. 1 illustrates the timeline of important contributions between the 19th and 20th century. During the 19th century, early discoveries were found by the five fathers of infrared thermography: William Herschel and his son John, Leopoldo Nobili, Macedonio Melloni, and Samuel P. Langley. Although the 19th century represents the origin of infrared thermography, it was during the 20th century that major technological advancements were made. These technologies were developed for various sectors, including the military, medicine, and building.
In the literature, it is commonly agreed that the birth of infrared thermography was in 1800, after the publication of William Herschel [8][9][10][11]. During an experiment, he observed the visible portion of the sunlight is not the only one that can increase the surface temperature of a target object [13]. Therefore, he concluded that the sunlight is certainly composed of radiation with a higher wavelength than the visible red light, influencing the etymology of infrared. The discovery of infrared radiation led to several inventions in the 19th century. One of these inventions is the thermopile of Nobili and Melloni presented in 1831 [14]. The thermopile was created initially by Leopoldo Nobili to measure temperature. Using the first prototype of the thermopile, his associate Macedonio Melloni found a way to measure radiant heat. After the invention of the thermopile, in 1840, John Herschel, son of William Herschel, produced one of the first thermograms from the sunlight [15]. The thermogram was obtained by a method called evapography, using a lens to focus the sunlight on alcohol-containing carbon particles. Samuel P. Langley invented a more advanced technique in 1880 to measure far-infrared radiation, a component of infrared radiation [16].
Inventions in the 19th century were merely preliminary prototypes leading to modern systems for the acquisition of infrared or thermal images. The first system able to acquire thermal images from a camera was invented by Kalman Tihanyi in 1929 [17]. The camera was used for nocturnal vision by the aircraft defense. Many infrared technologies were then developed in the military sector, particularly during World War II [18]. In 1959, a system called the Pyroscan was first installed to acquire thermal images for medical use at the Middlesex Hospital in London and the Royal National Hospital for Rheumatic Diseases in Bath (United Kingdom). Twenty-four years later, in 1983, the first commercial systems were created and used for various applications in infrared thermography [19].
Most acquisition systems of thermal images that have been developed since the 1980s work on the same fundamental principles [20,21]. As illustrated in Fig. 2, systems acquiring thermal images primarily consists of a sensor. The sensor receives a heat flux (in W/m 2 ), which is a combination of several radiative heat fluxes, that is: where is the transmissivity of the air between the target surface and the thermal sensor, the emissivity of the target surface, the longwave radiation from the target surface (in W/m 2 ), the longwave radiation from the background (in W/m 2 ), and the longwave radiation from the air (in W/m 2 ). The longwave radiation (in W/m 2 ) is a portion of the infrared radiation between 7 and 14 μm [10], which can M. Martin et al.  be expressed from the surface temperature of an object (in Kelvin) as: where is the Stefan-Boltzmann constant (= 5.67 ⋅10 −8 W/m 2 -K 4 ).
According to Eqs. (1) and (2), the surface temperature of the target surface = 4 √ ∕ (in Kelvin) depends on various variables. One of them is the emissivity of objects [22]. Another is the longwave radiation emitted by the background, corresponding to the skydome in most outdoor situations. The background can become more complex to define if thermal images are collected in an outdoor environment with obstructions. In this case, the longwave radiation emitted by the background can be measured from an aluminum foil placed on one of the target objects [23].
Due to the number of variables that need to be known when using an infrared camera, the accuracy of the surface temperature is often less than this obtained with a contact surface sensor. However, the surface temperature can be observed with a higher spatial resolution using thermal images collected by an infrared camera, as shown in Fig. 3. Therefore, thermal images enable us to evaluate how cool or hot certain elements in the built environment are compared to others. The thermal behavior of some features like trees or grass can be challenging to study with contact surface sensors, but not with thermal images. Thermal images can now be collected with a similar time resolution to contact surface sensors. Because thermal images are two-dimensional data, they require significantly more space to be stored in a computer.

Methodology
The objective of the review was to address one question in particular: What are applications and research opportunities in infrared thermography at multiple scales of the built environment? To find a response to this question, the review was conducted following the workflow described in Fig. 4. The first step in the workflow was to enter a list of keywords on the Google Scholar search engine. Among the papers resulting from the keyword-based search, one was read to understand if its content was about the built environment. The built environment refers to the outdoor environment within a city ranging in scale from the microscale to the mesoscale. Each paper, whose content corresponds to a study about the built environment, was then classified for its analysis with other contributions. The analysis was conducted after reading all the papers in the list and after using all identified keywords. From the results provided by the investigation, the objectives were to detect research opportunities and state some conclusions.
Most papers in the literature define a list of keywords they consider to be the most representative concepts discussed in their content. While reading each article, some of their keywords were considered to explore the literature further. The list of keywords used for analyzing the literature is shown in Table 1. If a paper contained one of these keywords in its list or its title, it was then considered to be potentially relevant to the review. To be fully applicable to the study, the paper also needed to address an issue of the built environment. The reason is that an article can contain one of these keywords in its list or title and discuss another topic not related to the built environment. For example, M. Martin et al.  one paper might include the term 'Infrared Thermography' in its list of keywords but study one of its medical applications. In contrast, some papers might not have a word in their list of keywords and still indirectly refer to it in their content. One of the reasons is that the terminology used in infrared thermography usually varies depending on the scale it is applied. When using a satellite to collect thermal images, the term 'Infrared Thermography' itself is rarely used, but 'Land surface temperature' or 'Remote sensing' instead. It explains why not all reviewed papers contain the term 'Infrared Thermography' in their list of keywords or titles.
For each relevant contribution to the built environment, a set of information was extracted from its content. One such example is the country where the study was conducted. This piece of information enabled us to determine where most studies in infrared thermography were completed and which regions might need further exploration. Another criterion was the scale at which infrared thermography was applied in the study. In addition to the scale, it was essential to determine the system used in the study to collect thermal images and the application in which the thermal images were intended to be used. The classification of selected contributions was reported in a spreadsheet to conduct the data analysis based on all these criteria.
The scale at which a reviewed study was conducted needed more efforts to be identified than other criteria. The reason is that there are multiple ways to define the scale of a study in the built environment. One of them is described by Oke [24] and has been primarily used in urban climate studies. It includes three essential scales, which are the mesoscale (10-200 km), the local-scale (0.1-50 km), and the microscale (>1 km). Another way to classify studies in the built environment by scales is explained by Hertwig et al. [25]. It has been used mainly by urban planners to distinguish their studies either as city-scale (10-100 km), neighborhood-scale (0.1-10 km), and building-scale (10-100 m).
Apart from comparing the number of studies for each identified class, the data analysis also consisted in understanding the evolution of infrared thermography in the built environment between 1980 and 2021. For this purpose, the chronology of selected contributions was established and analyzed to show how some limitations were overcome in the past or could be solved in the future, which explains why critique was an essential aspect of the review.

Results and discussion
During the literature review, it was observed that different infrared systems were used to study the built environment at one or several scales. The infrared systems are presented in Fig. 5. The system that enables the collection of thermal images at the largest scale is a satellite. Apart from the thermal images, which are usually used to measure the Land Surface Temperature (LST), a satellite can contain other sensors to collect data at the mesoscale or city-scale remotely. When thermal images are needed between the city-scale and neighborhood scales, an infrared camera is normally installed on an Aerial Vehicle (AV) such as an aircraft or a helicopter. The infrared camera can also be installed on different supports to collect thermal images at lower scales. It is installed on a rooftop observatory for neighborhood-scale studies, on a drone for studies between the neighborhood-scale and buildingscale, or with a tripod, handheld system, or smartphone for microscale studies. Fig. 6 illustrates the number of studies that have been conducted since 1980 in the built environment using infrared thermography at different and multiple scales. In general, a growing interest in the use of infrared thermography is observed to explore the built environment, which is certainly due to major advancements of this technology and reductions in its cost. More specifically, it is seen that the portion of studies at different scales does not seem to significantly vary whether the classification of Oke [24] or Hertwig et al. [25] is used. In either case, microscale or building-scale studies appear to be the most frequent, with a portion between 50% and 55% since 1980. The number of studies conducted at the mesoscale or city scale also looks relatively substantial. However, a few contributions were found in infrared thermography to explore the built environment at the local and neighborhood scales. Even less reviewed studies considered thermal images simultaneously collected at multiple scales. The small number of local, district, and multi-scale studies can be explained from the time and cost required to collect thermal images with adequate infrared systems. Indeed, it often requires more time and cost to collect thermal images from an AV or observatory than a satellite or mounted camera.
The small portion of local-scale or neighborhood-scale studies could be explained by the fact that they are more difficult and expensive to set up than those conducted at other scales. Launching a satellite to collect thermal images at the mesoscale or city scale might be costly, but thermal images obtained from a satellite can then be shared with the all scientific community at a relatively low price. The collection of the thermal images at the microscale or building-scale can also be achieved at a low cost whether drones, mounted cameras, handheld cameras, or smartphones are used for this purpose. In contrast, installing an infrared system to conduct a local-scale or neighborhood-scale study is always relatively expensive. Safety and confidentiality measures to consider for local-scale or neighborhood-scale studies are more important than for other scales. Fig. 7 shows applications that can be performed in the built environment by different infrared systems and at different scales. According to review papers, thermal images collected by a satellite can be used to either observe the UHI effect, analyze urban heat fluxes, or study urban descriptors. Observations of the UHI effect were also made at the local-scale or neighborhood-scale using Aerial Vehicles (AVs), rooftop observatories, or drones. In addition, to analyze urban heat fluxes, drones were used to detect building defects, evaluate building thermal performance, analyze UHI mitigation strategies, and detect faulty Photovoltaic (PV) panels at the microscale or building scale. Similar applications were performed simultaneously using mounted cameras, handheld cameras, and smartphones.

Studies conducted at the mesoscale or city-scale
Most studies aiming at observing the built environment at the mesoscale or city-scale have used remotely sensed data collected from satellites. Between the 1950s and 1960s, satellites were essentially used for military purposes [26]. During the 1970s, several satellites were launched in orbit to collect information for the scientific community, including thermal images. The thermal images were used in some preliminary studies during the 1980s to observe the LST. For example, the accuracy with which the LST can be measured from the Landsat 3 and National Oceanic and Atmospheric Administration (NOAA) 6 was evaluated by Price [27,28]. With thermal images obtained from the NOAA 6, Price discovered that the surface temperature could be measured with an accuracy of ±2-3 • C. Other studies, such as the one by Vukovich [29], used thermal images obtained from the Heat Capacity Mapping Mission (HCMM), another satellite, to observe the difference in the LST between urban and rural areas in St. Louis (USA). In the same city, Kidder and Wu [30] made similar observations but considering the snow-covered. Later in the 2000s, two MODerate Resolution Imaging Spectroradiometer (MODIS) satellites were launched to provide a variety of remote sensed data [31]. The first satellite, MODIS Terra, was launched in 1999. It provides the same remote sensed data as the MODIS Aqua, launched in 2002.
Tables 2-4 shows reviewed studies that were conducted at the mesoscale or city-scale using infrared thermography with Landsat, NOAA, or MODIS. The majority of these studies were conducted in China and North America, as illustrated in Fig. 8 and the review of Almeida et al. [5]. A non-negligible portion of reviewed studies, approximately 12%, was conducted in Europe. These observations imply that numerous contributions in understanding the built environment at mesoscale or city-scale can still be made in several countries, particularly those located in Africa and South America.
Among all reviewed studies conducted at the mesoscale or cityscale, the majority collected thermal images from Landsat according to Fig. 9. The reason seems to be that Landsat is more suitable for observing the UHI effect and inferring urban descriptors than its concurrent. Nevertheless, MODIS Terra and Aqua look to gain interest over the years, in particular, to analyze urban heat fluxes. This fact could be due to the significant amount of remotely sensed data provided by MODIS Terra and Aqua in addition to the LST. While there was a visible competition between MODIS and Landsat in the scientific community, few reviewed studies used NOAA to understand the built environment using infrared thermography.    . 10 shows the number of reviewed studies that were dedicated to observing the UHI effect, analyzing urban heat fluxes, and studying urban descriptors using thermal images collected from a satellite. According to these results, most reviewed studies aimed at observing the UHI effect using the LST measured with a satellite. A notable number of reviewed studies also used other remote sensed data in addition to the LST to analyze urban heat fluxes and study urban descriptors.

Observation of the UHI effect from the LST
During the 1990s and early 2000s, various studies proposed methods to observe the UHI effect from the LST measured using a satellite. Among these studies, Roth et al. [32] analyzed the UHI effect during the day and at night. The study conducted by Carnahan and Larson [33] focused more on the possible sinks of the UHI effect. Even though thermal images can be obtained for various locations using a satellite, Roth et al. [32] and Carnahan and Larson [33] assessed the UHI effect of one city only. As an improvement, Gallo et al. [34] showed how the UHI effect could be evaluated in several cities in the United States using the NOAA satellite. Data collected using this kind of satellite can be used to develop empirical models of the UHI effect as expressed by Streutker [36]. The models can predict the UHI effect at specific times when no data is available. As illustrated by Lo and Quattrochi [37], thermal images collected from a satellite are indeed limited over time. However, the amount of information each time is relatively significant compared with what can be provided by an empirical model.   most studies were conducted in China. For instance, Nichol [38] tried to evaluate the influence of the urban morphology of Hong Kong on the UHI effect. A more detailed study was conducted by Chen et al. [39] on the relation between the land use of Guangzhou and the UHI effect. In addition to the land use, Yang and Liu [42] retrieved biophysical characteristics using thermal images of Lanzhou to understand the formation of the UHI effect. Wang et al. [43] did not only consider thermal images to analyze the UHI effect in Beijing. They assumed the UHI effect could also be explained from the albedo, vegetation index, and broadband surface emissivity remotely sensed from MODIS. Using information measured from the Landsat Enhanced Thematic Mapper Plus (ETM+) satellite, Li and Yu [44] studied characteristics of the UHI effect in Wuhan. As a complement to the analysis based on measurements, they performed Computation Fluid Dynamics (CFD) simulations to understand how the UHI effect could be mitigated by providing better outdoor air circulation. It is relatively unusual that simulations of the UHI effect are performed in addition to a measurement-based analysis. Studies like Li et al. [46] in Shanghai and Wang et al. [63] in Shenzhen normally try to find spatial patterns and correlations using remotely sensed data from satellites to analyze the UHI effect.
Apart from the United States and China, other mesoscale studies of the UHI effect were conducted in other parts of the world during and after the 2010s. Bechtel [48] tried to identify spatial patterns of the surface temperature in Hamburg (Germany). In addition, measurements of the surface temperature were used for the classification of local climate zones following the definition established by Stewart and Oke [109]. Lazzarini et al. [53] used another kind of classification called impervious surface areas to analyze the relationship between the roughness of an urban area and the magnitude of the UHI effect in Abu Dhabi (United Arab Emirates). Despite the various classifications of an urban area that have been studied in the literature, the land use/cover seems to remain the most appropriate one to explain the UHI effect using thermal images as shown by Tomlinson et al. [52] in Birmingham (United Kingdom), Dihkan et al. [56] in Istanbul (Turkey), and Kikon et al. [59] in Noida (India). Thermal images provide information about the LST but not necessarily the ambient temperature. Because the ambient temperature is often considered to evaluate the magnitude of the UHI effect, Ho et al. [55] determined how it could be derived from thermal images obtained in Vancouver (Canada) using machine learning algorithms. A similar study was conducted by Coutts et al. [58] in Port Philip (Australia) using very high resolution airborne thermal M. Martin et al.  images. Instead of using the difference in the surface or ambient temperature between an urban area and its rural surroundings, Shirani-Bidabadi et al. [62] used another metric called the urban heat island ratio index to calculate the magnitude of the UHI effect in Isfahan (Iran).

Analysis of urban heat fluxes
The analysis of urban heat fluxes tries to understand the causes of the UHI effect more than assessing its magnitude from thermal images. From the early 2000s, various studies have attempted to evaluate the heat emitted by buildings, vegetation, and anthropogenic sources from thermal images obtained by satellites.
Among these studies, some contributed to the assessment of the netall wave radiation from remote sensing data obtained by a satellite. For example, Chrysoulakis [66] was one of the studies proposing a method to assess the net-all wave radiation, which consists of the downward shortwave radiation, the upward shortwave radiation, the downward longwave radiation, and the upward longwave radiation. A more sophisticated method was defined by Bisht et al. [68], in which the upward longwave radiation is measured from the LST and emissivity. The accuracy of the LST/emissivity method was checked by Wang et al. [70]. Tang and Li [72] describes how the net-longwave radiation can be estimated from the upward longwave radiation assessed using the LST/emissivity method and the downward longwave radiation measured from the top of atmosphere radiance. All necessary information to compute the net-all wave radiation was measured by Bisht and Bras [74] using the MODIS Aqua satellite. Both the Aqua and Terra MODIS satellites were used by Wu et al. [76], and [80] to test multiple predictive models of the net-all wave radiation.
Apart from the net-all wave radiation, sensible and latent heat fluxes can also provide useful information to understand the causes of the UHI effect. However, these two urban heat fluxes cannot be estimated from thermal images obtained by satellite only. In addition to thermal images, data need to be collected from a network of automatic weather stations, as originally formulated by Kustas and Norman [110]. Using thermal images from satellites and data from a network of automatic weather stations, Norman et al. [64] could show one of the first pictures of latent heat fluxes at the mesoscale. Both the sensible and latent heat flux were assessed by Ma et al. [65] and French et al. [67].
Although sensible and latent heat fluxes can be directly estimated, they need to be balanced with other urban heat fluxes to ensure their validity. In the study conducted by Ma et al. [65], sensible and latent heat fluxes are balanced with the net-all wave radiation and the ground heat flux, which might not be representative of all heat Table 2 Reviewed studies using thermal images to observe the UHI effect at the mesoscale or city-scale.

Study of urban descriptors
In parallel to the assessment of the UHI effect and the analysis of urban heat fluxes, various studies observed urban descriptors at the mesoscale, which can either be biophysical or geometrical. Urban descriptors were inferred from remotely sensed data, including thermal images obtained by satellite, to study their impact on the LST or UHI effect.
Biophysical descriptors correspond to the different types of surfaces that can be observed over a region, including the fraction of vegetation, pavement, and soil. In the 1990s, Carlson et al. [111] and Gillies and Carlson [112] observed that biophysical descriptors could be inferred from the LST and Normalized Vegetation Difference Index (NDVI). As illustrated by these studies, the correlation between the LST and biophysical descriptors was studied initially in rural areas.
Since the 2000s, biophysical descriptors have been inferred in urban areas. Weng et al. [82] was among the first in inferring the LST and the NDVI remote sensed data to understand the correlation between the portion of vegetation in a city and its UHI effect. From the LST, Lu and Weng [83] tried to define several types of land covers from forests to highly dense urban areas.
Several studies have attempted to evaluate the impact of land cover or biophysical descriptors on the LST or UHI effect using a different method. He et al. [84] applied an interpolation between urban and rural stations to assess the UHI effect while using land cover maps estimated from satellites as in Lu and Weng [83]. Zhou et al. [85] used a multi-linear regression model to study the impact of land cover on the LST, both obtained from a satellite. Instead of considering various biophysical descriptors, Xu et al. [86] focused on the effect caused by impervious surfaces. In contrast, Guo et al. [87] still considered several biophysical descriptors. Using an object-oriented segmentation approach, their impact was studied on clusters of the UHI effect. In addition, to assess the impact on the UHI effect, Sannigrahi et al. [88] tried to understand how some biophysical descriptors can help in mitigating this climatic hazard. A model combining a principal component analysis with an ordinary least squares regression was developed by Firozjaei et al. [89] to study the impact of biophysical descriptors on the LST. The relation between biophysical descriptors and the LST was analyzed by Ghosh et al. [90] using Geographic Information System (GIS) and statistical-based models.
Recent studies show that not only biophysical descriptors can be inferred from remotely sensed data but also geometrical ones. Among urban geometrical descriptors, sky view factors are essential in explaining the causes of the UHI effect. The lower the sky view factor is, the higher the magnitude of the UHI effect at night. The relation between the sky view factors and the LST was studied by Scarano and Sobrino [57] and Scarano and Mancini [61]. In addition to sky view factors, Hu et al. [91] analyzed how the LST is affected by many other urban geometrical descriptors. In contrast, Kim et al. [92] focused on extremely low sky view factors and their impact on the LST.

Studies conducted at the local-scale or neighborhood-scale
Compared to the mesoscale or city-scale, more infrared systems can be used to observe the built environment at the local or neighborhood scales. In Fig. 11, it is shown that the built environment was observed mainly using an infrared camera installed on a helicopter or observatory. The use of aircraft can utilize expensive platforms to collect thermal images at the local or neighborhood scales. Drones are a modern technology, which appears to be used in a few studies at the local or neighborhood-scale for the moment, but might gain more interest in the future due to its low cost in comparison to other systems.
As observed at the mesoscale or city-scale, and as illustrated in Fig. 12, most studies at the local-scale or neighborhood scale aim at observing the UHI effect and analyze urban heat fluxes using thermal images. The only difference is that the UHI effect can either be observed from the LST or UST at a local or neighborhood scale. This observation is explained by the fact that infrared systems like helicopters and  observatories enable the collection of thermal images from different perspectives, including an oblique or vertical view of an urban area. It implies that thermal images collected at the local or neighborhood scale can potentially provide more information about the magnitude or causes of the UHI effect than at other scales. Because of this, it is surprising that a few efforts have been made so far to investigate the UHI effect using infrared thermography at the local scale or neighborhood scale (see Table 5).

Observations of the UHI effect from the LST
In Section 4.1.1, it was discussed that many studies had collected thermal images to observe the LST or UHI effect of a city. Thermal images collected from a satellite have a limited resolution, which can distort observations of the LST at the local scale.
For this reason, various studies have collected thermal images either from an aircraft or helicopter. Eliasson [93] was among the first studies in observing the LST measured from two infrared cameras that were placed on an airplane. Instead of an aircraft, Voogt and Oke [94] preferred to use a helicopter to collect thermal images at the local scale.
These two studies were considered as references to many others that have been conducted since the 2000s. Saaroni et al. [95], for example, show how characteristics of the UHI effect at the local scale can be obtained from an infrared camera installed on a helicopter and a network of automatic weather stations. The view of thermal images collected from an aircraft or a helicopter is not necessarily planned but can also be oblique as shown in Lagouarde et al. [96]. From a plan view, Leuzinger et al. [97] observed the surface temperature of different species of trees in an urban area.
Aircraft and helicopters are constantly in motion, which does not facilitate the collection of thermal images over space and time. Concerning this problem, Lagouarde et al. [98] explains how time-series can be extracted from thermal images obtained from a helicopter. Honjo et al. [100] tried to reconstitute an LST map from a mosaic of thermal images taken at different positions from a helicopter. A similar map was used by Antoniou et al. [101] to validate the LST assessed from computational fluid dynamics.
It might be relatively expensive to use an aircraft or helicopter to collect thermal images at the local or neighborhood scales. For this reason, other studies explored the possibility of using drones for this purpose. To reconstitute thermal image at the local-scale or neighborhood-scale, drones need to travel at several points over the region of interest as explained by Lagüela et al. [99] and Fabbri and Costanzo [102].

Observations of the UHI effect from the UST
As shown by Lagouarde et al. [96], the Urban Surface Temperature (UST) can partially be obtained from thermal images obtained by an aircraft or helicopter. By UST, it is here referred to the surface temperature of façades, roofs, and streets in an urban area as formulated by Krayenhoff and Voogt [113].
Instead of using an aircraft or helicopter, studies measured the UST from an infrared camera installed at the rooftop of a building. This type of installation is often referred to as an observatory. An observatory with pan/tilt unit was installed by Adderley et al. [103], for instance, to collect hemispheric thermal images over 360 degrees. Using a similar infrared system, but without a pan/tilt unit, Morrison et al. [104] observed the UHI effect from the UST of an entire neighborhood.

Analysis of urban heat fluxes
Thermal images collected from an observatory can also assess urban heat fluxes at the local or neighborhood scales. Apart from the heat emitted by roofs and roads, which is better assessed using thermal images collected from an aircraft or helicopter, observatories enable to improve estimates of heat fluxes coming from building façades or any other vertical element in an urban area. Studies conducted by Richters et al. [105] and Morrison et al. [108] prove that estimates of the longwave radiation emitted by building façades can be improved using an observatory. Another urban heat flux, the sensible heat emitted by building façades, was observed by Sham et al. [106] using an observatory and a network of automatic weather stations. According to Dobler et al. [107], thermal images collected by an observatory also allow locating the heat emitted by HVAC systems.

Studies conducted at the microscale or building-scale
As mentioned at the beginning of Section 4, the majority of observations in the built environment using infrared thermography were made at the microscale or building-scale. Fig. 13 shows that a significant portion of these observations was made in Europe, in particular Spain and Italy. In Asia and North America, as well, several studies in the built environment were performed at the microscale or building-scale. As observed for studies conducted at the mesoscale or city-scale, South America and Africa are the regions where more efforts should be made in the future.
Most studies at the microscale or building-scale were performed using an infrared camera mounted on a tripod, as illustrated in Fig. 14. A non-negligible number of studies were conducted using handheld cameras or drones. Smartphones, however, do not seem to have been considered by many studies to collect thermal images of the built environment. A possible explanation would be that infrared cameras and software integrated into smartphones are not yet as performant as those included in mounted, handheld, or drone systems. Fig. 15 demonstrates that more than 85% of reviewed studies at the microscale or building-scale were conducted to detect defects of buildings or evaluate their thermal performance. The other 20% of studies were dedicated to analyzing mitigation strategies of the UHI effect, detecting defects on PV panels, observing techniques to renovate buildings, and analyzing urban heat fluxes. The latter application is the only one that can also be found in studies conducted at higher scales. It implies that studies performed at the microscale or buildingscale are more concerned by aspects that can affect the building energy efficiency. At the same time, those conducted at higher scales give more attention to phenomena impacting the outdoor environment.
The detection of defects appears to have been performed on residential buildings mainly and on historical ones a little. Fig. 16 also show that none of the reviewed studies considered defects that might appear on the envelope of office or commercial buildings. This gap can certainly be justified by the fact that most studies at the microscale or building-scale were conducted in Europe, where a considerable portion of the energy is consumed in residential buildings D'Agostino et al. [114]. However, it was reported by Berardi [115] that the energy demand in non-residential buildings increases in Europe, as well as in other regions of the globe. The fact that few observations were made on the energy performance of non-residential buildings can then become a considerable limitation in the building sector (see Tables 6-8).

Detection of defects on residential or historical buildings
In infrared thermography, defects of a building primarily refer to all imperfections on its envelope that can compromise its indoor thermal comfort and energy efficiency. Grinzato et al. [116] were among the first in listing all building defects that can be detected from an infrared camera at the micro-scale or building-scale. It includes plaster detachment on walls, insulation deficiencies, thermal bridges, and moisture. These defects were detected by Grinzato et al. [116] as anomalies in thermal images provided by the infrared camera.
Techniques to detect defects were originally studied on historical or old buildings' envelopes or structural elements. For example, Haralambopoulos and Paparsenos [117] tried to detect insulation deficiencies on the envelope of an old building located in Salonika (Greece). A similar method was used by Grinzato et al. [119,130], Al-Kassir et al. [122], Tavukçuoğlu et al. [124], and Kordatos et al. [131] to observe damages caused moisture on the façade of ancient buildings. Instead of considering the entire envelope of a building, Li et al. [118] studied defects caused by air gaps or moisture on ceramic tiles. Studies like Rosina et al. [120] and Edis et al. [138,141] also focused on damages caused by moisture on structural elements. Lerma et al. [139] determined

Table 6
Reviewed studies using thermal images to detect defects on residential or historical buildings at the microscale or building-scale.
Since the early 2000s, numerous studies have aimed at detecting locations on the envelope of a building where heat can escape from the indoor to the outdoor, that is, heat losses. One of the main source of heat losses are windows as demonstrated by Ocaña et al. [121], Vavilov [128], Barreira et al. [146], and Marino et al. [147]. Some windows can be difficult to observe from a mounted or handheld camera. For this reason, Martinez-De Dios and Ollero [125] used a drone to detect heat losses from the windows of a tall building. In addition to windows, Ribarić et al. [127] captured heat losses from heat exhausts using both thermal and Red-Green-Blue (RGB) images.
Some building defects are not necessarily detected at a specific instant. They are sometimes the result of degradation over time, which requires a multi-temporal analysis of thermal images for their detection. A multi-temporal analysis was performed by Lerma et al. [133] to better locate damages caused by moisture on the façade of a historical building. The same kind of analysis was performed by Fox et al. [142] and Bauer et al. [150] to detect cracks that might appear on residential buildings over time. A large and long experimental campaign was conducted by Barreira et al. [144] to assess all moisture-based defects that can appear on structural elements of the built environment. Fox et al. [145] reported various defects that can gradually appear outside or inside a building and detect by infrared thermography. Apart from imperfections that gradually appear over time, some degradation caused by a sudden event like debonds on an external wall can be seen from a multi-temporal analysis as proven by Lai et al. [143].
Although moisture appears to be the main cause of defects on the envelope of a building, it can also be provoked by other phenomena. For example, Paoletti et al. [136] studied damages resulting from an earthquake.
Instead of detecting defects appearing on the envelope of buildings, certain studies focused on damages caused by renovation or retrofitting. Among these studies, Avdelidis and Moropoulou [134] described several of these renovation methods applied on historical buildings, which includes surface cleaning, consolidation of stones, restoration of mortars, and examination of plaster mosaics. Another study is the one conducted by Hopper et al. [135], in which thermal bridges that might emerge after retrofitting are detected using infrared thermography.
Thermal bridges are among the most complex defects to detect from a thermal image. The reason is that it requires an assessment of the heat conduction through the envelope of a building as reported by Bianchi et al. [137] and O'Grady et al. [148]. Zalewski et al. [129], Taylor et al. [140], and O'Grady et al. [152] linked infrared thermography with computer simulation to analyze thermal bridges on the envelope of a building or specific structural elements. Thermal bridges can be quantified using an incidence factor as expressed by Baldinelli et al. [149]. Instead of using a single metric like the incidence factor, Tejedor et al. [153] detected thermal bridges from a 2D map of U-values assessed from an infrared camera. A 3D thermal model of a building was created by Rakha et al. [154] from a drone to detect thermal bridges on the envelope.

Evaluation of building thermal performance
The thermal performance of a building corresponds to its capability to maintain indoor conditions at a satisfactory level of thermal comfort Table 7 Reviewed studies using thermal images to evaluate building thermal performance at the microscale or building-scale.
Indeed, the U-value is typically obtained from thermal images of an opaque surface. It is relatively difficult to assess the U-value of transparent surfaces like the glass on windows. Despite this difficulty, Baldinelli and Bianchi [164], Maroy et al. [174], and Park et al. [193] attempted to infer the U-value of various types of glazing using an infrared camera and compared estimates with measurements collected by contact sensors.
Instead of using the U-value, other studies analyzed the thermal performance of buildings from 3D thermal models. As illustrated by Lagüela et al. [156], a 3D thermal model aims at capturing the thermal behavior of a building from all possible angles. In the literature, several techniques were explored to generate 3D thermal models. One of them consists of fusing and matching thermal images collected from different angles as defined by Lagüela et al. [157], González-Aguilera et al. [159], and Yang et al. [181]. Instead of fusing and matching a sequence of thermal images, Ham and Golparvar-Fard [160] and Wang et al. [163] generated a cloud of geolocalized points, whose surface temperature was captured by an infrared camera at different positions. Whether the 3D thermal model is obtained by fusion/matching or a cloud of points, Lagüela et al. [161] shows that it can be integrated into Building Information Modelling (BIM) to analyze the thermal Table 8 Reviewed studies using thermal images for other applications at the microscale or building-scale.

Author(s)
Year Country Infrared camera Model Support

Detection of faulty PV panels
Lee et al. [201] 2018 South Korea Not available Not available Drone Ismail et al. [202] 2019 United Arab Emirates FLIR Zenmuse XT Drone Et-taleby et al. [203] 2020 Morocco Not available Not available Drone Henry et al. [204] 2020 South Korea FLIR Vue Pro R Drone

Analysis of urban heat fluxes
Hoyano et al. [205] 1999 Japan Not available Not available Mounted Feng et al. [206] 2019 China FLIR Vue Pro 640R Drone Arjunan et al. [207] 2021 Singapore FLIR A300 Mounted performance of a building. After integrating the 3D thermal model into BIM, Lagüela et al. [166] assessed the U-value for each surface of the building. Ham and Golparvar-Fard [169] created an entire 3D map of U-values and integrated it into BIM. Instead of using BIM, Marshall et al. [179] used another building visualization model to fix the 3D map of U-values.
Recently, some efforts have been made to evaluate the thermal performance of a building using thermal images and artificial intelligence. Sen and Al-Habaibeh [186] could categorized different types of walls using an artificial neural network and the U-value calculated from thermal images. The U-value of different walls was also studied by Tejedor et al. [187] through time series analysis. An instance segmentation technique was used by Sadhukhan et al. [188] to estimate the U-value of various elements on a building, including doors, walls, and windows.

Analysis of strategies to mitigate the UHI effect
While the UHI effect is generally observed at the mesoscale or cityscale, strategies to mitigate its aggravation are usually studied at the microscale or building scale. The main reason is that it would be timeconsuming and expensive to set up an experiment on UHI mitigation strategies at the mesoscale or city scale.
Despite the emergency in finding solutions to mitigate the UHI effect, it was observed that a few reviewed studies were dedicated to their analysis using infrared thermography. Among the studies that used infrared thermography to analyze mitigation strategies of the UHI effect, Mastrapostoli et al. [195] and Cho et al. [200] could capture the effect and deterioration of cool roofs over time. An alternative to cool roofs is rooftop gardens, whose influence was studied by Monteiro et al. [196] using a sequence of thermal images collected at different positions. The UHI effect is not necessarily mitigated using strategies on the roof of buildings. Chui et al. [197] and Kolokotsa et al. [198], for instance, observed how certain materials enable to reduce the surface temperature of street pavements using infrared thermography. The surface temperature of the street pavement can also be reduced using shading devices as shown by Garcia-Nevado et al. [199].

Detection of faulty PV panels
Recently, various studies have tried to detect faulty PV panels using infrared cameras installed on drones. As for defects on the envelope buildings, Lee et al. [201], Ismail et al. [202], and Henry et al. [204] detect faulty PV panels from anomalies on thermal images. A more sophisticated method relying on machine learning was developed by Et-taleby et al. [203] to automatically detect faulty PV panels.

Analysis of urban heat fluxes
Among all applications of infrared thermography in the built environment, the analysis of urban heat fluxes is the only one that could be found at multiple scales. However, more studies can be found at higher scales than at the microscale or building scale. Among the few studies conducted at the microscale or building-scale, Hoyano et al. [205] analyzed the sensible heat emitted by a building over a typical day. Sensible heat fluxes were also studied by Feng et al. [206] from thermal images collected by a drone. Arjunan et al. [207] observed the operation of HVAC systems using infrared thermography; and, thus, indirectly assessed the anthropogenic heat that might be emitted from the use of air conditioning.

Multi-scale studies
Although many contributions have been made in infrared thermography at different scales of the built environment, a small number considered multiple scales in the same study. One of them is the study conducted by Gluch et al. [208] in the early 2000s. The objective was to compare thermal images obtained at the mesoscale with others collected at the microscale. A similar comparison was made by Golden and Kaloush [209] and Hartz et al. [210], but at the city-scale using a satellite and at the building-scale using a handheld infrared camera.
It might be abrupt to directly compare thermal images obtained at a very large scale with these collected at a smaller scale. For this reason, Yamazaki et al. [211] decided to study the UHI effect between the local-scale and microscale. Other studies like Kuo et al. [212] and Parlow et al. [213] preferred to consider thermal images obtained at the mesoscale and local scale. Bonafoni et al. [214] and Bonafoni and Tosi [215] developed a downscaling method to collect thermal images of the built environment from the city-scale to the building scale.

Research opportunities
During the review, several research opportunities were identified at different scales of the built environment. The next phase of research should explore the convergence of infrared radiation data with Internetof-Things (IoT), geospatial data, and other data sources found in the built environment. This foundation enables the use of infrared thermography for applications at the urban scale, such as energy modeling and the UHI effect. Although the UHI effect was observed using infrared thermography in numerous studies, in particular at the mesoscale, some improvements can still be made in the future.

Temporal data integration with Internet-of-Things (IoT) and other imaging systems
Infrared thermography images taken over a period of time results in the collection of surface temperature data with a temporal dimension. These data enable the extraction of behavior related to the dynamics of building envelopes, mechanical systems, and human behavior in buildings. Collection of the temporal dynamics from these systems empowers the convergence of data from other types of energy, indoor and outdoor environmental quality, and wearable measurements. Additionally, data from infrared data collection could be linked with other remote sensing systems such as visible light, broadband, and hyperspectral imaging to achieve high-level insights about numerous buildings simultaneously. An example of such a deployment is the Urban Observatory that was deployed in New York City, which was able to capture the energy consumption, lighting use, grid stability, and environmental conditions of hundreds of buildings on the Manhattan skyline at once [107]. This data fusion also enabled the evaluation of urban vegetative health, the ecological impacts of light pollution, and the technology adoptions habits of building occupants.

Geospatial data integration and digital twins
As large-scale data sourced from infrared thermography is essentially a form of geospatial data, the literature review exposed a notable gap of the lack of recognition of such data in the geospatial realm, its integration in GIS, and coupling it with other kinds of geoinformation, potentially uncovering new applications in the built environment. With digital twins increasingly supporting dynamic data and allowing accommodating diverse sources [216,217], a viable research direction would be to investigate the direct integration of latent thermal data in them, potentially facilitating new use cases, e.g., understanding urban vibrancy and thermal comfort [218,219]. For example, the standard CityJSON [220] enables extending urban digital twins with new types of information. A direction for future work would be researching an automated way to supplement the standard with static or dynamic information from thermal cameras and associate them with urban features that are already available in these datasets.
Another notable development at the urban scale and in the geospatial domain is the proliferation of street view imagery [221][222][223]. Since thermal cameras may be mobilized, a question that arises is whether we can develop a new research line that focuses on developing street-level thermal imagery, supplementing optical imagery, which has been the main focus of research so far [224].
As advances in computer vision provide means to process a large number of images and as these techniques are gaining momentum in urban studies [225][226][227][228], it might be worthwhile to use them to develop new mechanisms to process thermal imagery and reveal new applications.

Detailed and comparative analysis of urban heat fluxes at multiple scales
As discussed in Section 4, the analysis of urban heat fluxes is an application of infrared thermography that can certainly be used to explore the built environment at multiple scales. Nonetheless, it was observed that the level of detail with which urban heat fluxes were analyzed varies concerning the scale they were studied. At the mesoscale or city-scale, highly detailed observations of urban heat fluxes were made using thermal images and other remote data collected from satellites. At lower scales, only a few urban heat fluxes were considered by reviewed studies at the same time. Most of these studies essentially inferred the sensible and anthropogenic heat emitted by buildings from thermal images.
While urban heat fluxes were considered at different scales separately, none of the reviewed studies compared their estimates at multiple scales simultaneously. To perform this kind of comparison would also require the deployment of automatic weather stations at various scales. It would also be recommended to use and synchronize several infrared systems. Such a network of infrared systems and automatic weather stations have not been deployed over a city.
Therefore, there are two main research opportunities related to the analysis of urban heat fluxes at multiple scales. One possibility would be to perform a more detailed analysis of urban heat fluxes at lower scales than the mesoscale or city-scale by considering the latent heat emitted by vegetation or the anthropogenic heat released by traffic, for instance. Another opportunity would be to compare urban heat fluxes estimated from thermal images at multiple scales and evaluate their divergence.

Urban-scale building energy modeling
In recent years, there has been increasing interest in urban-scale building energy modeling (UBEM) because of its ability to simulate city-scale building energy performance to support sustainable development decision-making and urban planning. However, the credibility of UBEMs is often questionable due to a large number of assumptions in the modeling process [229]. Specifically, envelop thermal properties for UBEMs are often assumed using default or reference values. To this end, as found from this review, infrared thermography has been used to reduce the uncertainties in characterizing a building's envelope thermal properties. However, its application remained at the microscale involving determining a single building's thermal properties [169,179]. Therefore, investigations into using infrared technology at an urban scale to inform building thermal performance and reduce UBEM uncertainties would be a promising research direction. More specifically, it will be interesting to investigate the applicability of techniques and technologies from the local scales that can be used to scale up 3D thermal analysis that has been to date limited to the microscale.

Conclusion
In this review, several applications of infrared thermography in the built environment were presented at multiple scales. The review summarizes 197 contributions that were selected for their relevance and classified in accordance with several criteria, including the studied area, infrared system, scale, and application. Data analysis was conducted based on the classification and the chronology of contributions to detect the research gaps that could be addressed in the future. Apart from the research gaps or opportunities, the data analysis shows three main tendencies on applications of infrared thermography to explore the built environment at multiple scales.
Firstly, it is observed that the majority of reviewed studies used infrared thermography to evaluate the thermal performance of buildings, or detect their defects. These applications are often performed at the microscale or building-scale, which explains why a considerable portion of reviewed studies was conducted at this scale. However, a non-negligible part of reviewed studies was interested in other applications at the microscale or building-scale, including the analysis of UHI mitigation strategies, the detection of faulty PV panels, and the observation of urban heat fluxes. In the future, more applications of infrared thermography can certainly be found at the microscale or building-scale to understand the energy efficiency of buildings better. One of them could be creating building energy models from thermal images at the microscale or building scales. UBEMs could be generated using infrared thermography at multiple scales by extension of this application.
Secondly, the observation of the UHI effect is the most frequent application of infrared thermography at higher scales than the microscale or building-scale. This result is certainly justified by the extreme emergency to identify the consequences of intense urbanization, particularly during global warming and climate change. However, the magnitude of the UHI effect alone does not inform the scientific community on the causes of its aggravation nor on strategies to mitigate it. More detailed studies should be conducted on urban heat fluxes, especially at lower scales than the mesoscale, to better understand contributors and mitigators of the UHI effect.
Thirdly, it was pointed out that thermal images have been linked with a few other data sources to explore the built environment at multiple scales. So far, thermal images have essentially been linked with weather data to estimate urban heat fluxes at lower scales than the mesoscale or city-scale. The small interaction between thermal images and other data sources certainly limits the number of applications. In the future, this limitation could be overcome by integrating thermal images into an IoT and a digital twin platform. A linkage of data collected by an IoT and thermal images should better assess the building energy efficiency. If IoT data and thermal images were together included in a digital twin platform with geospatial data, the scientific community and practitioners would have better visualization tools to analyze the operation of a city and strategies to improve its sustainability.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.