1 Introduction

Scientific examination and accurate digital photographic recording of cultural heritage (CH) artworks are fundamental for their contextualization, documentation, and conservation. Non-invasive scientific analysis helps identify the materials used by artists and offers information about how the artwork was created without affecting its integrity. Further, it provides—quickly and effectively—information on the state of conservation of the artwork.

In this context, multispectral and hyperspectral imaging has emerged as popular methods for non-invasively investigating and documenting cultural heritage surfaces [1,2,3]. The methods refer to the collection of the spectrum of the reflected, transmitted, scattered, or emitted light for each pixel in the image of a scene. The collected data form the so-called spectral datacube, in which two dimensions represent the spatial extent of the scene and the third its spectral content. In heritage science, spectral imaging is employed to detect and study light diffusely reflected by an artistic surface [4,5,6,7] with the aim of revealing preparatory sketches and hidden details [8,9,10,11,12], retrieving accurate colour information and achieving pigment identification and mapping [13,14,15,16]. The method is easily applied to the study of polychrome surfaces of small- or medium-size objects in museum collections, as easel paintings, illuminated manuscripts or paper-based artefacts. More recently, it has been applied also outdoor on large mural surfaces [17,18,19] and archaeological sites, with the aid of avionic-based spectral cameras originally developed for earth-surface observations [20].

Spectral imaging can be implemented with many different approaches [21, 22]. When dealing with CH surfaces efficient macroscale imaging is typically achieved with two types of cameras: (i) a line-scanning imaging spectrometer (see, e.g. [9, 10, 13, 20]) and (ii) a filter-based imaging system (see, e.g. [8, 17, 23, 24]). The first approach produces the spectral datacube line-by-line by moving the object or the detector over one spatial coordinate: for each scanning step, the object surface is illuminated in a narrow strip and the scattered or emitted light is collected through a slit. A diffraction grating separates the various wavelengths dispersing them onto a two-dimensional detector. These types of spectral imaging systems are classified as hyperspectral because they are based on a continuous wavelengths separation, even though the pixelated detector discretizes the final spectrum into narrow spectral bands with a typical spectral resolution of ~ 2.5 nm in the visible range and ≤ 10 nm in the near-infrared and short-wave infrared region [4, 25]. The collection rate is rather fast, leading to an acquisition time for the entire spectral datacube in the order of a few minutes. The main drawback of this approach lies in the need of linearly moving either the object under analysis or the detection camera with the aid of a translating stage. This issue typically limits the application of such hyperspectral cameras to close-range applications and to the analysis of flat-field artistic surfaces.

The second approach relies on the so-called spectral scanning method. It constructs the spectral datacube as a stack of images, each one acquired by filtering the incoming light in a discrete set of spectral bands. As the spectrum is integrated in a few bands, this latter approach is referred to as multispectral imaging. The method typically employs a filter wheel or an electronically tuneable filter placed in front of a monochrome camera [2]. At present, the main technological challenge lies in the development of tuneable filters with high optical throughput and capable of scanning, at high spectral resolution, a wide spectral range. To solve this issue, Balas and co-authors [26] have recently developed a spectral scanning camera based on the use of a tuneable electro-optic filter that produces a spectral datacube covering the entire 370–1100 nm spectral range with a variable spectral resolution ranging from 6 to 12 nm in a few tens of seconds. In contrast to line scanning, the method has the advantage of being easily coupled to different optical systems in collection, allowing it to be used to study surfaces of variable sizes, from microscale [27, 28] to large surfaces [8, 23, 24, 29], and placed at variable distances [17].

A completely different approach, seldom employed in the CH field, is provided by Fourier transform hyperspectral imaging (FT-HSI). The method relies on interferometry and on the Fourier transform (FT) operation to retrieve spectral information. By means of an interferometer, the wavefront of the light coming from the object is split into two replicas, with a relative phase difference, which interfere on the detector. By acquiring images at different phase delays, this system ends up with a data cube in which each spatial point contains one interferogram. By means of the FT operation, this content can be converted to the spectral information according to the Wiener–Khinchin theorem [28]. The spectra that are retrieved by the FT operation are continuous, and for this reason, FT imaging is considered a hyperspectral technique.

FT-HSI has some advantages with respect to spatial and spectral scanning techniques. An extensive description of the FT method can be found in [30], while hereafter, only a synthesis of the benefits of the method is briefly reported.

Compared to point- or line-scanning techniques, FT-HSI (i) exhibits a higher signal-to-noise ratio (Fellget multiplex advantage [31]) thanks to the lack of the spatial aperture, which is needed by dispersive methods and (ii) does not require moving the imaging system with respect to the object under study resulting in greater flexibility in terms of field of view amplitude and image magnification.

Compared to the spectral scanning approach, FT-HSI (i) exhibits a higher light throughput due to the absence of spectral filters; (ii) allows the collection of the spectral information as a continuous function; (iii) displays a flexible spectral resolution since this parameter is set by the length of the collected interferogram, which can be changed according to the needs; and (iv) does not suffer for significant differences in transmission efficiency along the considered spectral range, an issue that is typical of liquid crystal tuneable filters [2].

Despite these advantages, the fact that both spectral and spatial information are acquired in parallel poses some challenges to FT-HSI. The first limitation is related to the parallel spatial acquisition (wide-field approach): it arises when imaging highly contrasted scenes, a problem which is also common in traditional photography. The integration time is the same for all the detector pixels and is chosen in order not to saturate the brightest points of the scene; hence, the signal-to-noise ratio of the darker regions is inevitably worsened. In contrast, spatial scanning approaches are immune to this issue, since the integration time can be adapted to the light intensity at each pixel within the field of view (FOV). The second drawback of FT-HSI emerges from the fact that each pixel collects the whole spectral information at once, and consequently, the measurement suffers from the uneven spectral responsivity of the measurement system throughout the considered spectral range. In particular, the spectral responsivity is proportional to the product between two parameters: the camera efficiency and the power spectral density of the illumination sources. Regarding the former, the quantum efficiency of CMOS and CCD detectors is typically high in the red/near-infrared region of the spectrum, but drops in the blue [32, 33]. On the other hand, the typical illumination sources used in CH reflectivity measurements are halogen lamps [2, 3, 9], which have higher emission in the near-infrared and weak power density in the blue region. These features unbalance the spectral responsivity in favour of the infrared components. While in the spectral scanning approach, the uneven spectral responsivity can be easily compensated by varying the exposure time for the different spectral bands, in FT-HSI a uniform spectral responsivity of the measurement system is needed to well reconstruct all spectral components.

In this paper, we describe a FT-HSI camera based on the TWINS ultra-stable common-path interferometer [34]. We discuss in detail how we have tackled the main limitations of the FT-HSI approach (i) by implementing the high dynamic range (HDR) method to virtually increase the camera dynamic range and (ii) by designing an illumination system with a higher spectral emission in the blue region using a low-cost equipment. Finally, we demonstrate the effectiveness of the TWINS-based FT-HSI camera to the study of an Indian painting from the “Album of Nasir al-din Shah” where pigment mapping was achieved with the aid of multivariate statistical analysis.

2 Materials and methods

2.1 The TWINS FT-HSI camera

Recently, Perri and co-authors have presented a high-performance FT-HSI camera particularly suitable for in situ experiments thanks to its inherent compactness and robustness [34]. The FT-HSI camera employs the TWINS interferometer, where phase-delay scanning is provided by a birefringent block with variable thickness. To achieve the imaging capabilities, the TWINS interferometer is placed in front of a monochrome imaging camera equipped with a camera lens.

In the present research, the TWINS-interferometer is made of α-BBO (transparency range from 200 nm to 3.5 µm); it is coupled to a monochrome CMOS silicon-based camera (6.78 mm × 5.43 mm in sensor size, 1280 × 1024 pixels) with 12-bits dynamic range and spectral sensitivity from 400 to 1100 nm. The camera is equipped with a 25 mm camera lens with a maximum aperture f/1.4. The resulting angular FOV of the FT-HSI system is ~ 16 degrees, and the working distance ranges from 1 m to ∞. Note that the addition of the TWINS interferometer does not alter the spatial resolution of the camera.

For diffuse reflectance measurements, the illumination sources, which will be detailed in a following section, are positioned at a minimum distance of 1 m from the surface under analysis and placed off-axis at an angle of about 45° with respect to the surface normal to avoid specular reflection to be collected by the camera. By varying the relative delay of the two interfering wavefronts, a set of images that contains, for each pixel, an interferogram encompassing many optical cycles is obtained. A standard experiment consists of acquiring 200 images varying the relative delay in steps of about 0.6 fs, for an overall phase delay of 57 optical cycles @ 600 nm. This time range corresponds to a spectral resolution of about 4 nm at 600 nm. Note that during this standard experiment, the focus and the aperture stop of the camera lens and the exposure time of the camera sensor are kept fixed.

For a correct measurement of the reflectance spectrum, two reference items are used: an in-scene standard white Labsphere Spectralon®, which provides an absolute standard with calibrated diffuse reflectance all over the visible spectrum, and a highly diffusive White Balance card (X-Rite M50101 ColorChecker) to perform flat-field correction. The Spectralon gives a > 95% diffuse reflectance that is reproducible and independent of viewing angle calibrated from 250 to 2500 nm wavelengths.

By Fourier transforming the recorded temporal datacube, we obtain the hyperspectral datacube, which we analysed with in-house software created within the MATLAB environment. An example of instrumentation setup for the acquisition of a X-Rite® ColorChecker Classic chart is shown in Fig. 1.

Fig. 1
figure 1

Experimental instrumentation and setting: arrangement of the illumination and detection instruments in front of a X-Rite® ColorChecker Classic chart. In this case, the in-scene standard reference sample is provided by the white sector of the ColorChecker, whose reflectance spectrum was measured with a benchtop spectrometer

Full size image

2.2 Materials employed for the characterization of the TWINS FT-HSI camera performance

To perform reflectivity experiments, we tested two sources of illumination: a 150 W halogen lamp (Dedolight DLH4 Spotlight) and a slide projector (KODAK EKTAGRAPHIC III Slide Projector). The latter mounts a dichroic halogen lamp (Osram 93,520 FHS 300 W 82 V GX5.3) in which the dichroic parabola removes a large fraction of the infrared light, along with an equalization optical filter (HOYA LB140). The spectral irradiance of the illumination sources has been estimated with a commercial spectrometer (TMCCD C10083CA-2100, Hamamatsu Photonics).

To check the correct reconstruction of reflectivity spectra in the visible range, we employed a set of coloured Labsphere Spectralon®. These are diffusive, tabulated colour standards typically used for calibration in the 360–830 nm range.

To evaluate the advantages provided by the HDR acquisition method, we employed the ColorChecker Classic chart, also known by its original name “Macbeth ColorChecker”. This is a rectangular card measuring 20.6 cm × 28.9 cm that includes 24 squared 5.1 cm × 5.1 cm coloured patches arranged in a 4 × 6 grid. The colour coordinates of the matte-coloured patches are tabulated to deliver repeatable colour results and accurately set the colour balance during an acquisition. Six of the coloured panels form a uniform grey scale with a reflectivity that scales from patch to patch. These grey patches were used for the HDR characterization.

2.3 HDR data acquisition method

The HDR method was implemented by acquiring a series of datacubes with varying exposure time. The number of acquisitions is N, which also designate the HDR order in the following. For each datacube acquisition, only the pixels with intensity above a given threshold are stored. Here, we set the threshold at the half dynamics: once a pixel is included in an acquisition, it is not inserted in other acquisitions. In this way, the pixels of the captured FOV are divided into different groups, each characterized by its own exposure time. Starting from the first acquisition with exposure time Δt, set so as not to saturate any pixel in the captured FOV, each of the following acquisitions is taken by doubling the exposure time. Thus, the i-th acquisition is taken with exposure time 2i−1·Δt. The final datacube is obtained by merging the pixels groups; each is multiplied by a scale factor that is the reciprocal of the exposure time.

Figure 2 shows the partitioning of pixels into the different exposure time-groups for HDR = 5 following this procedure. With our 12-bit sensor, each pixel group from the 1st to the 4th is distributed over half of the camera bit depth (2048 levels), and the last one (the 5th group) is distributed over the entire camera depth (4096 levels).

Fig. 2
figure 2

Example of partitioning of pixels into different exposure time groups with HDR = 5 and threshold set at half the dynamic. The table refers to a 12-bit sampling. The y-axis represents the counts without the HDR approach, while the bars of each group indicate the dynamic expansion due to the doubling of the selected exposure

Full size image

2.4 Multivariate data analysis methods for pigment mapping

When analysing the hyperspectral reflectance datacube of a polychromatic surface, the main expectation is to map and identify the pigments employed by the artist. However, the paints used in paintings seldom consist of pure pigments, where only one pigment is present in a given area of the artwork. More often, pigments are mixed and layered, and the resulting spectral signature is a combination of different pigments. To identify and group them, various unmixing methods must be applied to the hypercube: these algorithms, originally developed by the remote sensing community [14, 35, 36], enable decomposing each spectral signature into a set of pure components, referred to as spectral endmembers. Figure 3 shows a scheme of the two main spectral unmixing approaches that traditionally are used in the CH community and that were also exploited in this work.

Fig. 3
figure 3

Scheme of the procedures traditionally used for the analysis of an HSI dataset to obtain a composite image useful for pigment mapping. The unsupervised and the supervised methods are depicted where the latter is commonly aided by some guiding tools for the manual selection of spectral endmembers

Full size image

The unmixing process can be either unsupervised or supervised. In the first case, the algorithm automatically extracts the endmembers: it searches the minimum number of endmembers that reconstructs all the spectra of the dataset, with a given tolerance. Typical examples of unsupervised methods are the N-FINDR [37] or the vertex component analysis (VCA) [38]. These approaches are extremely powerful and unbiased, but might generate a large number of endmembers, some of which are redundant or difficult to interpret and thus require an experienced user to select the most appropriate endmembers. Other unsupervised unmixing methods rely on clusterization as, for example, K-means clustering [39].

Alternatively, the dataset can be analysed using a supervised approach, where endmembers are chosen manually with the help of guiding tools derived from the collected hyperspectral datacube itself. The most traditional tool is the RGB map, in which the spectral range between 400 and 700 nm is separated into the classic red, green and blue channels, which are then combined to create a photographic representation of the sample. Other colour combinations can be also used as guiding tools for endmember selection: one of the most widely used tools is the infrared false-colour (IRFC) image, which is inspired by the photographic acquisitions with the now obsolete Kodak EKTACHROME Infrared films. Starting from the hyperspectral datacube, the IRFC is created by associating a synthetic red colour to the infrared part of the spectrum (780–950 nm), green to the common red portion of it (630–780 nm), and blue to the green wavelengths (480–630 nm).

A further approach to select endmembers exploits the derivative method, which is a powerful tool to enhance tiny spectral differences among different material mixtures [40,41,42,43,44]. Spectral features are in fact more easily observable in the first-derivative spectrum, where a peak corresponds to an inflexion in the reflectance spectrum, i.e. a rapid change in reflectivity properties. It should be noted that a disadvantage of using the derivative method is the intrinsic increase in noise, which is commonly addressed by smoothing the spectral dataset through an appropriate pre-processing [45, 46]. In this context, the application of the first-derivative method to datacubes using the FT approach is less problematic since the FT operation includes an intrinsic spectral low-pass filter due to the fact that the interferogram is collected over a limited delay interval.

Once the endmembers have been identified, they are compared with all the other spectra present in the dataset using classification methods. For this purpose, the most common classification approach in CH is the spectral angle mapping (SAM) method [12, 47], where spectra are treated as vectors and the spectral similarity with a reference spectrum of interest is calculated as the angle they form with a reference spectrum. Pixels with similar emission spectra are grouped and displayed in a greyscale map (referred to as SAM map), ranging from white (value of the smallest angle and thus closest similarity) to black (value of the largest angle and thus minimal similarity). The SAM maps related to all spectral endmembers can be combined in a false-colour representation to obtain a composite image that provides information about the spatial distribution of alike materials. Note that, since it relies on the evaluation of the angular distance between spectra, the method compares spectral shapes, regardless of their amplitude.

Finally, the comparison of the endmembers with a library of reflectance spectra of known pigments [48] helps to identify the pigments used in the painting. Pigment assignment can be further assisted by complementary spectroscopy techniques as X-ray fluorescence and photoluminescence spectroscopy [9, 10, 12, 13, 15].

3 Characterization of the TWINS FT-HSI camera performance

3.1 Assessment of the most suitable illumination

As mentioned previously, the FT-HSI approach is characterized by the simultaneous acquisition of the entire spectrum: this calls for an equalized spectral responsivity. To meet this condition, we tested two different illumination sources: the 150 W Dedolight halogen lamp and the Kodak slide projector. The two sources, positioned at 45° with respect to the sample-camera axis, produce an irradiance of 8 W/m2 (equivalent to an illuminance of 1000 lm/m2) and 7.5 W/m2 (equivalent to an illuminance of 900 lm/m2), respectively.

Recall here that the overall spectral responsivity of an HSI system employed for reflectivity measurements depends on the spectral power density of the illumination source and on the efficiency of the collection device (the camera), which in our case consists of the CMOS detector, the camera lens and all the optical elements of the TWINS interferometer. Figure 4a shows the spectra of the illumination sources measured with a calibrated commercial spectrometer (TMCCD C10083CA-2100, Hamamatsu Photonics). It is evident that the light projector has a higher spectral contribution in the visible range (especially in the blue region) than the halogen lamp. As the spectral efficiency of our camera (given by the one of the CMOS detectors, of the camera lens and of all the optical elements of the TWINS interferometer) decreases in both infrared and blue region of the spectrum, the combination of both halogen lamp and projector allows us to achieve a more uniform overall spectral responsivity of the FT-HSI system when employed for diffuse reflectance measurements.

Fig. 4
figure 4

a Halogen lamp and projector illumination spectra, together with the FT-HSI camera efficiency (normalized to its peak value). b RGB image of the coloured Spectralon® set obtained from the reflectivity spectral cube of the measurement with the combined illumination of the halogen lamp and the projector. c Spectral standard deviation of the reflectivity of the white Spectralon® for the three different illumination methods. d–f Coloured Spectralon® set reflectivity spectra. Solid lines: measured spectra, obtained with the three illuminations. Each line is the average spectrum over the corresponding Spectralon® region. Dashed lines: the calibrated reflectivity spectra provided by NIST

Full size image

To assess the effectiveness in using this combined illumination, we have performed 3 tests on the coloured Spectralon® set using different illuminations: one with the halogen lamp, one with the slide projector and one combining the two illuminations. For these measurements, we positioned the lamps with the aim of balancing the halogen lamp and projector irradiances on the target. The integration time was chosen to fill the entire dynamic of the camera (20 ms for measurements with each individual lamp and 10 ms for experiments with the two illuminations together).

The results are shown in Fig. 4. The measurement obtained with the light projector allows a better reconstruction of the reflectance spectra in the 400–500 nm region than those obtained with the halogen lamp. This is evident when comparing the measured reflectivity spectra of the coloured Spectralon® to their tabulated reference (Fig. 4d–f). The result is even clearer when considering the spectral standard deviation (Fig. 4c), which is more than 2 times lower with respect to the measurement with the halogen lamp alone. On the other hand, the halogen lamp enables a better reconstruction of the spectra and a lower standard deviation in the 800–1000 nm region. The measurement with the halogen lamp and the slide projector together is an effective compromise in the overall 400–1000 nm spectral region.

An example of the effect of the test illumination during a hyperspectral acquisition of a model painting is presented in Supplementary Figure S1.

3.2 Assessment of the effectiveness of the HDR approach

A high-fidelity image reproduction may require a dynamic range larger than that provided by the camera; we will extend the dynamic range by the HDR approach.

To evaluate the effectiveness of the HDR approach for the FT-HSI acquisitions, we measured the reflectivity of the greyscale panels of the X-Rite® ColorChecker Classic chart (see the top panel of Fig. 5a) by employing the HDR approach. A general description of the chart is given in Sect. 2.2; in more detail, the reflectivity of grey panels—nominally uniform in the 400–1000 nm spectral range—is expected to be 0.910, 0.584, 0.357, 0.190, 0.088 and 0.031. We illuminated the target by combining the halogen lamp and the slide projector.

Fig. 5
figure 5

taken from X-Rite® ColorChecker Classic chart. It is obtained from our HDR = 5 acquisition by integrating the reflectivity spectra over the whole spectral range. Bottom: histograms of the spectral average reflectivity in the 6 panels for different HDR values (HDR 1 = no HDR) plotted in logarithmic scale; the reflectivity value is averaged over the detection spectral range. b Average reflectivity spectra of panel #1 for different HDR values; inset: expanded vertical scale around the average reflectivity, indicated by a dashed line in the main panel. c Standard deviation of the reflectivity spectra shown in panel (b) evaluated for pixels in region #1

a Top: black and white image of the grey scale panels (labelled from #1 to #6)

Full size image

As shown in Fig. 5, increasing the HDR order improves the reconstruction of the spectrum of the darkest patches. Further, the use of the HDR approach results in a lower spectral standard deviation of the reconstructed spectra, especially in the blue and in the infrared spectral regions, where the overall responsivity of the FT-HSI camera is low (see Fig. 4a).

The bottom panel of Fig. 5a shows the histograms of the average reflectivity for the 6 grey panels measured with HDR orders ranging from 1 (no-HDR, i.e. single exposure) to 5. The histogram, plotted on a logarithmic scale, shows nearly equally spaced peaks, which is consistent with the fact that the relative reflectivity reduces—from bright to dark patches—by factors of 1.56, 1.63, 1.882.16 and 2.83, respectively. In our measurements, as the HDR order increases, the peaks related to the darker grey panels (#1–#3 in Fig. 5a) are shifted towards lower average reflectivity values and their dispersion decreases. This is due to the optimised spectral characterization on the blue and infrared edge of the spectra (see panel (b)), and the reduction of the pixel-to-pixel fluctuation of the spectra, as evidenced by the standard deviation illustrated in panel (c). This demonstrates the effectiveness of the HDR approach. Indeed, increasing the effective dynamic range of the camera allows us to better reconstruct the reflectance, counteracting the noise of the system.

An example of the improvement obtained using the HDR approach on a FT-HSI measurement of a model painting is reported in Supplementary Figure S1.

4 In situ study of a painting for achieving pigment mapping

To showcase the capability of the TWINS FT-HSI camera to be used in in situ studies in the cultural heritage sector, we performed hyperspectral acquisitions on a painting of the Musée du Louvre, Paris, at the C2RMF as part of an ongoing collaboration. The selected painting is named “Portrait of Khwajah Abu Al-Hasan", a folio from the Indian “Album of Nasir al-din Shah” attributed to Mir Hashim and produced in the first half of the XVII century (Musée du Louvre, Département des Arts de l'Islam, Paris, Inv. OA7163—https://collections.louvre.fr/en/ark:/53355/cl010329620). The purpose of the study was to map the distribution of pigments present in the work of art. The painting is 22.2 cm wide and 33.4 cm high, but the imaged FOV is larger in order to accommodate also the reference items.

We performed hyperspectral imaging implementing the HDR-5 approach, while the illumination was provided by two 150 W halogen lamps since the slide projector was not available at the time of in situ analysis. The total irradiance on the painting was 16 W/m2 corresponding to 1200 lm/m2. The acquisition parameters of the TWINS camera were the standard ones mentioned in Sect. 2.1, which led us to a total imaging time of only 4 min with HDR-5. Our final pixel dimension is 0.4 µm/px.

The dataset was analysed with both unsupervised methods as well as supervised procedures while being guided with the tools mentioned in Sect. 2.4. While the results of the unsupervised analysis are briefly presented in Supplementary Figure S2, in the following, we detail the outcomes of the manual identification of the main reference spectra (spectral endmembers).

We first generated the RGB representation of the painting, shown in Fig. 6a, and the infrared false-colour (IRFC) image, shown in Fig. 6b. By comparing panels (a) and (b), we infer that some areas painted with pigments of similar colour display a different IRFC tint, denoting that they behave differently, mainly in the infrared portion of the spectrum, and thus that they are made of different pigment mixtures. To us, the leaves of the flowers around the portrait give the most prominent example of varied use of different pigments to create a similar green colour in the RGB image. On this basis, we selected three areas on different leaves showing dissimilar hues in the IRFC, as shown in Fig. 7a and b. The corresponding reflectivity spectra are shown in Fig. 7c. The SAM maps, displayed in Supplementary Figure S3, were then converted into binary images by setting 5°, 18°, and 15° as a threshold for the spectral angle for the blue, green and red image, respectively, and combined into a composite image (Fig. 7d). The latter image displays the distribution of the materials of interest and demonstrates the rich use of different shades of green to paint the decorative leaves in the painting.

Fig. 6
figure 6

RGB a and IRFC b representations of the painting. The RGB image was created by associating to the 400–500 nm band a synthetic blue colour, to the 500–600 nm a green colour and to the 600–700 nm band a red colour. The IRFC image was generated by associating a synthetic red colour to the infrared part of the spectrum (780–950 nm), green to the common red portion (630–780 nm) of it and blue to the green wavelengths (480–630 nm)

Full size image
Fig. 7
figure 7

Use of the RGB and IRFC comparison for the determination of the endmembers. a and b show a zoom of the RGB and IRFC image, respectively, with selected areas of interest highlighted with coloured circles. c Reflectivity spectra extracted from the selected areas of interest. d Composite map of the SAMs generated using the reflectivity spectra in (c) as endmembers and setting an angular threshold of 5°, 18°, and 15° for the blue, green and red channel, respectively. The colours correspond to those of the circles in panel (a) and (b)

Full size image

As a second tool for guiding the endmembers selection, we applied the first-derivative approach to the hyperspectral dataset to enhance small differences between similar spectra. As an example, we used this approach to distinguish between the red paints used for some flowers. Figure 8a depicts an overlay of two derivative maps (shown in Supplementary Figure S4) where the green colour indicates a peak in the first derivative of the spectrum around 565 nm, while the magenta colour a peak around 615 nm. Regions where two red pigments are characterised by reflectance spectra with different inflexion points become clearly evident, whereas this was not obvious from the observation of neither the RGB (Fig. 8b) nor the IRFC image. The reflectivity spectra of the corresponding region of interests (ROIs) are shown in Fig. 8c together with their first derivative. Using these spectra as endmembers, we created two new SAMs setting 6° as angular thresholds (Supplementary Figure S5) and combined them into the composite image shown in Fig. 8d.

Fig. 8
figure 8

Use of the derivative method for the determination of the endmembers. a Overlay of two derivative maps, where the green colour indicates a main peak in the first derivative of the reflectivity between 550 and 580 nm, while the magenta colour denotes a main peak in the first derivative between 610 and 630 nm. Through this map, it is possible to note which areas have pigments with an edge transition at specific wavelength. b Zoomed in region from the RGB image of the paint with circles around areas where the derivative maps highlighted pigments with different properties. c Reflectivity spectra (solid line) and correspondent first derivative spectra (dashed line) relative to the circled areas in (b). The same colour corresponds to the same area. d Composite map obtained from the SAM generated from the reflectivity spectra in (c)

Full size image

Finally, we chose other endmembers by inspecting the RGB image. We selected the ROIs of similar colours considered the most relevant for describing the entire painting. All the selected ROIs and the corresponding endmember spectra are reported in Fig. 9a and b. These endmembers are used to create the final SAMs, presented in Supplementary Figure S6. Each SAM is colour-coded with an arbitrary synthetic colour and is used to generate the composite image in Fig. 9c. Note that black colour signifies unassigned pixels or, as in the case of the background, unpainted paper. As can be observed, the method allows for an almost full segmentation of the pigments present in the painting.

Fig. 9
figure 9

Selection based on visual inspection of the remaining endmembers for the creation of the composite image for pigment mapping. a RGB image showing the selected areas of interest obtained via the previously described methods plus the visual inspection of the RGB image itself. b Endmembers spectra corresponding to the selected areas of interest. c Composite image of all the SAMs generated from the reference endmembers. The colours are all consistent with the selected areas in (a). Black colour means that no similarity was found

Full size image

5 Conclusions and future perspectives

In this work, we have presented the application of the high-performance Fourier transform hyperspectral camera (FT-HSI) based on the ultra-stable common-path interferometer TWINS. We discussed the pros and cons of our method, highlighting the disadvantages of the parallel approach in both the spectral and spatial domain. We have indeed suggested two viable solutions to these drawbacks, involving the simultaneous use of lamps with different emissions to spectrally balance the illumination of the painting, and the HDR data acquisition method to increase the signal-to-noise ratio from the darker regions in the FOV.

Note that the intent of our approach is to perform in situ experiments, such as in museums and archaeological sites, where the best available light sources might not be fully equalized. However, we demonstrate that even using a non-optimised illuminant, the robustness of our experimental method, which relies on the FT approach and the use of an absolute reference, ultimately provides a good final result. In fact, the FT produces an intrinsic low-noise dataset, while the use of reference samples leads to a differential measurement that is highly tolerant against the uneven responsivity of the detector, provided that linearity is preserved.

These methodologies were indeed applied to in situ studies performed primarily outside our laboratory. This was possible also thanks to the compactness and robustness of our hyperspectral camera, which was designed to be easily transported (in the present case study, it was carried in a backpack) and used outside the laboratory, with an acquisition time and light dose compliant with best practices in conservation science.

Besides, the richness of the dataset allowed us to perform both unsupervised and supervised analysis for endmembers selection, with the latter analysis driven by the immediacy of the RGB and IRFC images and the derivative method. In both cases, SAM was then applied to generate similarity maps for the purpose of visualizing the pigment distribution. The application of multivariate statistical analysis is made possible by the high quality of our hyperspectral dataset and is of fundamental importance for the investigation of artworks; in fact, based on the similarity maps, one can be guided in the identification of the main pigments and speculate on their nature. Moreover, the similarity maps can guide the application of other analytical techniques such as XRF or Raman point spectroscopy for reliable identification of materials.

Regarding the hyperspectral camera itself, an important upgrade could be to couple it with a macro-objective to be able to explore the complexity of detailed paintings or even to evaluate the presence of heterogeneous painting areas where pigments granules can be present. This would enhance the ability to better distinguish raw materials and finely tune the analysis through the endmembers selection. A further advancement could be the extension of the detection range beyond the near-IR to the SWIR region through the choice of suitable detectors and of birefringent materials for the TWINS. In this way, it would also be possible to perform studies on underlayers and preparatory drawings, not to mention the opportunity to access some of the vibrational lines of pigments, which would increase our ability to identify pigments.