Analysis of Two-Color Infrared Polarization Imaging Characteristics for Target Detection and Recognition

Abstract

Infrared polarization imaging has demonstrated significant advantages in target detection and recognition in natural environments. While there have been numerous research achievements on medium-wave/long-wave infrared polarization in a single band, there is a lack of related conclusions on the characteristics of wide spectral two-color infrared polarization imaging. To address this gap, this article employs a theoretical model in infrared polarization imaging to investigate the influence of temperature, incident angle, and refractive index on polarization degree and polarization angle. These findings have been substantiated through discussions on experimental results. We conducted a comparative analysis between intensity imaging and polarization imaging within the framework of traditional single-band infrared imaging. We then delineated the benefits of dual-color infrared polarization imaging. Finally, we harnessed image registration fusion algorithms to amalgamate multiple-band images. Our findings demonstrate that this fusion technique not only enables penetration through water mist but also enhances target visibility, providing theoretical support for all-weather target detection and identification.

1. Introduction

When electromagnetic radiation is reflected and emitted by targets on the Earth’s surface and in the atmosphere, their own properties and the basic laws of optics give rise to polarization characteristics. Different objects or states (such as refractive index, roughness, water content, physical and chemical characteristics, etc.) exhibit different polarization states, which are closely related to the wavelength and form a polarization spectrum [1,2,3,4]. Infrared polarization imaging technology combines the advantages of infrared imaging and polarization imaging, utilizing not only the intensity information from target scene radiation and reflection but also the polarization information from target scene radiation and reflection. This technology enhances the detection and identification capabilities of imaging systems in complex backgrounds and holds great potential for further development [5,6,7,8,9,10,11,12].

In 2020, Yang M. et al. developed a polarization analyzer for medium-wave infrared cameras, enabling the acquisition of degree of polarization and polarization angle images at a frame frequency of 45 fps. This technology is particularly suitable for capturing the infrared polarization characteristics [13] of objects in dynamic scenes. In 2022, Shi et al. proposed a Stokes analytical model for rough surface infrared polarization (Dop) using a shadow function based on the bidirectional reflection distribution function of surface microelements (BRDF). Experimental results confirmed the consistency between the infrared polarization characteristics of the target and the theoretical analysis results [14]. In the same year, Liu et al. introduced a fusion method based on multi-decomposition latent low-rank representation (LatLRR) and combined it with a double simplified pulse-coupled neural network (D-SPCNN) to enhance detail layer fusion. This approach effectively preserves image details and achieves superior performance [15]. In 2014, Zhang et al. conducted a detailed study on the long-wave infrared polarization characteristics of various metal targets, including aluminum plates, iron plates, and aluminum plates coated with black paint or yellow-green camouflage. They analyzed and investigated the variation patterns of long-wave infrared polarization and polarization angle with respect to measured temperature [16]. In 2021, Li et al. proposed a novel road detection and tracking scheme that combines the differences in polarization characteristics, intensity, and time information in the long-wave infrared spectrum and background. They also created a manually annotated focal plane segmentation long-wave infrared road scene dataset, known as the LWIR DoFP road scene dataset (LDDRS). Experimental results demonstrated that this method outperformed two state-of-the-art real-time semantic segmentation networks, FANet-34 and SwiftNet, with improvements of 1.4% and 2.1% in terms of IoU, respectively [17]. In recent years, significant advancements have been made in infrared polarization detection, particularly in the areas of lens technology, detector design, and polarizing devices. However, due to limitations in the research and development of broadband infrared detectors, there is currently little research on the imaging characteristics of dual-color infrared and polarization for medium wave (3–5 microns) and long wave (8–14 microns) integrated optical paths [18].

Currently, there is a wealth of research on medium/long wave infrared polarization in a single band, but there is a lack of relevant conclusions regarding the characteristics of two-color infrared polarization imaging. Building upon the theory of traditional infrared polarization imaging modeling and experimentation, this study explores the characteristics of double-color infrared polarization imaging. A comparative analysis is conducted between traditional infrared intensity imaging and polarization imaging. Furthermore, we utilized an improved SIFT (scale-invariant feature transform) algorithm for image registration and a twin convolutional neural network image fusion algorithm based on similarity learning. The effectiveness of image fusion using multiple spectra after registration is verified, demonstrating its ability to penetrate water mist and enhance target visibility. These findings provide theoretical support for all-weather photoelectric detection and recognition technology.

2. Theoretical Model

The infrared polarization of ground objects is influenced by various factors, including their intrinsic properties, external environmental conditions, and detector conditions [19]. The infrared radiation that affects the polarization characteristics of the detected target consists of two components: one is emitted by the target itself, and the other is reflected from the target in response to environmental radiation. These components can be analyzed using Kirchhoff’s radiation theory and Planck’s blackbody radiation law [20,21]. According to Fresnel’s law of reflection, the reflected/spontaneous radiation from the target exhibits different polarization characteristics, which are influenced by factors such as incident angle, detection angle, and surface physicochemical properties [22].

To quantitatively analyze the infrared polarization characteristics of objects, it is necessary to develop a modeling approach. In this modeling approach, idealized assumptions are made: it is assumed that the infrared radiation emitted by the target and its surroundings follows Planck’s blackbody radiation law; the target surface is assumed to be perfectly smooth, and the infrared reflection and thermal radiation at the interface adhere to Fresnel’s law and Kirchhoff’s law; the ambient radiation is assumed to be natural light, and the radiation reflected into the detector is described as a point light source; the detector and polaroid responses are idealized without considering the influence of the air medium. The simplified analysis model of infrared polarization characteristics is illustrated in Figure 1.

According to Planck’s blackbody radiation law, the radiation source M_λb:

$$M_{\lambda b}=\frac{2\pi hc}{{\lambda }^5}\frac{1}{\exp (\frac{hc}{\lambda kT})-1}=\frac{c_1}{{\lambda }^5}\frac{1}{e^{c_2/\lambda T}-1}$$

(1)

Then, the radiation amount I of a radiation source with a temperature of T is in a certain infrared band (T):

$$I_b(T)={\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}{M}_{\lambda b}}\cdot d\lambda ={\displaystyle {\int }_{{\lambda }_1}^{{\lambda }_2}\frac{c_1}{{\lambda }^5}}{(e^{\frac{c_2}{\lambda T}}-1)}^{-1}d\lambda$$

(2)

where h is the Planck constant, $h=\left(6.626176\pm 0.000036\right)\times 10^{-34}\left(\mathrm{W}\cdot {\mathrm{s}}^2\right)$; k is the Boltzmann constant, $k=(1.380662\pm 0.000044)\times 10^{-23}(\mathrm{W}\cdot \mathrm{s}\cdot {\mathrm{K}}^{-1})$; c is the speed of light, $c=(2.99792458\pm 0.000000012)\times 10^8(\mathrm{m}\cdot {\mathrm{s}}^{-1})$; c₁ is the first radiation constant, $c_1=3.741832\times 10^8(\mathrm{W}\cdot \mathrm{c}{\mathrm{m}}^{-2}\cdot \mathrm{\mu }{\mathrm{m}}^4)$; c₂ is the second radiation constant, $c_2=1.438786\times 10^4(\mathrm{\mu }\mathrm{m}\cdot \mathrm{K})$; λ is the wavelength (μm), and T is the absolute temperature (K).

The thermal radiation emitted by an ideal blackbody radiation source is characterized by natural light, which can be decomposed into two mutually perpendicular polarization directions:

$$I_p(T)=I_s(T)=\frac{1}{2}{I}_b(T)$$

(3)

where I_p(T) is the amount of radiation in the horizontal direction, and I_s(T) is the amount of radiation in the vertical direction.

According to Fresnel’s reflection law, the expression of reflectivity is as follows:

$$R_p({\theta }_1)={\left|\frac{n_2\cos {\theta }_1-n_1\cos {\theta }_2}{n_2\cos {\theta }_1+n_1\cos {\theta }_2}\right|}^2$$

(4)

$$R_s({\theta }_1)={\left|\frac{n_1\cos {\theta }_1-n_2\cos {\theta }_2}{n_1\cos {\theta }_1+n_2\cos {\theta }_2}\right|}^2$$

(5)

where R_p is the reflectance in the horizontal direction, and R_s is the reflectance in the vertical direction. θ₁ represents the incidence angle, and θ₂ represents the refractive angle. n₁ represents the air refractive index, and n₂ represents the target refractive index.

Assuming no infrared radiation passes through the target, according to Kirchhoff’s law, the emission rate of the target is equal to its absorption rate when the target is in thermal equilibrium:

$${\epsilon }_p({\theta }_1)={\alpha }_p({\theta }_1)=1-R_p({\theta }_1)$$

(6)

$${\epsilon }_s({\theta }_1)={\alpha }_s({\theta }_1)=1-R_s({\theta }_1)$$

(7)

where ε represents emissivity, and α represents absorption rate.

The infrared radiation emitted by the target comprises two components: infrared reflection and its inherent thermal radiation.

$$I_p(T_o,T_e,{\theta }_1)=\frac{1}{2}{I}_b(T_0){\epsilon }_p({\theta }_1)+\frac{1}{2}{I}_b(T_e)R_p({\theta }_1)$$

(8)

$$I_s(T_o,T_e,{\theta }_1)=\frac{1}{2}{I}_b(T_0){\epsilon }_s({\theta }_1)+\frac{1}{2}{I}_b(T_e)R_s({\theta }_1)$$

(9)

The calculation of the degree of polarization (Dop) and angle of polarization (Aop) is defined as follows [23]:

$$Dop=\left|\frac{I_p(T_o,T_e,{\theta }_1)-I_s(T_O,T_e,{\theta }_1)}{I_p(T_o,T_e,{\theta }_1)+I_s(T_O,T_e,{\theta }_1)}\right|$$

(10)

$$Aop=\arctan (\frac{E_s(T_o,T_e,{\theta }_1)}{E_p(T_o,T_e,{\theta }_1)})=\arctan (\sqrt{\frac{I_s(T_o,T_e,{\theta }_1)}{I_p(T_o,T_e,{\theta }_1)}})$$

(11)

Among them, E_s and E_p are the components of the electric field vector in the vertical and horizontal directions, respectively.

The refractive index of the air, n₁ = 1, and the refractive index of the target is set to n₂ = 4.56 − 28.5i. The wavelength range considered is in the medium-long wave band, from 3 to 14 µm. The ambient temperature is T_e = 10 °C, and the target temperature T_o varies from −20 to 40 °C. Figure 2a,b illustrate the infrared polarization of the target with respect to the background temperature difference T at different incidence angles.

As shown in Figure 2a, the degree of polarization of the target increases with a greater temperature difference between the target and its surroundings. When the temperature difference is zero, the target loses its polarization. Furthermore, the degree of polarization of the target increases with a larger incident angle. However, even at a target temperature of 40 °C and an incident angle of 70°, the polarization degree of the target is only 0.04, considering the set target refractive index. At this point, we suspect the important factor affecting the polarization degree of the target to be the refractive index and will try to prove it.

Figure 2b demonstrates that the polarization angle of the target gradually decreases as the target temperature increases. Additionally, the degree of polarization angle reduction is greater with a larger incident angle. However, the variation in the polarization angle of the target is not significant, indicating that it is less influenced by temperature and incident angle.

The target temperature was set to 25 °C, and the ambient temperature was 10 °C. The incident angle, denoted as θ₁, was set to 45°. The real part of the target refractive index, denoted as nr, was varied from 1 to 5, while the imaginary part, denoted as ni, was varied from 0 to 20. The target refractive index was calculated as n₂ = n_r − n_i·i. The polarization change curve of the target with varying refractive index is shown in Figure 3a,b.

As depicted in Figure 3a, the degree of polarization increases as the real part of the refractive index increases when the imaginary part of the target refractive index is close to zero. When the real part of the refractive index equals 5, the degree of polarization exceeds 0.07. However, an increase in the imaginary part of the refractive index leads to a sharp decrease in the degree of polarization, becoming the primary factor influencing polarization. When the imaginary part of the refractive index reaches 20, the degree of polarization drops below 0.01. This observation also elucidates that, under identical conditions, the degree of polarization in metals is consistently lower than that in insulators. Figure 3b demonstrates that, as the imaginary part of the target refractive index approaches zero, the polarization angle gradually decreases with an increase in the real part of the refractive index. Conversely, when the imaginary part of the refractive index is greater than zero, the polarization angle increases with the imaginary part, indicating that both the real and imaginary parts of the refractive index impact the polarization angle.

3. Experiments

3.1. Long-Wave Infrared Imaging

Long-wave infrared images of three distinct targets (1.2.3) were captured at varying distances (D) and solar heights (H) to assess the contrast between the targets and the sky background. Imaging experiments were conducted during nighttime and under different visibility conditions (VIS). The detection wavelength is 8–14 μm, the resolution is 640 × 512, and NETD (Noise equivalent temperature difference) is ≤50 mk. The experimental setup and scenario are illustrated in Figure 4.

The radiation intensity of the brighter portion of the target and the sky background in the long-wave infrared image was extracted to calculate the contrast (c) data using the formula $\mathrm{C}=\left|{\mathrm{I}}_{\mathrm{O}}-{\mathrm{I}}_{\mathrm{b}}\right|/({\mathrm{I}}_{\mathrm{O}}+{\mathrm{I}}_{\mathrm{b}})$, where I_o represents the target radiation intensity, and I_b represents the background radiation intensity, as illustrated in Figure 5.

Figure 5a–c present the line diagram depicting the contrast of three targets at different times, corresponding to varying solar altitude angles and distances. The figure reveals that the contrast between the target and the background increases with a higher sun altitude angle. This is due to the stronger sunlight at noon, resulting in higher target temperatures and greater radiation energy. The contrast, however, remains relatively stable with increasing distance. Figure 5d–f illustrate the change in target contrast with distance at a similar solar altitude angle but different visibility conditions. It is evident from the figure that lower visibility leads to lower contrast, and the contrast does not significantly change with increasing detection distance. The mean contrast value of the three target materials shows no significant difference.

3.2. Two-Color Infrared Polarization Imaging

The distances were increased to 200–1000 m, and the bridge and lighthouse targets at sea were observed at different times (corresponding to different solar altitude angles). The bridge and lighthouse represent two different materials, metal and concrete, respectively. The experimental setup and detection targets are depicted in Figure 6a–c. The detection wavelength is 3–14 μm, the resolution is 640 × 512, and NETD is ≤40 mk. The polarizing element adopts a wide spectral band infrared gate polarizer, with an extinction ratio between 150:1 (3 µm) and 300:1 (14 µm) and an average transmittance greater than 70%. The polarizer is placed inside a high-speed rotating wheel and can achieve a rotational speed of 120°/8 ms. The two-color infrared contrast between the two targets was measured, and the results are presented in Figure 7.

As illustrated in the figure, it can be observed that the contrast remains relatively stable as the distance increases, indicating that distance has minimal impact on the contrast. However, as the solar altitude angle increases, the contrast also increases, which aligns with the findings of previous long-wave infrared imaging experiments. This can be attributed to the fact that higher solar altitude angles result in greater light intensity and higher radiant heat from the target, leading to a larger difference in radiance between the target and the sky background and, consequently, a greater contrast between the target and the background in the image. In comparison to the previous long-wave infrared image, the two-color infrared image generally exhibits higher contrast between the target and the background. The contrast in the long-wave infrared image is concentrated below 40%, whereas the contrast in the two-color infrared image can exceed 70%. Therefore, two-color infrared imaging offers more advantages in target detection and recognition compared to single-band long-wave infrared imaging.

Infrared radiation intensity, degree of polarization, and polarization angle were measured throughout a 24 h period, encompassing both day and night. The detection target and background are shown in Figure 6d. The results are depicted in Figure 8.

As depicted in Figure 8a,d, the radiation intensity of the ship target surpasses that of the sea surface background. These intensity data serve as a foundation for distinguishing between the target and the background, with the temperature rise leading to an increased disparity in radiation intensity between the two. In Figure 8b,e the polarization of the ship target is significantly higher than that of the sea surface background, particularly during daylight hours, resulting in an average contrast increase from 0.035 in Figure 8d to 0.217 in Figure 8e. This heightened contrast can effectively aid in target identification. Due to the horizontal observation, as the solar altitude angle rises, so does the incident angle, leading to an increased temperature difference between the target and the background, consequently causing the degree of polarization to exhibit an upward trend. The polarization angle information, as shown in Figure 8c,f, experiences an increase due to the sensitivity of the polarization angle to external environmental interference and system noise. However, the contrast of the polarization angle between the target and the background tends to decrease with rising temperature. In order to obtain more stable and reliable polarization angle information, it is imperative to impose stricter requirements on external environmental conditions and system structural performance.

Images of distant islands under early morning sea fog were captured, revealing the target’s visible light, two-color infrared intensity, polarization degree, and polarization angle, as illustrated in Figure 9.

The two-color infrared image exhibits a lower definition compared to the visible image due to the significant infrared absorption by water vapor. Nevertheless, the polarization degree and polarization angle images enhance the contrast and sharpness of the target in comparison to the original infrared image. In terms of daytime haze imaging, infrared polarization imaging serves as a valuable tool to compensate for the information deficiency in conventional infrared imaging.

As depicted in Figure 10, infrared imaging exhibits excellent detection capabilities during nighttime. The addition of polarization information in the infrared image proves particularly effective in detecting artificial targets such as buildings, windows, and metal support poles while reflecting low polarization characteristics for non-artificial targets like trees and land. In terms of nighttime imaging, infrared polarization not only enables all-weather detection and recognition but also aids in distinguishing between man-made and non-artificial objects based on their high-temperature and low-temperature signatures. Moreover, it facilitates camouflage detection and target recognition in special scenarios.

3.3. Multispectral Segment Fog Transmission Imaging and Fusion Processing

Utilizing the polarization properties of fog transmission, we employed a visible light polarization camera, as well as short-wave and long-wave infrared cameras, to detect vehicles under water mist conditions. Among them, visible light polarization cameras can achieve real-time acquisition of images of various polarization components within the visible light range, obtaining polarization angle and degree of polarization images for subsequent fusion work. The shortwave infrared camera achieves 9500–1700 nm imaging with a resolution of 640 × 512. To capture multi-angle and multidimensional information about the target, detection was conducted during different times of the day, specifically noon (strong light), dusk (strong light), and evening (low light), with a detection range of 50–150 m. The camera’s front water mist generation device was utilized to simulate real haze conditions. The experimental setup and the device are illustrated in Figure 11.

In order to achieve registration between visible light and infrared images, it is necessary to address the disparity in perspective resulting from the use of different image acquisition equipment. To ensure spatial alignment and consistency between the two modalities, transformations such as scale, rotation, and translation are applied. For this purpose, a multimodal image registration method [24] utilizing an enhanced SIFT (scale-invariant feature transform) algorithm is employed. The registration process is depicted in Figure 12 below.

Firstly, the saliency analysis technique is employed to identify crucial information within the image, resulting in the generation of a saliency map. Utilizing the saliency map, the regions of interest and non-interest are distinguished for both the visible light and infrared images. Subsequently, the SIFT detection algorithm is utilized to extract feature points from different regions of the visible light and infrared images. These extracted feature points are then described using an enhanced SIFT feature point descriptor generation method, which enhances the discriminative power of the feature points. Finally, based on the similarity of the feature point descriptors, corresponding feature points are identified on the visible light and infrared images. The RANSAC (Random Sample Consensus) is employed to refine and purify the matched feature points, ultimately yielding the matching results between the infrared and visible light images.

As depicted in Figure 13, the original image is decomposed into a low-frequency approximation image and four high-frequency detail images using a local extreme value multiscale decomposition technique. Each high-frequency image is further processed through a non-downsampled direction filter, resulting in four different direction filter detail coefficient matrices. The detail layers are then fed into a pre-trained twin network to extract deep features. The weighted normalized cross-correlation is computed between the resulting detailed feature map and the high-frequency coefficients. By combining the resulting weighted normalized intercorrelation matrix with the high-frequency coefficients, the fused high-frequency direction coefficients are obtained. Additionally, visual salience mapping is employed to fuse the low-frequency approximation images. Finally, the fused base and detail layers are subjected to a multiscale inverse transformation to generate the fusion image [25].

The fusion and enhancement processed images are presented in Figure 14. Information extraction was performed on the fused image, and the mean of eight indicators of the image was obtained [26,27], as shown in

Table 1. Test data.

SD	EI	EN	AG
49.843	68.792	7.297	6.623
RMSE	PSNR	CE	MI
0.135	56.723	1.247	2.331

.

From the table, it is evident that larger standard deviation (SD) values and values for edge intensity (EI) and average gradient (AG) indicate a higher level of contrast and clearer details in the fused image. Furthermore, the entropy (EN) value can serve as an indicator of the information content within the fused image. Conversely, smaller values of root mean square error (RMSE) and cross-entropy (CE) signify that the fused image contains relatively minimal error and distortion. A reduced disparity between images implies greater information extraction from the source image.

In addition, higher values of peak signal-to-noise ratio (PSNR) signify a closer resemblance of the fused image to the source image, translating to a smaller amount of error introduced during fusion. Moreover, an elevated mutual information (MI) value in the fused image indicates a greater inheritance of information from the source image, resulting in an improved quality of the fused image.

The fusion image, which combines visible light, polarization, near-infrared, and long-wavelength infrared, exhibits exceptional adaptability to various environmental conditions, as corroborated by both subjective and objective evaluations. It not only excels in penetrating haze to reveal target details but also consistently delivers clear imaging under diverse lighting conditions throughout the day.

4. Discussion

In the simulation of infrared polarization characteristics, various factors such as target temperature, temperature difference between the target and the environment, incidence angle, and the real and imaginary parts of the refractive index of the target object have different degrees of influence on the degree of polarization and polarization angle. As the target temperature increases, the degree of polarization (Aop) decreases, while an increase in the temperature difference between the target and the background leads to an increase in the degree of polarization (Dop). When the incident angle (θ) increases, the degree of polarization increases, but the polarization angle decreases at a faster rate. When the imaginary part of the refractive index of the target is very small, the degree of polarization increases with the increase of the real part. In this case, the polarization angle gradually decreases with the increase of the real part of the refractive index. However, an increase in the imaginary part of the refractive index significantly reduces the degree of polarization, becoming the main factor affecting it. The polarization angle increases with the increase of the imaginary part.

In the experiment of long-wave infrared polarization imaging, the contrast between the target and the background increases as the solar altitude angle increases and decreases with decreasing visibility. By studying the multi-target imaging characteristics of long-wave infrared polarization at different times of the day, we also explore the characteristics of dual-color infrared polarization imaging. The contrast of the long-wave infrared image is less than 40%, while the contrast of the two-color infrared image can exceed 70%. Therefore, compared to single-band long-wave infrared imaging, two-color infrared imaging has more advantages in target detection and recognition. The difference in radiation intensity between the target and the background increases during the day, providing a basis for target and background identification. The average contrast in polarization information increases from 0.035 to 0.217, which can be better utilized for target identification. From the perspective of daytime haze imaging, dual-color infrared polarization imaging can compensate for the lack of information in traditional infrared imaging. From the perspective of night imaging, two-color infrared polarization can be used for all-weather detection and recognition and is more effective in distinguishing between man-made and non-man-made targets, high-temperature and low-temperature objects, as well as camouflage target detection and recognition in special scenes. This highlights the advantages of dual-color infrared polarization imaging in target detection and recognition.

Finally, an enhanced SIFT multimodal image registration method and a multispectral fusion enhancement algorithm are employed to register and fuse visible, near-infrared, long-wave infrared, and polarized images. Utilizing multiple sets of experimental data, we derived the mean values for eight image evaluation indicators. The outcomes consistently demonstrated that the fused image consistently attained optimal values across various indicator assessments. It effectively retained the original image’s information content while enhancing image contrast and enriching the details of the target. It has been demonstrated that the combination of images from multiple spectral bands can not only penetrate through water mist but also enhance the visibility of targets, thereby providing theoretical support for all-weather target detection and recognition. In the future, photoelectric detection technology will continue to evolve towards the integration of multiple spectral bands and the integration of multidimensional information.

5. Conclusions

Compared to the imaging characteristics summarized in current research findings, infrared medium wave, infrared long wave, and polarization all exhibit favorable effects. However, the imaging characteristics of wide-band infrared polarization detected simultaneously by medium wave and long wave have been less explored [28,29]. This study demonstrates its advantages in various climatic conditions, multiple time periods, diverse targets, and varying distances. The imaging characteristics of the wide-band medium and long waves are comprehensively analyzed. Due to the complexity of the experimental environment and the influence of factors such as detector response, the experimental cycle was condensed, resulting in a relatively small amount of data. In the future, we will continue to collect data under more diverse conditions, enhance the accuracy of the model, and reinforce the reliability of both the model simulation results and experimental findings.