Next Article in Journal
Dual Homogeneous Patches-Based Band Selection Methodology for Hyperspectral Classification
Previous Article in Journal
Effective Factors for Reducing Land Surface Temperature in Each Local Climate Zone Built Type in Tokyo and Shanghai
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Technical Note

Using Double-Layer Patch-Based Contrast for Infrared Small Target Detection

1
Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan 430079, China
2
Hubei Research Center for Educational Information, Central China Normal University, Wuhan 430079, China
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Remote Sens. 2023, 15(15), 3839; https://doi.org/10.3390/rs15153839
Submission received: 1 June 2023 / Revised: 24 July 2023 / Accepted: 31 July 2023 / Published: 1 August 2023
(This article belongs to the Section Remote Sensing Image Processing)

Abstract

:
Detecting infrared (IR) small targets effectively and robustly is crucial for the tasks such as infrared searching and guarding. While methods based on the human vision system (HVS) have achieved great success in this field, detecting dim targets in complex backgrounds remains a challenge due to the multi-scale framework and over-simplified disparity calculations. In this paper, infrared small targets are detected with a novel local contrast measurement named double-layer patch-based contrast (DLPC). Firstly, we crafted an elaborated double-layer local contrast measure, to suppress the background, which can accurately measure the gray difference between the target and its surrounding complex background. Secondly, we calculated the absolute value of the grayscale difference between the target and the background in the diagonal directions as a weighting factor to further enhance the target. Then, an adaptive threshold on the DLPC was employed to extract the target from the IR image. The proposed method can detect small targets effectively with a fixed-scaled mask template while being computationally efficient. Experimental results in terms of background suppression factor (BSF), signal-to-clutter ratio gain (SCRG) and receiver operating characteristic (ROC) curve on five IR image datasets demonstrated that the proposed method has better detection performance compared to six state-of-the-art methods and is more robust in addressing complex backgrounds.

1. Introduction

Infrared (IR) small target detection is a crucial research field with a wide range of applications, including early warning systems, missile tracking systems and precision guidance [1,2,3,4]. However, several disadvantageous factors make it challenging to achieve optimal detection performance. These factors can be classified into two types: the characteristics (e.g., small, having no concrete shape or texture) of the small target [5,6,7,8], and the adverse effects of its context (e.g., high-brightness background, background edge and pixel-sized noises with high brightness (PNHB)) of different types of components in infrared images [9,10,11]. Because of these obstacles, it is untrivial to develop an IR small target detection method with high detection rates and low false alarm rates.
Several infrared small target detection models have been proposed in recent decades such as morphology-based models [12,13], HVS-based models [4,8,14,15,16,17,18] and deep learning-based models [6,19,20,21]. The morphological methods are concise and easy to apply, but cannot deal with heavy clutter and noise due to the lack of local difference computation [13]. The HVS-based methods focus on the local contrast more than the absolute brightness of the dim targets, thus generally computing and enhancing the heterogeneity of the target and its surrounding background with well-designed sliding windows consisting of central and background cells. For example, the local contrast method (LCM) [22] calculates the ratio between the maximum gray value of the center cell and the average gray value of the background cell to obtain local contrast. The multiscale patch-based contrast measure (MPCM) [14] measures the difference between the average gray value of the center patch and the background patch and then figures out the product of the gray value difference in the diagonal direction to further enhance the target. Unlike LCM and MPCM, the relative local contrast measure (RLCM) [23] adopts the average value of the previous i-th maximum gray value and takes the ratio of the center cell to the background cell as an enhancement factor. The multiscale local homogeneity measure (MLHM) [24] and the homogeneity-weighted local contrast measure (HWLCM) [25] not only compare the grayscale difference between the target area and surrounding background, but also consider the standard deviation of the target and surrounding background, respectively. Compared with the newly emerging deep learning methods which tackle the dim target detection problem with pixel-based classification [6] or bounding box regression modules [19], the HVS-based modules are more predictable, widely applicable, edge-precise and off-the-shelf (i.e., free from labeling or training), thus being the de facto golden methods in real applications. Our work is also in this scope. Moreover, by abandoning the globally used multi-scale framework [22,23,26], the proposed method avoids “expansion effects” [27] and, in addition, gains better detection results with a well-crafted contrast enhancement calculation.
In this paper, we propose a new method named double-layer patch-based contrast (DLPC) for small target detection. The method employs a three-step approach. First, the method calculates a double-layer local contrast measure with a fixed-scale sliding window to suppress the background. Second, it uses a directional local contrast calculation method that we crafted to enhance the targets’ saliency. By combining these two measures in the DLPC calculation, a saliency map is obtained that accurately highlights the target. Finally, it uses an adaptive threshold to segment the target from the saliency map. Our experimental results demonstrated that the proposed DLPC method outperforms six state-of-the-art methods in terms of background suppression factor (BSF), signal-to-clutter ratio gain (SCRG) and receiver operating characteristic (ROC) curves. The main contributions of our work are two-fold: (1) A novel single-scale double-layer local contrast measure was developed, which can effectively and efficiently mitigate the influence of background clutter and enhance the target signal; (2) The robust and well-performed target enhancement operations function even in complex and cluttered backgrounds.
It is noteworthy that DNGM [28] shares similarities with the proposed method, as both techniques aim to improve the sliding window structure. However, DNGM calculates the disparity between the central cell and the double-layer background separately, thereby resulting in reduced detection performance under complex backgrounds. Specifically, when the target is adjacent to a high-brightness background with a gray value greater than that of the target, the DNGM method may miss detection. Comparatively, our proposed method evaluates the distinction between the center layer and the whole surrounding area by analyzing the gray difference between the double-layer surrounding area. Firstly, our method calculates the minimum difference between the gray value of the double-layer surrounding area and the target patch. The larger difference is then chosen to suppress the background. Subsequently, by computing the absolute value of the gray difference on the diagonal between the center patch and double-layer surrounding area and then selecting the minimum value, the method can more robustly enhance the target area under complex backgrounds than DNGM.
The structure of the paper is as follows. In Section 2, we give an overview of the proposed method as well as a detailed description of each component. Section 3 presents the comparative experimental results and discussion, with comparison to six state-of-the-art methods. The conclusions are given in Section 4.

2. Methodology

To address the “expansion effects” introduced by the commonly utilized multi-scale framework in most HVS-based IR small target detection methods, as well as to enhance the automatic detection capability amidst heavy clutter and noise, we have proposed a novel method for IR dim target detection. This method comprises two components: DLPC and threshold operation. The DLPC utilizes a two-step framework. Firstly, we have developed a new and sophisticated fixed-scale sliding window that measures the disparity between a given region and its surrounding area. Subsequently, the measure is enhanced or eliminated based on carefully designed calculations for the target or background region separately. Finally, an adaptive threshold is employed to extract the target regions. An overview of our method can be seen in Figure 1.

2.1. Double-Layer Local Contrast Measure

The size of infrared small targets is generally acknowledged to fall within the range of 2 × 2 to 9 × 9 pixels [29]. Due to the variations in target sizes across different scenes, predicting the size of the object is challenging. To overcome this challenge, most previous methods utilized a multi-scale framework, which unfortunately introduced additional obstacles such as “expansion effects” and efficiency concerns. To tackle these issues, a new sliding window for measuring the local contrast is proposed, as depicted in Figure 2.
Instead of simply dividing the mask template into the center (T) and its surrounding regions, our proposed sliding window takes it a step further by dividing the surrounding region into two distinct layers: the middle layer and the background layer. The middle layer serves as the transition area between the target and the background layer, which consists of 8 middle patches ( M i , i = 1, …, 8). Each patch within the sliding window maintains a fixed size of 3 × 3. Similarly, the background layer is subdivided into 16 background patches ( B j , j = 1, …, 16). To calculate the double-layer local contrast measure, the difference between the target and the surrounding area is first determined and defined as follows:
d M i = m T m M i , ( i = 1 , , 8 ) d B j = m T m B j , ( j = 1 , , 16 ) ,
With the following notations:
  • d M i : the difference between the gray mean value of the center layer and the i-th middle patch;
  • d B j : the difference between the gray mean value of the center layer and the j-th background patch;
  • m T : the gray mean values of the center layer;
  • m M i : the gray mean values of the i-th middle patch;
  • m B j : the gray mean values of the j-th background patch.
Then, the minimum gray difference between the target and each layer patch is calculated, which is defined as
m d M = min i = 1 , , 8 d M i m d B = min j = 1 , , 16 d B j ,
where m d M and m d B are the minimum values of d M i and d B j . The larger value out of m d M and m d B is selected as the local contrast measure D 1 :
D 1 = max ( m d M , m d B ) , if max ( m d M , m d B ) > 0 0 , e l s e .
The proposed method effectively suppresses background interference by utilizing the maximum operation to perform scale selection. Figure 3 showcases various target sizes and their surrounding areas. When the size of the target is 3 × 3, the target is situated in the center layer, while the middle and background layers belong to the background area. Consequently, the gray values between the middle and background layers exhibit similarity, resulting in m d M m d B . Hence, the larger minimum difference D 1 is considered an effective local contrast measure, capturing the gray difference between the target and the surrounding background. For targets larger than 3 × 3 but smaller than 9 × 9, as depicted in Figure 3b, the middle layer comprises both the background and a portion of the target pixels. Consequently, the gray value of the middle layer is higher than that of the background layer, leading to D 1 taking the gray difference between the target and the background layer. The above operation demonstrates the capability of the proposed method to accurately measure the local contrast for targets of different sizes.

2.2. Target Enhancement

Precise detection of small targets in complex backgrounds is crucial, necessitating both effective target enhancement and background suppression. In contrast to other methods that intuitively calculate the disparity between different layers and enhance it using a simple mathematical function, our method takes the diagonal minimum absolute gray difference between the central layer and each of the other two layers and then uses this disparity to enhance the local contrast. We firmly believe that this adjustment aligns more coherently with the attention schema of the HVS. To be specific, we define the disparity between the target and its surrounding areas in different directions as
D M i = d M i × d M 9 i , ( i = 1 , . . . , 4 ) D B j = d B j × d B 17 j , ( j = 1 , . . . , 8 ) ,
with the following notations:
  • ×: the multiplication operation;
  • D M i : the contrast measure between the center layer and the middle layer along the i-th direction;
  • D B j : the contrast measure between the center layer and the background layer along the j-th direction.
Then, the minimum difference between the target and the surrounding background in different directions is calculated:
M D M = min i = 1 , , 4 D M i M D B = min j = 1 , , 8 D B j ,
MDM and MDB are the minimum values of D M i and D B j . Furthermore, an enhancement factor to boost the target effectively under a complex background is defined as
D 2 = min M D M , M D B .
By multiplying the enhancement factor D2 with the double-layer local contrast measure D1, the DLPC is introduced as
D L P C = D 1 × D 2 .
The aforementioned calculation process accurately enhances the target while simultaneously suppressing the background. Particularly, the proposed method effectively addresses the limitations of the DNGM method. Figure 4 illustrates a scenario where the target is positioned adjacent to the high-brightness background (HB), causing the DNGM method to struggle with detection due to the presence of brighter background clutter near the actual target. In the DNGM method, the minimum difference of the average gray value between the target and the double-layer neighborhood background is calculated, followed by a product operation, leading to missed detection in this case. In contrast, our proposed method calculates the minimum difference between the average gray value of the target and the middle layer as well as the background layer. It then selects the larger minimum difference as the double-layer local contrast measure, thus avoiding situations where the target contrast is zero in similar cases. Secondly, the target enhancement operation calculates the absolute value of the gray difference between the target and the surrounding area on the diagonal and selects its minimum value. This process helps enhance the target while suppressing the background to a certain extent. As a result, the proposed method demonstrates superior detection capabilities in complex scenarios.
The DNGM method experiences a decrease in the contrast between the target and surrounding background as the target size increases, resulting in a higher false alarm rate in complex backgrounds. Figure 5 serves to illustrate the detection results of DNGM and DLPC under various thresholds in complex backgrounds. Here, k represents an adjustable parameter in the threshold operation, s × s denotes the target size, the red box indicates the target detected by the algorithm and the yellow box is the true target label. In the first scene in Figure 5, a ground scene with multiple areas of high brightness is presented, where the target size is 5 × 5. When k = 60, DNGM is able to detect the true target but mistakenly identifies two high-brightness areas as targets. On the other hand, when k = 70, DNGM fails to detect the true target and instead generates a false detection. In comparison, the proposed method at k = 70 wrongly detects a high-brightness area as a target while also detecting the true target. However, when k = 80, the proposed method accurately detects the target without any false detections. Thus, when the target size ranges from 3 × 3 to 9 × 9, the detection performance of the DNGM method in complex backgrounds declines as the target size increases, limiting the alternative threshold space. In contrast, the proposed method exhibits a larger and more robust alternative threshold space in complex backgrounds.

2.3. Threshold Operation

Through DLPC calculations, the saliency map can be obtained. Then, to segment the target from the infrared image, an adaptive threshold is adopted, which is defined as
T = μ + k × σ ,
where the following notations are used:
  • μ : the mean value of the saliency map;
  • σ : the standard deviation of the saliency map;
  • k: the adjustable parameter.
The saliency map’s pixel values are positively correlated with the likelihood of target presence. By setting a threshold, pixels with values higher than the threshold are identified as the target region.

2.4. Detection Ability Analysis

Detecting small targets in infrared images poses a significant challenge due to the existence of background clutter, noise and high-brightness pixel-size noises. To evaluate the effectiveness of the proposed algorithm, it is essential to analyze its performance on different components of the image. Without loss of generality, we chose the center pixel of the central layer as the target for analysis.
If the pixel belongs to the target region (represented by TT), its gray value will be significantly larger than its surrounding background region; then, we obtain
m d M T T , m d B T T > 0 , and M D M T T , M D B T T > 0 ,
D L P C T T > 0 ,
which illustrates that the target area is effectively enhanced.
When dealing with a pixel that belongs to the background region (represented by BB), three main cases should be considered. First, in cases where the background is homogeneous, the pixel’s gray value tends to be similar to its surrounding areas. This similarity results in a minimal difference between the central layer and the surrounding area, which is approximately 0, i.e.,
D L P C B B 0 ,
which denotes that background areas can be eliminated.
Secondly, in cases where the background is heterogeneous and the pixel is located in the noise region (represented by N), the mean operation of the proposed method effectively suppresses the noise, making the pixel’s average gray value lower than that of the real target, i.e.,
0 < D L P C N < D L P C T T .
Third, when the pixel is at the background edge (represented by BE), the minimum difference between the average gray value of the target and the patches along the edge is close to 0, i.e.,
D L P C B E 0 .
The previous investigation highlights the effectiveness of our proposed method in minimizing the impact of the background and enhancing the detectability of small infrared targets in various environments. Consequently, our algorithm stands out as a robust and dependable approach for detecting small infrared targets.

3. Experimental Results and Analysis

This section presents a detailed description of the dataset and evaluation metrics, followed by an analysis of the performance of various algorithms based on experimental results. To verify the effectiveness and efficiency of our proposed method, we compared it with six state-of-the-art methods, namely LCM [22], MPCM [14], RLCM [23], DNGM [28], TLLCM [30] and NTFRA [31]. All experiments were conducted using MATLAB R2020b on a computer equipped with a 2.6 GHz Intel Core i5 CPU and 16 GB RAM.

3.1. Datasets

The experiments conducted in this study utilized five real infrared image sequences, showcasing a variety of scenes, such as sky–cloud, sea–sky and ground scenes, as shown in Figure 6. Sequence 1 is represented by an example of a sky–cloud scene that contains a large area of high-brightness background, dark areas of holes in heavy clouds and irregular background edges. This sequence was comprised of 70 images with a size of 256 × 200. Sequence 2 presented a challenging scenario in which it is difficult to distinguish the target from the background due to the target’s extremely dim nature. Furthermore, there are several continuous pixel-sized noises with high brightness (PNHB). This sequence contained 60 images with a resolution of 320 × 240. These two sequences, provided by our anonymous research partners, have been used in our previous publications, such as [3,14]. Sequence 3 was a subset of the Single-frame Infrared Small Target (SIRST) dataset [19], which comprises a variety of scenes, including sky scenes with complex backgrounds and ground, architecture and sea scenes. It consisted of 184 single-frame images of varying sizes, which were selected manually by excluding the overly simple cases. We obtained data 7 and data 16 from [32] and used them as Sequence 4 and Sequence 5, respectively, considering the number of targets and the typical background scenes. Sequence 4 was a set of ground scenes where targets were often immersed in high-brightness backgrounds, making target detection more difficult. It was comprised of 150 infrared images, each of which had a resolution of 256 × 256. Sequence 5 included 200 images with a resolution of 256 × 256. Several images in this sequence contained high-brightness areas that contrasted with the background region. In all of the above image sequences, only one bright target existed in each frame.

3.2. Evaluation Metrics

To validate the proposed approach, we conducted experiments using three commonly used evaluation metrics: background suppression factor (BSF), signal-to-clutter ratio gain (SCRG) and receiver operating characteristic (ROC) curve. BSF measures the effectiveness of background suppression and is defined as
B S F = δ i n δ o u t ,
with the following variables:
  • δ i n : the standard deviation of the raw image;
  • σ : the standard deviation of the saliency map;
  • k: the adjustable parameter.
The SCRG measures the improvement in the signal-to-clutter ratio achieved by the proposed method, which is defined as
S C R G = S C R o u t S C R i n , S C R = P t μ b δ ,
with the following notations:
  • P t : the maximum gray value of the target;
  • μ b : the gray mean of the surrounding background;
  • δ : the standard deviation of the raw image.
S C R i n and S C R o u t are the SCR values of the raw image and the saliency map, respectively. Higher BSF and SCRG values imply a better algorithm capacity for background suppression and target enhancement.
The ROC curve is a graphical representation of the trade-off between true positive rate and false positive rate, and is commonly used to evaluate the performance of detection algorithms. It is composed of two main components: the probability of detection rate (PD) and the false alarm rate (FAR). The PD is the probability that a target will be detected correctly, while the FAR is the probability of falsely detecting a non-target. The two variables are calculated as
P D = number of detected true targets total number of real targets ,
F A R = number of detected false targets number of tested frames .

3.3. Experimental Results and Comparisons

Figure 7 illustrates the ability of the proposed method to effectively detect targets of varying sizes from Sequence 3. The results indicated that the method was capable of detecting not only targets ranging from 3 × 3 to 9 × 9, but also larger targets. Figure 8 presents the 3D mesh view of example images and saliency maps generated by different algorithms. For Sequence 1, MPCM enhanced dark hole-like areas in heavy clouds, whereas RLCM and TLLCM had some residual background. NTFRA significantly enhanced some non-background areas but was as excellent at eliminating most of the background as the proposed method. In Sequence 2, MPCM, RLCM and TLLCM produced a considerable amount of background clutter. The saliency map generated by NTFRA was a flat surface, indicating that the target was not enhanced. In Sequence 3, MPCM, RLCM and TLLCM still had significant residual backgrounds. In Sequence 4 and Sequence 5, NTFRA had plenty of background residual, while MPCM and RLCM also had slight background clutter.
In the above-mentioned five image sequences, LCM could effectively enhance the target and suppress the background to a certain extent, but had limited ability to suppress the background. DNGM, on the other hand, had slight background residue but exhibited good performance. In contrast, the proposed algorithm accurately enhanced the target and effectively suppressed the background area.
The ROC curves for various algorithms are displayed in Figure 9. For Sequence 1, TLLCM and LCM exhibited lower detection rates and higher false alarm rates than the other algorithms. The thick cloud in the sequence contained dark regions with holes, causing MPCM to incorrectly identify these regions as targets despite the low contrast between the targets and the surrounding background. Additionally, when the target was near a high-brightness background area with a greater gray value, DNGM excluded the target as part of the background, thereby failing to detect all the targets. Overall, NTFRA, DNGM, RLCM and MPCM yielded favorable results. However, when the false alarm rate was zero, the proposed method demonstrated a detection rate of 100%, indicating superior performance over other methods. In Sequence 2, Figure 9 illustrates that the PD for NTFRA was zero, regardless of the FAR value, indicating that NTFRA became ineffective when the image had a low SCR and the target was particularly dim. None of the algorithms could detect all the targets in the image sequences, since some images contained multiple consecutive high-brightness pixels with greater contrast than the dim targets. For Sequence 3, MPCM, RLCM and DNGM exhibited good performance. In Sequences 4 and 5, due to the complex background in high-brightness regions, NTFRA had a high false alarm rate and a low detection rate. Across all five image sequences, the proposed algorithm’s ROC curve appeared in the upper-left corner, indicating higher detection rates and lower false alarm rates.
The evaluation metrics of BSF and SCRG are presented in Table 1 and Table 2, which show the BSF and SCRG values of each algorithm. The highest BSF and SCRG values are highlighted in bold and underlined in the tables. The proposed algorithm achieved the maximum BSF and SCRG values for Sequence 1, Sequence 4 and Sequence 5. For Sequence 2, the proposed algorithm had the highest SCRG value and its BSF value was second only to that of NTFRA. In Sequence 3, the BSF and SCRG values of the proposed algorithm were suboptimal. These results demonstrate that the proposed algorithm has superior background suppression and target enhancement capabilities.
Table 3 presents the time consumption of each algorithm, showing that the proposed algorithm was the second fastest, following only DNGM, in terms of time consumption. Overall, the experimental results demonstrate that the proposed algorithm outperforms the other comparison algorithms in terms of detection performance, as measured by the evaluation metrics of BSF, SCRG and ROC curves.
In general, the experimental results demonstrate that our method outperforms the state-of-the-art comparative methods in enhancing the target area and suppressing heavy background noise and clutter, both qualitatively (as shown in Figure 8) and quantitatively (as shown in Table 1 and Table 2). Additionally, as illustrated in Figure 9, the highest top-left ROC curve indicates a better trade-off between the sensitivity and the specificity of the proposed method, which alleviates the dilemma of selecting an appropriate threshold for the next step. However, there are still specific negative cases observed. The first challenge involves high-brightness noisy pixels randomly appearing amidst homogenous backgrounds. Identifying them from real targets without additional information [33] is nearly impossible, as they both appear as textureless bright or dark dim regions in the IR images. However, since we aimed to develop a universally applicable IR small detection method, introducing domain knowledge falls outside the scope of this work and will be considered in future research. Secondly, as our proposed method focuses on local contrast based on the human visual system, it cannot handle the rich semantic information conveyed by some high-resolution IR images. Addressing this may require more complex processing workflows that incorporate local contrast measures and deep learning methods, as seen in, e.g., [19].

4. Conclusions

In this study, we present a novel method that eliminates the need for a multi-scale framework in order to detect infrared (IR) small targets under various scenarios. Our method offers several significant advantages compared to existing approaches. Firstly, the proposed single-scale double-layer local contrast measure can effectively and efficiently enhance the dim target area of varied sizes while suppressing the clutter and noise of the background. Secondly, we deliberately crafted a target enhancement calculation coupled with a sliding window which makes DLPC perform well in some difficult situations, such as HB and increased targets’ sizes. The experimental results demonstrate that the proposed method outperforms other state-of-the-art methods in the presence of complex backgrounds. This indicates its wide-ranging applicability in IR security and guidance systems. However, as discussed in the previous section, there is still room for improvement in our future work. For instance, incorporating domain knowledge and leveraging local and global semantic image context can be explored to further enhance the performance of our method.

Author Contributions

Conceptualization, L.L., Y.W. (Yantao Wei) and H.Y.; methodology, L.L. and H.Y.; software, L.L. and Y.W. (Yue Wang); validation, L.L., Y.W. (Yue Wang) and D.C.; writing—original draft preparation, L.L., Y.W. (Yantao Wei) and H.Y.; writing—review and editing, H.Y. and Y.W. (Yantao Wei); visualization, Y.W. (Yue Wang); supervision, H.Y. and Y.W. (Yantao Wei); project administration, H.Y.; funding acquisition, H.Y. and Y.W. (Yantao Wei). All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Natural Science Foundation of China, grant number 62277029, and the Fundamental Research Funds for the Central Universities, under grants CCNU22JC025 and CCNU22JC011.

Data Availability Statement

Datasets underlying the results presented in this paper are available at https://github.com/YimianDai/sirst (Image Sequence 3) and https://www.scidb.cn/en/detail?dataSetId=720626420933459968&dataSetType=journal (Image Sequence 4 and 5).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Qin, Y.; Li, B. Effective infrared small target detection utilizing a novel local contrast method. IEEE Geosci. Remote Sens. Lett. 2016, 13, 1890–1894. [Google Scholar] [CrossRef]
  2. Deng, H.; Sun, X.; Liu, M.; Ye, C.; Zhou, X. Entropy-based window selection for detecting dim and small infrared targets. Pattern Recognit. 2017, 61, 66–77. [Google Scholar] [CrossRef]
  3. Shi, Y.; Wei, Y.; Yao, H.; Pan, D.; Xiao, G. High-boost-based multiscale local contrast measure for infrared small target detection. IEEE Geosci. Remote Sens. Lett. 2017, 15, 33–37. [Google Scholar] [CrossRef]
  4. Dai, Y.; Wu, Y.; Zhou, F.; Barnard, K. Asymmetric contextual modulation for infrared small target detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA, 3–8 January 2021; pp. 950–959. [Google Scholar]
  5. Zhang, T.; Li, L.; Cao, S.; Pu, T.; Peng, Z. Attention-Guided Pyramid Context Networks for Detecting Infrared Small Target Under Complex Background. IEEE Trans. Aerosp. Electron. Syst. 2023, 1–13. [Google Scholar] [CrossRef]
  6. Li, B.; Xiao, C.; Wang, L.; Wang, Y.; Lin, Z.; Li, M.; An, W.; Guo, Y. Dense Nested Attention Network for Infrared Small Target Detection. IEEE Trans. Image Process. 2023, 32, 1745–1758. [Google Scholar] [CrossRef]
  7. Zhang, M.; Zhang, R.; Yang, Y.; Bai, H.; Zhang, J.; Guo, J. ISNET: Shape matters for infrared small target detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 877–886. [Google Scholar]
  8. Zhu, H.; Ni, H.; Liu, S.; Xu, G.; Deng, L. Tnlrs: Target-aware non-local low-rank modeling with saliency filtering regularization for infrared small target detection. IEEE Trans. Image Process. 2020, 29, 9546–9558. [Google Scholar] [CrossRef]
  9. Gao, C.; Meng, D.; Yang, Y.; Wang, Y.; Zhou, X.; Hauptmann, A.G. Infrared patch-image model for small target detection in a single image. IEEE Trans. Image Process. 2013, 22, 4996–5009. [Google Scholar] [CrossRef]
  10. Han, J.; Liu, C.; Liu, Y.; Luo, Z.; Zhang, X.; Niu, Q. Infrared small target detection utilizing the enhanced closest-mean background estimation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 14, 645–662. [Google Scholar] [CrossRef]
  11. Lu, R.; Yang, X.; Li, W.; Fan, J.; Li, D.; Jing, X. Robust infrared small target detection via multidirectional derivative-based weighted contrast measure. IEEE Geosci. Remote Sens. Lett. 2020, 19, 7000105. [Google Scholar] [CrossRef]
  12. Tom, V.T.; Peli, T.; Leung, M.; Bondaryk, J.E. Morphology-based algorithm for point target detection in infrared backgrounds. In Proceedings of the Signal and Data Processing of Small Targets 1993, Orlando, FL, USA, 22 October 1993; SPIE: Bellingham, WA, USA, 1993; Volume 1954, pp. 2–11. [Google Scholar]
  13. Bai, X.; Zhou, F. Analysis of new top-hat transformation and the application for infrared dim small target detection. Pattern Recognit. 2010, 43, 2145–2156. [Google Scholar] [CrossRef]
  14. Wei, Y.; You, X.; Li, H. Multiscale patch-based contrast measure for small infrared target detection. Pattern Recognit. 2016, 58, 216–226. [Google Scholar] [CrossRef]
  15. Xia, C.; Li, X.; Zhao, L.; Shu, R. Infrared small target detection based on multiscale local contrast measure using local energy factor. IEEE Geosci. Remote Sens. Lett. 2019, 17, 157–161. [Google Scholar] [CrossRef]
  16. Guan, X.; Peng, Z.; Huang, S.; Chen, Y. Gaussian scale-space enhanced local contrast measure for small infrared target detection. IEEE Geosci. Remote Sens. Lett. 2019, 17, 327–331. [Google Scholar] [CrossRef]
  17. Zhang, L.; Peng, L.; Zhang, T.; Cao, S.; Peng, Z. Infrared small target detection via non-convex rank approximation minimization joint l 2, 1 norm. Remote Sens. 2018, 10, 1821. [Google Scholar] [CrossRef] [Green Version]
  18. Zhang, L.; Peng, Z. Infrared small target detection based on partial sum of the tensor nuclear norm. Remote Sens. 2019, 11, 382. [Google Scholar] [CrossRef] [Green Version]
  19. Dai, Y.; Wu, Y.; Zhou, F.; Barnard, K. Attentional local contrast networks for infrared small target detection. IEEE Trans. Geosci. Remote Sens. 2021, 59, 9813–9824. [Google Scholar] [CrossRef]
  20. Wang, K.; Du, S.; Liu, C.; Cao, Z. Interior Attention-Aware Network for Infrared Small Target Detection. IEEE Trans. Geosci. Remote Sens. 2022, 60, 1–13. [Google Scholar] [CrossRef]
  21. Xu, H.; Zhong, S.; Zhang, T.; Zou, X. Multiscale Multilevel Residual Feature Fusion for Real-Time Infrared Small Target Detection. IEEE Trans. Geosci. Remote Sens. 2023, 61, 1–16. [Google Scholar] [CrossRef]
  22. Chen, C.P.; Li, H.; Wei, Y.; Xia, T.; Tang, Y.Y. A local contrast method for small infrared target detection. IEEE Trans. Geosci. Remote Sens. 2013, 52, 574–581. [Google Scholar] [CrossRef]
  23. Han, J.; Liang, K.; Zhou, B.; Zhu, X.; Zhao, J.; Zhao, L. Infrared small target detection utilizing the multiscale relative local contrast measure. IEEE Geosci. Remote Sens. Lett. 2018, 15, 612–616. [Google Scholar] [CrossRef]
  24. Nie, J.; Qu, S.; Wei, Y.; Zhang, L.; Deng, L. An infrared small target detection method based on multiscale local homogeneity measure. Infrared Phys. Technol. 2018, 90, 186–194. [Google Scholar] [CrossRef]
  25. Du, P.; Hamdulla, A. Infrared small target detection using homogeneity-weighted local contrast measure. IEEE Geosci. Remote Sens. Lett. 2019, 17, 514–518. [Google Scholar] [CrossRef]
  26. Han, J.; Ma, Y.; Zhou, B.; Fan, F.; Liang, K.; Fang, Y. A robust infrared small target detection algorithm based on human visual system. IEEE Geosci. Remote Sens. Lett. 2014, 11, 2168–2172. [Google Scholar]
  27. Li, Y.; Li, Z.; Liu, B.; Liu, Y. Effective and robust infrared small target detection based on directional improved structure tensor. Infrared Phys. Technol. 2022, 127, 104420. [Google Scholar] [CrossRef]
  28. Wu, L.; Ma, Y.; Fan, F.; Wu, M.; Huang, J. A double-neighborhood gradient method for infrared small target detection. IEEE Geosci. Remote Sens. Lett. 2020, 18, 1476–1480. [Google Scholar] [CrossRef]
  29. Wang, X.; Peng, Z.; Kong, D.; He, Y. Infrared dim and small target detection based on stable multisubspace learning in heterogeneous scene. IEEE Trans. Geosci. Remote Sens. 2017, 55, 5481–5493. [Google Scholar] [CrossRef]
  30. Han, J.; Moradi, S.; Faramarzi, I.; Liu, C.; Zhang, H.; Zhao, Q. A local contrast method for infrared small-target detection utilizing a tri-layer window. IEEE Geosci. Remote Sens. Lett. 2019, 17, 1822–1826. [Google Scholar] [CrossRef]
  31. Kong, X.; Yang, C.; Cao, S.; Li, C.; Peng, Z. Infrared small target detection via nonconvex tensor fibered rank approximation. IEEE Trans. Geosci. Remote Sens. 2021, 60, 5000321. [Google Scholar] [CrossRef]
  32. Hui, B.; Song, Z.; Fan, H.; Zhong, P.; Hu, W.; Zhang, X.; Lin, J.; Su, H.; Jin, W.; Zhang, Y.; et al. A Dataset for Dim-Small Target Detection and Tracking of Aircraft in Infrared Image Sequences; Science DB: Beijing, China, 2019. [Google Scholar]
  33. Zhao, E.; Dong, L.; Dai, H. Infrared Maritime Small Target Detection Based on Multidirectional Uniformity and Sparse-Weight Similarity. Remote Sens. 2022, 14, 5492. [Google Scholar] [CrossRef]
Figure 1. Flowchart of DLPC.
Figure 1. Flowchart of DLPC.
Remotesensing 15 03839 g001
Figure 2. The composition of the proposed sliding window, which consists of the center layer, middle layer and background layer.
Figure 2. The composition of the proposed sliding window, which consists of the center layer, middle layer and background layer.
Remotesensing 15 03839 g002
Figure 3. An example of different-sized targets and their surrounding areas. (a) The target size is 3 × 3. (b) The target size is larger than 3 × 3 and less than 9 × 9.
Figure 3. An example of different-sized targets and their surrounding areas. (a) The target size is 3 × 3. (b) The target size is larger than 3 × 3 and less than 9 × 9.
Remotesensing 15 03839 g003
Figure 4. The target is adjacent to the high-brightness background, where HB denotes the high-brightness background.
Figure 4. The target is adjacent to the high-brightness background, where HB denotes the high-brightness background.
Remotesensing 15 03839 g004
Figure 5. Detection results of DNGM and DLPC at different thresholds. The red boxes indicate the targets detected by the algorithms and the yellow boxes indicate the true targets.
Figure 5. Detection results of DNGM and DLPC at different thresholds. The red boxes indicate the targets detected by the algorithms and the yellow boxes indicate the true targets.
Remotesensing 15 03839 g005
Figure 6. Examples of the five image sequences. (ae) Sequences 1–5.
Figure 6. Examples of the five image sequences. (ae) Sequences 1–5.
Remotesensing 15 03839 g006
Figure 7. Targets (in red boxes) of different sizes in Sequence 3. (a) 3 × 3. (b) 5 × 5. (c) 7 × 7. (d) 9 × 9. (e) 16 × 9.
Figure 7. Targets (in red boxes) of different sizes in Sequence 3. (a) 3 × 3. (b) 5 × 5. (c) 7 × 7. (d) 9 × 9. (e) 16 × 9.
Remotesensing 15 03839 g007
Figure 8. The 3D mesh views of example images and saliency maps for different algorithms: (a1e1) 3D mesh view of example images; (a2e2) LCM; (a3e3) MPCM; (a4e4) RLCM; (a5e5) TLLCM; (a6e6) DNGM; (a7e7) NTFRA; (a8e8) the proposed algorithm.
Figure 8. The 3D mesh views of example images and saliency maps for different algorithms: (a1e1) 3D mesh view of example images; (a2e2) LCM; (a3e3) MPCM; (a4e4) RLCM; (a5e5) TLLCM; (a6e6) DNGM; (a7e7) NTFRA; (a8e8) the proposed algorithm.
Remotesensing 15 03839 g008
Figure 9. ROC curves of different algorithms. (ae) Sequence 1–Sequence 5.
Figure 9. ROC curves of different algorithms. (ae) Sequence 1–Sequence 5.
Remotesensing 15 03839 g009
Table 1. BSF values of different algorithms. The best results are in bold and underlined.
Table 1. BSF values of different algorithms. The best results are in bold and underlined.
SequenceLCMMPCMRLCMTLLCMDNGMNTFRAProposed
1240.941845.991080.152184.673772.891474.694215.13
258.49950.69403.05626.122359.66Inf2604.84
3314.593164.051459.163032.865308.441191.454990.02
4356.983990.881315.133579.496760.54825.147668.34
5466.244056.401012.373217.085972.861921.526642.68
Table 2. SCRG values of different algorithms. The best results are in bold and underlined.
Table 2. SCRG values of different algorithms. The best results are in bold and underlined.
SequenceLCMMPCMRLCMTLLCMDNGMNTFRAProposed
11.7424.4411.9528.1550.392.1657.53
20.9746.1318.5929.47120.20NaN132.83
31.9336.8615.5334.3360.629.1257.31
41.5326.837.0723.0043.900.8050.58
51.9726.026.1320.6739.462.8943.94
Table 3. Average consumption time of different algorithms (in seconds).
Table 3. Average consumption time of different algorithms (in seconds).
SequenceLCMMPCMRLCMTLLCMDNGMNTFRAProposed
10.15160.15651.37610.97920.13341.14130.1373
20.15710.16502.17291.49250.13821.76750.1425
30.16590.17221.56581.08390.14581.74140.1512
40.16680.16931.89311.31460.14368.40820.1485
50.16760.17521.91411.33340.14963.25810.1546
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Liu, L.; Wei, Y.; Wang, Y.; Yao, H.; Chen, D. Using Double-Layer Patch-Based Contrast for Infrared Small Target Detection. Remote Sens. 2023, 15, 3839. https://doi.org/10.3390/rs15153839

AMA Style

Liu L, Wei Y, Wang Y, Yao H, Chen D. Using Double-Layer Patch-Based Contrast for Infrared Small Target Detection. Remote Sensing. 2023; 15(15):3839. https://doi.org/10.3390/rs15153839

Chicago/Turabian Style

Liu, Liping, Yantao Wei, Yue Wang, Huang Yao, and Di Chen. 2023. "Using Double-Layer Patch-Based Contrast for Infrared Small Target Detection" Remote Sensing 15, no. 15: 3839. https://doi.org/10.3390/rs15153839

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop