Next Article in Journal
Assessing within-Field Corn and Soybean Yield Variability from WorldView-3, Planet, Sentinel-2, and Landsat 8 Satellite Imagery
Previous Article in Journal
Remote Sensing Image Retrieval with Gabor-CA-ResNet and Split-Based Deep Feature Transform Network
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

N-YOLO: A SAR Ship Detection Using Noise-Classifying and Complete-Target Extraction

1
Logistics Engineering College, Shanghai Maritime University, Shanghai 201306, China
2
Naval Academy Research Institute, F-29240 Lanvéoc, France
3
School of Medical Information Engineering, Guangzhou University of Chinese Medicine, Guangzhou 510006, China
*
Author to whom correspondence should be addressed.
Remote Sens. 2021, 13(5), 871; https://doi.org/10.3390/rs13050871
Submission received: 9 February 2021 / Revised: 21 February 2021 / Accepted: 22 February 2021 / Published: 26 February 2021

Abstract

:
High-resolution images provided by synthetic aperture radar (SAR) play an increasingly important role in the field of ship detection. Numerous algorithms have been so far proposed and relative competitive results have been achieved in detecting different targets. However, ship detection using SAR images is still challenging because these images are still affected by different degrees of noise while inshore ships are affected by shore image contrasts. To solve these problems, this paper introduces a ship detection method called N-YOLO, which based on You Only Look Once (YOLO). The N-YOLO includes a noise level classifier (NLC), a SAR target potential area extraction module (STPAE) and a YOLOv5-based detection module. First, NLC derives and classifies the noise level of SAR images. Secondly, the STPAE module is composed by a CA-CFAR and expansion operation, which is used to extract the complete region of potential targets. Thirdly, the YOLOv5-based detection module combines the potential target area with the original image to get a new image. To evaluate the effectiveness of the N-YOLO, experiments are conducted using a reference GaoFen-3 dataset. The detection results show that competitive performance has been achieved by N-YOLO in comparison with several CNN-based algorithms.

1. Introduction

Synthetic aperture radar (SAR) is an active side-looking radar that can overcome weather interference and provide high-resolution images. SAR images have been then considered more suitable for ship detection than optical images. SAR ship detection is an has important application in the field of marine surveillance and has received much attention recently [1,2].
In recent years, increasing scholars began to study SAR ship recognition method based on neural networks. Some scholars use the two-stage method to detect ships. Cui et al. [3] proposed a dense attention pyramid network to detect multiscale ships, and Lin et al. [4] also proposed a squeeze and excitation Faster R-CNN [5] to improve detection accuracy. Zhao et al. [6] applied fast region convolutional neural network (R-CNN) [7] to ship detection in synthetic aperture radar (SAR) image. These two-stage methods can often achieve higher detection accuracy, but their detection computational speed is often slower than that of one-stage methods. Therefore, in order to ensure the real-time effect of recognition, some scholars use the one-stage method to inspect ships. Wei et al. [8] designed a high-resolution SAR ship detection network based on HR-SDNet. Wang et al. [9] applied transfer learning based on SSD [10] to improve accuracy. Wang et al. [11] proposed an RetinaNet-based [12] detection for the ship in GaoFen-3 images. Mao et al. [13] firstly used a simplified U-Net to extract features and proposed an anchor-free SAR ship detection framework. These one-stage methods have faster detection speed, but they are not effective in detecting small targets (the small target refers to the kind of target with a very small proportion in the image area). You Only Look Once (YOLO) [14] is also a one-stage method with fast recognition speed, and the latest version of YOLOv5 has made targeted improvements to the problem of different target sizes. Therefore, and in order to obtain faster recognition speed, this paper proposes a SAR ship recognition method based on YOLOv5.
Compared with offshore ships, the detection of inshore ships can be more challenging. Although the pixel value of a ship is much higher than that of the ocean, its texture features and gray level are very similar to those of coastal and shore buildings. Recently, most of the related work on inshore ship detection is aimed at optical remote sensing images. Xu et al. [15] present a detection method based on robust invariant generalized Hough transform to detect and segment out the inshore ship using high-resolution remote sensing imagery. Lin et al. [16] present a fully convolutional network, the shape and context information are utilized to detect the inshore ship. Liu et al. [17] used Harris corner detection for ship foredeck detection. These methods carry out inshore detection based on the extraction of contour and edge. However, when it comes to the SAR image, it is hard to extract the contour and edge feature accurately due to the existence of intrinsic multiplicative speckle noise. Zhai et al. [18] proposed a inshore ship recognition method based on a super pixel generation algorithm, including salient region detection and final false alarm elimination mechanism. This method can achieve better detection results, but the overall structure is slightly complex, and there is still room for real-time improvement. Cui et al. [19] preprocessed a SAR image by using similar pixels according to the different scattering mechanisms of shore and ship, and finally used threshold processing and morphological filtering. This method has a certain effect on the recognition of inshore ships, but the calculation is a little heavy, and the final threshold processing and filtering will lose small target ships. Fu et al. [20] proposed a ship detection method based on FBR-Net, Cui et al. [21] proposed a ship detection method based on CenterNet, both of which are anchor-free detection methods to reduce the influence of background environment. This method can reduce the interference of background environment to ship detection to a certain extent, but it depends on the accuracy of regression. Because the basic candidate frame is abandoned, the regression error may be very large.
When considering SAR images, the existence of noise is always a problem faced by ship recognition, and a large amount of noise interferes with ship recognition. CFAR is a commonly used preprocessing method for ship recognition in SAR images, which is used to overcome the interference of background clutter and noise. [22] Common CFAR detection algorithms in recent years include the cell averaging CFAR (CA-CFAR) [23], greatest of CFAR (GO-CFAR) [24], smallest of CFAR (SO-CFAR) [25] and the order statistic CFAR (OS-CFAR) [26]. The CA-CFAR detector is used in the homogeneous clutter background, and the commonly used two-parameter CFAR algorithm is based on the CA-CFAR detector using normal distribution. The GO-CFAR detector and the SO-CFAR detector are proposed to solve the clutter edge. The OS-CFAR detector is designed based on sort processing technique in digital image processing, which has a good performance with the strong interfering target presence. However, they cannot be used alone to deal with complex and changeable background situations. Some scholars use other filters for experiments. Liu et al. [27] proposed a ship detection method based on whitening filter, which can improve the contrast between ship and background clutter, thus improving the accuracy of ship recognition. However, when the speckle is very bright and close to the ship, this method cannot separate the ship from the speckle. Liu et al. [28] proposed a method based on adaptive bandwidth for ship recognition, which can obtain small bandwidth in the ship area and large bandwidth in the background area, thus smoothing the background of the image. However, the premise of this method is to extract the target area of the ship by using local mean, and the extraction work will be affected when the ship inshore or has obvious speckle. Other researchers have chosen a notch filter to filter SAR [29,30,31], because notch filter can deal with multiple interference components or periodic noises at the same time. However, one of the most important parameters of notch filter is to set the domain size with the same weight. If this parameter is too small, it is not conducive to noise equalization in a wider range. If this parameter is too large, the image details cannot be obtained. For SAR images, there are great differences in noise distribution level and noise types, so it is almost impossible to set a parameter to be applicable to all images to obtain common and good results. Because the noise levels and types of SAR images are quite different, this kind of filtering method using filters has the same disadvantages when processing SAR images, that is, it cannot deal with the noise in all images well.
The research developed in this paper introduces a new SAR ship detection method so-called N-YOLO, which is based on the classification of noise level and the processing of noise. It consists of three parts, the first one is a noise level classifier (NLC), the second one is the SAR target potential area extraction (STPAE) module, the third one is the identification module based on YOLOv5. By applying the NLC classifier, images are divided into three levels according to the level of noise and sent to different modules. Images affected by high-level noise are sent to YOLOv5 for detection, and other images are sent to STPAE module. In STPAE module, CA-CFAR is used to detect the preliminary target area in order to extract the potential target area. In order to prevent some dark pixels on the target from being missed by CA-CFAR, the expansion operation is used to fill and expand the target area acquired by CA-CFAR. In YOLOv5-based recognition module, firstly, the image extracted by STPAE module is combined with the original image to obtain a new image. In the new image, there are fewer noise and the ship and coast are highlighted, thus reducing the impact of coast and noise on ships. The new image is then sent to YOLOv5 for recognition. To evaluate the performance of N-YOLO, we conducted several experiments on the GaoFen-3 dataset, in which the images were taken by GaoFen-3 satellite in China. The detection results show that our method is efficient for detecting multiscale ships in SAR images, compared with several CNN-based methods, e.g., YOLOv5 and G-YOLOv5. The major contributions of this article are summarized as follows:
1) A novel detection method called N-YOLO for detecting ships in SAR images.
2) A three-step framework that first contains a NLC module to distinguish images with different noise levels. Secondly, a STPAE module to extract the complete potential target area, and thirdly, a module based on YOLOv5 to identify the ship from the image with highlighted target and less noise.
3) Experiments on the reference GaoFen-3 dataset demonstrate that the called N-YOLO detects ships with competitive results in comparison with some classical and special CNN-based methods.

2. Methods

Let us successively introduce the three components of our N-YOLO approach, that is, the NLC module, STPAE module and YOLOV5-based target discrimination.
The architecture of N-YOLO is shown in Figure 1. The influence of noise on SAR images varies greatly. The objective of the NLC module is to classify the noise level. If the image is affected by medium-level noise or low-level noise, the original image is sent using path1 to two processes. On the one hand, the image is sent to the STPAE module in which the image is prescreened with CA-CFAR, and then the whole potential target area is obtained by dilation operation. On the other hand, the other branch retains and outputs the original image. Images obtained from the two branches are then combined. If the pixel value of a given position on the two images is not null, the pixel value of this point on the combined image will be assigned as 1, otherwise, it will be assigned as null. The combined image will be sent to YOLOv5 network for ship detection. If the image is affected by high-level noise, it will be sent to YOLOv5 for detection through path2.

2.1. Classify the Noise Level

When considering the GaoFen-3 dataset, images are affected by different levels and kinds of noises. Among all kinds of noises, salt and pepper noise is the most common and has the greatest influence on ship identification. Salt and pepper noise, also known as impulse noise, which randomly changes some pixel values, denotes a noise produced by image sensor, transmission channel and decoding processing. In order to better deal with the influence of salt and pepper noise, we divided the noise into three grades according to its influence. The average pixel value is calculated as follows:
V = i = 1 n j = 1 n v i j n 2 ,
in which V is the average pixel value of the whole image, v i j is the pixel value at coordinates i,j in the picture and n 2 is the total number of pixels in the image.
In order to improve ship detection affected by high-level noise, we introduced an NLC module to classify and process images, as shown in Figure 2. For images affected by low-level noise and medium-level noise, they are sent to the STPAE module for processing. Images affected by high-level noise are sent to YOLOv5 for detection.
The threshold value T is selected by an empirical method. According to the images affected by different noise levels and the results obtained by CA-CFAR processing, we applied an empirical method to obtain the interval of different noise levels. We set the average pixel value range of images affected by low-level noise to be [ 0 , 30 ) . Accordingly, the average pixel values of the images affected by medium-level noise and those affected by high-level noise are [ 30 , 80 ] and ( 80 , 255 ] , respectively. Therefore, we set the threshold T to 80. If the threshold is higher than 80, some images affected by high-level noise will be sent to STPAE, which can affect the overall training results and improve the missed detection rate. If the threshold is lower than 80, some images affected by medium-level noise cannot remove noise interference, and some images affected by shore interference cannot remove shore interference.

2.1.1. Low-Level Noise

The images affected by low-level noise are shown in Figure 3. This kind of image has less noise and sparse distribution, which has little influence on the ship recognition task. The average pixel value range of such images is less than 30.
The average pixel values of each image in Figure 3 are listed in Table 1. It can be seen from Table 1 that the average pixel values of these four images are all less than 30, so they all belong to images affected by low-level noise. There are uniformly distributed salt and pepper noises in these four images, but the noise influence is slight, which hardly affects ship identification.

2.1.2. Medium-Level Noise

The images affected by medium-level noise are shown in Figure 4. The noise density of this kind of image is not too large and the distribution is not too dense, which will have some influence on the ship recognition task. The average pixel value of this kind of image is between 30 and 80.
The average pixel values of each image in Figure 4 are listed in Table 2. It can be seen from Table 2 that the average pixel values of these four images are between 30 and 80, so they all belong to images affected by medium-level noise. There are uniformly distributed and dense salt and pepper noises in this kind of images, which will have some influence on ship recognition. However, the potential target region extraction module and the YOLOv5-based recognition module can filter out the noise and improve the recognition accuracy.

2.1.3. High-Level Noise

This kind of picture shown in Figure 5 is disturbed by severe noise, and the noise in this kind of picture is very dense and uniform, which brings great challenges to ship recognition. The average pixel value of this kind of image is greater than 80.
The average pixel values of each image in Figure 5 are listed in Table 3. It can be seen from Table 3 that the average pixel values of these four images are all greater than 80, so they all belong to images affected by high-level noise. This kind of picture is greatly affected by noise, and if the potential target extraction module and the recognition module based on YOLOv5 are directly used to recognize it, the effect is not good; not only is the rate of missing detection high, but also the training effect is poor.

2.2. Extract the Complete Target Area

In order to extract the complete target area from SAR images, this paper introduced a STPAE module, which consists of CA-CFAR and dilation operation.
In SAR images, the gray intensity of ship is higher than that of surrounding sea clutter. CA-CFAR can generate a local threshold value to detect bright pixels via a sliding window. CA-CFAR divides the local area into three windows: center region of interest’s (ROI) window, the guard window and the background clutter’s window, as shown in Figure 6.
CA-CFAR first calculates the average pixel value of the region of interest ( μ R O I ) and the average pixel value of clutter ( μ c ), and then multiplies the average value of clutter by a coefficient α . The obtained value is the adaptive threshold T. Finally, the μ R O I are compared with the threshold T. If the μ R O I are greater than the threshold T, the ROI pixels are marked as bright pixel in an output binary image J, otherwise it is marked as a dark pixel. Assuming that the dimensions of the input SAR image I and the output binary image J are both X × Y, where x = { 0 , 1 , , X 1 } , y = { 0 , 1 , , Y 1 } .The I and J can be defined as
I = { { I ( x , y ) } x = 0 x = X 1 } y = 0 y = Y 1 = [ I ( 0 , 0 ) I ( 0 , Y 1 ) I ( 1 , 0 ) I ( 1 , Y 1 ) I ( X 1 , 0 ) I ( X 1 , Y 1 ) ] ,
J ( I , T ) = { { J ( I , x , y , T ) } x = 0 x = X 1 } y = 0 y = Y 1 .
The CA-CFAR binary pixel J ( I , x , y , T ) can be calculated with
J ( I , x , y , T ) = f ( x ) = {     t r u e ,       i f μ R O I ( x , y ) α μ c = μ R O I ( x , y ) T 1 f a l s e ,     o t h e r w i s e ,
The SAR ship images for which J ( I , x , y , T ) = t r u e will be extracted and sent to the next stage for expansion operation. Using the prescreening proposed method can greatly reduce the workload of subsequent recognition work, maintain a constant false alarm rate. At the same time, it will not miss all possible ships in the image.
The flow chart of STPAE module is shown in Figure 7. After we sent the original SAR image, we first calculate the adaptive threshold when the sliding window traverses each point on the image. The adaptive threshold can be defined as
t h r e s h o l d = z × α ,
where z is the average value of surrounding pixels and α is the adaptive coefficient. The size of α depends on the size of the clutter window. Then, compare the pixel value of each point with its adaptive threshold. If the pixel value of this point is greater than its adaptive threshold, assign 1 to the corresponding position of the prescreened picture; otherwise, assign 0 to the corresponding position of the prescreened picture. Next, the prescreened pictures are sent to the expansion operation. Through the expansion operation, the highlighted pixels are expanded around, thus the potential target areas extracted in the previous step are filled and expanded, to avoid the partial areas of some targets being lost by the previous operation due to the low pixel value. Finally, the obtained image covering the complete target area is sent to the next stage.

2.3. Ship Identification Based on YOLOv5

In the recognition stage, firstly, the extracted image of potential target area is combined with the original image, and the preprocessed image with bright target and less noise points is obtained. Compare the pixels in the same position on the two images. If the pixels in the original image are greater than the threshold value T and the pixels in the image obtained by STPAE module are greater than 0, then the point in the new image is assigned 1, otherwise, the point is assigned 0.
The process of combining the above two images is shown in Figure 8. If two conditions are met, that is, the pixel value in the original image is greater than the threshold value T c , and the pixel value at the corresponding position in the extracted potential target image is 1, then the pixel value of this point in the obtained new image is 1, as shown by point 2 in Figure 8. Otherwise, even if one of the conditions is met, the pixel value of this point in the new image will be null. As shown in point 1 in Figure 8, the pixel value in the original image is greater than the threshold value t, but the pixel value of the corresponding position in the extracted potential target image is null, so the pixel value of this point in the new image is set to null. By analogy, we can get a new image combined with the above two images. Compared with the original image, most of the noise is filtered out and the target is highlighted and enhanced. Finally, the new image will be sent to YOLOv5 for ship identification.

3. Experiments

This section the experiments evaluation of the function of the NLC module and the impact of the whole method through some experiments.

3.1. Experimental Datasets

We carried out a series of experiments on GaoFen-3 dataset to verify the proposed YOLOv5-based ship detection method. Some samples of inshore ships and ships in images affected by noise are shown in Figure 9.
12,000 images extracted from the GaoFen-3 dataset are randomly divided into two parts, i.e., a training set and a testing set, with the proportion of 6:1. All experiments are implemented using a TensorFlow framework on windows with a Nvidia Quadro p5000 graphics card support.
Some images from the GaoFen-3 dataset have three channels, while others have only one channel, so the experiment uses one channel for all images.

3.2. Evaluation Criterions

Experimental results are shown in Table 4 and Table 5 respectively. Some indices are used to evaluate the experimental results of the different methods, e.g., recall rate, precision rate, F score (F1), and average precision (AP). The following equations define these indices:
p r e c i s i o n   r a t e ( p ) = T P T P + F P ,
R e c a l l   r a t e ( R ) = T P T P + F N ,
F 1 = 2 P R P + R ,
A P = 0 0 P ( R ) d R ,
where TP, FP, and FN represent true positives, false positives, and false negatives, respectively. Precision rate refers to the proportion of ground truth ships predicted by networks in all predictions. Recall rate refers to the proportion of ground truth ships predicted by networks in all ground truth ships. F1 is a comprehensive indicator used for judging the performance of different networks by combining precision rate with recall rate together. AP describes the area under precision–recall (PR) curves and it also illustrates comprehensive performance of different methods.

3.3. Noise Level Classifier Impact

When STPAE module processes images are affected by different noise levels, the effects are quite different.
The images affected by low-level noise are shown in Figure 10. This kind of image has less noise and sparse distribution, which has little influence on the ship recognition task. With our method, these tiny noises will be filtered out well. The images affected by medium-level noise are shown in Figure 11. The noise distribution in this kind of image is dense and uniform, which has certain influence on ship recognition. Using our method, the noise in this kind of images will be removed in large quantities, thus greatly improving the accuracy of ship recognition. The kind of image shown in Figure 12 is disturbed by high-level noise, and the noise in this kind of image is very dense and uniform, which brings great challenges to ship recognition. The STPAE is not efficient when dealing with such images. After combining the extracted image of potential target area with the original image, an image with noise in the center and almost no ship will be obtained.
It can be seen from Figure 10 and Figure 11 that if the noise interference in the original image is not severe, the proposed method can get better results; otherwise, if the noise interference in the original image is too serious, the target will be lost in the new image. The emergence of this situation not only reduces the recall rate and causes many missed inspections, but also has a negative impact on the training in the process of sending to YOLOv5 training, resulting in low overall recognition accuracy. Therefore, the NLC module can be applied to classify SAR images according to the noise level. Images affected by high-level noise are sent to YOLOv5 for detection, while other images are sent to STPAE for processing and then sent to YOLOv5 for recognition.
In order to verify the effectiveness of the NLC module, a set of comparative experiments are carried out to verify it. We compare the recognition with YOLOv5, the recognition with STPAE and the YOLOv5-based recognition module without NLC module classification, and the recognition of images affected by different noise levels with the N-YOLO. The test results are shown in Table 4.

3.4. Comparison with Other CNN-Based Methods

In order to prove the filtering effect of N-YOLO, we used the typical filtering method (Gaussian filtering) as preprocessing before YOLOv5 detection. As shown in Table 5, the experiments we conducted using our method, YOLOv5, and G-YOLOv5 respectively. Figure 13 shows the PR curves of the different CNN-based methods tested on several ships

4. Discussion

It can be concluded from Table 4 that the highest accuracy is the result of training directly sent to STPAE and YOLOv5-based detection module without passing through NLC module. Compared with the training result directly using YOLOv5, its precision is 7% higher, but its recall rate is 12.75% lower. This is because the images affected by high-level noise will produce a mass of noise in the middle of the image and lose the target after being sent to STPAE, which will not only improve the missed detection rate, but also affect the overall training results in the training process. In contrast, using the method proposed in this paper (classified by NLC module), the recall rate is greatly improved. The recall rate of images affected by high-level noise after training is as high as 92.36%, which is very close to 92.65% of the YOLOv5, and the recall rate of images affected by low-level noise after training also reaches 86.42%. Compared with the YOLOv5, the accuracy of the proposed method is greatly improved. Among them, the accuracy of images affected by medium and low-level noise after training reaches 76.5%, which is 5.7% higher than that of the YOLOv5. The accuracy of images affected by high level noise after training is 67.46%, which is 3.34% lower than that of the first method. Among the 12,000 images in the training set, there are 1744 images affected by high level noise and 10256 images affected by medium and low-level noise. According to the ratio of the two, N-YOLO has improved the accuracy and decreased the false detection rate.
Experiments show that using NLC can not only improve the detection accuracy, but also increase the missed detection rate less, thus improving the overall detection performance. At the same time, images affected by different noise levels can be prevented from interfering with each other in the training process.
It can be seen from Table 5 that the precision of the last two methods has been improved to varying degrees compared with the first method, and the precision of the method proposed in this paper has been improved the most. In terms of recall rate, the first two methods are almost the same and superior to the latter two methods, while YOLOv5 is the best. Because the latter two methods preprocess the images, the details of small targets are destroyed, resulting in missing detection.
Figure 13 shows the PR curves of the CNN-based methods. The navy blue line is the PR curve obtained by using YOLOv5 training. The light blue line is the PR curve obtained by non-NLC. The green line and yellow line are PR curves of images affected by high-level noise and medium/low-level noise, respectively, which are trained by our method. The red line is the PR curve obtained from the contrast experiment, which is first filtered by Gaussian and then sent to YOLOv5 for training.
The PR curve of non-NLC comes to a sharp decrease with an increase in recall rate compared with YOLOv5. It might be because of the insufficient characteristics extracted by non-NLC, which leads to weak discrimination for ships. Furthermore, the PR curve of non-NLC is lower than those of other methods when recall rate is higher than about 0.5. In addition, the PR curve of h-level is higher than that of others when recall rate is greater than 0.9.
Figure 14 shows the detection results of the different methods as applied to four different ships situations. These four situations are as follows: offshore ships affected by medium/low-level noise (the first row of Figure 14), offshore ships affected by high-level noise (the second row of Figure 14), inshore ships affected by high-level noise (the third row of Figure 14), and inshore ships affected by medium/low-level noise (the fourth row of Figure 14). It can be seen from the first line of Figure 14 that the effects of the four detection methods are almost the same for the first situation. Compared with the original method, the detection accuracy of the latter two methods is slightly improved, among which G-YOLOv5 is improved by 1%, N-YOLO is improved by 2%. For the second situation, compared with the original method, the detection accuracy of G-YOLOv5 is equal to the original method, and N-YOLO is improved by 4%. For the third situation, compared with the original method, the detection accuracy of G-YOLOv5 is reduced to a certain extent, and G-YOLOv5 also has a false detection. In this picture, the detection accuracy of N-YOLO is improved by 7% on average compared with the original method. For the last situation, G-YOLOV5 not only failed to reduce the noise interference, but also the target became blurred, so the detection accuracy dropped significantly and there were four missing detections. For this image, the detection accuracy of N-YOLO is slightly improved compared with the original method. Among them, the detection accuracy of the ship in the lower left corner increased by 15%. However, although N-YOLO did not miss the detection, it mistakenly identified a ship in the lower right corner.

5. Conclusions

The research developed in this paper introduced a new ship detection method of the maritime environment in SAR imagery, consisting of NLC module, STPAE module, and YOLOv5-based discrimination. The NLC module classifies the images according to the noise level, and the images affected by high-level noise are sent to YOLOv5 for detection, while the rest of the images are sent to the STPAE module. The STPAE module uses CA-CFAR and expansion operation to extract the target potential region and expand and fill it. In the recognition stage, firstly, the extracted image of potential target area is combined with the original image, and then the image with bright target and less noise is obtained, which is sent to YOLOv5 for recognition. Compared with sending pictures directly to classical target detection networks (such as YOLOv5), the N-YOLO has better detection performance. Experiments show that N-YOLO has a good effect on ship recognition in SAR images. The proposed method can reduce the interference of noise and shore to ship identification, and has a wide application prospect in the field of marine monitoring. The N-YOLO still partially damages the ship edge information, and the future work will focus on better protecting the edge information.

Author Contributions

Conceptualization, G.T. and Y.Z.; methodology, G.T.; software, G.T.; validation, G.T. and Y.Z.; formal analysis, G.T.; investigation, G.T.; resources, G.T.; data curation, G.T.; writing—original draft preparation, G.T. and C.C.; writing—review and editing, G.T and C.C.; visualization, Y.Z.; supervision, G.T.; project administration, G.T.; funding acquisition, S.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded in part by the GuangDong Basic and Applied Basic Research Foundation under Grant 2020A1515110503, in part by the Medical Science and Technology Research Foundation of Guangdong under Grant A2020334, in part by the Young Talent Training Project of Guangzhou University of Chinese Medicine under Grant QNYC20190110.

Data Availability Statement

Restrictions apply to the availability of these data. Data was obtained from National Satellite Ocean Application Service (NSOAS) and are available from website of http://dds.nsoas.org.cn/ (accessed on 25 February 2021).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Stasolla, M.; Mallorqui, J.J.; Margarit, G.; Santamaria, C.; Walker, N. A Comparative Study of Operational Vessel Detectors for Maritime Surveillance Using Satellite-Borne Synthetic Aperture Radar. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2016, 9, 2687–2701. [Google Scholar] [CrossRef] [Green Version]
  2. Liu, W.; Ma, L.; Chen, H. Arbitrary-Oriented Ship Detection Framework in Optical Remote-Sensing Images. IEEE Geosci. Remote Sens. Lett. 2018, 15, 937–941. [Google Scholar] [CrossRef]
  3. Cui, Z.; Li, Q.; Cao, Z.; Liu, N. Dense Attention Pyramid Networks for Multi-Scale Ship Detection in SAR Images. IEEE Trans. Geosci. Remote Sens. 2019, 57, 8983–8997. [Google Scholar] [CrossRef]
  4. Lin, Z.; Ji, K.; Leng, X.; Kuang, G. Squeeze and Excitation Rank Faster R-CNN for Ship Detection in SAR Images. IEEE Geosci. Remote Sens. Lett. 2019, 16, 751–755. [Google Scholar] [CrossRef]
  5. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1137–1149. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Zhao, Y.; Zhao, L.; Xiong, B.; Kuang, G. Attention Receptive Pyramid Network for Ship Detection in SAR Images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 13, 2738–2756. [Google Scholar] [CrossRef]
  7. Girshick, R.J.C.S. Fast R-CNN. arXiv 2015, arXiv:1504.08083. [Google Scholar]
  8. Wei, S.; Su, H.; Ming, J.; Wang, C.; Yan, M.; Kumar, D.; Shi, J.; Zhang, X.J.R.S. Precise and robust ship detection for high-resolution SAR imagery based on HR-SDNet. Remote Sens. 2020, 12, 167. [Google Scholar] [CrossRef] [Green Version]
  9. Wang, Y.; Wang, C.; Zhang, H. Combining single shot multibox detector with transfer learning for ship detection using Sentinel-1 images. In Proceedings of the 2017 SAR in Big Data Era: Models, Methods and Applications (BIGSARDATA), Beijing, China, 13–14 November 2017; pp. 1–4. [Google Scholar]
  10. Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Berg, A.C. SSD: Single Shot MultiBox Detector. In Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, 11–14 October 2016. [Google Scholar]
  11. Wang, Y.; Wang, C.; Zhang, H.; Dong, Y.; Wei, S.J.R.S. Automatic Ship Detection Based on RetinaNet Using Multi-Resolution Gaofen-3 Imagery. Remote Sens. 2019, 11, 531. [Google Scholar] [CrossRef] [Green Version]
  12. Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal Loss for Dense Object Detection. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2999–3007. [Google Scholar]
  13. Mao, Y.; Yang, Y.; Ma, Z.; Li, M.; Su, H.; Zhang, J. Efficient Low-Cost Ship Detection for SAR Imagery Based on Simplified U-Net. IEEE Access 2020, 8, 69742–69753. [Google Scholar] [CrossRef]
  14. Redmon, J.; Divvala, S.; Girshick, R.B.; Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. arXiv 2016, arXiv:1506.02640. [Google Scholar]
  15. Xu, J.; Sun, X.; Zhang, D.; Fu, K. Automatic Detection of Inshore Ships in High-Resolution Remote Sensing Images Using Robust Invariant Generalized Hough Transform. IEEE Geosci. Remote Sens. Lett. 2014, 11, 2070–2074. [Google Scholar] [CrossRef]
  16. Lin, H.; Shi, Z.; Zou, Z. Fully Convolutional Network With Task Partitioning for Inshore Ship Detection in Optical Remote Sensing Images. IEEE Geosci. Remote Sens. Lett. 2017, 14, 1665–1669. [Google Scholar] [CrossRef]
  17. Liu, G.; Zhang, Y.; Zheng, X.; Sun, X.; Fu, K.; Wang, H. A New Method on Inshore Ship Detection in High-Resolution Satellite Images Using Shape and Context Information. IEEE Geosci. Remote Sens. Lett. 2014, 11, 617–621. [Google Scholar] [CrossRef]
  18. Zhai, L.; Li, Y.; Su, Y. Inshore Ship Detection via Saliency and Context Information in High-Resolution SAR Images. IEEE Geosci. Remote Sens. Lett. 2016, 13, 1870–1874. [Google Scholar] [CrossRef]
  19. Cui, X.; Su, Y.; Chen, S. A Saliency Detector for Polarimetric SAR Ship Detection Using Similarity Test. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2019, 12, 3423–3433. [Google Scholar] [CrossRef]
  20. Fu, J.; Sun, X.; Wang, Z.; Fu, K. An Anchor-Free Method Based on Feature Balancing and Refinement Network for Multiscale Ship Detection in SAR Images. IEEE Trans. Geosci. Remote Sens. 2020, 59, 1331–1344. [Google Scholar] [CrossRef]
  21. Cui, Z.; Wang, X.; Liu, N.; Cao, Z.; Yang, J. Ship Detection in Large-Scale SAR Images Via Spatial Shuffle-Group Enhance Attention. IEEE Trans. Geosci. Remote Sens. 2021, 59, 379–391. [Google Scholar] [CrossRef]
  22. Ai, J.; Tian, R.; Luo, Q.; Jin, J.; Tang, B. Multi-Scale Rotation-Invariant Haar-Like Feature Integrated CNN-Based Ship Detection Algorithm of Multiple-Target Environment in SAR Imagery. IEEE Trans. Geosci. Remote Sens. 2019, 57, 10070–10087. [Google Scholar] [CrossRef]
  23. Novak, L.M.; Hesse, S.R. On the performance of order-statistics CFAR detectors. In Proceedings of the 1991 Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers, Pacific Grove, CA, USA, 4–6 November 1991. [Google Scholar]
  24. Bisceglie, M.D.; Galdi, C. CFAR detection of extended objects in high-resolution SAR images. IEEE Trans. Geosci. Remote Sens. 2005, 43, 833–843. [Google Scholar] [CrossRef]
  25. Kuttikkad, S.; Chellappa, R. Non-Gaussian CFAR techniques for target detection in high resolution SAR images. In Proceedings of the Image Processing (ICIP-94), Austin, TX, USA, 13–16 November 1994. [Google Scholar]
  26. Hofele, F.X. An innovative CFAR algorithm. In Proceedings of the International Conference on Radar, Beijing, China, 15–18 October 2001. [Google Scholar]
  27. Liu, T.; Zhang, J.; Gao, G.; Yang, J.; Marino, A. CFAR Ship Detection in Polarimetric Synthetic Aperture Radar Images Based on Whitening Filter. IEEE Trans. Geosci. Remote Sens. 2020, 58, 58–81. [Google Scholar] [CrossRef]
  28. Liu, Z.; Bai, X.; Sun, C.; Zhou, F.; Li, Y. Multi-Modal Ship Target Image Smoothing Based on Adaptive Mean Shift. IEEE Access 2018, 6, 12573–12586. [Google Scholar] [CrossRef]
  29. Gao, G.; Shi, G. CFAR Ship Detection in Nonhomogeneous Sea Clutter Using Polarimetric SAR Data Based on the Notch Filter. IEEE Trans. Geosci. Remote Sens. 2017, 55, 4811–4824. [Google Scholar] [CrossRef]
  30. Gao, G.; Gao, S.; He, J.; Li, G. Ship Detection Using Compact Polarimetric SAR Based on the Notch Filter. IEEE Trans. Geosci. Remote Sens. 2018, 56, 5380–5393. [Google Scholar] [CrossRef]
  31. Marino, A.; Hajnsek, I. Ship Detection With TanDEM-X Data Extending the Polarimetric Notch Filter. IEEE Geosci. Remote Sens. Lett. 2015, 12, 2160–2164. [Google Scholar] [CrossRef] [Green Version]
Figure 1. The architecture of N-YOLO.
Figure 1. The architecture of N-YOLO.
Remotesensing 13 00871 g001
Figure 2. NLC: Classify and process images according to noise level.
Figure 2. NLC: Classify and process images according to noise level.
Remotesensing 13 00871 g002
Figure 3. Images affected by low-level noise. Average pixel value (a) 21.96, (b) 20.28, (c) 25.13, (d) 28.56.
Figure 3. Images affected by low-level noise. Average pixel value (a) 21.96, (b) 20.28, (c) 25.13, (d) 28.56.
Remotesensing 13 00871 g003
Figure 4. Images affected by medium-level noise. Average pixel value (a) 55.01, (b) 68.22, (c) 32.24, (d) 64.59.
Figure 4. Images affected by medium-level noise. Average pixel value (a) 55.01, (b) 68.22, (c) 32.24, (d) 64.59.
Remotesensing 13 00871 g004
Figure 5. Images affected by high-level noise. Average pixel value (a) 127.80, (b) 125.96, (c) 113.14, (d) 122.60.
Figure 5. Images affected by high-level noise. Average pixel value (a) 127.80, (b) 125.96, (c) 113.14, (d) 122.60.
Remotesensing 13 00871 g005
Figure 6. CA-CFAR window configuration. The mean pixel value of the clutter and ROI are calculated as μ c and μ R O I . The guard window is used to prevent the corruption to the μ c .
Figure 6. CA-CFAR window configuration. The mean pixel value of the clutter and ROI are calculated as μ c and μ R O I . The guard window is used to prevent the corruption to the μ c .
Remotesensing 13 00871 g006
Figure 7. Flow chart of the STPAE module.
Figure 7. Flow chart of the STPAE module.
Remotesensing 13 00871 g007
Figure 8. Combine the extracted image of the potential area with the original image.
Figure 8. Combine the extracted image of the potential area with the original image.
Remotesensing 13 00871 g008
Figure 9. Samples of inshore and offshore ships in GaoFen-3 dataset. (a) and (b) show ships in images affected by noise. (c) and (d) show inshore ships where much interference exists caused by shore.
Figure 9. Samples of inshore and offshore ships in GaoFen-3 dataset. (a) and (b) show ships in images affected by noise. (c) and (d) show inshore ships where much interference exists caused by shore.
Remotesensing 13 00871 g009
Figure 10. Under low-level noise interference, the new image generated by the STPAE module and combine operation.
Figure 10. Under low-level noise interference, the new image generated by the STPAE module and combine operation.
Remotesensing 13 00871 g010
Figure 11. Under medium-level noise interference, the new image generated by the STPAE module and combine operation.
Figure 11. Under medium-level noise interference, the new image generated by the STPAE module and combine operation.
Remotesensing 13 00871 g011
Figure 12. Under high-level noise interference, the new image generated by the STPAE module and combine operation.
Figure 12. Under high-level noise interference, the new image generated by the STPAE module and combine operation.
Remotesensing 13 00871 g012
Figure 13. PR curves of CNN-based methods tested on several ships.
Figure 13. PR curves of CNN-based methods tested on several ships.
Remotesensing 13 00871 g013
Figure 14. Visual detection results of CNN-based methods on offshore ships. The first column is the detection result of YOLOv5, the second column is the detection result of YOLOv5 after preprocessing with Gaussian filter, and the third column is the detection result of the N-YOLO.
Figure 14. Visual detection results of CNN-based methods on offshore ships. The first column is the detection result of YOLOv5, the second column is the detection result of YOLOv5 after preprocessing with Gaussian filter, and the third column is the detection result of the N-YOLO.
Remotesensing 13 00871 g014
Table 1. Average pixel value of each image in Figure 3.
Table 1. Average pixel value of each image in Figure 3.
Figure 3abcd
Average pixel value21.9620.2825.1328.56
Table 2. Average pixel value of each image in Figure 4.
Table 2. Average pixel value of each image in Figure 4.
Figure 4abcd
Average pixel value55.0168.2232.2464.59
Table 3. Average pixel value of each image in Figure 5.
Table 3. Average pixel value of each image in Figure 5.
Figure 5abcd
Average pixel value127.80125.96113.14122.60
Table 4. Comparison of experimental results of several methods.
Table 4. Comparison of experimental results of several methods.
MethodPrecisionRecallF1AP
YOLOv50.70800.92650.8030.9097
Non-NLC0.77800.7990.78830.7985
h-level0.67460.92360.77970.9158
m/l-level0.7650.86420.81160.8553
Table 5. Detection Results of CNN-Based Methods.
Table 5. Detection Results of CNN-Based Methods.
MethodPrecisionRecallF1AP
YOLOv50.70800.92650.8030.9097
h-level0.67460.92360.77970.9158
m/l-level0.7650.86420.81160.8553
G-YOLOv50.72750.90690.80740.884
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Tang, G.; Zhuge, Y.; Claramunt, C.; Men, S. N-YOLO: A SAR Ship Detection Using Noise-Classifying and Complete-Target Extraction. Remote Sens. 2021, 13, 871. https://doi.org/10.3390/rs13050871

AMA Style

Tang G, Zhuge Y, Claramunt C, Men S. N-YOLO: A SAR Ship Detection Using Noise-Classifying and Complete-Target Extraction. Remote Sensing. 2021; 13(5):871. https://doi.org/10.3390/rs13050871

Chicago/Turabian Style

Tang, Gang, Yichao Zhuge, Christophe Claramunt, and Shaoyang Men. 2021. "N-YOLO: A SAR Ship Detection Using Noise-Classifying and Complete-Target Extraction" Remote Sensing 13, no. 5: 871. https://doi.org/10.3390/rs13050871

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop