An Improved Kuhn Munkres Algorithm for Ship Matching in Optical Satellite Images

Ship matching based on multiple satellite images can effectively grasp the trajectories of the ship with long time intervals, providing technical support for marine regulation and surveillance. Ship detection and ship association are two steps for ship matching based on satellite images. However, traditional unsupervised moving object detection methods only apply to the scenes where the camera is stationary. In addition, it cannot segment the ships from the background, interfering with the data association step. Most data association algorithms rely on the target's location information and are not suitable for ship matching based on satellite remote sensing images with long time intervals, because the ship's position has significantly changed. Therefore, an unsupervised automatic ship matching method is proposed in this article. In the ship detection part, the ship is segmented based on the proposed preprocessing algorithm and the GrabCut algorithm to remove the interference caused by background movement; in the data association part, the Kuhn Munkres algorithm is improved by similarity comparison and iteration to find the optimal matching of ships. Compared with other matching methods, the algorithm proposed in this article is not affected by the moving background, which performs a better ship matching accuracy with a lower time cost.


I. INTRODUCTION
S HIP monitoring and surveillance is an essential tool to ensure the daily operation of sea areas and maintain maritime security interests [1], so it is increasingly valued by various countries [2], [3]. Most of the existing ships' positioning and tracking methods are based on video data, which can only obtain short trajectories of the moving ship within a short time due to the slow movement of ships. Unlike tracking based on video data, satellite images are taken at longer intervals. On the basis of this, ship matching can locate and associate multiship among multiview remote sensing images with long time intervals, which is of great significance to the tracking and monitoring of maritime ships. With the development of satellite networking, it has become possible to achieve ship matching under large spatial and temporal spans by using multiple satellite remote sensing images. Manuscript  Similar to multiobject tracking methods, the primary methods of multiship matching are to use the matching-by-detection framework, which simplifies the ship matching to the ship detection and ship association.
Depending on the relative positions of the camera and the background, moving object detection can be divided into two categories: the camera is stationary relative to the background (detection under stationary background); the camera is moving relative to the background (detection under moving background) [4]. Due to the difference in satellite movement and shooting angle, ship detection based on optical satellite images belongs to detection under moving background. Common unsupervised object detection methods include background subtraction methods [5], [6], optical flow methods [7], [8], probabilistic methods [9], [10], [11], etc. While these methods can quickly locate the position of a moving object, they are not suitable for ship detection based on the satellite imagery, as they are only effective against a stationary background. The commonly used supervised object detection method is deep learning [12], [13], [14]. Although it is suitable for detection under moving background, it requires a large number of samples for training and takes a long time. In addition, segmenting the ships from the background is essential to help the subsequent data association step. However, the above method cannot split the ship, so it cannot remove the background interference in the data association step. Therefore, developing an efficient unsupervised ship detection method that can segment ships from the background is key to improving ship detection efficiency and increasing the accuracy of data association.
Data association determines whether two images belong to the same ship by the features. Unlike matching techniques based on video or sequence images under stationary background, in a moving background, differences in the resolution, spectral information, shooting angles, and sensor parameters of the shooting satellites can lead to differences in the imaging of the ship, such as the shape, color, and texture [15], increasing the difficulty of associate ship data. Therefore, a ship association method suitable for different imaging conditions is the key issue in satellite image-based ship matching. Many data association methods are based on ship position information, such as fitting the track using least squares [16], predicting the predicted position of the ship in the next frame using the Kalman filter and determining the ship trajectory using the intersection over union [17], and using elements such as centroid distance to achieve ship association [18]. However, for satellite images with a long time interval, the ship's position may change greatly, making it impossible to use the location information for association. For this problem, ship matching can be considered using extracted feature points, such as using the SIFT algorithm to establish the association of ships between different frames [19], [20]. However, when the resolution of optical remote sensing images is low, the imaging of ships becomes blurred. There need to be more feature points to be found for matching, which reduces the accuracy of the feature point matching algorithm. Kuhn Munkres (KM), a classical data association algorithm in target tracking [21], [22], makes data association independent of location information and image resolution. So it has been widely applied in pedestrian multitarget tracking [23], [24] and multitarget moving object tracking [25]. Although the KM algorithm can solve the data association problem of multiple objects, compared with other objects, ships in satellite remote sensing images have two characteristics: 1) differences in satellite imaging mechanisms cause the same ships to be imaged differently in different images; 2) when the image resolution is low, the ships are blurred, which tends to cause a large number of ships to be imaged similarly in the same image. These limitations of large intraclass variation and small interclass variation tend to make more values with little difference in the metric matrix, which makes the KM algorithm assign the wrong association objects to the ships.
In summary, there are two limitations in ship matching based on satellite images: first, in ship detection, although the existing deep learning methods can take into account both moving and stationary backgrounds, they also have high time and labor costs, and cannot segment ships from complex backgrounds; second, in terms of data association, the large intraclass differences and small interclass differences of ships make many similar values appear in the metric matrix, which leads to the KM algorithm being prone to incorrect matching.
To overcome the above limitations, this article aims to construct an unsupervised ship matching method based on optical satellite remote sensing images to detect and associate the multiships in a moving background. Specifically, 1) an unsupervised ship detection method is proposed, which first uses the proposed preprocessing algorithm to improve the computational efficiency and accuracy of the GrabCut algorithm to segment ships from the background and then removes false alarms to obtain ship detection results; 2) an improved KM algorithm is proposed for data association among different ships, which adds similarity judgment and iteration to the original KM algorithm to overcome the limitation of many similar values in the metric matrix, and the optimal ship association results are finally obtained.

II. UNSUPERVISED SHIP AUTO-MATCHING METHOD
In this article, an unsupervised ship matching framework is proposed to realize the automated process of ship detection association of optical remote sensing images under moving background, and the overall flow of the method is shown in Fig. 1. The proposed ship matching framework is mainly divided into ship detection and ship association. In the ship detection part, first, the region of interest (ROI) is extracted by the saliency map to reduce the computational effort, then the foreground is segmented using the GrabCut algorithm; finally, the false alarm objects are removed to obtain the ship detection results. In the ship association part, the detected ship targets are first extracted and fused with multiple features, after which similarity is calculated from ship features to construct the metric matrix; and lastly, the ship matching results are generated by the improved KM algorithm.

A. Ship Detection
Because of the long interval between satellite images, the ship's background may change and bring interference to the subsequent ship association. In order to exclude background interference and improve subsequent matching accuracy, the foreground target needs to be segmented in the ship detection part. The whole detection process is divided into three parts: first, extract the ROI where the ship may exist, then segment the foreground in the ROI, and finally, remove the false alarm object.
1) Extract ROIs: First, histogram equalization is performed on the original remote sensing image to enhance the detailed information of small ships in the image. Then, significance detection is performed on the detection results. Next, threshold segmentation and morphological operations are performed on the significance map to obtain the foreground of significant presence in the image. Finally, the land is removed by removing large connected domains to obtain the final binary image of the ROI.
2) Segment the Foreground: Due to the difference in the imaging mechanism of multisource remote sensing images, it will cause changes in the imaging of ships and backgrounds, which increases the difficulty of ship matching. The complex background will also bring interference to ship detection. Therefore, it must be wholly separated from the background to extract the ship features accurately. Separating the ship from the background can effectively remove the interference of complex background and background changes, which is beneficial to the subsequent ship matching. The traditional object detection algorithm will have the problem of segmenting the background and foreground in the detection frame, so the GrabCut algorithm [26] is mainly used in this article to segment foreground targets in the ROI.
The GrabCut algorithm makes use of texture (color) information and boundary (contrast) information in images and can obtain good segmentation results with only a tiny amount of user interaction [27], which is widely used in the field of computer vision, such as foreground segmentation and stereo vision [28]. However, due to the small proportion of ships in the whole remote sensing image, the original GrabCut algorithm could not accurately segment the ships and sea surface. Moreover, due to the large size of the remote sensing image, a large amount of calculation was needed, resulting in a low time efficiency of the algorithm. Therefore, a preprocessing method is proposed to improve the accuracy of the GrabCut algorithm in separating ships and the sea surface. The specific steps are as follows.
1) Expanded image: The upper left corner of the image is defined as the origin point, and black pixels are used to expand the surroundings of the ROI binary image and remote sensing image at the same time. 2) Locating the foreground target: Obtain a foreground coordinate from the expanded binary image of interest, clockwise from the upper left corner: (x 1 , y 1 ), (x 2 , y 1 ), (x 2 , y 2 ), (x 1 , y 2 ). 3) Clipping remote sensing images: Clipping remote sensing images in the range of (x 1 -250, y 1 -250), (x 2 +250, y 1 -250), (x 2 +250, y 2 +250), and (x 1 -250, y 2 +250) for fine segmentation within this range. 4) Segmentation of foreground: A rectangle with (249, 249) as the upper left corner point, (y 2y 1 +2) as the height, and (x 2x 1 +2) as the width is used as the input of the GrabCut algorithm to segment the cropped remote sensing images until all the foregrounds in the ROI are segmented. As shown in Fig. 2, the first line is the images segmented by the original GrabCut algorithm, and the second line is the results after preprocessed algorithm. It can be seen that the preprocessed algorithm has significantly improved the accuracy of the GrabCut algorithm.
3) Remove False Alarm Targets: In addition to ships, the segmentation results also include some false alarm targets. The E-HOG feature of the target can be extracted for ship detection [29], since the shape of the ship is approximately a left-right symmetrical strip, and the shape of the false alarm target is  primarily irregular. We simplify the E-HOG feature to remove false alarm targets.
As shown in Fig. 3, the simplified E-HOG gradient direction is divided into nine intervals. The target is rotated to the horizontal direction for normalization, and then the block's feature vectors are accumulated by direction to highlight the gradient amplitude of the target in each direction. The standardized ship shape is approximately a rectangle with the width greater than the height, so the ship will have the largest gradient amplitude in the vertical direction (the Z4 direction), whereas the gradient amplitude in other directions is small. As shown in Fig. 4, the first row shows various target images, the second row shows the normalized target edge images, and the third row shows the simplified E-HOG feature histogram. There are apparent differences between the gradient histograms of false alarm targets and ship targets. The targets satisfying (1) and (2) simultaneously are defined as ship targets, whereas other targets are defined as false alarm targets where i = 0, 1, 2, 3, 5, 6, 7, 8 denotes the remaining direction intervals except the Z4 direction, V i denotes the gradient amplitude on direction interval i, V 4 denotes the gradient amplitude on direction interval 4, and V i denotes the gradient amplitude's average value on the gradient remaining direction intervals except the Z4 direction.

B. Ship Association
By excluding the interference of the background through the ship detection step, the focus of the association algorithm can be made to focus on the ship itself. The ship association algorithm  proposed in this article is divided into two steps. First, extract ship features and construct a metric matrix based on detection results. Then, input the metric matrix into the improved KM algorithm to complete ship matching, as shown in Fig. 5.

1) Feature Extraction and Similarity Measurement:
If the same ship is regarded as the same kind and different ships are regarded as different classes, due to the difference in imaging machines and other reasons, the ships have the characteristics of significant intraclass differences and minor interclass differences, so it is not easy to accurately represent the ships with only a single feature. Therefore, the shape, color, and local features of ships are extracted and fused for similarity metrics, and the specific features are shown in Table I.
According to the above feature extraction, each ship has three feature vectors corresponding to its shape, color, and local features. The ship feature vectors between the target image and the image to be matched are metricized for similarity using Euclidean distance. Thus, the metric matrix of each feature is constructed and normalized according to the following equation: where D m,n denotes the similarity score between ship m in the first image and ship n in the second image after normalization, D m,n denotes the calculated original Euclidean distance, D max denotes the maximum value in the matrix, and D min denotes the minimum value in the matrix. Then, the shape, color, and local similarity of the proposed ships are fused according to (4) to obtain the metric matrix between the ships of the two images. The closer the similarity score is to 1, the more similar the ships are where Score represents the similarity score between two ships, and D shape , D colour , and D local represent the distance of shape features, color features, and local features between the two ships, respectively. In the metric matrix of the two images, the number of rows is the number of ships in the first image. The number of columns is the number of ships in the second image. The number in the matrix represents the similarity score between the corresponding ships.
2) Improved KM Algorithm: The KM algorithm aims to solve the optimal matching problem of weighted bipartite graphs in data association [21], [22]. As shown in Fig. 6, each vertex of the bipartite graph is given the top label. i and j denote the target numbers to be matched, respectively, and are consecutive integers. Let holds. However, the accuracy of the KM algorithm is primarily influenced by the accuracy of the input metric matrix [30]. Although multiple feature fusions are used to calculate the metric matrix in this article, in practical application, the presence of many similar ships leads to a large number of similar values in the metric matrix, resulting in inaccurate matching results.
To improve the matching accuracy of the KM algorithm, the most critical mission is to compare and judge the similarity values in the metric matrix to find the most matching one with the target ship from multiple similar ships. The similarity value in the metric matrix means that the corresponding ships cannot be distinguished by shape, color, and local features, and this is where pixel-by-pixel matching is a good complement. Pixel-by-pixel matching can compare the similarity of two ships in detail to the maximum extent. Therefore, template matching is introduced into the KM algorithm for ship pixel-by-pixel comparison in this article.
Template matching is a commonly used method for singleobject tracking [31]. When it is used alone for many-to-many object matching, it requires a large number of calculations with low time efficiency. Therefore, adding a similarity calculation step to the KM algorithm by using template matching can both improve the accuracy of data association and overcome the problem of the low time efficiency of template matching. The specific steps of the improved KM algorithm are as follows.
Step 1. Constructing a weighted bipartite graph: The weighted bipartite graph is constructed according to the similarity matrix, and the specific steps are as follows. In the preliminary matching result, the ship a i in the target image matches the ship b j in the image to be matched. Next, the normalized association method is used to calculate the template similarity score S i,j between a i and b j .
In this step, the template similarity is only calculated for the preliminary matching results generated by the KM algorithm. That is, the template similarity is calculated for the ships that are most likely to match each other. This step avoids the disadvantage that template matching needs to traverse all the images to be matched and greatly reduces the computational effort, thus significantly reducing computation time.
Step 3. Generate fine-match results: The similarity matrix AB is updated using (5) for the similarity score S i,j . The KM algorithm is judged again until the number of pending objects is not greater than 1. When the number of pending objects is equal to 1, this pending object is regarded as an object disappearance. When the number of pending objects is zero, the matching ends and the final ship matching result is generated where α and β are the two thresholds for determining the template similarity, and β < α. The overall flow of the algorithm is shown in Table II.

III. EXPERIMENTAL RESULTS AND ANALYSIS
To separately evaluate the performance of the improved KM algorithm and the accuracy of the overall ship matching framework for matching in a moving context, experiments are conducted in Sections III-A and III-B, respectively.
In the ship detection section, the ship detection results are evaluated using precision and recall, and the formulas for precision and recall are as follows: recall = T P/(T P + F N) where T P is the number of detected real ship targets, F P is the number of detected nonship targets, and F N denotes the number of ships that are not detected.
In the data association section, the method of this article is compared with the template matching method, the original KM algorithm, and the game theoretic hypergraph matching (GTHM) [32]. The template matching degree above 0.8 is considered as a successful match. GTHM is a framework for refining the correspondence of multisource image features to achieve matching between multisource images, and its effectiveness in feature point matching is experimentally demonstrated. The property that GTHM also can match multiple source images without relying on the target's location information applies to the experimental data in this article, and it can achieve ship matching in satellite images with long time intervals. Therefore, the GTHM method is selected as a comparison method for experiments in the field of feature point matching.
In the ship association section, the methods' accuracy is evaluated by two metrics: the number of correctly matched sample pairs (NCM) and the matching accuracy (MP). The MP is computed as follows: where NTM is the number of all sample pairs.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.

TABLE II PSEUDOCODE OF THE IMPROVED KM ALGORITHM
A. Effectiveness Evaluation of the Improved KM Algorithm 1) Experimental Data: In order to evaluate the performance of the improved KM algorithm, 56 images were intercepted as experimental data in the Esri Living Atlas platform. The ship images are preprocessed to eliminate the influence of complex backgrounds as shown in Table III, where the first 27 pairs are different images of the same ship, and the 28th and 29th pairs are two ships without matching objects.
The color, resolution, and brightness of the same ships differ in images taken at different times or from different platforms, leading to differences between ships in the same pair. Furthermore, some ships have highly similar structure, size, and color, which makes it challenging to match ships.
2) Ship Matching Results: The first ship images in the 27 ship pairs are divided into group A, numbered as a 1 , a 2 , . . . , a 27 , and the second ship images in the 27 ship pairs are divided into group B numbered as b 1 , b 2 , . . . , b 27 . The 28th pair is divided into group A, numbered asa 28 , and the ships in pair 29 are assigned to group B, numbered as b 29 . So that there should be 27 matching pairs of ships in both groups A and B. In addition, one ship in each group has no matching ship in the other group (a 28 and b 29 ). The metric matrix is constructed from the extracted features and input into the improved KM algorithm. Comparing the method in this article with the template matching method, the original KM algorithm, and GTHM, the specific results of matching are shown in Table IV. Table IV shows that among all 28 ship pairs, the template matching method has two incorrectly matched pairs (the 23rd and the 28th). The incorrect matching is because the two mismatched pairs of ships are very similar in orientation, size, and internal local texture, resulting in a high template matching score. The original KM algorithm has three mismatched pairs (a 28 with b 22 , a 22 with b 23 , and a 23 with b 29 ). It is worth noting that a 28 and b 29 should not have correct matching objects, but the original KM algorithm still assigns matching objects to them. The error occurs because there are many similar values in the metric matrix, which interferes with the KM algorithm. GTHM with four matching error pairs (the 3rd, the 6th, the 27th, and the 28th), and most of the error matching ships could only find a few features, making the algorithm has significant errors. In general, the template matching algorithm and the original KM algorithm have poor differentiation between similar ships, and are unable to mark vanishing ships and emerging ships in practical applications. GTHM is limited by the number and quality of feature points that can be found, and is not suitable for smaller or more ambiguous ships.
The proposed improved KM algorithm matches correctly for all 28 ship pairs. Although many ships have extremely close similarity scores, the matching results of the improved KM algorithm are continuously corrected through multiple iterations of the KM algorithm. Note that the improved KM algorithm does not assign any matching objects to a 28 and b 29 , indicating that the algorithm can mark vanishing and new objects, which can better complete the ship matching between images.
3) Accuracy Evaluation: The number of correct matches (NCM) and matching accuracy (MP) are used to evaluate the four methods, which are shown in Fig. 7.     Fig. 7 shows that the improved KM algorithm has a significant improvement in matching accuracy compared with other methods. This is due to the improved KM algorithm correcting the matching results of the original KM algorithm, which can better discriminate different ships and assign the best match to them in object disappearance, object similarity, and object imaging change. The other algorithms are not robust when the object changes, appears, or disappears.
The running times of various algorithms are shown in Table V. The running time of the template matching algorithm and GTHM far exceeds the other two algorithms, because those algorithms need to traverse each object in group A and group B to calculate the similarity, resulting in a higher time cost for them. Although the improved KM algorithm takes slightly longer to run than the original KM algorithm, the time required is much smaller than the template matching and GTHM, indicating that the algorithm ensures efficiency while improving accuracy.

B. Ship Matching Under Moving Background
To further verify the effectiveness of the proposed ship association algorithm under moving backgrounds, three images of the OVS satellite and the three images of the GF 1D and the GF 6 satellites are used for two experiments. The OVS satellite uses push-broom imaging, and the satellite position varies greatly relative to the background; the GF 1D and GF 6 images are heterogeneous remote sensing images, and the shooting angles and imaging mechanisms are different. The two sets of experimental data verify the effectiveness of ship detection and association based on video satellite images and multispectral remote sensing images under different resolution application scenarios.
The three images of the OVS satellite were all taken on August 24, 2020, with a 1 s time interval and 1.98 m spatial resolution. The specific information is shown in Table VI.
The two multispectral images used in the experiment are from GF 1D, and one scene is from GF 6. All images were taken on November 4, 2020, with a difference in shooting time of 9 s to 2 min approximately, and the spatial resolution is 8 m. The detailed information is shown in Table VII. Since the images occupy a large area and only a tiny part of the sea contains ships, three experimental areas were selected from the overlapping part. Experimental areas 1 and 2 contain more ships than experimental area 3, and experimental area 3 has a more complicated background.

1) Ship Detection Results:
The ROIs extracted from the three images of OVS are shown in Fig. 8. The sea surface in the image is unevenly illuminated, covered by clouds and shadows, and the ships are dense and similar. Compared with the first two images, there are apparent differences in the image (c), such as the color of the sea surface and ships have changed, making it challenging to match ships.
The ROIs extracted for every three experimental areas of the GF 1D and GF 6 images are shown in Fig. 9. The images and ROI of experimental area 1 are shown in (a-1, b-1) and (a-2, b-2). The images and ROI of experimental area 2 are shown in (c-1, d-1) and (c-2, d-2). The images and ROI of experimental area 3 are shown in (e-1, f-1), (e-2, f-2). The backgrounds of study area 1 and study area 2 are relatively simple, and the background of study area 3 contains noise such as land and harbor. From the significance detection results, it could be seen that the proposed algorithm could remove the noise such as land better and highlight the ships better at the same time.
The above experimental results based on OVS and GF data show that the extracted ROI can highlight the areas where the ships are located more accurately in the moving background, which can meet the experimental needs. The ROI excludes the large land and clouds, and only includes the ships and the small scattered clouds and land.
After removing the false alarms by calculating the E-HOG features, the ship detection results of two groups of experiments  Table VIII, where "-" indicates no corresponding ship, and the 19th is the ship trails, not the ship target.
The ship detection results of the three experimental areas with GF data are shown in Table IX. Pairs 1-15 are the ships extracted from experimental area 1, pairs 16-24 are the ships extracted from experimental area 2, and pairs 25-31 are the ships extracted from experimental area 3.
As shown in Tables VIII and IX, the two sets of experiments show some differences in imaging the same ship, such as imaging clarity and ship orientation. The imaging of small ships such as yachts is more blurred and can be confirmed almost exclusively by the wake traces. From the corresponding ship images, it can be seen that the difficulty of ship association is mainly reflected in three aspects: first, the same ship in the three images does not have the same features; second, a small part of ships incompletely extracted the hull due to the differences in the contrast ratio; and finally, numerous ships are comparable.
2) Ship Association Results: The ship detection results of each OVS images are grouped into one group, and three groups of ship images A, B, and C are obtained. Feature extraction, fusion, and normalization are performed on the three groups of images to produce three metric matrices to match the ships in groups A and B and groups B and C, respectively, and the matching results are obtained. Comparing the method in this article with the template matching method, the original KM algorithm, and the GTHM, the specific results of matching are shown in Tables X and XI.
As shown in Tables X and XI, the template matching algorithm incurs a total of two pairs of matching errors (pairs 2 and 6 of groups B and C). These two ships disappear in group C, but template matching did not find this situation, thus showing that template matching does not adapt well to the case of disappearing ship targets. A total of 11 pairs of matching errors occurred in GTHM. The reason for the poor matching results is that this method is based on feature points to match the images, and when the ship imaging changes, it is not possible to find enough feature points to match the ship because the ship image is small. The original KM algorithm incurs five pairs of matching errors, especially when the object disappears or when there are similar ships that are prone to mismatching. This is because the KM algorithm assigns an object with the best similarity as the best match when the match does not conflict, so a false match is prone to occur when this object disappears. The improved KM algorithm is less prone to this problem because the algorithm makes a judgment of the initial matching result in response to the disappearance of the object. It can also be seen from the experimental results that the improved KM algorithm only has matching errors in the 20th pair of ships, which is caused by the incomplete segmentation of the hull when the ship detection algorithm segments the ships.
The matching results of the GF data are shown in Table XII. They show that the matching method proposed in this article achieves correct matching for all 31 pairs of ships, which is better than the other three algorithms. Among the three comparison methods, GTHM has the lowest accuracy because the resolution   of both the GF 1D and GF 6 data used in this set of experiments is 8 m, which is lower than that of the first two sets of experimental data. This leads to blurrier imaging of the ships in this set of experimental data, which causes GTHM to fail to extract enough feature points for describing the ships and seriously affects its matching accuracy. Similarly, for template matching, the blurred imaging makes the ship details unclear, leading to the matching accuracy decrease. The original KM algorithm and the improved algorithm in this article are both based on matching the extracted shape, color, and local features, which reduces the impact brought by the blurred imaging problem when the resolution is reduced. The algorithm proposed in this article is improved on the basis of the original KM algorithm, and it can be seen from the matching results that this improvement overcomes Comparing the results of the two sets of experiments, we found that template matching, the original KM algorithm, and GTHM all showed significantly more false matches based on the GF data than the experiments based on OVS. This is mainly because the resolution of the GF data is lower than that of OVS data, and the decrease in resolution leads to blurred ship imaging and unclear details, so more false matches occur. The improved KM algorithm proposed in this article can effectively overcome this problem. The improved KM algorithm finds the optimal match by iteratively comparing and judging the similarity of ships, which applies to the situation where the resolution changes.
3) Accuracy Evaluation: For the ship detection part, precision and recall are used to evaluate the ship detection results. The ships are classified and evaluated in terms of accuracy by  Table XIII.
From Table XIII, it can be found that both the precision and recall of large ships are significantly higher than those of small  ships. This is due to the smaller proportion of small ships in the whole image, resulting in only a few pixel blocks. Therefore, the detection method is not sensitive to smaller ships of yacht type, resulting in more missed detections of small ships and lower recall values. The accuracy is higher for both large and small ships in terms of the precision values.
NCM and MP are used to evaluate the matching methods in the framework, and the MP of the four algorithms is shown in Fig. 10. Similar to the matching experiments based on the Esri Living Atlas platform, the proposed improved KM algorithm has the highest MP among the four algorithms. It is verified that the proposed algorithm can also obtain accuracy in practical situations, achieve better matching even when the ship imaging changes and similar ships exist. It is also proved that the proposed algorithm can adapt to the decline of image resolution and can realize ship matching in medium-and high-resolution remote sensing images.
The running times required by the four algorithms are shown in Table XIV. From the table, we find that method GTHM takes the longest time. Because this algorithm takes much time to calculate the feature points of each image, and then iterates through the images to calculate whether there are matching feature points between images, which requires a high time cost. Similarly, template matching also requires traversal between images to calculate the similarity, so it also takes a certain amount of time. The improved KM algorithm proposed in this article has slightly more running time than the original KM algorithm. However, the slight increase in running time is acceptable considering the increase in matching accuracy. In contrast, the template matching and GTHM do not improve the matching accuracy while the running time increases.
In general, the proposed algorithm shows a more significant advantage in the ship matching stage. Compared with GTHM, the improved KM algorithm does not rely on local feature points, but combines global features and local features to calculate the feature vector of the ship as a whole, so that the ship image can achieve better ship matching results even when the features such as texture details are more blurred. Compared with the template matching algorithm, the proposed algorithm can reduce the traversal of images and has high time efficiency while ensuring accuracy. Compared with the original KM algorithm, the proposed algorithm can correct the error matching and significantly improve the matching accuracy.

IV. DISCUSSION
This article proposes an unsupervised automatic ship matching method based on optical satellite images. To address the problem that existing target detection algorithms cannot segment ships from the background, the GrabCut algorithm is used to segment ships, and a preprocessing method is proposed to improve the performance of the GrabCut algorithm. To address the problem that the accuracy of KM algorithm decreases due to too many similar values in the metric matrix, this article introduces the template similarity calculation into the KM algorithm to improve the ship matching accuracy. It can be seen from Fig. 2 that the proposed preprocessing algorithm improves the integrity of the GrabCut algorithm in segmenting ships and separating them from the background, because the original GrabCut algorithm works on the whole image of the input, whereas the proposed preprocessing algorithm dynamically divides the GrabCut algorithm's working range according to the location of each ship. As shown in Table XIII and Fig. 10, the improved KM algorithm outperforms the template matching, the original KM algorithm, and the GTHM algorithm in terms of accuracy. The improvement in accuracy of the improved KM algorithm originates from the introduction of template similarity. The shape, color, and local features of similar ships may be very similar, resulting in a large number of similar values in the metric matrix, which decreases the KM algorithm's accuracy. And the introduction of template similarity enhances the fineness of ship feature extraction, thus distinguishing similar ships. As shown in Table XIV, the running time of the improved KM algorithm is much smaller than that of the template matching and GTHM algorithms, and slightly larger than that of the original KM algorithm. In terms of time efficiency, although it takes longer to calculate the template similarity, the improved KM algorithm narrows the range of ships that need to calculate the template similarity, significantly improving the time efficiency.
Although the improved KM algorithm has better experimental results in matching ships based on optical remote sensing images, there are some limitations, such as not considering ship occlusion. When a ship is concealed, the obscuring object is usually cloud layers. In most cases, the ship will be completely concealed when the thick cloud layer appears, and partial obscuration occurs less often. When a ship is completely obscured, it is generally not detected by the detection algorithm, and there is no need to match the concealed ship. Partial occlusion is also not taken into account since it occurs less often. When thin clouds appear, ships' imaging becomes blurred, similar to the resolution reduction of remote sensing images. The improved KM algorithm can adapt to this situation to a certain extent.

V. CONCLUSION AND OUTLOOK
In this article, an unsupervised ship matching method of optical remote sensing images is proposed to solve the ship matching problem, which achieves the ship's automatic detection and association of optical remote sensing images under moving background. A preprocessing algorithm is proposed to increase the computational efficiency and segmentation accuracy of the GrabCut algorithm, which can detect ships and segment them from the background simultaneously. An improved KM algorithm is proposed to improve the accuracy of ship association by adding similarity comparison and iterative steps. To verify the effectiveness of the proposed ship matching method, this article conducts experiments based on the sample data intercepted by the Esri Living Atlas platform, OVS images, and GF data. The results show that the improved KM algorithm significantly improves the accuracy with guaranteed execution efficiency, and can realize ship matching of medium-and high-resolution remote sensing images. Compared with deep-learning-based ship tracking methods, the unsupervised ship matching method proposed in this article does not require samples for training and is time-efficient, enabling ship localization and tracking over long time and large spatial spans.
There are still some limitations in this study. For example, similar to tracking based on detection, the accuracy of the detection part has a large impact on the overall method accuracy. With the development of the satellite network, more and more multisource satellite remote sensing images can provide data support for ship matching. When there are enough images in the study area at a certain period, the research method in this article can be extended to multiple images. At this time, it is necessary to additionally consider the data association problem caused by the target's reappearance. Then, the ship tracking based on the satellite network can be realized.