Image Processing of Manganese Nodules Based on Background Gray Value Calculation

To troubleshoot two problems arising from the segmentation of manganese nodule images-uneven illumination and morphological defects caused by white sand coverage, we propose, with reference to features of manganese nodules, a method called “background gray value calculation”. As the result of the image procession with the aid this method, the two problems above are solved eventually, together with acquisition of a segmentable image of manganese nodules. As a result, its comparison with other segmentation methods justifies its feasibility and stability. Judging from simulation results, it is indicated that this method is applicable to repair the target shape in the image, and segment the manganese nodule image in a short time. Also, it could be used to synchronously process a large number of manganese nodules on different conditions in an image, laying a good foundation for automatic underwater manganese nodule survey. Even if the target in the image is slightly distorted, the statistical data of manganese nodules are still accurate. Moreover, other methods cannot be fully applied to the segmentation of manganese nodule images; in another word, the effectiveness and stability of this method are proved.


Introduction
Natural manganese nodules, containing dozens of precious metal elements like cobalt and nickel, are widely used in such sectors as aviation, aerospace and electronic information [Zhuang, Che, Sun et al. (2019) ;Zhuang, Sun, Zhi et al. (2017)]. Since the seabed at 2,000 to 6,000 meters deep is home to manganese nodules, it is imperative to analyze the images obtained by means of autonomous underwater vehicle (AUV), and in turn, to define the reserves and distribution of manganese nodules in a specific area. As to image segmentation, it often plays an indispensable role in statistical tasks on manganese nodules [Awad, Ali and Suganthan (2017)]. In the process of performing statistical tasks, uneven illumination and morphological incompleteness caused by white sand cover of manganese nodules are the key problems to be solved. Because of the bad underwater environment, images become blurred [Petit, Capelle-Laize and Carre (2016)]. So, image enhancement is needed [Huang, Wang and Song (2018)]. In addition, due to the inevitable uneven illumination of the photographed image, it is necessary to adopt effective methods to impose more uniform illumination upon the whole image. As to uneven illumination, some solutions were proposed [Zhang, He and Guo (2020); Li, Zou, Sun et al. (2020); Hassan, Mostfa and Salehi (2016); He (2011)]. However, these methods are only used to deal with uneven illumination of document images, and are unable to restore the morphology of manganese nodules. Despite that, Wang et al. [Wang, Dong and Pan (2016)] have found while the Retinex algorithm is conducive to solve uneven illumination, it still cannot restore the morphology of manganese nodules. The image segmentation method proposed by Diels et al. [Diels, Van and Keresztes (2017); Liu, Zhou, Wang et al. (2018); Wang, Wu, Wang et al. (2018); Jing and Guo (2019); Kaur (2014)] is mostly done without the target being covered. So, these methods mentioned above are not appropriate for manganese nodules covered by white sand. Experiments show that the manganese nodule image, segmented by the Otsu method [Yuan, Wu and Peng (2015); Malarvel, Sethumadhavan, Bhagi et al. (2017)], maximum entropy method [Ma, Jia, Zhao et al. (2019)], and the iterative threshold method [Gui and Zeng (2020)], failed to meet the requirements for subsequent processing due to poor segmentation effects and incomplete target morphology. With respect to the recovery of object morphology, existing literature is usually advocate processing by corrosion expansion algorithm. However, the gap between the manganese nodules is small, and the corrosion expansion algorithm fills the gap between the manganese nodules or causes a whole piece of manganese nodules to be divided into pieces. There are few methods applicable to process images of manganese nodules that can simultaneously solve uneven illumination and restore the morphology of manganese nodules. As such, for the image segmentation of manganese nodules, we propose a method in this paper to simultaneously solve uneven illumination and restore the morphology of manganese nodules. First, we convert the original image to a grayscale image. Second, we calculate and subtract the background gray value of the image from the grayscale image. Then, we adjust the gray value of the image as shown in the block diagram in this paper. Finally, we take these steps once more. In the first time, the process solves uneven illumination. Then, the image which solves uneven illumination will be executed again to restore the morphology of manganese nodules. As the end, several images processed by MATLAB simulation [Qun, Heng and Fang (2019)] are segmented to verify the feasibility and stability of the proposed method.

Selection of image samples and design of method flow
As shown in Fig. 1, this paper selects the manganese nodule images taken by AUV as the research samples. It unveils that the image processing of manganese nodules is usually held back by difficulties including: a) uneven illumination which inevitably exists in images due to different angles of light sources b) the fact that manganese nodules are generally covered by white sand, and the gap between them seems very small. In the process of morphological restoration, the gap between manganese nodule particles will be filled.
Therefore, it is difficult to restore the morphology of manganese nodules by using conventional image processing methods without affecting the gap between the manganese nodules. In a word, uneven illumination and morphological recovery of manganese nodules in image segmentation dominate image processing of manganese nodules. Image enhancement seems to be a cornerstone to eliminate uneven illumination and restore the morphology of manganese nodules.   Fig. 2, once the image of manganese nodules is illuminated uniformly through c and d steps, the morphology of manganese nodules in the image can be restored if above-mentioned operations are performed again. After the manganese nodule image is binarized, the obtained image is segmented. Then, the feasibility and stability of this method are verified after the comparison in terms of image segmentation effects.

Image preprocessing
In this paper, image preprocessing includes image enhancement and Gaussian filtering, the former of which aims to improve the contrast between manganese nodules and the background. The image taken by the AUV is generally a color image. For the convenience of subsequent processing, we grayscale the color image. Then, perform grayscale stretching on the obtained grayscale image. Its gray histogram before and after grayscale stretching are shown in (2) in which, c is the scale and λ is a scaling coefficient. that F (x, y) must meet the conditions as below: (x, y) = 1 (3) After enhancing the image, filter the gray image first to mitigate the influence of debris, lower down the gray value of some white sand on manganese nodules, and then lay the foundation for the calculation of background gray value and the generation of artifacts. The formula is as follows: in which, G (x, y) is the gray value of the filtered image at (x, y). The size of the filter's range of action depends on σ. The larger value of σ leads to the more blurred image.

Elimination of uneven illumination and restoration of manganese nodule morphology 4.1 Background gray value computation
In order to prevent the background of the image from excessive artifacts while adjusting the gray value, this section talks about how to eliminate the influence of uneven illumination and make the background gray value of the image more uniform. First, calculate the background gray value of a gray image. Second, subtract the background gray value from the original image to eliminate its influence and generate the more uniform background gray value of the image. Finally, adjust the gray value of the image with reference to the block diagram, so as to keep manganese nodules clearly displayed in the image. The first implementation of this process is to solve uneven illumination. As a start, traverse all the pixels in the image by a mask of N*N, and then record, in turn, the n points with the highest gray value in the mask. Assign the recorded gray value to LP where 1≤P≤n, and the set L is formed in the order from large to small, creating the set L L= {L1, L2, … , Ln}. For the sake of interference points in the image, we remove the maximum gray value Ln and the minimum gray value L1 before calculating the background gray value. The remaining points in the set are summed and taken as the mean of the background gray value of the central point G (x, y) of the mask. The number of points is n, n=8. As shown in Fig. 4, however, different sizes of masks will have a great impact on the calculation of the background, so the subsequent task is to determine the value.  Fig. 4 shows that the smaller mask could help calculate the background gray value more accurately. However, subtracting from the grayscale image is not conducive to both the elimination of illumination unevenness and subsequent steps to generate artifacts. But the mask fails if too large. An overlarge mask is also of no use to eliminate uneven illumination. In order to find a suitable mask, we record the size of the mask and its running time as shown in Tab. 1. According to Tab. 1, the larger mask entails the longer calculation time. So, we set N=21 to calculate the gray value. Next, we subtract the background gray value from the original image to solve uneven illumination.

Solution to problem of uneven illumination
In order to solve uneven illumination and create the more uniform background gray value of the image, we subtract the grayscale image from the background gray value. After subtraction, the contrast between the manganese nodules and the surrounding environment in the subtracted image will be reduced in the region where the background gray value of the image is small. Therefore, it is necessary to adjust the gray value between the manganese nodules and the background. Respectively set X, Y as the gray value of the grayscale image and the background gray at (x, y), and E as the gray value of the image after the gray value adjustment. The contrast adjustment block diagram is shown in Fig. 5.

Figure 5:
Contrast adjustment block diagram in which, K is a multiple of magnification. After preprocessing, the gray value of white sand covered with manganese nodules is lower than that of white sand in background. The original method [He (2011)] requires the suppression of artifacts, but while dealing with uneven illumination, we need to make the image generate artifacts appropriately to prepare for the morphological restoration of manganese nodules. However, at the same time, noise generation and image distortions should be suppressed. As such, the value of K is as follows: in which, B1 and B2 are constant, and their values are from 1.6 to 2 and 1 respectively. Likewise on, the values of these two constants depend on both the size of the region to be repaired and the contrast between the target and the background. After the image contrast is adjusted, the grayscale of the image is stretched. As shown in Fig. 6, the results of this method are compared with those of the Retinex algorithm and the original method. As revealed in Fig. 6, all three methods above can be used to solve uneven illumination. However, only the method proposed in this paper can help to improve the contrast between manganese nodules and the surrounding environment, conducive to image segmentation. Furthermore, this method has an advantage over Retinex algorithm and the original method in that the morphology of manganese nodules can be again restored with the aid of this method. Although uneven illumination has been dissolved, the morphology of manganese nodules has not been completely restored. Therefore,, we next need to use this method again to restore the morphology of manganese nodules, and then test the results of this paper in image segmentation.

Morphological restoration of manganese nodules
After solving uneven illumination, we need to restore the morphology of manganese nodules. As shown in Fig. 7, the gray value of the area covered by white sand is higher than that of the area without white sand. On a single manganese nodule, the pixels with higher gray value of white sand are mostly surrounded by the pixels with lower gray value, and the covered white sand scatters on a single manganese nodule. In view of this feature, we adopt the method again to calculate the background gray value of the image.  (2) Next, the image is subtracted from its background gray value, and the gray value of the image is readjusted with reference to Fig. 5. In this way, the morphology of manganese nodules can be restored. As shown in Fig. 8, if we use algorithm of corrosion and expansion to restore morphology directly at this time, the following problems will arise: (1) Corrosion algorithm is applied to the image for restoring the morphology of manganese nodules. Although the morphology of manganese nodules is restored, the gap between individual manganese nodules disappears and the interference increases. Eventually, it is difficult to separate the manganese nodules from the interfering pixels by the expansion algorithm. (2) The expansion algorithm is applied to the image for interference removal. Although the interference is excluded, due to the white sand cover, the whole manganese tuberculosis will split into many pieces which are removed together with the interference pixels by the small target removal algorithm. Compared with other methods, this method not only ensures the gap between individuals, but also prevents the partial loss of manganese nodules after morphological restoration. It is shown that this method is applicable to the morphological restoration of manganese nodules, and is a feasible and stable method. It should be noted that the size of the mask has a greater influence on the results than on the solution to the problem of uneven illumination. As shown in Fig. 9, too large or too small masks can affect the restoration of morphological manganese nodules. Figure 9: Effect of different masks on restoration: (a) Original image; (b) Overlarge mask; (c) Moderate mask; (d) Too small mask As can be seen from Fig. 9, excessive masking will lead to the disappearance of gaps between manganese nodules, and too small masking will lead to incomplete restoration of morphological manganese nodules. After solving the two core problems, we now need to segment the processed image to verify the stability and feasibility of this method.

Binarization of image
Since binary images containing only manganese nodules are needed, we are required to check whether the processed images can present the desired results in image segmentation after the aforesaid operations are completed. If the image needs to keep more details, we use the Sauvola algorithm [Li, Zou, Sun et al. (2020)] to binarize the image. The principle of the algorithm is that the mean and standard variance of the gray value in the mask of r*r size are calculated to determine the threshold of the mask center (x, y). In order to eliminate a small part of the interference, we increase the r value as much as possible. The average gray value of a mask centered on (x, y) is: The variance in the mask is: According to Eqs. (5) and (6), the threshold of image at point (x, y) can be obtained as follows: ( , ) = ( , ) * [1 + * ( ( , ) − 1)] (8) in which, the dynamic range of standard deviation is R, R=128. And the correction parameter is K, K=0.5. If the image needs to be quickly segmented, the threshold can be set as 0. As shown in Fig.  10, after traversing the gray value of the pixel point at the target and combining it with the gray histogram, we have found that the gray value at the target is far lower than the gray value of other regions, and most of its gray value is zero if histogram specification is followed. From the comparison of the images, we can see that the morphology of manganese nodules has been greatly restored, but the interference still exists. Next, we will make use of image segmentation to eliminate the interference and test the effectiveness and stability of the method.  Because the morphology of most manganese nodules has returned to normal, algorithm of corrosion and expansion and small target removal algorithm can be used to remove the interference. The interference in the image is mostly slender and the number of pixels in the joint with manganese nodules in the image is relatively scarce. So, we use the expansion algorithm to disjoin manganese nodules. After the experiment with different masks, the structural element B=[0, 1, 0; 1, 1, 1; 0, 1, 0] is selected, and then the binary image is expanded three times, followed by the small target removal algorithm applied to remove the interference from the image. Finally, the corrosion algorithm is adopted with the aid of the flat disc structure element (disc radius=4). The time of image segmentation is shown in Tab. 2. The image processing results are shown in Fig. 12.

Image segmentation
As can be seen from Fig. 12, compared with the results of image segmentation in the case of direct binarization and image without restoration by this method, the image processed by this method not only solves uneven illumination, but also retains the gap between manganese nodules. The manganese nodule image processed by this method has a good effect on image segmentation, which proves that this method can be applied to the segmentation of manganese nodule image, and it is feasible and stable. Although the morphology of manganese nodules is slightly distorted, it will not have much influence on the identification of the distribution of manganese nodules in this area. However, manganese nodules buried almost entirely in the sand failed to be repaired by this method. Finally, we compare this method with classical segmentation methods, such as Otsu method, maximum entropy method, and the iterative threshold method. Among them, Otsu is a segmentation algorithm that determines the threshold of image binarization. Suppose the segmentation threshold is T, the ratio of the number of target pixels to the total number of pixels is W0, and the average gray value of the image is μ0. The ratio of the number of background pixels to the total number of pixels is W1, and the average gray value is μ1. The average gray value of the entire image is μ, and the variance is g. In an image of size M*N, if there are N0 pixels with a value lower than T, and N1 pixels with a value greater than the threshold T, then here comes the following formula: = 0 * 0 + 1 * 1 (9) = 0 ( 0 − ) 2 + 1 ( 1 − ) 2 (10) In which W0=N0/M*N, W1=N1/M*N. W0+W1=1. Based on the two formulas above, the following formula can be obtained: = 0 1 ( 0 − 1 ) 2 (11) After the whole image is traversed, it is elicited that the value of g can be maximized, and this maximum is the threshold T. Finally, the time of images segmentation is shown in Tab. 3. The image processing results are shown in Fig. 13.  = ∑ =0 (13) The corresponding entropy of the target area is as follows: The entropy corresponding to the background area is: The entropy function of the entire image can be obtained by combining the above formulas:   The iterative threshold method contains steps as follows: (a) selecting average value T0 of the grayscale image as the initial threshold. (b) recording the average gray value as T1, and the average gray value of remaining pixels as T2, for pixels with gray value lower than or equal to T0. (c) calculating new threshold T=(T1+T2)/2. (d), keeping threshold still as T if T=T0, and otherwise T0=T. Next, repeat steps (a)-(c). Finally, the time of images segmentation is shown in Tab. 5. The image processing results are shown in Fig. 15.  Because the method proposed in this paper can directly make the threshold value T=0 during the process of binarization, the duration of threshold value calculation is omitted. Therefore, the process of image binarization takes a short time. In terms of execution time, it is close to the iterative threshold method, which is better than the typical Otsu method. But it is about twice as slow as the maximum entropy method. Regarding segmentation effects, however, the method proposed in this paper can solve uneven illumination and morphological defects of manganese nodules, so it can meet the segmentation requirements of manganese nodules in images. As to the other three methods, the effect of uneven illumination was not considered during the segmentation process. While segmenting an image with uniform illumination, we found that the shape of the target cannot be repaired. Therefore, their segmentation results are not applicable to further statistics on manganese nodules in the manganese nodules image. It is proved that the method proposed in this paper is stable and feasible for manganese nodule image processing.

Conclusion
In order to realize the segmentation of manganese nodule images, it is necessary to solve two core problems: a) Uneven illumination. b) Morphology of manganese nodules. As such, with reference to features of manganese nodule image, a method is proposed in this paper and its feasibility and stability are tested in image segmentation. The general process is as follows: First, preprocess the gray image, and execute the method presented in this paper to get the image which eliminates the uneven illumination. Second, use this method to restore the morphology of manganese nodules. Comparison with other methods proves the applicability and stability of this method in eliminating the uneven illumination and repairing the morphology of manganese nodules. Finally, the image with only manganese nodules is obtained by segmentation, which verifies the feasibility and stability of this method. However, the morphology of manganese nodules treated by this method is slightly distorted. In a word, the method is effective and stable by processing several images, and the manganese nodule image processed by this method has achieved good results in image segmentation. The method presented in this paper can synchronously process a large number of manganese nodules with different conditions in an image, which lays a good foundation for automatic underwater manganese nodule survey. How to create statistics on the manganese nodules is what we need to study in the future.

Conflicts of Interest:
The authors hereby make the statement that they have no conflicts of interest to the report regarding this study.