Detection of surface defects and subsurface defects of polished optics with multisensor image fusion

Surface defects (SDs) and subsurface defects (SSDs) are the key factors decreasing the laser damage threshold of optics. Due to the spatially stacked structure, accurately detecting and distinguishing them has become a major challenge. Herein a detection method for SDs and SSDs with multisensor image fusion is proposed. The optics is illuminated by a laser under dark field condition, and the defects are excited to generate scattering and fluorescence lights, which are received by two image sensors in a wide-field microscope. With the modified algorithms of image registration and feature-level fusion, different types of defects are identified and extracted from the scattering and fluorescence images. Experiments show that two imaging modes can be realized simultaneously by multisensor image fusion, and HF etching verifies that SDs and SSDs of polished optics can be accurately distinguished. This method provides a more targeted reference for the evaluation and control of the defects of optics, and exhibits potential in the application of material surface research.

the detection in small sampling areas. Efficient and fast detection of SDs of large optics can be realized by digital evaluation system with wide-field scattering microscope [7,8]. But SSDs are covered under the surface, which are difficult to directly detect by classic detection systems for SDs. Specific detection methods for SSDs have been developed. Destructive ones such as acid etching [9], dimpling [10], etc. They expose SSDs through physical or chemical means, and will cause irreversible damage to the optics. Nondestructive methods based on optical imaging include total internal reflection microscope (TIRM) [11], confocal laser scanning microscope(CLSM) [12,13], and optical coherence tomography(OCT) [14], etc. They illuminate the detection area in different ways, receive the optical signal modulated by SSDs, and will not cause damage. CLSM with fluorescence imaging is widely used for SSD detection in recent years [15,16]. The optics are doped with tiny fluorescent materials during grinding and polishing, and they can generate fluorescence under the excitation of lasers. These materials may come from the cooling fluid used in polishing [15], or may be artificially doped quantum dots [17]. They are buried in the pits, scratches and other mechanical damage in subsurface layer. It is worth noting that stronger fluorescence can be excited with quantum dots used as fluorescent materials, but they are also contaminants for precision optics, and are difficult to remove. In addition, studies have shown that this kind of SSDs with fluorescence characteristics is closely related to the laser damage of optics [18,19].
The surface of polished optics is relatively smooth, and most of the defects are removed, but there are still some residues randomly distributed on the surface and subsurface. Since SDs and SSDs may be stacked in space, it is a major challenge to quickly detect and accurately distinguish them by non-destructive detection. CLSM with fluorescence imaging has capabilities of high resolution and three-dimensional (3D) reconstruction. But the system is complex, and its field of view is quite small, which is usually used for small-range detection of hundreds of microns. The large-range detection can be realized by wide-field scattering imaging, but SDs and SSDs cannot be distinguished. Herein, a multisensor image fusion detection method is proposed, which combines wide-field scattering and fluorescence imaging. The sample is illuminated by a laser, and a microscope system is placed in a direction perpendicular to the sample. Scattering and fluorescence lights are split by the system, and received by two image sensors at the same time. Multisensor images are processed by spatial registration and feature-level fusion, realizing the identification and extraction of SDs and SSDs.

Multisensor imaging system
Polishing is the last processing process of most optics, SDs and SSDs are greatly affected by polishing. The surface and subsurface structure of a polished optics is shown in Fig. 1a. From top to bottom, there are redeposition layer (or hydrolyzed layer, Beilby layer, polishing layer), subsurface layer, deformed layer and the bulk [20]. According to the chemical action theory and thermal surface flow theory, the redeposition layer is produced by the hydrolysis of the polishing slurry and the material surface. Contaminants such as cerium and iron remain in this layer, and the thickness of this layer is about tens to hundreds of nanometers. According to the mechanical grinding theory, polishing particles enter the redeposition layer and then slide on the material, resulting in polishing dots and polishing scratches. They and a small amount of cracks remaining during grinding together constitute mechanical SSDs. Affected by the pulling force of polishing, contaminants are quickly buried in these SSDs, covering all or part of SSDs under the redeposition layer. Therefore, there is SSD that extends below the surface and is partially covered [21,22]. The former is called extended SSD and the latter is called covered SSD in this paper.
The principle of dark-field scattering imaging for defects detection is shown in Fig. 1b. The surface of the sample is illuminated by the incident light obliquely at a certain angle, and the light is directly reflected when there are no defects on the surface. If there are defects, the scattering light is generated and can be received by the microscopy system (not shown in the figure), so a dark-field image with bright defects can be obtained. Since transmissive optics can be penetrated by the incident light, covered SSDs and bulk defects [23] will also be illuminated and generate scattering light, but the strength is relatively weak. Fluorescence imaging is similar to scattering imaging. As shown in Fig. 1c, the surface is illuminated by the excitation light, then the fluorescence is generated by contaminants with strong light absorption. These contaminants are all over the surface and subsurface. The ones buried in SDs are removed after cleaning; and the ones buried in the redeposition layer and SSDs are retained and become the main source of fluorescence. Because of the deep mechanical SSDs, they provide places for a large number of contaminants to gather, so the fluorescence is strong and concentrated. The depth of the redeposition layer is very shallow, there are few contaminants and they are evenly distributed, so the fluorescence from the redeposition layer is weak, showing as a uniform background.
As shown in Fig. 1d, SDs and extended SSDs are the main types of defects in scattering image, and they are hard to distinguish. Extended SSDs and covered SSDs are the main types of defects in fluorescence image, and they are also hard to distinguish. The mechanism of laser-induced damage of optics with different types of defects is different, and it is of great significance to distinguish them for damage research. The optical field is modulated by SDs, inducing local field enhancement. In addition, SDs cause fracture-induced subbandgap absorption and ultimately induce damage [24]. Covered SSDs provide many attachment points for contaminants (such as Fe, Cu, Ce), and the thermal-absorption of the contaminants causes localized high temperature. The higher the area proportion of covered SSDs, the lower the laser damage threshold of the optics [25]. Extended SSDs have both fracture-induced subbandgap absorption properties and thermal-absorption properties. From the perspective of defects detection and removal, SDs and extended SSDs are easy to be detected and removed in subsequent processing; while covered SSDs are not easy to be found and removed. The defects information collected by one imaging method is very limited, so multisensor imaging includes both scattering and fluorescence modes is proposed. Scattering and fluorescence lights are collected by two independent sensors simultaneously to improve efficiency. Different types of defects are identified from the multisensor images, so they can be reduced in a targeted manner in subsequent processing.
The sample used in the experiment is an optical window of polished fused silica with a size of 100 × 100 mm and a thickness of 5 mm. The fluorescent materials come from the water-soluble oil coolants used during diamond grinding [15]. Since such coolants are widely used in the processing of most optics, no additional fluorescent materials are added. The sample needs to be ultrasonically cleaned (Micro-90 cleaner used in this paper) before multisensor imaging to remove contaminants adhering to the surfaces of the optics. A detection system designed based on the principle of multisensor imaging is shown in Fig. 1e. Two image sensors are used in the microscope to take scattering and fluorescence images simultaneously, and a laser is used as the light source for two imaging modes. When the sample is illuminated by an ultraviolet laser, fluorescence in the visible light can be generated from the SSDs [19,25]. Therefore, an ultraviolet quasi-continuous laser is used as the excitation light in the system. The laser of Gaussian beam is modulated into a uniform flat-top beam by a shaper, so the energy density of the illuminated area is basically the same, covering the field of view of the imaging system. Reflected and transmitted lights are absorbed by beam traps to reduce stray light. Scattering and fluorescence lights generated by defects are received by an objective (4 × , NA 0.13), and after being split by a 409 nm dichroic mirror (transmission wavelength: 415-850 nm), they enter different tube lenses (f = 150 mm) and image sensors respectively. The sensors are ultraviolet-enhanced CCD and electron-enhanced CCD (pixel size: 13.3 μm, resolution: 1024 × 1024). Both sensors are located at a position conjugate to the object plane, and take the scattering and fluorescence images on the same imaging area. Because the fluorescence is very weak, a watt-level 355 nm laser (short wavelength with stronger energy) is used as the excitation, and a highly sensitive electron-enhanced CCD is used as the image sensor, ensuring that the fluorescence can be excited and detected. The focusing and scanning control system is used to control the 3D movement of the sample by a XYZ stage, and adjusts the posture of the sample to keep it in focus positions.
Image processing includes preprocessing, registration and fusion. The scattering and fluorescence images are first preprocessed after reading into the computer, including denoising, background homogenization, and distortion correction. Then the defects will be highlighted from the background, which is conducive to the subsequent image processing. Next, two images will be registered and fused to get images characterizing SDs, extended SSDs and covered SSDs respectively.

Image registration and fusion
Image registration refers to the process of matching multiple images of the same scene to make their features correspond. These images with overlapping regions may be taken at different times, different conditions, or different sensors. Even if the same objective lens, tube lens and image sensors are used in the multisensor imaging system, the positions of image planes will be slightly different due to the difference in the detection wavelength. In addition, there will inevitably be differences in the positions of the imaging devices in the optical path, especially the image sensors. These factors will cause the difference of the imaging range of the target.
As shown in Fig. 2a.
ii, the imaging results of polishing scratches and polishing dots are represented by straight lines and rectangles. These defects are extended SSDs, which exist in both scattering and fluorescence images. Due to the difference between the two imaging systems, there are differences in the position, size and rotation angle of the same defects in the two images. The unregistered and registered superimposition images are shown in Fig. 2a.iii & 2a.iv (the defects in the scattering image and the fluorescence image are set to red and green respectively for display). If the two images are not registered, the same defect cannot be overlapped in the superimposition image. In the subsequent image fusion process, it is easy to misjudge the spatial location of such defects. Therefore, image registration become one of the key steps before image fusion.
The flow chart of image registration is shown in Fig. 2b. First, the feature points are selected in the scattering and fluorescence images respectively, and then the parameters of the transformation matrix are estimated based on affine transformation. Finally, the image is resampled and interpolated to complete the registration. After the system is set up, multisensor imaging system will no longer change, so the affine transformation matrix is also fixed. Therefore, after the matrix is estimated for the first time, the parameters can be used directly to complete image registration.
Image fusion is a multi-level image processing that uses the temporal or spatial correlation and complementarity of two (or more) images to get more accurate fusion images. For the multisensor image fusion detection system, the images taken by two sensors are based on different imaging principles, having different physical meanings. The information of the scattering and fluorescence images is both complementary and redundant, and the purpose of image fusion is to get images that characterize different types of defects. According to the characteristics of the images of defects, feature-level fusion is used in this paper, and the flow chart is shown in Fig. 2c. First, the contours of all defects in the scattering and fluorescence images are extracted, and the coordinates of the contour points are recorded. Then the coordinates are used for feature matching, and the defects are divided into three types. Finally, the defects are classified and extracted, getting images that characterize SDs, extended SSDs and covered SSDs respectively.

Imaging and identification of typical defects
The imaging results of SDs and covered SSDs are shown in Fig. 3a-3f, they are superimposition images of scattering and fluorescence images. The red areas in Fig. 3a-3b are surface scratches. They only exist in the scattering image, indicating that they are SDs. The green areas in Fig. 3c-3d are polishing points, and in Fig. 3e-3f are polishing scratches. These four defects are all covered SSDs, which are covered under the redeposition layer and cannot generate obvious scattering lights, only existing in the fluorescence image. The sample is etched with HF acid to verify the effectiveness of multisensor imaging for SSDs detection. As shown in Fig. 3g-3h, there are SD (shallow surface scratch) and covered SSD (polishing scratch) in the superposition image before etching. The shallow SD is completely etched away after a 3 μm-deep etch, so it disappears in the bright field microscope. The covered SSD is fully exposed after etching, which can be directly observed in the bright field image.
The imaging result of an extended SSD is shown in Fig. 4a. This defect exists in both the scattering and fluorescence images, and the overlapping area is displayed in yellow (red in the scattering image, green in the fluorescence image, and the overlapping area becomes yellow after superimposition). To get an image characterizes SDs, the extended SSD is needed to be removed from the scattering image. The removal result after the pixel-level processing is shown in Fig. 4b. The defect in the fluorescent image is first expanded, and then the expanded area is subtracted from the scattered image. It can be seen from the result that the defect is not completely removed, but a "doughnut" remains, which is physically meaningless. Although the "doughnut" can be removed by adjusting the expansion parameters, but it is difficult to set parameters suitable for . c d e f Superposition images, there are covered SSDs in the figures (polishing points and scratches in green areas). g h Superposition image before etching and its bright image after etching, there is a SD (shallow surface scratch) which disappears after etching, and a covered SSD (polishing scratch) which is exposed after etching  Fig. 4c. It can be seen that the extended SSD is completely removed, and the remaining defects in the figure are SDs.
The signal strengths of SSDs in scattering and fluorescence images are discussed in detail. A covered SSD at site 1 is shown in Fig. 5a, and local enlarged scattering and fluorescence images of it are in Fig. 5b-5c; an extended SSD at site 2 is shown in Fig. 5d, and local enlarged images of it are in Fig. 5e-5f. Both images are captured under the same power of laser with same exposure and gain, so the strengths of scattering and fluorescence signals can be reflected by gray values. We select sampling lines on each of the two defects, and draw their grayscale curves. In the scattering image shown in Fig. 5g, the peak gray value of the covered SSD is about 2800, which is very close to the background; the peak gray value of the extended SSD reaches 65,535 (16-bit image), which is overexposed and significantly different from the background. Although covered SSDs can generate scattering signals, the strength is relatively weak, and the scattering images are dominated by extended SSDs and SDs. In the fluorescence image shown in Fig. 5h, the peak gray values of covered SSD and extended SSD are around 2000 and 1000, respectively. Both are significantly different from the background. Therefore, covered SSD and extended SSD are dominant in fluorescence images. It is difficult to capture and distinguish all types of defects by a single imaging method, so it shows the advantages of multisensor image fusion.  Fig. 5b and Fig. 5e). h Contrast of the gray values of the two defects in fluorescence image (the sampling lines are the connection lines of the green arrows in Fig. 5c and Fig. 5f ) The superimposition images taken in three sites are shown in Fig. 6a -6c. There are extended SSDs in all three images, and image fusion is needed to extract different types of defects. There is a scratch that exists in both the scattering and fluorescence images in Fig. 6a, and after it is removed from the scattering image, a SDs image is obtained as shown in Fig. 6d. There is also a scratch that exists in both the scattering and fluorescence images in Fig. 6b, and after it is extracted, an extended SSDs image is obtained as shown in Fig. 6e. There are two polishing dots in both the scattering and fluorescence images in Fig. 6c, and after they are removed from the fluorescence image, a covered SSDs image is obtained as shown in Fig. 6f. After the defects are classified and extracted by image fusion, the scratches and pits in the images can be identified, and the sizes of the defects can be calculated with calibration [26], finally realizing the quantitative evaluation of various defects.

Methods
This section presents briefly the algorithms for image registration and fusion. In the beginning, preprocessing is introduced. Whether it is a scattering image or a fluorescence image, the image pixels where the defect is located have different gray-scale from the surrounding background, which is shown as a bright spot on a dark background. However, the noise generated during image acquisition and the uneven background generated by the illumination will adversely affect the extraction of defects. In the previous paper, we have introduced the preprocessing algorithms such as Top-Hat algorithm, gray-scale converting and medium filter [26].  Fig. 6a is removed). e Extended SSDs image (the extended SSD in Fig. 6b is reserved). f Covered SSDs image (the extended SSD in Fig. 6c is removed) The core step of image registration, affine transformation matrix estimation is introduced next. For two images to be registered for the same target, if the distance between a pair of feature points in the two images is small enough, it is considered that the points correspond to the same position in the target object. This featurebased registration calculates the spatial transformation model between the images by extracting the positional relationship between a pair of feature points. The analysis and processing of the entire image is transformed into few points, which greatly reduces the amount of calculations. In the multisensor image fusion detection system, the polished dots appear as bright spots under a dark background in the scattering and fluorescence images, and the shapes are generally regular with obvious closed area characteristics, which are suitable choices for feature points. The fluorescence and the scattering image are used as fixed image and moving image respectively, and the feature points coordinate sets of the fixed image x i , y i and the moving image   After the feature points are determined, the spatial transformation model parameters between the fixed and moving images are calculated by affine transformation. The model of affine transformation can be expressed as: where (x , y and (x' , y') are the pixel coordinates of the feature point pair in the fixed image and the moving image respectively; a 1 , a 2 , a 3 , a 4 are the transformation parameters of scale, rotation, flip and shear; t x , t y are the translation parameters. Substituting the coordinates{(x i , y i )} and x ′ i , y ′ i into formula (1), the affine transformation matrix M can be calculated. Since there are 6 unknowns in formula (1), at least 3 pairs of noncollinear feature points are needed. The moving image is resampled and interpolated according to the affine transformation matrix, and its coordinate system is mapped to the coordinate system of the fixed image.
The images are binarized after registration, and then they are processed by modified algorithm of feature-level fusion. The algorithm consists of three steps: a. Contour feature extraction; b. Feature matching and classification; c. Defects extraction. Next, they will be introduced in detail.

a. Contour feature extraction
The contour features in the image are extracted at first in the process of feature-level image fusion, and a feature space is retained, reducing memory and time consumption. Traversal searching is used to extract contour features of the defects, and the steps are as follows: i. First search the image from top to bottom, from left to right, and find the first white pixel as the contour point of the first defect, and record its coordinates as (x 1 , y 1 ). ii. Take the first contour point as the center, and start the search for the second contour point clockwise in the 8-neighborhood with (x 1 , y 1 ) as the starting point, and denote the second contour point as (x 2 , y 2 ).The basis for judging the contour point is: if the four adjacent points of a certain point are white points, it is not a contour point, otherwise it is a contour point. iii. Take the second contour point as the center, repeat the step in ii. Until it returns to (x 1 , y 1 ), it means that the traversal of all contour points of the first defect has been completed, and these contour points are denoted as set are a total of n defects in the scattering image, the contour point coordinate set of the j-th defect is denoted as S j (j=1,2,…,n). b. Feature matching and classification The commonly used feature matching method is template matching. If the morphological similarity of the features in two images is greater than a threshold, the two features are considered to belong to the same target. The defects are mainly linear scratches and dotted polished dots, and there will be a lot of similar features in the image. And the scattering image and the fluorescence image are taken by different imaging mechanisms, even the same defect may show different topography in the two images. Therefore, template matching is not suitable for the feature matching of defects. The spatial information of the defects is used for feature matching in this paper. As shown in Fig. 7c, if the contour of a defect in the scattering image does not overlap with any contour in the fluorescence image, the defect belongs to Type I (SD, which only exists in the scattering image). If the contour of a defect in the fluorescence image does not overlap with any contour in the scattering image, the defect belongs to Type II (covered SSD, which only exists in the fluorescence image).
If the contour of a defect overlaps on the two images, the defect belongs to Type III (extended SSD, which exist in both scattering and fluorescence images).
The key to feature matching is to accurately determine whether the contour overlaps in the two images. The judgment criterion used in this paper is: if there are two points on the contour of a defect that are located within the area contained by the contour of another defect, the two defects are spatially overlapped. For example, take one point (x F , y F ) in the first defect F 1 to determine whether it locates within the irregular area contained by the first defect S 1 . As shown in Fig. 7d.i, the rough judgment is carried out first. The maximum and minimum values of the abscissa and ordinate in S 1 are denoted as x max , x min , y max , y min respectively. According to the coordinates of the above four points, a circumscribed rectangle of S 1 can be constructed. If (x F , y F ) is outside this rectangle, that is: Then (x F , y F ) must be outside the irregular area of S 1 , and the coarse judgment is completed. On the contrary, if (x F , y F ) is within the circumscribed rectangle, the Ray Casting Algorithm [27] is used for fine judgment. As shown in Fig. 7d.ii, a horizontal straight line through (x F , y F ) is drawn, and the number of times that the straight line intersects the contour of S 1 are calculated. If the number of intersection points on the left and right sides of the point are both odd, then the point is inside the contour, otherwise it is outside. The number of intersections on the left and right sides are 3 and 1 respectively shown in Fig. 7d ii, so it can be judged that (x F , y F ) is in the irregular area contained by S 1 . The implementation of fine judgment are provide in references [27,28]. Compared with the point-by-point comparison in the connected domain, this feature-level matching only traverses the points on the contour, reducing the time required for feature matching. And the number of judgments is greatly reduced by two rounds of judgment. After all the points in F 1 are traversed, if there are two points located (x F > x max ) (x F < x min ) y F > y max y F < y min in the irregular area contained by S 1 , it means that the two defects F 1 and S 1 are overlapped. Comparing all the features in F i and S j , all the overlapped defects can be indentified and marked, which belong to type III. After the marked defects removed from S j , the remaining ones belong to Type I. And after the marked defects removed from F i , the remaining ones belong to Type II. c. Defects extraction After the classification of all defects is completed, the final step of image fusion is to extract defects to get fused images with accurate physical meaning. The image in Fig. 7e.i is the superposition result of the scattering and fluorescence images in Fig. 7c. This image retains the information taken by two sensors to the greatest extent, and Type I, Type II and Type III defects are set to red, green and yellow respectively. Fig. 7e.ii-7e. iv are fusion images after defects extraction. A single image does not cover all the information taken by two sensors, but each one characterizes a type of defect and has a more accurate physical meaning.

Conclusion
SDs can be detected by scattering imaging, but the detection results will be interfered by SSDs; SSDs can be detected by fluorescence imaging, but it is difficult to distinguish between the extended SSDs and covered SSDs. Based on the scattering and fluorescence imaging principles of polished optics, a multisensor image fusion detection method for SDs and SSDs is proposed. Two image sensors are used for wide-field imaging in two modes at the same time, which has the advantages of large imaging range and high detection efficiency. The scattering and fluorescence images are processed by registration and fusion algorithms, and after contour extraction, feature matching and classification, three types of defects are extracted. The method provides a rich reference for the quality evaluation of the optical surface processing, which is beneficial to improve the processing technology, reducing various defects in a more targeted manner.