High-precision Multichannel Solar Image Registration Using Image Intensity

Solar images observed in different channels with different instruments are crucial to the study of solar activity. However, the images have different fields of view, causing them to be misaligned. It is essential to accurately register the images for studying solar activity from multiple perspectives. Image registration is described as an optimizing problem from an image to be registered to a reference image. In this paper, we proposed a novel coarse-to-fine solar image registration method to register the multichannel solar images. In the coarse registration step, we used the regular step gradient descent algorithm as an optimizer to maximize the normalized cross correlation metric. The fine registration step uses the Powell–Brent algorithms as an optimizer and brings the Mattes mutual information similarity metric to the minimum. We selected five pairs of images with different resolutions, rotation angles, and shifts to compare and evaluate our results to those obtained by scale-invariant feature transform and phase correlation. The images are observed by the 1.6 m Goode Solar Telescope at Big Bear Solar Observatory and the Helioseismic and Magnetic Imager on board the Solar Dynamics Observatory. Furthermore, we used the mutual information and registration time criteria to quantify the registration results. The results prove that the proposed method not only reaches better registration precision but also has better robustness. Meanwhile, we want to highlight that the method can also work well for the time-series solar image registration.


Introduction
Solar multichannel observations are extremely important for studying solar activity. However, the observation data obtained from different instruments or channels have various resolutions and different fields of view (FOV). So, the images must be registered before analyzing them. Image registration aims to find the greatest similarity between an image to be registered (called the moving image) and a reference image (called the fixed image), and further to obtain a transformation parameter vector μ(r, s, x, y), i.e., rotation angle r, scale s, and shift x and y along with the X and Y directions. Furthermore, the parameter vector is used to align the two images. But, how to find the parameter vector is still an enormous challenge.
Cross correlation is a common practice in solar image registration (Feng et al. 2012;Yang et al. 2015). But because the fixed image energy is usually not constant and varies with the feature position and different spatial scales, the cross correlation method cannot usually succeed. Another drawback is that the region of the cross correlation heavily relies on the size of the moving image and the amplitudes of the fixed image. Featurepoint matching can register the images with different resolutions and rotations and has been introduced into solar image registration (Yang et al. 2020;Deng et al. 2021). Its registration accuracy depends on the number of feature points between the pair of images. However, the feature points of solar images are unstable, fewer in different channels, and have obvious differences. Therefore, the accuracy of the feature-point matching method is greatly uncertain. Phase correlation is another solar image registration method (Hrazdíra et al. 2020(Hrazdíra et al. , 2021 that is used to measure the solar differential rotation. In this paper, we proposed a novel high-precision registration method to align multichannel solar images using image intensity similarity metrics (Styner & Brechbuhler 2000). The intensity similarity metrics use the statistical characteristics of the intensity distribution between the fixed image and the moving image to find an optimum vector (called transform parameters). Our registration method adopts the coarse-to-fine strategy to find the optimum parameters. In the coarse registration step, we used the regular step gradient descent (RSGD; Eikvil et al. 2005;Mambo 2018) algorithm as optimizer to continuously iterate the transformation parameters to bring the normalized cross correlation (NCC; Lewis 1995;Yu et al. 2021) similarity metric to a maximum. The coarse transformation parameters are then taken as the initial values of the Powell-Brent (PB;Powell 1964Powell , 1975Brent 2013) algorithm to minimize the Mattes mutual information (MMI; Mattes et al. 2001Mattes et al. , 2003Rahunathan et al. 2005) similarity metric for high-precision transformation parameters.
The paper is structured as follows. Section 2 explains the NCC and MMI similarity metrics and the RSGD and PB algorithms. Section 3 describes our method and illustrates the registration processing using a pair of images with different channels and FOV. In Section 4, two pairs of solar images from different instruments and channels are selected to further illustrate the accuracy of our method. Section 5 evaluates our registration results by quantitative comparison to other registration methods. Finally, our conclusion is given in Section 6.

Methodology
The aim of image registration is to quantify the similarity between the fixed image and the moving image. Our proposed method is iterative registration processing. The iterative processing requires a pair of images, a similarity metric, and an optimizer. The similarity metric quantifies the similarity of the pair of images, and the optimizer maximizes (or minimizes) the similarity metric. The process repeatedly calculates the similarity metric of the pair of images according to the parameter vector μ and the optimizer continuously adjusts the transformation vector until the metric reaches a given threshold or convergence.

NCC Similarity Metric and RSGD Optimizer
The NCC method overcomes the drawbacks mentioned above regarding cross correlation by normalizing the fixed image and the moving image to unity to yield a cosine-like correlation coefficient (Lewis 1995). The RSGD algorithm acts as an optimizer to find a parameter vector μ that makes the NCC similarity metric between the moving image and the fixed image reach a maximum. It is a gradient descent method, and continuously updates μ according to the gradient at the current position μ k , i.e., the current transform parameter vector, through iterations, where μ k+1 represents the next parameter vector. ∂S NCC /∂μ is the gradient at position μ k and a k denotes the decaying step size or learning rate, which is a nonnegative value. The value gets smaller when the gradient changes its direction determined by the inner product of ∂S NCC /∂μ. where κ is a relaxing factor and set to 0.5. If the gradient value or the relaxing factor is less than a given value, or the iteration number reaches a threshold, the coarse registration process will be achieved, and the coarse registration vector μ will be obtained.

MMI Similarity Metric and Powell-Brent Optimizer
The mutual information (MI) metric measures the similarity between the moving image and the fixed image from the perspective of information entropy. The smaller the image registration error, the smaller the joint entropy, and the greater the MI metric value. A high MI value implies that the images are likely better registered. The MMI algorithm is a more effective method to represent the MI metric. Calculating the similarity metric of the pair of images using MI is described as a maximization problem, but the aim of MMI is to minimize the negative MI. MMI uses the joint probability distribution of the pixels to measure the similarity of the pair of images. B-spline kernels are used to compute the probability density functions. MMI uses a single set of pixel locations for persistent optimizing, instead of continuously drawing a new set at each iteration. The samples are used to calculate the probability density and the bins of the samples are used to calculate the uncertainty (Rahunathan et al. 2005).
The PB algorithm is taken as an optimizer to minimize MMI by performing a line search. It is an iterative multidirectional search that exactly performs an iterative fitting of a parabola to each transformation parameter (scale, rotation, and shift) profile to minimize MMI, ( ) denotes the initial position of the kth iteration, in which m 0 0 ( ) is the coarse registration parameters. λ i is a step size that brings MMI into the minimum. The processing uses the Brent linear search method along the four directions given by ) as a new direction to obtain μ (k+1) using the line search. If the Euclidean distance ||μ (k+1) − μ (k) || is less than a given threshold ò, the fine registration step reaches convergence and the iteration stops. μ (k+1) is the final transformation vector. Otherwise, the algorithm again selects the four directions from d i k ( ) (i = 2, 3, 4, 5) to continuously iterate the process until ||μ (k+1) − μ (k) || ò. This step can avoid the linear correlation of the foregoing four directions that could make the algorithm fail to reach convergence. Detailed descriptions of the PB algorithm are given by Powell (1975) and Brent (2013).

Implementation Process
We chose a pair of images with different resolutions and FOV obtained by the Goode Solar Telescope (GST; Cao et al. 2010) at Big Bear Solar Observatory on 2012 May 22 at 17:17:10 UT. The two images observed in the Hα and TiO channels are shown in Figures 1(a) and (b), respectively. We see that their resolution and FOV are different according to the umbra and several pores. Next, we will use the two images as an example to illustrate the registration processing. Here, the TiO image is taken as the moving image and the Hα image as the fixed image.

Image Pyramid
Our method requires us to use image intensity information multiple times for finding the optimum registration parameters. Therefore, we generated multiscale Gaussian pyramid images with different scales using Gaussian convolution kernels to improve the convergence speed. Here, we set a three-layer pyramid with the first layer being 16× downsampling of the original image, the second layer being 4× downsampling, and the third layer being the original image.

Coarse Aligning
The registration processing starts from the first layer of the image pyramid, and then the result obtained by the first layer is used as the initial parameter of the second layer until the final coarse alignment is completed. In the first layer, the parameter r 0 of the initial vector μ 0 in Equation (1) is set to 0, s is equal to 1, and the shift (x 0 and y 0 ) is set to the distance from the geometric center of the TiO image to the geometric center of the Hα image. The minimum step size is set to 1 × 10 −5 , the minimum gradient value is 1 × 10 −6 and the maximum iteration number is 75. If the step size is less than 1 × 10 −5 , or the gradient value is less than 1 × 10 −6 , or the iteration numbers reach 75, then the coarse process will stop. Finally, the coarse transformation parameters are obtained. The TiO image is magnified 1.055 times, counterclockwise rotated 0.442°, and shifted 446.528 and 366.166 pixels along the x-and y-axes. The transformed TiO image is shown in Figure 1(c) where the Hα image is stitched together to better compare the alignment result. The TiO image is on the right side and the Hα image is on the left side. In Figure 1(c), we see that the bright bridge in box B has been coarsely aligned to the Hα image. But, the pores in box A are not completely aligned and the pores in the TiO image have a slight divergence. This means that the transformed TiO image is not accurately aligned to the Hα image.

Fine Aligning
The coarse registration parameter μ(1.055, −0.442, 446.528, 366.166) is taken as the initial value of the fine alignment. The PB algorithm is considered as the optimizer to find the transformation parameter μ that makes the MMI reach the minimum. The Euclidean norm ò is set to 1 × 10 −9 . If the Euclidean distance ||μ (k+1) − μ (k) || is less than ò, the fine registration process will stop. Finally, the fine registration vector μ(1.354, −0°.997, 482.667, 353.257) is obtained. Figure 1(d) shows the fine registration result. Similar to Figure 1(c), the Hα image and the registered TiO image are also stitched together. Compared with the features in the two boxes of Figure 1(c), the penumbral fiber structures are precisely aligned besides the bright bridge, and the pores are better overlapped with each other in box A.

Results
We selected two pairs of images with different channels and different FOV to further demonstrate the applicability and robustness of the proposed method. The pair of images is the TiO image obtained by GST and the continuum image provided by the Helioseismic and Magnetic Imager (HMI; Schou et al. 2012) on board the Solar Dynamics Observatory (SDO; Pesnell et al. 2012). The other pair of images is observed by GST in the Hα blue-and red-wing channels.

GST TiO Image and SDO/HMI Image
The TiO image was observed on 2015 January 5 at 19:12:43 UT, and its pixel resolution is 0 0376. The other is the continuum image observed by SDO/HMI at the same time whose pixel resolution is 0 6. The full-disk continuum image is shown in Figure 2(a), and the TiO image is shown in Figure 2(b). Here, the TiO image is the moving image and the continuum image is the fixed image.
The pixel resolution of the TiO image is about 15 times higher than that of the continuum image. To improve the convergence speed, we completed image preprocessing of the pair of images. We first decreased by 15 times the sample rate of the TiO image to generate a new downsampled image. Furthermore, we calculated the NCC coefficient between the downsampled TiO image and the full-disk solar image to locate a coarse region to be aligned in the full-disk image according to the NCC maximum position. The coarse region to be aligned is marked with a gray box in Figures 2(a) and (c) shows much richer image details about the coarse region. Here, this coarse region is taken as the fixed image and the downsampled TiO image as the moving image. Here, we note that in the step we  only calculated the NCC maximum to obtain the coarse position in the full-disk image to improve the convergence speed. The process is different from calculating the NCC gradient value in the coarse step.
The final transformation vector μ is obtained. The pixel resolution of the TiO image is 17.568 times that of the continuum image, and the rotation angle is 19°. 465. The registered TiO image and the continuum region are overlapped and shown in Figure 2(d). The pores and penumbral filaments pointed to by the red arrows are better aligned together. If the pixel resolution (0 6) of the full-disk image is taken as a reference, the pixel resolution of the TiO image is 0 0342. Figures 3(a) and (b) show the Hα blue-(100 Å) and red-(80 Å) wing images closer to the solar edge obtained by the GST on 2015 July 14 at 18:15:09 UT. Their pixel resolution is 0 029. We can see that the bright bridge and penumbral filaments in the sunspot are deformed due to the projection effect. Compared to Figure 3(a), panel (b) has slight rotation angles and shifts. The blue-wing image is taken as the fixed image and the red-wing image as the moving image. The fine alignment vector μ(1.003, −6.101, −4.361, 0.264) is obtained by our method. According to the registration parameters, one sees that the pixel resolution of the blue-and red-wing images is approximately equal. But, the red-wing image must rotate counterclockwise 6°.101, and shift −4.361 and 0.264 pixels along the x-and y-axes. The final result and the stitched image are shown in Figures 3(c) and (d), respectively. The solar features in the red-wing image, such as umbral points and penumbral filaments pointed to by the three red arrows, are accurately joined to the blue-wing image, demonstrating that the moving image and the fixed image are better aligned.

Discussion
In this section, we first discuss why the coarse-to-fine method is proposed, and then compare our method to the scaleinvariant feature transform (SIFT) and phase correlation methods. Figures 4 and 5 Table 1. The "..." symbols represent that the corresponding method fails to align the data set. All experiments were completed on a Windows 10 64 bit PC with an Intel Core i7 2.60 GHz processor (6 cores and 12 threads) and 16 Gbyte RAM. The registration methods are implemented on the Python 3.8.2 platform.

Coarse-to-fine Registration
The advantage of the RSGD algorithm as an optimizer is that its search direction and step are determined by the gradient that takes the impact of all registration parameters into account simultaneously. If one of the parameters (i.e., r, s, x, y) results in a wrong direction, the impact would be averaged by all parameters. Thus, the RSGD algorithm is insensitive to the initial values of the parameters. But its inherent drawback is that the candidate parameters obtained by the algorithm are close enough to the global optimum parameters, although in most cases the global optimum parameters cannot be obtained even though we attempt to add the iteration number of the algorithm.
Compared to the RSGD algorithm, the PB algorithm takes every parameter into account in turn. The computational cost is relatively lower than the derivative-based methods, such as RSGD, which causes it to quickly approach the optimum solution area. But its limitation is that every parameter is taken into account separately. Thus, every parameter contributes a great impact in every iteration and makes the algorithm easily fall into a local optimum. Therefore, the selection of the initial parameters is crucial to whether PB can obtain the globally optimum values without falling into a local optimum. So, we used the advantage of the RSGD algorithm, i.e., insensitive initial parameters, to obtain a coarse area and then used the PB algorithm to achieve a high-precision registration.
Figures 4(f) and 5(f) present two registration results using the PB algorithm combining MMI (PB-MMI), respectively. Compared to the registration result of our methods (see Figures 4(c) and 5(c)), we can find that the registration accuracy of PB-MMI is relatively lower. But, if we only focus on the red box in Figure 4(f), the alignment result seems to be good. This is because PB-MMI falls into a local optimum. The registration result shown in Figure 5(f) seems to be successful. But, if we focus on the small pore pointed to with a red arrow in the bottom left of Figure 5(f), two images that contain the same pore fail to align with each other. Compared to the results in Table 1, the registration time of the coarse-to-fine method is comparable to that of the PB-MMI method, but our MI values have been significantly improved. The fact that the PB algorithm easily fails or the registration accuracy is relatively low is related to its optimization process. However, adopting the coarse-to-fine registration strategy not only considerably decreases the registration time but also improves the registration accuracy.

Comparison and Evaluation
We selected the two classical solar image registration methods, SIFT and phase correlation, to evaluate and compare the performance with different methods. We selected the library provided by the OpenCV-Python package 7 to complete the SIFT experiments. For phase correlation, we used the Python library 8 that implements an FFT-based method (Reddy & Chatterji 1996). The FFT-based phase correlation method can align the images with different translations, rotations, and scales.
We tested the SFIT method in five pairs of images, and got four decent alignments and a failed case, which is illustrated in Figure 5. As listed in Table 1, the MI values of our method are slightly higher than that of the SIFT method, and the registration times of our method are significantly superior to the SIFT method. Figure 4(d) shows the registration result aligned by the SIFT method. The registration accuracy is comparable to our method (Figure 4(c)). SIFT does not work well if the pair of images have distinct image features, for example, Figure 5(d). This is because the pair of images (Figures 5(a) and (b)) present different image features that mean few feature points are successfully matched by SIFT. The three pairs of images shown in Figures 1, 3, and 5 are successfully aligned by the phase correlation method. Figure 5(e) shows a successful case. The registration accuracy is comparable to Figure 5(c). But, the fact that only three pairs of images were aligned successfully proves that the robustness and stability of phase correlation are inferior to our method.

Conclusion
We proposed a coarse-to-fine registration method based on image intensity similarity metrics. We first used the RSGD algorithm as an optimizer to bring the NCC similarity metric to the maximum for obtaining a coarse registration vector. Subsequently, the coarse vector is taken as the initial value in the fine registration step to further align the moving image to the fixed image. In the fine registration step, the PB algorithm is taken as the optimizer and MMI as the similarity metric. We selected five pairs of images with different resolutions, rotation angles, and shifts observed by GST and SDO/HMI to evaluate the registration results. Meanwhile, the MI and registration time criteria are used to quantify the registration results obtained by our method, SIFT, and phase correlation. The registration results and the quantitative values prove that our proposed method has much better alignment accuracy and robustness, and its efficiency is also very high. Moreover, we want to highlight that our method not only registers multichannel images but also resolves the alignment problem of time-series images in the same channel.