Application of the Kimmel method for two-layer matrix color separation systems

. In this work, interpolation by the Kimmel method is applied for the developed color separation systems based on matrix photodetectors. Author's two-layer receivers with an equal distribution of layers of the visible and infrared spectra over the sensitive surface are given. To compare the interpolation result, it was proposed to use a matrix photodetector with a modified standard RGBG system on RGBIr. This system shows what a multispectral receiver would be like without the use of multilayer silicon technologies. Thus, it was possible to show that the use of two-layer color separation systems simplifies the interpolation algorithm by a factor of two. The developed systems can be useful in various fields of science and technology. The use of such multispectral matrices is necessary especially in agriculture and forestry for monitoring the environment and detecting fire


Introduction
Fires are the most acute problem in agriculture and timber industry. Modern technologies are fighting against them: quadrocopters, shooting with matrix photodetectors, digital image processing, image recognition -fire [1][2][3][4][5]. The latter can be most accurately detected by infrared spectrum recording receivers [6,7]. But for the correct recognition of the object of study and the surrounding area where the survey takes place, surveys in the visible spectrum are also necessary [8,9].
The author has developed patterns for matrix photodetectors with the function of operating in the visible and infrared ranges [10,11]. The essence of the proposed patterns is to combine the capabilities of Bayer patterns and multilayer systems. The latter have high noise performance due to the large number of layers in one cell, the Bayer-type layers are placed apart in space and complicating the process of interpolation and finding the primary colors for each cell of the matrix photodetector. The proposed modification makes it possible to separate two-layer cells in space, which will reduce noise relative to multilayer matrices of this type, as well as simplify interpolation methods with respect to Bayer patterns. The purpose of the study is the application of interpolation by the Kimmel method for the developed multispectral photodetectors. Within the framework of the purpose, the following tasks are set: the application and comparison of interpolation for different receiver patterns.
Receiver-1 was proposed for the comparison following the example of the standard Bayer RGBG pattern ( Fig. 1, a), but with the last green pixel replacement by an infrared cell ( Fig. 1, b). Thus, it will be possible to adequately compare a simple spatial system with registration of the visible and infrared spectra with the developed two-layer patterns of the same capabilities. The next receiver-2 is developed in such a way that 50% of the entire sensitive surface is covered by each primary and infrared color (Fig. 1, c). The pattern consists of a pair of two cells: the first is a pair of blue and red layers, the second is a pair of green and infrared.
Receiver-3 also consists of layers, where each layer covers 50% of the entire sensitive surface (Fig. 1, d). The pattern consists of a pair of two cells: the first is a pair of red and infrared layers, the second is a pair of blue and green.
These receivers allow to obtain a multispectral image in one frame. The developed systems can be used in various fields of science and technology. The use of such multispectral matrices is especially necessary in agriculture and forestry for monitoring the environment and detecting fires.
In the figures, the upper layers are shown much smaller than the lower ones, although this is technologically incorrect. Such a ratio is given only for a visual representation of the operation principle of the proposed type matrix and illustrating the fact that with the exclusion of one layer, it is possible to increase the sensitive area of the next. One for a more detailed analysis, it is necessary to carry out studies of the light flux pussing through the matrix with the function of registering IR radiation.
The Kimmel method consists of three steps [12]: 1) finding green color; 2) finding red and blue colors using the found green; 3) correction of expressions.
In Kimmel's method, the green color is found through linear interpolation. The remaining calculations are made through the derivatives of the color D and the weights of these colors E, as in the adaptive method. The weights E are calculated in a linear combination with the probability that the desired pixel belongs to the same image object as the one under study.

Interpolation of Receive-1
A receiver of this type is presented as an example if a multispectral array photodetector with an IR filter was obtained based on a standard RGBG spatial color separation system. Thus, it will be possible to qualitatively compare the algorithms of the developed two-layer receivers with this system.
Let's consider the RGBIr pattern interpolation algorithm. For cell 7, we need to find red, green and blue: Determine E: Find R, B and G for cell 7: For cell 11, we need to find blue, green and IR radiation: Determine E: Find B and G for cell 11: Let us determine the IR radiation for cell 11 using the adaptive method For cell 6, we need to find the red color, green color and IR radiation: Dx(G22)=(G21-G23)/2 Determine E: Find R and G for cell 6: Find the IR radiation for cell 6 using the adaptive method For cell 10, we need to find the red color, blue color and IR radiation: Find R for cell 10 by interpolating red with green and blue: Find the IR radiation for cell 10 using the adaptive method IR23=(IR12+IR32+IR14+IR34)/4 As we can see, the interpolation algorithm of this receiver is quite voluminous and contains many different calculations.

Interpolation of Receive-2
Let's consider the algorithm for receiver-2. This receiver is balanced and should allow to reduce the algorithm for determining the missing colors in each cell. For cell 6, we need to find the green color and IR radiation. Let's find D and E using different colors: The value of D is determined by the layer of the cell: Other values of E are assigned the value of the layer in the cell: Find the green color by interpolating through the red and blue colors: Comparing the edge-adaptive method [21] and interpolation along the diagonal of green through blue and red, the edge-adaptive one is better and simpler. When interpolating the green color diagonally, we first need to find B and R for cells 1,3,9,11, which complicates finding the green color. There is one drawback in the interpolation method through green color: it is necessary to interpolate the B and R values for cells 2, 4, 10, 12. Therefore, for cell 7 it is better to use the adaptive method. It can be seen from the given algorithm that for half of the cells, the values of E and D do not require calculation.

Interpolation of Receive-3
This receiver is also considered to be balanced in terms of the colors distribution over the sensitive surface of the matrix photodetector. Such an arrangement of layers should also allow to reduce the algorithm for determining the missing colors in each cell. For cell 7, we need to find the red color and IR radiation: Finding E with Red: Find R for cell 7 by interpolating red with green and blue: Find the IR radiation for cell 7 using the adaptive method For cell 11, we need to find blue and green color: Find blue and green color in cell 11: As expected, the algorithm for this receiver-3 is reduced by almost two times compared to receiver-1, and there is no search for colors along the diagonal. The developed two-layer color separation systems based on the silicon ability to absorb a certain wavelength depending on the depth of the potential well are presented. This technology makes it possible to incorporate ultraviolet and infrared radiation channels into standard patterns of visible colors. Due to this, multispectral matrix photodetectors are obtained, requiring new approaches in interpolation.

Conclusion
The Kimmel interpolation method was used, on the example of two-layer matrices with cells for registration of the visible and infrared ranges. This method is based on adaptive interpolation, therefore, in the course of work, some calculations were carried out using this method. The RGBIr system is inferior to two-layer systems, because its pattern consists of four pixels, and not two, as in the developed systems. All this leads to the fact that the algorithm, while processing a standard pattern, turns out to be 2 times more complicated. From the obtained equations, we can conclude that using such modified interpolation methods, it is possible to speed up the image processing process by about 2 times.
All this points to the competitiveness of the development of two-layer multispectral matrix photodetectors. And the possibility of registering visible and infrared ranges on one receiver makes it promissing for implementation in monitoring forests and agricultural fields to detect fires.