CONFORMING IDENTIFICATION OF THE FUNDAMENTAL MATRIX IN THE IMAGE MATCHING PROBLEM

Abstract The article considers the conforming identification f the fundamental matrix in the image matching problem. The method consists in the divisi on of the initial overdetermined system into lesser dimensional subsystems. On these subsys tems, a set of solutions is obtained, from which a subset of the most conforming solutions is defined. Then, on this subset the resulting solution is deduced. Since these subsystems are for med by all possible combinations of rows in the initial system, this method demonstrates hig h accuracy and stability, although it is computationally complex. A comparison with the meth ods of least squares, least absolute deviations, and the RANSAC method is drawn.


Formulation of fundamental matrix
identification problem A model of a pinhole camera is used, assuming that both images are obtained by cameras with the same matrix of internal parameters: where f is a focal length of cameras, (u0, v0) are coordinates of the main points of the cameras in the coordinate systems associated with the cameras. Let M be a point in the global coordinate system. The coordinate vector of point M in the global coordinate system is related to the coordinate vectors of this point m1 and m2 in the coordinate systems of the first and second cameras by the equation [4]: Here R1, R2 are the 3×3 matrices describing the rotation of the coordinate systems of the first and second cameras relative to the global one, and t1 = [t1,x, t1,y, t1,z] T , t2 = [t2,x, t2,y, t2,z] T are coordinates of the origin of the global coordinate system in the coordinate systems of the first and second cameras, respectively.
The matrix R is formed as The corresponding points on two images are related by a fundamental matrix [4]. For points whose coordinates are given by 3×1 -vectors m1 = [u1, v1, 1] T and ' ' 2 2 2 , ,1 The equation for calculating the fundamental matrix has the form: ξ are errors related with inaccurate calculation of the coordinates of the corresponding points. Estimate с of vector c can be obtained by solving the system (2) using the LSM, the LADM, RANSAC [5], or conforming identification. The number of observations, on which the system (2) is formed, is extremely small, therefore, we apply the conforming identification method.

Formulation of conforming identification problem
We consider the problem of estimating the vector с of linear model parameters: where y and X are N×1-vector and N×M -matrix observed in the experiment, and ξ = [ξ1, ξ2,…, ξN] T is N×1-vector of unknown errors. The matrix X is composed of rows xi, The task of identification is to calculate vector of estimates ĉ using observations y and X.
If there is no a priori error information, then the leastsquares method is usually used: It is known that LSM estimates are unbiased and effective under the usual assumptions. However, with a small number of observations, these assumptions prove to be unreliable because of the insufficient statistical stability of probabilistic characteristics. The method of conforming identification is based on the assumption that the solutions obtained on the subsystems that are most free of noise will be closer to each other (i.e., conformed), and the task is to determine such a subsystem. Here is a brief description of the identification algorithm based on the principle of conformity of estimates.
A certain set of subsystems of small dimension is "extracted" from the initial system (3): Each subsystem (4) contains the rows of the initial system (3). Let us further refer to these subsystems as lower-level subsystems. In this case, the set of lowerlevel subsystems contains subsystems with square M×Mmatrices Xk. By calculating an estimate ˆk c from available observations Xk, yk for each subsystem (4), we can obtain Each l th higher-level subsystem (5) contains a set of lower-level subsystems (4), on which the corresponding set Θ(l) of estimates .. is calculated: To characterize the conformity of sets Θ(l), a function of mutual proximity of the estimates is introduced: where , ,, are the estimates obtained on lower-level subsystems contained in the l th higher-level subsystem. Indices i and j in the right side of the expression (6) take all possible pairs of values. The set Θ(l) of estimates , , 1, , The hypothesis is that the most conformed subsystem is the most noise-free, so the task is to find the index l of the higher-level subsystem: ( ) min ( ), 1, , At the l th higher-level subsystem, we can either calculate an estimate or a "cloud" of estimates [2].
Admittedly, the implementation of the described conforming identification algorithm requires large computational resources. So computing time for identification problem with parameters N = 10, M = 5, P = 9 is 0,01 seconds and computing time for identification problem with parameters N = 18, M = 9, P = 16 is 793.530322 seconds. This is due to the lack of a priori information and a small number of observations. That is why it becomes necessary to use an efficient parallel algorithm to decrease the computation time.

Parallel algorithm of conforming identification
In paper [6] a parallel algorithm of conforming identification was considered. Sets of higher-level subsystems were formed on different processors. Subsystems indices are formed sequentially because it is necessary to form all possible indexes, which leads to the downtime of the processors. It is worth noting that the indices of the subsystems of the subsystems of the upper and lower levels can be formed once. It will eliminate the downtime of the processors in further calculations of the solutions. Fig. 1 shows the general block diagram of the proposed algorithm and detailed diagrams of the algorithm stages, where Fig. 1a shows general block diagram, Fig. 1b demonstrates data loading and subsystem generation, Fig. 1c shows the calculation of the higher-level subsystem with the least conformity coefficient for each thread, and Fig. 1d shows the selection of the higher-level subsystem with the least conformity coefficient and calculation of the solution.
Because of the need to combine all the results of calculations, the master-slave communication topology and MPI are used.
For the above algorithm, the speedup and efficiency characteristics were calculated by solving the identification problem with parameters N = 18, M = 9, P = 16. The results are shown in Table 1. The calculations were performed on a supercomputer "Sergei Korolev". One node with two processors, which have 4 cores each, was used.
The data in Table 1 shows that constructed parallel algorithm is well scalable. Its execution time decreases almost linearly proportionally to the number of threads used. This algorithm is quite efficient because the calculation is evenly distributed among the threads. With the increase in the dimension of the original system (the number N of rows), the execution time of the program is significantly increased.

Experimental study of accuracy and reliability
When modeling the initial data, the following parameters are used: 1 2 960 0 960 0 540 960 0 0 1 Fig. 1

. Block diagram of the algorithm
When forming the elements of the matrix R2 in (1), the angles α2, β2, γ2 are set in the interval [0°, 8°]. Vector t2 is specified as follows: 2 cos( ) sin( ) , [5,6], With the above mentioned parameters, 100 sets of points m1 and m2 are generated. In the formed sets of points Gaussian noise with SNR = 70 and mean equal to zero is added. Then a random gross error is added. For each modeled set of corresponding points, a system of linear equations is formed. Next, estimatesĉ of the fundamental matrix coefficients are calculated using the LSM, the LADM, and the conforming identification method. Each element of the resulting matrix F is normalised: Table 2 gives an example of the obtained coefficients of fundamental matrices calculated using the LSM, the LADM, and the method of conforming identification (CIM). To determine the reliability of methods for each system, a set of 500 corresponding points m1 and m2 is generated. For each pair of corresponding points, distances to epipolar lines , =1,500 , where dki is the distance from i th test point to the epipolar line for k th set of points m1 and m2, K = 500. Table 3 shows the maximum and minimum values of dk for each method.  (7), histograms are formed. Тhe interval of possible values is divided into 20 intervals ∆dl, 1, 20 l = and for each interval, the probabilities ˆl p for the values of the criterion dk∈∆dl are determined: The graphs in Fig. 2 show that the conforming identification method has higher accuracy and reliability in comparison with the LSM and LADM. In the problem of fundamental matrix identification, the least-squares solution will give a more accurate solution than the LADM.
The study of accuracy and reliability of the CIM and RANSAC methods was also carried out. Fig. 3 shows a histogram of distances and distribution function of dk . for both methods.
The graphs in Fig. 3 demonstrate that the CIM slightly exceeds the RANSAC method in accuracy and reliability. Also, unlike RANSAC method, the conforming identification method does not require setting the threshold value and the number of iterations.
The algorithm was applied to real images form the set "Temple of the Diskouroi". To find the feature points in the images the algorithm SURF [7] from OpenCV library was used. Figure 4 shows the selected images and the obtained corresponding points.
Using the selected feature points, we obtain a system of linear equations to which an extra gross error was added, and the fundamental matrix is further calculated. Similarly, for the generated test points the distance di, and then parameter d were calculated. The conformity coefficients for the CIM and RANSAC equal 8803,506 and 12297,313, respectively.

Conclusion
Experimental studies demonstrate that for the problem of the fundamental matrix identification the method of conforming identification ensures a more accurate solution, and has a higher reliability as compared to the LSM and the LADM. The conforming identification method slightly exceeds the RANSAC method in terms of accuracy and reliability. Also, the CIM does not require setting the threshold value and the number of iterations.