ANALYSIS OF GROUND SAMPLING DISTANCE OF CONVERGENT IMAGES FOR KEYPOINTS DETECTION FOR CLOSE-RANGE PHOTOGRAMMETRY

: Keypoint detection for image matching is an important step in close range photogrammetry. It essentially depends upon ground sampling distance (GSD) of an image. For a convergent image, GSD variations on the either side of image axis are not equal. Moreover, GSD also varies according to camera position placed at constant distance from an object. This paper investigates and analyses the GSDs of convergent images for various geometrical configurations of camera positions on a circular arc for a building corner. GSD expressions and rates of GSD change are derived for left and right edges of a convergent image. Both GSD and rate of GSD change are characterized by non-linear mathematical functions of camera FOV, and its position on circular for a given corner. Experiments are conducted to acquire varying number of convergent images on circular arcs of different radius. Keypoints for convergent images are influenced more by rate of GSD change than the GSD. The study determines a critical value of 28 for rate of GSD change. The correct matching of keypoints in two images is limited within FOVs corresponding to the critical value. An example demonstrating the correct keypoint matching for two images is presented.


INTRODUCTION
Close range terrestrial photogrammetry (CRTP) is one of the popular method for high-resolution 3d modelling of complex building structures such as heritage (Martínez et al., 2013). CRTP acquires multi-view convergent images and creates 3D model using common keypoints among all images. For building corners consisting of two non-coplanar planes, CRTP typically uses a network of convergent geometry images at constant radial distance from the object corner (Forstner andWrobel, 2016, García-Gago et al., 2014). Operators like scale invariant feature transform (SIFT) are used for locating the keypoints and corresponding matched points in the two images. The detection of keypoints is influenced by factors like image scale and spatial resolution of the images (Mikolajczyk and Schmid, 2005, Westoby et al., 2012, Chen et al.,2018. For a normal image, key point detection by feature opeartors performs better as the change in the image scale is less than 1.5 (Mikolajczyk and Schmid, 2005). Also, number of corresponding keypoints do not vary, even if normal images are acquired at different distances. On the other hand, spatial resolution, which is a measure of resolving and identifying two features, is a function of ground sampling distance (GSD). For a non-topographic application, Costa et al (2016) successfully demonstrated that more number of keypoints are detected for low GSD images. On the other hand, GSD also influences the accuracy of the 3D model generated (Flyability, 2022).
GSD for convergent images are not same as that of normal images for CRTP. Figure 1 below shows acquisition of two images by convergent geometry and one image by normal geometry capturing a building corner. Images acquired at camera positions C1 and C2 are examples of convergent geometry, whereas normal geometry image is acquired at camera position S. The camera optical axis at C1 and C2 are pointing at building corner. However, at S optical axis of camera is perpendicular to wall. For the normal geometry, GSD varies symmetrically from the centre to both ends of an image as the distances from camera to wall, SL1 and SR1, are equal. On the other hand, for convergent geometry (at C1 and C2), distances, dL1, dR1, dL2, and dR2 are different even the images are acquired at same distance from the building corner. Figure 2 below shows normal geometry capturing a building plane. It is evident from the figure 1 and figure 2 that for both convergent and normal geometry images GSD is minimum at centre and increasing towards edges. However, for the same checker board positions on the object, the dimensions in the convergent image of building corner are reducing non-uniformly when compared to the normal image. It is apparent that though F and H are the edges of same convergent image, the GSD at F and H are different. Similarly, GSD at E and G are not equal. This reveals that for a convergent image: (1) GSD varies within an image non-uniformly from centre to edges, (2) GSD variation is different for various camera positions (Thomson et al., 2020), and (3) GSD variation is not symmetric about the optical axis of a convergent image and consequently GSD at two edges of a convergent image are different. These three factors may impact the keypoint detection for a pair of convergent images. Moreover, for view planning, where a group of convergent images is captured at a particular distance around a corner on a camera network for 3D modelling, effect of GSD on keypoint detection can be more severe. This paper investigates the effect of GSD variation for convergent images on keypoint detection. The study is supported by field data and detailed analysis. Based on the experiment and simulation, this study derives a criterion for favourable portions in images for keypoint detection and associated processes. The paper is organized into four sections. After the introduction in this section, second section derives and simulates GSD terms for convergent geometry and compare with that of normal geometry. More over the section also analyses keypoints variation with GSD variation and illustrates the results. The paper also demonstrates an example showing portions of matched keypoints in two images due to influence of GSD variation. The fourth section presents conclusion.

GSD OF CONVERGENT IMAGES
2.1 GSD for normal geometry image Figure 3a shows a normal image acquired using a camera of field of view ( 2 ), at a normal distance d from the object.
(3a) (3b) Figure 3. (3a) Normal geometry (image and object planes are parallel) (after Wolf, 2018), (3b) GSD for normal geometry On the object surface, the minimum sampling distance appears at centre and it gradually increases towards edges. Defining the minimum and maximum GSDs as b R and SE R , as shown in the figure3 above, the mathamatical expressons of two terms can be written as: where θ  = Instantaneous field of view of camera (IFOV) For a camera having x p number of pixels in horizontal direction, IFOV can be expressed as: As IFOV is very small, tangent of IFOV in expression (1) is considered as IFOV. With this assumption and substituting IFOV from equation (3), equation (1) can be redefined as: GSD for all pixels of a stereo image captured by camera of 64° FOV at a normal distance of 6m from the object surface are calculated using equation (4). The ratio of GSD at edges to centre is 1.39, which signifies that GSD at edge degrades to approximately 1.4 times of GSD at the centre for a normal image.

GSD of convergent geometry image
Unlike normal images, the GSD of a convergent image does not vary uniformly (as discussed in previous section). Forthcoming discussion derives the GSD for a convergent image. Let point O be a building corner defined by two plane walls OU and OV meeting at a specific angle. Figure Figure 4 shows the geometry of a convergent image corresponding to a position C on the circular path and its GSDs on planes OU and OV.

Figure 5. Left and right GSDs of convergent image
As shown in the figure above, the left and the right edges of the acquired image will meet at D and E on the left and right planes, respectively. It is evident that GSDs at points D and E may not be equal as the distance OD and OE are not equal. To derive the GSD expressions at short distances, trigonometry and GSD at edge of a normal image are used. Accordingly, line AB is drawn perpendicular to the line OC, which defines the angular position of camera with respect to line OP by angle ψ. For triangles DOA and OBE, all angles are marked. Image pixels at points D and E are highlighted and the GSD at two points are indicated by L R and R R , respectively.
Line OC has a length equal to d . The triangle CAB presents the geometry of a normal image. Length of AB, AC, and BC are: Applying sine rule for the ∆DOA and ∆OBE give the expressions of AD and BE as: Similarly, R R needs to be projected onto OE to derive R R . As mentioned earlier, GSD at edge of a normal image is available at point A. Thus, the L R is projection of the GSD at edge at point A to point D.
Substituting variables AC, and AD in above expression and further simplification leads to: Projecting the L R on to line by sin rule gives (refer figure 5): Substituting L R from equation 9 in the above expression and further simplification derives: Similarly, the GSD at right edge, R R can also be derived. First, we define variable R R , which is projection of GSD at edge and parallel to line OB. Secondly, the R R is projected onto OE to Projecting R R on to line by sine rule in ∆OBE gives: It is evident from equations (10) and (12) It can be observed that when camera position is located in left half of the circular arc, GSD at left edge is superior to GSD at right edge at equal angular distance from the optical axis of the camera and vice versa. GSD of normal image is always superior compared to that of the convergent images. Moreover, for a given corner geometry, GSDs at left and right edges of a convergent images vary according to camera position and also degrades at different rates towards image edges. For example, for a convergent image acquired by a camera position at 39.428° on the circular path, GSD at the left edge is approximately 4-5 times higher than the GSD at the right edge. However, the former is also 5 times lower than the GSD at edge of a normal image. In other words, GSD at right edge is 25 times lower than that of the normal image. It can also be observed that GSD of left edge curve and GSD of right edge curve are mirror reflection for the camera position that are equidistant from mid-point on the circular path in opposite directions.

Rate of GSD change
For a camera positions ( ψ ), the rate of GSD change with respect to FOV (  ) can be written as: Figure 7 below illustrates the rate of GSD change w.r.to FOV variation for three camera positions. Rate of GSD change w.r.to FOV is of similar trend as that of the GSD variation (as shown in figure 6). The rate of GSD change for each of the three images is minimum and equal to zero at centre of the optical axis. However, from the left to right edges of the three images, the rates of GSD change vary in ranges of 28-155, 55-55, and, 155-28, respectively. Next section describes the impact of GSD on keypoint detection for an image.

FIELD EXPERIMENTS AND ANALYSIS FOR KEYPOINTS
In this section, the impact of GSD on keypoint analysis is presented by experiments. As shown in figure 8, a 90° corner site is selected. Convergent images are acquired on radial lines spaced at equal angular interval for a set of images. Experiments are conducted for 8 sets of images starting from 3 to 8 images in a set for 6m, 8m and 10m radial distances. To mark the camera positions on the circular path in field, two perpendicular lines, that are represented by lines OP and OQ (as shown in figure 4) are drawn on field. Thereafter, radial lines are extended from the building corner by series of markers. On each of the lines, these camera positions are marked at the three distances (6m, 8m and 10m). Thus, a total of 33 camera positions are used for the experimental analysis for each of the three distances. On these positions, images are acquired by Canon EOS 7D Mark-II camera (64° FOV, 5472 x 3648 pixels), which is a consumer grade DSLR camera (Canon, 2021). The camera is mounted on a pole and a bubble is used to align the camera horizontally. Moreover, to maintain the same illumination, images are captured during same time period on clear days.
We observed that for a camera position of convergent image, the rate of GSD variation w.r.to FOV influences number of keypoints detected. Following discussion is for a set of image data containing three images located at 39.428°, 45°, and 50.571°, captured at 6m radial distance. These three images are referred as left, center, and right images in the forthcoming discussion. From the 3 images, at a time two images are used for detection of common matching keypoints. Using an open source feature matching algorithm (Garg, 2018), which provides improved performance of SIFT operator, keypoints are detected. The algorithm first detects the keypoints and creates their descriptors in two images using SIFT. Secondly, to find good quality corresponding matches between two images among the detected keypoints outliers are removed by k-nearest neighbor (KNN) and best buddies similarity methods (BBS). The distance between the descriptors is used as criteria in KNN and BBS for outlier removal.
We observed that keypoints can be detected up to certain value of FOV on both sides of optical axis of camera. Figures 9 and 10 plot number of detected keypoints for every 1º change of FOV from the centre of the image for images at camera positions ψ = 39.4286º and ψ = 45º, respectively. Here after, the FOVs in left and right sides of the optical centre of an image is termed as FOV of left portion of image and FOV of right portion of image, respectively.  figure 10 shows the variation of keypoints detected up to FOV of 28º towards edges on either side of optical axis for the center image. For the former case ( ψ = 39.4286º), the rate of GSD change for the right portion of image is measured as 28.12 that corresponds to the point in the curve, beyond which no or negligible keypoints are detected. This value of rate is defined as critical value for the left image. Whereas for the latter case, critical value for rate of GSD change is measured as 28.81 for both the edges of the center image. The equal number of keypoints on both sides of the optical axis are detected due to the rate of symmetrical variation of GSD variation about the optical axis of camera.
Corresponding to these values of rate of GSD change, we determined the FOV values in left and right portion of image from the figure 7. Moreover, we also determine these values from the equations 13 and 14. We address the former values as experimental values, and latter values as estimated values.  Matching keypoints in two images are connected by yellow lines. These matched keypoints can be used for image registration or similar processes requiring common keypoints. Regions lying out of the black lines, showing FOV values in each image contain keypoint, which match incorrectly. The incorrect matches of keypoints are due to inconsistency in GSDs of two images in these regions. Therefore, in an image one can ignore the regions beyond the FOV values for determining the corresponding points (matching keypoints).

CONCLUSIONS
This study presents an analysis on GSD of convergent images for keypoint detection on the planar surfaces of a building corner for close range terrestrial photogrammetry. For convergent images acquired at a constant distance from the corner object, the study derives mathematical expressions of GSD. Unlike normal image geometry, convergent geometry shows different GSDs at left and right edges of an image. These GSDs terms are named as GSD at left edge and GSD at right edge. The GSDs are functions of camera positions w.r.to corner edges, camera FOV, and object geometry. However, with angular position of the camera, GSD shows an eccentrically large values on one side. For a juxtaposition of two camera locations around the midpoint of a circular arc located at constant distance from an object corner, the left and right GSD terms can be interchanged. In other words, at the midpoint of circular path, GSDs are equal for a convergent image. On the other hand, the GSDs are larger for a convergent geometry compared to that of the normal geometry. For a commonly used consumer grade camera for CRTP, the GSD terms for a 90° corner follow square rule with respect to normal images, i.e. if the GSD of a convergent image at the centre is x times the GSD of normal image, the values of GSDs at left and the right edges of the convergent image are to be higher by a likely factor of square of x.
For a convergent images acquired for a building corner, variation of GSD change impacts more compared to GSD itself for keypoint detection for a convergent images acquired for a building corner. For detection of keypoints in a convergent image, the rate of GSD change value close to 28 in the image is critical for keypoint detection. The features in the image lying within this FOV corresponding to value of 28, they are more likely to be detected.
The results from this study can be used to select target location for better image matching for control points for close range photogrammetry. Moreover, the method presented in this study for keypoint detection for convergent images can be exploited for image registration, object detection, relative orientation, view planning, change detection, and 3D modelling of buildings. Authors envision to establish the expressions of GSD of convergent images for other non-planar surfaces like tree, rock hill etc.