PEL: A Predictive Edge Linking algorithm☆
Introduction
Edge detection is a very important and fundamental first step in many computer vision and image processing applications. A traditional edge detection algorithm [1], [2], [3], [4] takes a grayscale image as input and produces a binary edge map (BEM) as output, where an edge pixel (edgel) is marked (e.g., its value in the edge map is 255), and a non-edge pixel is unmarked (e.g., its value in the edge map is 0).
The binary edge maps produced by traditional edge detectors are usually of low quality, consisting of gaps between the edgels, unattended edgels and noisy notch-like structures, ragged and multi-pixel wide edgel formations, etc. An example of such an edge map with low quality artifacts is shown in Fig. 1 for the famous Lena image. This edge map was obtained by the OpenCV implementation of the widely-used Canny [2] edge detector (cvCanny) [5], which is the fastest known Canny implementation. To obtain this edge map, the input image was first smoothed by a Gaussian kernel with = 1.5 (using cvSmooth from OpenCV), and cvCanny was called with low and high threshold values set to 20 and 40 respectively, and the Sobel kernel aperture size set to 3. Fig. 1 also shows the close-up views of two separate sections of the edge map to illustrate the low quality artifacts, which can be grouped in three categories as follows: (1) There are discontinuities and gaps between edgel groups as can clearly be seen in the close-up views of the two enlarged sections of the edge map. Some of these gaps need to be filled up. (2) There are noisy, unattended edgel formations and notch-like structures. This is more evident in the close-up view of the upper-left corner of the edge map. These noisy artifacts needs to be removed. (3) There are multi-pixel wide edgel formations in a staircase pattern especially around the diagonal edgel formations (both 45 degree and 135 degree diagonals). Such formations can be seen in many places in the edge map, and they need to be thinned down to 1-pixel wide chains.
To improve the modal quality of the binary edge maps produced by traditional edge detectors, edge linking methods have been proposed in the literature [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24]. The goals of these methods are commonly to remove noisy edgel formations and clean up the edge map, and to fill in gaps between edgels to form longer edgel groups.
Snyder et al. [7] is one of the first researchers to present a method to deal with errors in edge detector results. The authors propose a method to close gaps in edge maps while preserving edge structure and connectedness based on the concept of a chamfer map. Xie [6] presents a method to link edge pixels for the purpose of line segment detection. The method makes use of the concepts of horizontal edge element and causal neighborhood window to realize edge linking and consequently line segment detection. Given a binary edge map, the author performs linking of pixels aligned on the same image line into what is called a horizontal edge element, which is then converted to a line segment. The algorithm performs poorly especially in highly textured regions. Basak et al. [8] also consider the problem of line detection via edge linking and propose two neural network architectures.
Farag and Delp [9] define edge linking as a graph search problem. Beginning at a start node, they use the search algorithm to reach a goal node. The search makes use of the gradient magnitude and angles, and the swath of edge information estimated by the zero crossings of Laplacian of Gaussian operator. They propose a linear path metric function to guide the search. To limit the search space, the algorithm only looks at three neighbors for transitioning, which reduces the running time but affects the accuracy of the algorithm. The biggest problem with this algorithm is the large number of parameters that needs to be adjusted. Also some statistical parameters and a priori information about the edge map needs to be known for the algorithm to work correctly. Zhu et al. [10] present an algorithm to link discontinuous edge segments. They model an edge map as a global potential field with energy dispositions at detected edges. A directional potential function measures the energy charges and guides the linking process to fill the gaps between the broken edge fragments. Their method can fill small gaps between broken edge fragments successfully, but produces unacceptable results when the gaps are large or the image is noisy.
Saber et al. [11] propose a method for segmentation of the color images. They model the segmentation map as a Gibbs random field and use the gradient magnitude of the 3-channel color image to find the regions. The boundaries of the closed regions constitute the final linked edge map. Maeda et al. [13] also concentrate on image segmentation and propose incorporating a linking algorithm based on a directional potential function into an edge preserving smoothing filter. They show that the unnecessary details of the image are smoothed out before region growing is performed. Hajjar and Chen [12] propose an edge linking algorithm and its VLSI implementation for real-time edge linking. Their method is based on the break points’ direction and weak level points and try to fill the gaps between edge groups. Ghita and Whelan [14] also propose an algorithm to close the gaps in an edge image by local analysis of the edge break points (terminators). They mark the edge termination points, and determine the connection path between different edge termination points by an analysis of the local edge structure. Their algorithm requires a single pass through the edge map and can be applied without a priori information. Shih and Cheng [15] apply mathematical morphology to edge linking to fill in the gaps between edge segments. Broken edge segments are extended along their slope directions by using the adaptive dilation operation with suitable elliptical structuring elements. They also apply thinning and pruning as a post-processing step. Although their algorithm is shown to fill gaps in the edge map of an ellipse, it is not clear how it can be applied without a priori information about the edge map.
Sappa and Vintimilla [18] present an algorithm that takes in an edge map and the original intensity image, and generates closed contours using graph theoretical approaches. In the process, they close small gaps and remove spurious edge pixels, but their algorithm is more tailored towards image segmentation than edge linking. Lu and Chen [16] apply ant colony optimization to compensate broken edges. They propose four moving policies for ants and apply a finite number of iterations to fill up the broken edge segments. Jevtic et al. [19] propose a similar ant based edge linking algorithm to combine broken edges. Wang and Zhang [17] propose the use of application-specific local neighborhood to compute edge direction and geodesic distance for measuring proximity between candidate edge points to be linked. Lin et al. [20] make use of an edge linking algorithm with directional edge-gap closing to produce complete edge-links that are then used for lane detection.
Flores et al. [21] present a method for the segmentation of intensity images that combines an optical contouring technique with edge linking. Ji et al. [22] present a method for segmentation of satellite imagery. Their idea is to make use of the gradient magnitude and directions and perform a heuristic search to link the original edge points. Their algorithm is also tailored toward image segmentation. Guan et al. [23] recently proposed a Partial Differential Equation (PDE)-based method to link the edges to obtain closed contours for image segmentation. They then show their algorithm’s performance on the segmentation of cell images.
We can classify most of the edge linking algorithms found in the literature in two categories: (1) Those that try to fill the gaps between broken edge groups in the edge map, which is only part of the edge linking problem that we attempt in this paper, (2) those that use edge linking to find closed contours for image segmentation, which is outside the scope of this paper.
To overcome the problems associated with binary edge maps from the get-go, authors in [25] propose a completely new edge and segment detection algorithm called Edge Drawing (ED). Rather than detecting edgels individually, which causes the above-mentioned problems in the first place, the authors conceive edge detection as a boundary detection problem and model it similar to childrens dot completion games, where the child is given a set of marked anchor points (numbered dots) in a picture and is asked to connect the dots to reveal the hidden picture boundaries. To emulate a dot completion game, ED first computes a set of anchor points, which are pixels that are definitely assumed to be edgels, and then uses an efficient method called the Smart Routing (SR) to join these anchors. Since ED draws the boundaries by attaching contiguous pixels one after the other, it outputs not just a binary edge map, but a set of edge segments, each of which is a contiguous chain of pixels.
Recently, Akinlar and Chome [24] proposed an edge linking algorithm named CannySR that computes a binary edge map using the Canny edge detection algorithm and then uses the Smart Routing step of Edge Drawing to convert Canny’s binary edge map to a set of edge segments. CannySR is shown to produce good results, but it is limited to working only in the context of the Canny edge detector. That is, for CannySR to work correctly, it requires the original grayscale image and the Gaussian smoothing kernel used to smooth the image before Canny edge detection. This is not a problem when the original image is available and we know how it was smoothed before edge detection, but in the general case where we only have the binary edge map and we do not know how it was obtained, CannySR does not work. Ideally, one wants to have an edge linking algorithm that requires only the binary edge map to work without needing to know how it was obtained.
In this paper we propose an edge linking algorithm that has the following goals:
- (1)
Close small gaps (1-pixel gaps in our case) between edgel groups.
- (2)
Clean-up noisy, unattended and notch-like structures from the edge map.
- (3)
Thin down multi-pixel wide staircase edgel formations to 1-pixel wide chains.
- (4)
Convert the binary edge map into a set of edge segments, each of which is a clean, contiguous, 1-pixel wide chain of pixels.
Traditional edge linking algorithm proposed in the literature mostly restrict their attention to the first goal with some emphesis on the second and third goals without considering the last goal, which we believe is very important. This is due to the fact that having a set of edge segments rather than a binary edge map enables us to perform many higher-level processing jobs such as line [26], arc, circle, ellipse [27], polygon and general shape detection, image registration and segmentation, and edge segment validation [28] among many others.
In this paper we propose an edge linking algorithm that satisfies all four goals listed above. The proposed algorithm just takes in a binary edge map generated by any traditional edge detection algorithm and converts it to a set of edge segments; filling in one pixel gaps in the edge map, cleaning up noisy edgel groups and thinning multi-pixel wide edgel formations in the process. It walks over the edge map based on the predictions generated from its past movements; thus the name Predictive Edge Linking (PEL) [37]. We give the details of PEL in Section 2, and evaluate its performance both qualitatively using visual experiments and quantitatively within the precision-recall framework of the Berkeley Segmentation Dataset and Benchmark (BSDS 300) [29], [30] in Section 4. Section 5 concludes the paper.
Section snippets
Predictive Edge Linking (PEL)
In this section we describe the details of our Predictive Edge Linking (PEL) algorithm, which takes as input only the binary edge map (BEM) produced by a traditional edge detector and returns a set of edge segments; filling in one pixel gaps in the edge map, cleaning up noisy edgel groups and thinning multi-pixel wide edgel formations in the process.
The pseudocode for PEL is given in Algorithm 1. PEL only takes a BEM as input and returns a set of edge segments (ES) as output. The algorithm
Discussion
In this section we compare and contrast the edge detection and linking methods utilized in this paper, i.e., Canny, ED, and PEL, to make PEL’s contributions clearer.
Fig. 10 depicts the typical processing pipeline for PEL and ED [25]. As seen from Fig. 10(a), PEL is a pure edge linking algorithm in that it takes in as input a binary edge map produced by a traditional edge detection algorithm such as Canny, Nalwa and Deriche [1], [3], [4], and converts the edge pixels to edge segments, filling
Experimental results
In this section, we evaluate the performance of PEL both qualitatively using visual experiments, and quantitatively within the precision-recall framework of the Berkeley Segmentation Dataset and Benchmark (BSDS 300) [29], [30]. We use Canny [2] for edge detection since it is the most widely used binary edge detection algorithm. We compare an contrast PEL’s performance to that of CannySR [24], which is a recently proposed edge linking algorithm to convert Canny’s binary edge maps to edge
Conclusions
In this paper we propose an edge linking algorithm named Predictive Edge Linking (PEL) that takes in a binary edge map generated by any traditional edge detection algorithm and converts it to a set of edge segments; filling in one pixel gaps in the edge map, cleaning up noisy edgel groups and thinning multi-pixel wide edgel formations in the process. PEL starts at an arbitrary edgel in the edge map and walks over the neighboring edgels until the end of an edgel chain is reached. During a walk,
References (37)
Edge linking by using causal neighborhood window
Pattern Recogn. Lett.
(1992)- et al.
Closing gaps in edges and surfaces
Image Vis. Comput.
(1992) - et al.
Edge Linking by sequential search
Pattern Recogn.
(1995) - et al.
Edge linking by a directional potential function (DPF)
Image Vis. Comput.
(1996) - et al.
Fusion of color and edge information for improved segmentation and edge linking
Image Vis. Comput.
(1997) - et al.
Segmentation of natural images using anisotropic diffusion and linking of boundary edges
Pattern Recogn.
(1998) - et al.
Adaptive mathematical morphology for edge linking
Inform. Sci.
(2004) - et al.
Edge detection improvement by ant colony optimization
Pattern Recogn. Lett.
(2008) - et al.
Edge linking and image segmentation by combining optical and digital methods
Optik
(2013) - et al.
Edge drawing: a combined real-time edge and segment detector
J. Vis. Commun. Image Represent.
(2012)
EDLines: a real-time line segment detector with a false detection control
Pattern Recogn. Lett.
EDCircles: a real-time circle detector with a false detection control
Pattern Recogn.
Theory of edge detection
Proc. Roy. Soc. Lond. Biol. Sci.
A computational approach to edge detection
IEEE Trans. Pattern Anal. Mach. Intell.
On detecting edges
IEEE Trans. Pattern Anal. Mach. Intell.
Using Canny’s criteria to derive a recursively implemented optimal edge detector
Int. J. Comput. Vis.
On edge and line linking with connectionist models
IEEE Trans. Syst. Man Cybernet.
Cited by (17)
Implementation of real-time image edge detector based on a bump circuit and active pixels in a CMOS image sensor
2018, Integration, the VLSI JournalCitation Excerpt :Eqs. (1) and (2) are differentials of brightness gradients in horizontal and vertical directions for a pixel at the point (x, y). Most existing edge detection algorithms compare brightness level between four or more pixels and a target pixel [2,3,8,9]. The algorithms are based on image signal processing, which proceeds several stages after image capture.
Fast Image Edge Detection based on Faber Schauder Wavelet and Otsu Threshold
2017, HeliyonCitation Excerpt :Extract the FSDWT extrema coefficients as described in the previous sections. The PEL (Predictive edge algorithm) [32] is used to link edge pixels. We have chosen this algorithm owing to its low complexity and effectiveness.
Edge detection using multi-scale closest neighbor operator and grid partition
2024, Visual ComputerReconstruct Damaged Sinhala Handwritten Characters by Guided Edge Linking Approach
2023, ICARC 2023 - 3rd International Conference on Advanced Research in Computing: Digital Transformation for Sustainable Development
- ☆
This paper has been recommended for acceptance by K. Chung.