Edge- and region-based segmentation technique for the extraction of large, man-made objects in high-resolution satellite imagery

doi:10.1016/j.patcog.2004.03.001

Pattern Recognition

Volume 37, Issue 8, August 2004, Pages 1619-1628

https://doi.org/10.1016/j.patcog.2004.03.001 Get rights and content

Abstract

A new object-oriented segmentation approach with special focus on shape analysis was developed for the extraction of large, man-made objects, especially agricultural fields, in high-resolution panchromatic satellite imagery. The approach, a combination of region- and edge-based techniques, includes new methods for the evaluation of straight edges, edge preserving degradation, and edge-guided region growing.

Introduction

Automatic image analysis of natural scenes has always been a challenging task. This is especially true for the analysis of remote sensing data where, e.g. sensor characteristics and varying atmospherical conditions influence the quality of the imagery. However, the visible objects in these images such as houses, streets, or agricultural fields are important data for Geographical Information Systems (GIS) which are widely used by public authorities and other institutions as a tool for decision-making and planning. Some GIS applications, e.g. the monitoring of land-cover classes or the assessment of damages after natural disasters, depend on the precise detection of the boundaries of agricultural fields. Only recently, new satellite sensors such as IKONOS or QuickBird provide image data with a ground sampling distance in the panchromatic band of up to 1 or $0.7 m$ , respectively, a sufficient spatial resolution for the desired tasks.

Due to the complexity of object characteristics, the extraction of field boundaries in high-resolution imagery requires sophisticated methods that are able to identify each agricultural field with exact boundaries as a single unit despite superfluous details and possibly low contrast to neighboring regions. In this context, it was necessary to develop a new segmentation technique with special emphasis on the consideration of shape information which will be presented in this paper.

Section snippets

Task and model definition

If large regions with anthropogenic influence, such as agricultural fields, are to be detected automatically within data of high spatial resolution (Fig. 1(a)), there exist three main difficulties:

1.
many small details can cause high-gray value variations within the objects which result in over-segmentation;
2.
the brightness contrast to neighboring objects can be low which is responsible for under-segmentation; and
3.
complex geometric shapes with long straight boundary parts that are not necessarily

Segmentation technique

The described model is optimally realized by an approach that combines edge- and region-based methods. Edge-based methods are able to detect long, straight edges while gaps within these edges can be closed by means of region-based approaches. Region-based techniques can determine the homogeneity of objects while uncertainties in detecting the exact boundary positions can be reduced by previously extracted edges. There exist a number of approaches that propose a variety of different

Results

The technique was tested on images from four different sensors with different spatial resolution (MOMS-02: $4.5 m$ ; IRS-C: $5.8 m$ ; IKONOS: $1 m$ ; aerial imagery: $1 m$ after digitalization). The test areas are primarily located in agricultural regions in Zimbabwe and Germany, but an urban example is presented as well. In the following section, the results and comparisons to other methods are presented while Section 4.2 focuses on the important question of parameter selection. An other application to urban

Conclusion and outlook

A new approach for the extraction of large, man-made objects in satellite image data was developed and presented in this paper. The key information for this task is shape knowledge. Its inclusion is necessary due to a high amount of superfluous details within objects and especially a low contrast between adjacent objects in the image data. Thus, the technique is divided into two parts where essential shape information is extracted in the first part to control a new region growing algorithm in

Acknowledgements

The MOMS scientific program and this investigation were funded by the German Bundesministerium für Bildung und Forschung (BMBF).

About the Author—MARINA MUELLER received her doctorate from the Faculty of Technology at the University of Bielefeld, Germany, in 2000. She is currently research scientist at the Institute of Photogrammetry and Remote Sensing, University of Karlsruhe, Germany. Her research interest focus on image analysis, neural networks, fuzzy logic, and expert systems.

References (22)

R.M. Haralick et al.
Surveyimage segmentation techniques
Comput. Vision, Graphics, Image Process.
(1985)
J.-P. Gambotto
A new approach to combining region and edge detection
Pattern Recogn. Lett.
(1993)
S.B. Abramson et al.
Evaluation of edge-preserving smoothing filters for digital mapping
ISPRS J. Photogram. Remote Sensing
(1993)
P. Soille
Morphological Image Analysis: Principles and Applications
(1999)
G. Venturi, C. Siffredi, S. Testa, Statistical integration of edge detection and region growing, Proceedings of the...
A. Bhalerao, R. Wilson, Multiresolution image segmentation combining region and boundary information, Theory and...
T. Pavlidis et al.
Integrating region growing and edge detection
IEEE Trans. Pattern Anal. Mach. Intell.
(1990)
T.W. Ryan et al.
Extraction of shoreline features by neural nets and image processing
Photogram. Eng. Remote Sensing
(1991)
M. Salotti, C. Garbay, Cooperation between edge detection and region growing: the problem of control, Proceedings of...
M. Tabb et al.
Multiscale image segmentation by integrated edge and region detection
IEEE Trans. Image Process.
(1997)

D. Comaniciu, P. Meer, Mean shift: a robust approach toward feature space analysis, IEEE Trans. Pattern Anal. Mach....

Cited by (122)

Case instance segmentation of small farmland based on Mask R-CNN of feature pyramid network with double attention mechanism in high resolution satellite images
2023, Computers and Electronics in Agriculture
Accurate spatial information of farmland in small farms is very important to provide operable information to farmers, managers and decision makers. However, small farms have small area, irregular shape, and use a variety of planting crops, which makes their boundaries blurred, and the standard edge detection algorithm cannot accurately segment the farmland boundary. Therefore, the automatic delimitation of fields in small farms is a challenging task. Aiming at the above problems, this paper proposes an example segmentation method of Mask R-CNN based on dual attention mechanism feature pyramid network (DAFPN) to describe small farms. DAFPN is composed of two attention modules: spatial attention module (SPA) and channel attention module (CHA) to enhance its feature extraction ability. Spatial attention module (SPA) generates spatial attention map by using the spatial relationship of features, and generates information to be emphasized or suppressed in spatial location; The channel attention module (CHA) learns an adaptive channel merging method based on the attention mechanism. Our proposed DAFPN can be easily inserted into the existing FPN model. We have conducted extensive experimental analysis on very high resolution (VHR) satellite images based on the Mask R-CNN deep learning framework of DAFPN. The standard COCO dataset evaluation index and F1-score evaluation strategy are used to compare the algorithm. AP₅₀, AP₇₅ and F1-score reach 82.86%, 55.51% and 70.90% respectively, which is 8.7%, 8.31% and 6.87% higher than Mask R-CNN respectively. Our results highlight the ability of Mask R-CNN based on DAFPN to accurately depict small farms in VHR satellite images, which lays a foundation for the automatic segmentation of small farms.
Using a semantic edge-aware multi-task neural network to delineate agricultural parcels from remote sensing images
2023, ISPRS Journal of Photogrammetry and Remote Sensing
This paper presents a semantic edge-aware multi-task neural network (SEANet) to obtain closed boundaries when delineating agricultural parcels from remote sensing images. It derives closed boundaries from remote sensing images and improves conventional semantic segmentation methods for the extraction of small and irregular agricultural parcels. SEANet integrates three correlated tasks: mask prediction, edge prediction, and distance map estimation. Related features learned from these tasks improve the generalizability of the network. We regard boundary extraction as an edge detection task and extract rich semantic edge features at multiple levels to improve the geometric accuracy of parcel delineation. Moreover, we develop a new multi-task loss that considers the uncertainty of different tasks. We conducted experiments on three high-resolution Gaofen-2 images in Shandong, Xinjiang, and Sichuan provinces, China, and on two medium-resolution Sentinel-2 images from Denmark and the Netherlands. Results showed that our method produced a better layout of agricultural parcels, with higher attribute and geometric accuracy than the existing ResUNet, ResUNet-a, R2UNet, and BsiNet methods on the Shandong and Denmark datasets. The total extraction errors of the parcels produced by our method were 0.214, 0.127, 0.176, 0.211, and 0.184 for the five datasets, respectively. Our method also obtains closed boundaries by one single segmentation, leading to superiority as compared with existing multi-task networks. We showed that it could be applied to images with different spatial resolutions for parcel delineation. Finally, our method trained on the Xinjiang dataset could be successfully transferred to the Shandong dataset with different dates and landscapes. Similarly, we obtained satisfactory results when transferring from the Denmark dataset to the Netherlands dataset. We conclude that SEANet is an accurate, robust, and transferable method for various areas and different remote sensing images. The codes of our model are available at https://github.com/long123524/SEANet_torch.
A machine learning approach for identifying and delineating agricultural fields and their multi-temporal dynamics using three decades of Landsat data
2022, ISPRS Journal of Photogrammetry and Remote Sensing
High spatial and temporal resolution satellite imagery are essential data for land cover discrimination and mapping of vegetation dynamics, offering insights into the number, extent, and condition of agricultural fields. However, an accurate account of the number, location, and variability of fields can be challenging to obtain in a timely manner, particularly at scale or in regions where ground-supporting data are not available, limiting the capacity for food production and water use assessment and planning. To bridge this capacity gap, a convolutional neural network approach was adopted and combined with two clustering techniques: the density-based spatial clustering of applications with noise (DBSCAN) and spectral clustering, to provide an account of agricultural fields and their extent across a region lacking ground based information. A random forest classification was also employed to discriminate crop types. Using an annual maximum normalized difference vegetation index (NDVI) derived from 2018 Landsat-8 data, the approach was applied to classify the shape and delineate the extent of agricultural fields across an agricultural region in Saudi Arabia that had an area under irrigation exceeding 2,300 km². When assessed against manually identified center-pivot fields (CPFs), the method achieved 97.4% producer’s and 98.0% user’s accuracies on an object basis, and 81.4% producer’s and 85.4% user’s accuracies on a pixel basis for identifying non-CPFs (i.e., tree crop plantations and other non-woody crops). The over- and under-segmentation error for CPFs was 1.5% and 1.0%, respectively, with intersection over union errors reported as being 3.5%. The framework showed stability when retrospectively applied to Landsat data from the year 2000, returning 97.5% producer’s and 96.6% user’s accuracies for CPF identification. In order to characterize the temporal dynamics of agricultural development over the past three decades, an analysis of field behavior between 1988 and 2020 was subsequently undertaken. The analysis indicated that the number of CPFs in the study region increased from 45 (covering 20 km²) in 1988 to 5,080 CPFs by 2016 (covering 2,368 km²), followed by a recent reduction to 3,700 CPFs in 2020 (covering 1,581 km²). Through the multi-temporal analysis, individual fields were able to be characterized in terms of their expansion, contraction and activity throughout the study period. Overall, the proposed method was simple to train, efficient in dealing with large datasets, relied on limited in-situ records to a very small degree, and has the potential to be applied to larger national scales, providing an ongoing assessment of important agroinformatic metrics.
Delineation of cultivated land parcels based on deep convolutional networks and geographical thematic scene division of remotely sensed images
2022, Computers and Electronics in Agriculture
Extraction of cultivated land information from high spatial resolution remote sensing images is increasingly becoming an important approach to digitization and informatization in modern agriculture. The continuous development of deep learning technology has made it possible to extract information of cultivated land parcels by an intelligent way. Aiming at fine extraction of cultivated land parcels within large areas, this article builds a framework of geographical thematic scene division according to the rule of territorial differentiation in geography. A deep learning semantic segmentation network, improved U-net with depthwise separable convolution (DSCUnet), is proposed to achieve the division of the whole image. Then, an extended multichannel richer convolutional features (RCF) network is involved to delineate the boundaries of cultivated land parcels from agricultural functional scenes obtained by the former step. In order to testify the feasibility and effectiveness of the proposed methods, this article implemented experiments using Gaofen-2 images with different spatial resolution. The results show an outstanding performance using methods proposed in this article in both dividing agricultural functional scenes and delineating cultivated land parcels compared with other commonly used methods. Meanwhile, the extraction results have the highest accuracy in both the traditional evaluation indices (like Precision, Recall, F₁, and IoU) and geometric boundary precision of cultivated land parcels. The methods in this article can provide a feasible solution to the problem of finely extracting cultivated land parcels information within large areas and complex landscape conditions in practical applications.
Automated delineation of agricultural field boundaries from Sentinel-2 images using recurrent residual U-Net
2021, International Journal of Applied Earth Observation and Geoinformation
Delineation of agricultural fields is desirable for operational monitoring of agricultural production and is essential to support food security. Due to large within-class variance of pixel values and small inter-class difference, automated field delineation remains to be a challenging task. In this study, a strategy is proposed to effectively address this issue. Firstly, a framework was developed using the Canny operator connected with the Watershed segmentation algorithm (CW) to quickly label the training dataset, which minimizes the workload of dataset generation in comparison with the commonly used manual vectorization. Secondly, a CW-trained deep semantic segmentation network, recurrent residual U-Net, was selected to mine the low level and deep semantic features. Finally, a boundary connecting method (to integrate fragmented boundaries) was used to generate the agricultural field boundary. The proposed methods are tested over smallholder agricultural landscape in Heilongjiang province, China, using Sentinel-2 imagery. Compared with the U-Net (overall accuracy (OA) 82.18%), the residual U-Net (ResU-Net, with OA 85.78%), traditional object-based image analysis (OBIA, with OA about 82%), and the existing 10-m resolution global land cover map (FROM-GLC10), the proposed method shows an improved performance (OA 89.28%, and Kappa 0.85). The successful application of the proposed method suggests that the recurrent residual U-Net model has great universality in agricultural field boundary extraction, and the automated technique has the potential of being applied to other regions.
Extracting water channels from aerial videos based on image-to-BIM registration and spatio-temporal continuity
2021, Automation in Construction
Citation Excerpt :
Many research works have been done in the field of ROI extraction. Researchers in geomatics [8,9] developed approaches to extracting man-made objects such as buildings and roads from remote sensing orthoimages; in civil engineering, research efforts have been made to segment 2D visual data into pixel regions containing various structures of interest [10–12]. Existing ROI extraction methods rely on human's prior knowledge of the objects of interest [13], which brings two aspects of limitations.
Unmanned aerial vehicles (UAV) are increasingly being used in water supply infrastructure inspection, resulting in a large volume of aerial visual assets (static photographs or videos). How to efficiently extract water channels from such assets for the benefits of hazard detection remains a challenge. This paper provides an aerial video processing approach for the extraction of various water channel structures, which includes a main algorithm based on the registration of video frames into a building information model (BIM), and a complementary algorithm enabled by aerial video spatio-temporal continuity. Experimental results demonstrate the effectiveness of the proposed approach, which can reliably extract water channels with least manual intervention. The results reveal the promise of exploiting readily-available information from BIM to automate aerial video processing. Future research is suggested to further explore the generalizability of the approach concerning variant flight settings and external environments.

View all citing articles on Scopus

About the Author—KARL SEGL received his doctorate from the Faculty of Engineering at the University of Karlsruhe, Germany, in 1996. He is currently a research scientist at the GeoForschungsZentrum Potsdam, Germany, in the Remote Sensing Section of the Division Kinematics and Dynamics of the Earth. His research interests focus on new methodological developments for hyperspectral data analysis, pattern recognition, image correction, and sensor design.

About the Author—HERMANN KAUFMANN is head of the Remote Sensing Section of the GeoForschungsZentrum in Potsdam and holds a chair at the University of Potsdam. He obtained his doctorate at the Ludwig-Maximilians-University of Munich, Germany, in geology and remote sensing. In 1992, he gained his inauguration in the field of remote sensing at the Faculty of Engineering, University of Karlsruhe, Germany. In his 25 years of experience he has been principal investigator of a large number of national and international projects funded by various governmental and industrial institutions.

View full text

Edge- and region-based segmentation technique for the extraction of large, man-made objects in high-resolution satellite imagery

Abstract

Introduction

Section snippets

Task and model definition

Segmentation technique

Results

Conclusion and outlook

Acknowledgements

Comput. Vision, Graphics, Image Process.

Pattern Recogn. Lett.

ISPRS J. Photogram. Remote Sensing

Morphological Image Analysis: Principles and Applications

Integrating region growing and edge detection

IEEE Trans. Pattern Anal. Mach. Intell.

Extraction of shoreline features by neural nets and image processing

Photogram. Eng. Remote Sensing

Multiscale image segmentation by integrated edge and region detection

IEEE Trans. Image Process.