FormalPara Overview

Pixel-based classification can include unwanted noise. Techniques for object-based image analysis are designed to detect objects within images, making classifications that can address this issue of classification noise. In this chapter, you will learn how region growing can be used to identify objects in satellite imagery within Earth Engine. By understanding how objects can be delineated and treated in an image, students can apply this technique to their own images to produce landscape assessments with less extraneous noise. Here, we treat images with an object delineator and view the results of simple classifications to view similarities and differences.

FormalPara Learning Outcomes

  • Learning about object-based image classification in Earth Engine.

  • Controlling noise in images by adjusting different aspects of object segmentation.

  • Understanding differences through time of noise in images.

  • Creating and viewing objects from different sensors.

Assumes you know how to

  • Import images and image collections, filter, and visualize (Part I).

  • Perform basic image analysis: select bands, compute indices, create masks, classify images (Part II).

  • Create a function for code reuse (Chap. 1).

  • Perform pixel-based supervised and unsupervised classifications (Chap. 6).

1 Introduction to Theory

Building upon traditional pixel-based classification techniques, object-based image analysis classifies imagery into objects using perception-based, meaningful knowledge (Blaschke et al. 2000; Blaschke 2010; Weih and Riggan 2010). Detecting and classifying objects in a satellite image are a two-step approach. First, the image is segmented using a segmentation algorithm. Second, the landscape objects are classified using either supervised or unsupervised approaches. Segmentation algorithms create pixel clusters using imagery information such as texture, color or pixel values, shape, and size. Object-based image analysis is especially useful for mapping forest disturbances (Blaschke 2010; Wulder et al. 2004) because additional information and context are integrated into the classification through the segmentation process. One object-based image analysis approach available in Earth Engine is the Simple Non-Iterative Clustering (SNIC) segmentation algorithm (Achanta and Süsstrunk 2017). SNIC is a bottom-up, seed-based segmentation algorithm that assembles clusters from neighboring pixels based on parameters of compactness, connectivity, and neighborhood size. SNIC has been used in previous Earth Engine-based research for mapping land use and land cover (Shafizadeh-Moghadam et al. 2021; Tassi and Vizzari 2020), wetlands (Mahdianpari et al. 2018 and 2020, Amani et al. 2019), burned areas (Crowley et al. 2019), sustainable development goal indicators (Mariathasan et al. 2019), and ecosystem services (Verde et al. 2020).

2 Practicum

2.1 Section 1: Unsupervised Classification

In earlier chapters (see Chap. 6), you saw how to perform a supervised and unsupervised classification. In this lab, we will focus on object-based segmentation and unsupervised classifications—a clean and simple way to look at the spectral and spatial variability that is seen by a classification algorithm.

We will now build a script in several numbered sections, giving you a chance to see how it is constructed as well as to observe intermediate and contrasting results as you proceed. We will start by defining a function for taking an image and breaking it into a set of unsupervised classes. When called, this function will divide the image into a specified number of classes, without directly using any spatial characteristics of the image.

Paste the following block into a new script.

A block of code for 1.1 unsupervised k-means classification. The code has different comments which explains its different parts. The functions used in the code are dot sample, e e dot clusterer dot weka K means, dot cluster, and dot select, and dot rename.

We will also need a function to normalize the band values to a common scale from 0 to 1. This will be most useful when we are creating objects. Additionally, we will need a function to add the mean to the band name. Paste the following functions into your code. Note that, the code numbering skips intentionally from 1.2 to 1.4; we will add Sect. 1.3 later.

A block of code for 1.2 simple normalization by maxes function and 1.4 simple add mean to band name function. The function used in the code for 1.2 is dot divide.

We will create a section that defines variables that you will be able to adjust. One important adjustable parameter is the number of clusters for the clusterer to use. Add the following code beneath the function definitions.

A block of code for 2 parameters to function calls and 2.1 unsupervised K means classification parameters. The code under 2.1 is var number of unsupervised clusters equals 4.

The script will allow you to zoom to a specified area for better viewing and exists already in the code repository check points. Add this code below.

A block of code for 2.2 visualization and saving parameters. For different images, you might want to change the min and max values to stretch. Useful for images 2 and 3, the normalized images. Var center object Y N equals true.

Now, with these functions, parameters, and flags in place, let us define a new image and set image-specific values that will help analyze it. We will put this in a new section of the code that contains “if” statements for images from multiple sensors. We set up the code like this because we will use several images from different sensors in the following sections; therefore, they are preloaded, so all that you have to do are to change the parameter “whichImage”. In this particular Sentinel-2 image, focus on differentiating forest and non-forest regions in the Puget Sound, Washington, USA. The script will automatically zoom to the region of interest.

A block of code for 3 statements, 3.1 selecting image to classify. The code has different comments which explain its different parts. The functions used in the code are e e dot image, print, e e dot geometry dot polygon, map dot add layer, and dot select.
A block of code for 3 statements, 3.1 selecting image to classify. The code has different comments which explain its different parts. The functions used in the code are e e dot image, print, e e dot geometry dot polygon, map dot add layer, and dot select.

Now, let us clip the image to the study area we are interested in, then extract the bands to use for the classification process.

A block of code for 4 image pre-processing. The functions used in the code are dot clip, dot select, map dot add layer, and dot select.

Now, let us view the per-pixel unsupervised classification, produced using the k-means classifier. Note that, as we did earlier, we skip a section of the code numbering (moving from Sects. 4 to 6), which we will fill in later as the script is developed further.

A block of code for 6 execute classifications, 6.1 per pixel unsupervised classification for comparison. The functions used in the code are map dot add layer, dot select, dot random visualizer, and print.

Then, insert this code, so that you can zoom if requested.

A block of code for 7 zoom if requested. The function used in the code is map dot center object.

Code Checkpoint F33a. The book’s repository contains a script that shows what your code should look like at this point.

Run the script. It will draw the study area in a true-color view (Fig. 11.1), where you can inspect the complexity of the landscape as it would have appeared to your eye in 2016, when the image was captured.

Fig. 11.1
A screenshot of the google earth engine with a map in satellite view on the main screen.

True-color Sentinel-2 image from 2016 for the study area

Full size image

Note harvested forests of different ages, the spots in the northwest part of the study area that might be naturally treeless, and the straight easements for transmission lines in the eastern part of the study area. You can switch Earth Engine to satellite view and change the transparency of the drawn layer to inspect what has changed in the years since the image was captured.

As it drew your true-color image, Earth Engine also executed the k-means classification and added it to your set of layers. Turn up the visibility of layer 6.1 Per-Pixel Unsupervised, which shows the four-class per-pixel classification result using randomly selected colors. The result should look something like Fig. 11.2.

Fig. 11.2
A screenshot of the google earth engine with a satellite map in four-class per-pixel classification result using randomly selected colors on the main screen.

Pixel-based unsupervised classification using four-class k-means unsupervised classification using bands from the visible spectrum

Full size image

Take a look at the image that was produced, using the transparency slider to inspect how well you think the classification captured the variability in the landscape and classified similar classes together, then answer the following questions.

Question 1. In your opinion, what are some of the strengths and weaknesses of the map that resulted from your settings?

Question 2. Part of the image that appears to our eye to represent a single land use might be classified by the k-means classification as containing different clusters. Is that a problem? Why or why not?

Question 3. A given unsupervised class might represent more than one land use/land cover type in the image. Use the Inspector to find classes for which there were these types of overlaps. Is that a problem? Why or why not?

Question 4. You can change the numberOfUnsupervisedClusters variable to be more or less than the default value of 4. Which, if any, of the resulting maps produce a more satisfying image? Is there an upper limit at which it is hard for you to tell whether the classification was successful or not?

As discussed in earlier chapters, the visible part of the electromagnetic spectrum contains only part of the information that might be used for a classification. The short-wave infrared bands have been seen in many applications to be more informative than the true-color bands.

Return to your script and find the place where threeBandsToDraw is set. That variable is currently set to B4, B3, and B2. Comment out that line and use the one below, which will set the variable to B8, B11, and B12. Make the same change for the variable bandsToUse. Now, run this modified script, which will use three new bands for the classification and also draw them to the screen for you to see. You will notice that this band combination provides different contrasts among cover types. For example, you might now notice that there are small bodies of water and a river in the scene, details that are easy to overlook in a true-color image. With numberOfUnsupervisedClusters still set at 4, your resulting classification should look like Fig. 11.3.

Fig. 11.3
A screenshot of the google earth engine with a satellite map depicting different regions and bodies in distinct colors.

Pixel-based unsupervised classification using four-class k-means unsupervised classification using bands from outside of the visible spectrum

Full size image

Question 5. Did using the bands from outside the visible part of the spectrum change any classes so that they are more cleanly separated by land use or land cover? Keep in mind that the colors which are randomly chosen in each of the images are unrelated—a class colored brown in Fig. 11.2 might well be pink in Fig. 11.3.

Question 6. Experiment with adjusting the numberOfUnsupervisedClusters with this new dataset. Is one combination preferable to another, in your opinion? Keep in mind that there is no single answer about the usefulness of an unsupervised classification beyond asking whether it separates classes of importance to the user.

Code Checkpoint F33b. The book’s repository contains a script that shows what your code should look like at this point. In that code, the numberOfUnsupervisedClusters is set to 4, and the infrared bands are used as part of the classification process.

2.2 Section 2: Detecting Objects in Imagery with the SNIC Algorithm

The noise you noticed in the pixel-based classification will now be improved using a two-step approach for object-based image analysis. First, you will segment the image using the SNIC algorithm, and then, you will classify it using a k-means unsupervised classifier. Return to your script, where we will add a new function. Noting that the code’s sections are numbered, find code Sect. 1.2 and add the function below beneath it.

A block of code for 1.3 seed creation and S N I C segmentation function. The functions used in the code are e e dot algorithms dot image dot segmentation dot seed grid, e e dot algorithms dot image dot segmentation dot S N I C and add bands.

As you see, the function assembles parameters needed for running SNIC (Achanta and Süsstrunk 2017; Crowley et al. 2019), the function that delineates objects in an image. A call to SNIC takes several parameters that we will explore. Add the following code below code Sect. 2.2.

A block of code for 2.3 object-growing parameters to change and 2.4 parameters that can stay unchanged. Some of the variables in the code are var S N I C underscore super pixel size equals 16, and so on.

Now, add a call to the SNIC function. You will notice that it takes the parameters specified in code Sect. 11.2.2 and sends them to the SNIC algorithm. Place the code below into the script as the code’s Sect. 11.2.5, between Sects. 4 and 6.

A block of code for 2.5 S N I C clustering. The functions used in the code are print, dot reproject, print, map dot add layer, dot select, dot random visualizer, dot select, map dot add layer, dot map, print, dot select, print, and map dot add layer.

Now, run the script. It will draw several layers, with the one shown in Fig. 11.4 on top.

Fig. 11.4
A screenshot of the google earth engine with random S N I C clusters, with randomly chosen colors for each cluster.

SNIC clusters, with randomly chosen colors for each cluster

Full size image

This shows the work of SNIC on the image sent to it—in this case, on the composite of bands 8, 11, and 12. If you look closely at the multicolored layer, you can see small red “seed” pixels. To initiate the process, these seeds are created and used to form square or hexagonal “superpixels” at the spacing given by the parameters passed to the function. The edges of these blocks are then pushed and pulled and directed to stop at edges in the input image. As part of the algorithm, some superpixels are then merged to form larger blocks, which is why you will find that some of the shapes contain two or more seed pixels.

Explore the structure by changing the transparency of layer 5 to judge how the image segmentation performs for the given set of parameter values. You can also compare layer 5.7–layer 3.1. Layer 5.7 is a reinterpretation of layer 3.1 in which every pixel in a given shape of layer 5.3 is assigned the mean value of the pixels inside the shape. When parameterized in a way that is useful for a given project goal, parts of the image that are homogeneous will get the same color, while areas of high heterogeneity will get multiple colors.

Now, spend some time exploring the effect of the parameters that control the code’s behavior. Use your tests to answer the questions below.

Question 7. What is the effect on the SNIC clusters of changing the parameter SNIC_SuperPixelSize?

Question 8. What is the effect of changing the parameter SNIC_Compactness?

Question 9. What are the effects of changing the parameters SNIC_Connectivity and SNIC_SeedShape?

Code Checkpoint F33c. The book’s repository contains a script that shows what your code should look like at this point.

2.3 Section 3: Object-Based Unsupervised Classification

The k-means classifier used in this tutorial is not aware that we would often prefer to have adjacent pixels to be grouped into the same class—it has no sense of physical space. This is why you see the noise in the unsupervised classification. However, because we have re-colored the pixels in a SNIC cluster to all share the exact same band values, k-means will group all pixels of each cluster to have the same class. In the best-case scenario, this allows us to enhance our classification from being pixel-based to reveal clean and unambiguous objects in the landscape. In this section, we will classify these objects, exploring the strengths and limitations of finding objects in this image.

Return the SNIC settings to their first values, namely:

A block of code for the super pixel seed location spacing, in pixels, larger values cause clusters to be more compact, setting this to 0 disables spatial distance weighting, connectivity either 4 or 8, and either square or hex.

As code Sect. 6.2, add this code, which will call the SNIC function and draw the results.

A block of code for 6.2 S N I C unsupervised classification for comparison. The functions used in the code are dot map, print, dot select, map dot add layer, dot random visualizer, and print.

When you run the script, that new function will classify the image in layer 5.7, which is the recoloring of the original image according to the segments shown in layer 5. Compare the classification of the superpixels (6.3) with the unsupervised classification of the pixel-by-pixel values (6.1). You should be able to change the transparency of those two layers to compare them directly.

Question 10. What are the differences between the unsupervised classifications of the per-pixel and SNIC-interpreted images? Describe the tradeoff between removing noise and erasing important details.

Code Checkpoint F33d. The book’s repository contains a script that shows what your code should look like at this point.

2.4 Section 4: Classifications with More or Less Categorical Detail

Recall the variable numberOfUnsupervisedClusters, which directs the k-means algorithm to partition the dataset into that number of classes. Because the colors are chosen randomly for layer 6.3, any change to this number typically results in an entirely different color scheme. Changes in the color scheme can also occur if you were to use a slightly different study area size between two runs. Although this can make it hard to compare the results of two unsupervised algorithms, it is a useful reminder that the unsupervised classification labels do not necessarily correspond to a single land use/land cover type.

Question 11. Find the numberOfUnsupervisedClusters variable in the code and set it to different values. You might test it across powers of two: 2, 4, 8, 16, 32, and 64 clusters will all look visually distinct. In your opinion, does one of them best discriminate between the classes in the image? Is there a particular number of colors that is too complicated for you to understand?

Question 12. What concrete criteria could you use to determine whether a particular unsupervised classification is good or bad for a given goal?

2.5 Section 5: Effects of SNIC Parameters

The number of classes controls the partition of the landscape for a given set of SNIC clusters. The four parameters of SNIC, in turn, influence the spatial characteristics of the clusters produced for the image. Adjust the four parameters of SNIC: SNIC_SuperPixelSize, SNIC_Compactness, SNIC_Connectivity, and SNIC_SeedShape. Although their workings can be complex, you should be able to learn what characteristics of the SNIC clustering they control by changing each one individually. At that point, you can explore the effects of changing multiple values for a single run. Recall that the ultimate goal of this workflow is to produce an unsupervised classification of landscape objects, which may relate to the SNIC parameters in very complex ways. You may want to start by focusing on the effect of the SNIC parameters on the cluster characteristics (layer 5.3) and then look at the associated unsupervised classification layer 6.

Question 13. What is the effect on the unsupervised classification of SNIC clusters of changing the parameter SNIC_SuperPixelSize?

Question 14. What is the effect of changing the parameter SNIC_Compactness?

Question 15. What are the effects of changing the parameters SNIC_Connectivity and SNIC_SeedShape?

Question 16. For this image, what is the combination of parameters that, in your subjective judgment, best captures the variability in the scene while minimizing unwanted noise?

3 Synthesis

Assignment 1. Additional images from other remote sensing platforms can be found in script F33s1 in the book’s repository. Run the classification procedure on these images and compare the results from multiple parameter combinations.

Assignment 2. Although this exercise was designed to remove or minimize spatial noise, it does not treat temporal noise. With a careful choice of imagery, you can explore the stability of these methods and settings for images from different dates. Because the MODIS sensor, for example, can produce images on consecutive days, you would expect that the objects identified in a landscape would be nearly identical from one day to the next. Is this the case? To go deeper, you might contrast temporal stability as measured by different sensors. Are some sensors more stable in their object creation than others? To go even deeper, you might consider how you would quantify this stability using concrete measures that could be compared across different sensors, places, and times. What would these measures be?

4 Conclusion

Object-based image analysis is a method for classifying satellite imagery by segmenting neighboring pixels into objects using pre-segmented objects. The identification of candidate image objects is readily available in Earth Engine using the SNIC segmentation algorithm. In this chapter, you applied the SNIC segmentation and the unsupervised k-means algorithm to satellite imagery. You illustrated how the segmentation and classification parameters can be customized to meet your classification objective and to reduce classification noise. Now that you understand the basics of detecting and classifying image objects in Earth Engine, you can explore further by applying these methods on additional data sources.