Image Segmentation

This document describes an algorithm implemented using the Insight Toolkit ITK www.itk.org. This paper is accompanied with the source code, input data, parameters and output data that the authors used for validating the algorithm described in this paper. This adheres to the fundamental principle that scientific publications must facilitate reproducibility of the reported results.


Introduction
Recall in edge detection, we found the boundaries of objects based on the change of pixel intensities.These boundaries divided, or segmented, an image into different regions.Here we discuss a region segmentation, where the image is segmented into closed regions based on region characteristics such as color and object classification.In this lecture, we will be focusing on the first approach.Region segmentation allows us to isolate individual parts of an image, while edge detection cannot.

Definition of Segmentation
Formally, an image I is partitioned into N regions if it satisfies the following properties: As we are segmenting the image by color, we need predicates (constraints) to determine the extent of regions.To simplify the discussion, let's begin by assuming out images are greyscale.Some examples of predicates are: • All pixels in a region must have the same intensity.
• All pixels in a region must be within a set intensity interval.
• The standard deviation of intensities in a region must be less than a certain value

Thresholds and Histograms
When the image only contains one object, let's say a grey block on a black tablecloth, the greyscale image is a split into two regions.One region (the object) is represented by some intensity value, while the other (background) is represented by a different intensity value.When we plot the pixel intensities on a histogram, we see two peaks.Thus to get the two regions, we can define a single threshold T in between the two peaks of the histogram to segment the image.This special type of thresholding results in a binary image.
Notice that binary images can contain more than two regions (e.g.many identical objects on a background).
For images that cannot be segmented into a binary image, the pixel intensity of the segmented image, J(x, y), depends on the relation of the corresponding pixel intensity in the original image, I(x, y), to a range of thresholds.
We can decompose the image into a histogram of color intensities and choose thresholds at the valleys of the histogram (remember in a histogram we can count the number of pixels in each interval) to obtain the regions.If there are N regions, there are N − 1 thresholds, and we end up with N binary images, where the region of interest is represented by an intensity of 1 while everything else 0. From these images, we can assign each region a label and build a complete segmented image.The sharpness of a peak is defined as the ratio of the area of the peak to the rectangle formed by its width and height: N W * P .Tall peaks are measured by the ratio of peak height to the average height of the adjacent valleys: Va+V b 2 * P .The peakiness is defined as a product of these factors.If the peakiness is greater than some threshold, then the peak will be used for segmentation.
Figure 4: Components of a Histogram Peak

Connected Components
We N regions and N binary images, so how do we combine them into one image?Notice that we can process the binary images over a sliding window of time, meaning at the first instance, we segment the image based on the first threshold, and the at the second instance, the first and second thresholds, so on... and at the last instance, the last threshold.Let's assign each instance of time an id, and apply the sliding window.Every time we apply the thresholding, we also perform flood fill on the image (setting pixels in region to be 1), and then assign all 1 pixels the id.The end result is a fully segmented image.This algorithm runs in O(N * P ), where P is the number of pixels.
Figure 5: Recursive Algorithm The issue with a recursive algorithm is that we are constrained by the memory of the call stack.We can also apply a two-pass sequential algorithm.This algorithm is based on equivalence classes, which is like an id of a region.Let's assume in a binary image we have two regions.If we suddenly find a connection, then then we set the regions' equivalence classes equal to each other.In the end, we reassign regions with equivalent equivalence classes to a master equivalence class.
Figure 6: Sequential Algorithm The final image segmentation algorithm is presented below:

Region Growing
Often image segmentation yields too many regions (see Figure 1) that we want to merge into larger, more representative regions.Imagine we have the initial segmented image, or seed segmentation.We can refine it through region growing algorithms.The simplest region growing algorithm is split and merge.We have a predicate (e.g.same pixel intensity) that determines if two regions are merged or split.There are also a lot of other region growing algorithms, and many of them involve graph theory.

Applications
The applications of image segmentation are numerous.First, because image segmentation divides the image into different regions with similar properties, it can aid in locating objects and classify objects.In traffic control systems or autonomous vehicles, it can help pinpoint objects in the surroundings.In the medical field, image segmentation allows doctors to see different areas of the body and observe blood flow.
In this lecture we covered only the basics of image segmentation.Modern segmentation is usually accomplished using convolutional neural networks or robust computer vision algorithms, but many of the concepts are the same.

Figure 1 :
Figure 1: Comparison of region segmentation algorithms based on object classification.Notice that regions are completely separated from each other by boundaries.

Figure 9 :
Figure 9: Demonstration of the Split Merge Algorithm

Figure 10 :
Figure 10: Entire Image Segmentation Process