Real–time memory efficient SLIC accelerator for low–power applications

Abstract

Superpixel segmentation is one of the popular segmentation methods used in computer vision. One of the major algorithms in superpixel segmentation literature is the Simple Linear Iterative Clustering (SLIC) algorithm. SLIC is an iterative algorithm that data–dependency of its two main steps, i.e., assignment and update, challenges the parallel implementation of the original structure. Also, there are lots of coefficients computed in each step which are needed for the next step and require large throughput. This makes the SLIC algorithm inefficient from memory point of view. The memory inefficiency prevents this algorithm from being used in low–power applications such as augmented reality (AR). In this manuscript, we propose a new structure for SLIC algorithm to improve the memory efficiency. This new structure modifies the Assignment step of SLIC to ignore main non required parameters. To overcome this problem, we propose a new memory-efficient structure for SLIC algorithm. In the basic SLIC algorithm, the main memory consuming parameters are Label and Distance Index which the second one is removed in proposed structure. This new technique works based on simultaneous calculation and comparison of Distance Index of pixels and leads to 25% memory saving. SLIC algorithm works in CIELAB color space which requires more memory space and throughput compared to RGB color space. Proposed structure works in RGB color space rather than CIELAB color space, by implementing a real–time (single clock) RGB–to–CIELAB convertor. This modification resulted to another 37% memory saving. The overall structure is 62% efficient compared to the basic SLIC algorithm and the most efficient one in the literature, to the knowledge of authors. To minimize the execution time of proposed structure, a new structure is proposed for Update step of SLIC algorithm. This memory-efficient SLIC algorithm is implemented and validated on a Stratix–III FPGA family. The implementation results validate real–time behavior of the proposed structure (24 frames per second).

Appendix: : RGB-to-CIELAB

In this section color conversion from RGB to CIELAB color space is represented in details. CIELAB color space, defined by International Commission on Illumination (CIE), is a perceptually uniform color space. In this color space, each pixel is expressed by three components L, a and b where L indicates lightness and a and b are color channels. Since CIELAB color space consists all perceivable colors, it is able to estimate human color perception correctly.

Conversion of RGB color space to CIELAB is achieved from following steps: In the first step, RGB values of each pixel are normalized to be in the range of [0, 1]. Then obtained values of RGB are transformed to sRGB color components using following formula:

$$ h(u) = \left\{ \begin{array}{lll} \frac{u}{12.92} & \quad , & \quad u \le 0.04045\\ \left( \frac{u + 0.055}{1.055} \right)^{2.2} &\quad , & \quad otherwise \end{array} \right. $$

(6)

where u is RGB color dimensions. In the second step, sRGB to CIEXYZ conversion is performed by:

$$ \left[ \begin{array}{c} X\\ Y\\ Z \end{array}\right] = \left[ \begin{array}{ccc} 0.4124 & 0.3576 & 0.1805 \\ 0.2126 & 0.7152 & 0.0722 \\ 0.0193 & 0.1192 & 0.9505 \end{array} \right] \left[ \begin{array}{c} R_{linear}\\ G_{linear}\\ B_{linear} \end{array} \right] $$

(7)

Where R_linear, G_linear and B_linear are sRGB color components. The third step carries out L, a and b values using following equations:

$$ \begin{array}{@{}rcl@{}} L &=& 116 \left( f\left( \frac{Y}{Y_{n}}\right)\right) - 16 \\ a &=& 500\left( f\left( \frac{X}{X_{n}}\right) - f\left( \frac{Y}{Y_{n}}\right)\right) \\ b &=& 200\left( f\left( \frac{Y}{Y_{n}}\right) - f\left( \frac{Z}{Z_{n}} \right)\right) \\ f(v) &=& \left\{\begin{array}{lll} 7.787v + \frac{16}{116} & \quad , & \quad v \le 0.008856 \\ \sqrt[3]{v} &\quad , & \quad otherwise \end{array} \right. \end{array} $$

(8)

With respect to an illuminant D65 standard white point, the values of X_n, Y_n and Z_n are:

$$ \begin{array}{@{}rcl@{}} X_{n} &=& 0.9505 \\ Y_{n} &=& 1.0000 \\ Z_{n} &=& 1.0890 \end{array} $$

(9)

For simplicity of hardware implementation, the division operation in L, a and b calculation at (8) can be applied to coefficients in the second step. In this case the (7) and (8) are changed as following:

$$ \left[\begin{array}{c} X\\ Y\\ Z \end{array}\right] = \left[\begin{array}{ccc} 0.4339 & 0.3762 & 0.1899 \\ 0.2126 & 0.7152 & 0.0722 \\ 0.0177 & 0.1095 & 0.8728 \end{array} \right] \left[\begin{array}{c} R_{linear}\\ G_{linear}\\ B_{linear} \end{array} \right] $$

(10)

$$ \begin{array}{@{}rcl@{}} L &=& 116(f(Y) - 0.138) \\ a &=& 500(f(X) - f(Y)) \\ b &=& 200(f(Y) - f(Z)) \\ f(v) &=& \left\{ \begin{array}{lll} 7.787v + \frac{16}{116} & \quad , & \quad v \le 0.008856 \\ \sqrt[3]{v} & \quad , & \quad otherwise \end{array} \right. \end{array} $$

(11)