Fast colorization based image coding algorithm using multiple resolution images

Abstract

This paper proposes a representative pixel (RP) extraction algorithm and chrominance image recovery algorithm for the colorization-based digital image coding. The colorization-based coding methods reduce the color information of an image and achieve higher compression ratio than JPEG coding; however, they take much more computing time. In order to achieve low computational cost, this paper proposes the algorithm using the set of multiple-resolution images obtained by colorization error minimizing method. This algorithm extracts RPs from each resolution image and colorizes each resolution image utilizing a lower resolution color image, which leads to the reduction of the number of RPs and computing time. Numerical examples show that the proposed algorithm extracts the RPs and recovers the color image fast and effectively.

1 Introduction

The image colorization technique can recover a full-color image from a luminance image and several representative pixels (RPs) which have the chrominance values and their positions [1–4]. Levin et al. proposes a well known colorization algorithm, which achieves a high colorization performance using appropriately given RPs. However, this algorithm requires high computing cost and fails to colorize a whole image if only a few RPs are given. The authors have proposed the colorization algorithm, which can restore a color image from only a few RPs and takes a low computational cost [4].

The development of the colorization technique has led to the generation of a new color image coding method called colorization-based color image coding [5–12]. Because the luminance image can be compressed by standard coding methods such as JPEG coding and because the chrominance information is represented by a small number of RPs, the colorization-based image coding can compress full-color images more than these methods. The colorization-based image coding algorithms proposed in [10, 12] have achieved higher coding performance than JPEG2000, which is a highly optimized standard using many entropy coding techniques together. Therefore, the colorization-based image coding has potential to become standard image coding for the next generation. The performance of the colorization based coding is evaluated based on the number of RPs and the quality of a restore image, and therefore, the objective of this paper is to propose a new coding method which requires less RPs and gives more accurate chrominance information. In order to reduce the amount of information to represent the RPs, the method [5] assumes that all pixels in a line segment have only one color and proposes the algorithm where the RPs are described as a set of color line segments. In [9], the algorithm finds a set of two RPs which give the similar effect for colorization result, and the redundant RPs are deleted. In [12], under the assumption that the chrominance image is given as a sparse linear combination of the basis constructed from the luminance image, the algorithm achieves a high compression performance using the theory of compressed sensing [13, 14]. However, these algorithms require a high computing time depending on the image size. To reduce computing time, the method [5] proposes an algorithm to extract RPs from a single low-resolution image. This paper takes the same approach to reduce the computational costs and provides an RPs extraction algorithm using multiple low-resolution images. Though a low resolution image used in [5] is obtained by a simple downsampling, this paper gives low resolution images which are optimized such that a good color image is recovered in the colorization phase.

This paper proposes a new colorization-based image coding algorithm consisting of the extraction RPs phase (coding) and the colorization phase (decoding). In the phase of extracting RPs, the algorithm extracts RPs using error feedback between the original image and the colorized image similar to [15]. In order to achieve a low computational cost and to reduce the amount of information to represent RPs, the algorithm extracts RPs from a set of multiple resolution images obtained by multiple downsampling and colorization error minimizing. In the phase of colorization, the colorization algorithm colorizes multiple-resolution images. Performance of the colorization algorithm affects the performance of the colorization-based image coding, that is, the colorization using a few color pixels leads to a high coding performance. To propose a fast and precise coding algorithm, this paper modifies the colorization algorithm proposed in [4] and introduces a multiple resolution scheme to reduce the computational cost. Figures 1 and 2 show the outline and concept of the proposed algorithm, respectively. The major contribution of this paper is to propose a colorization-based image coding algorithm which requires less computational time and achieves a high coding performance.

Fig. 1

Outline of the proposed colorization-based image coding

Fig. 2

Concept of the proposed algorithms

This paper is organized as follows. In Section 2.2, we give a colorization algorithm based on the algorithm proposed in [4]. Section 2.3 proposes the RPs extraction algorithm which uses recovery error feedback, and it is applied to multiple resolution images. In Section 2.4, a multi-resolution color image recovery algorithm is proposed. We consider the implementation of saving images with RPs in Section 2.5. Numerical examples show that the proposed algorithm extracts the RPs and recovers the color image fast and effectively in Section 3.

2 Main works

2.1 Notation

We provide here a brief summary of the notations used throughout the paper. (A) _i,j and (a) _i denote the (i,j)-element of a matrix A and the ith element of a vector a. ∥·∥₂ and ∥·∥ _F denote the l ₂ norm of a vector and the Frobenius norm of a matrix, respectively. 0 _k ∈R ^k is denoted as a k-dimension zero vector. We use diag(A ₁,…,A _m ) to denote a block diagonal matrix consisting of A ₁, …, A _m and |S| to denote the number of elements in a set S. We denote the ceiling function and the floor function by ⌈a⌉ and ⌊a⌋, respectively.

2.2 Colorization algorithm

In order to provide a colorization-based image coding, this paper applies the colorization algorithm proposed in [4] because it can recover a full color image from a luminance image using only a few color pixels. The algorithm restores chrominance images C _b and C _r independently to obtain a color image, and this subsection gives a chrominance image restoration method.

Let vectors x∈R ^mn and y∈R ^mn denote a desired chrominance image and a rasterized-given luminance image, respectively, where m and n denote height and width of the image. Define U∈R ^(m−1)×m, V∈R ^m(n−1)×mn, $\bar {U} \in \boldsymbol {R}^{(m-1)n\times mn}$ and D∈R ^{(2mn−m−n)×mn} as

$$U_{i,j} = \left\{ \begin{array}{rl} 1, & \text{if \(i=j\)} \cr -1, & \text{if \(i+1=j\)} \cr 0, & \text{otherwise} \end{array} \right., $$

$$V_{i,j} = \left\{ \begin{array}{rl} 1, & \text{if \(i=j\)} \cr -1, & \text{if \(i+M=j\)} \cr 0, & \text{otherwise} \end{array} \right., $$

$$\bar{U} = \text{diag}(U,\ldots,U), $$

and

$$ D=\left[\bar{U}^{T}\ V^{T}\right]^{T}, $$

((1))

respectively. The matrices $\bar {U}$ and V denote vertical and horizontal difference operators. Then D x denotes the differences between the neighbor pixels of a whole image. Let us consider the following ℓ ₂ norm minimizing colorization problem,

$$ \begin{aligned} &\underset{\boldsymbol{x}}{\text{Minimize}} \qquad \| D\boldsymbol{x} \|_{2}^{2} \\ &\mathrm{Subject \ to} \qquad (\boldsymbol{x})_{i} = c_{i},\forall i \in \cal{C}, \end{aligned} $$

((2))

where c _i is a constant corresponding to a given color value, and the index set denotes a given set of vector indices of given color pixels. This problem recovers a color image by minimizing the sum of differences between neighbor pixels. In order to colorize an image more appropriately, [4] assumes that the differences between neighbor pixels of a chrominance image have relationship with those of a color image, which is defined by the following equality,

$$ (D\boldsymbol{x})_{i} = \alpha f{((D\boldsymbol{y})_{i})}, $$

((3))

where α is an unknown constant, and f:R→R is a given function. Then, we obtain the following problem from (2) under the assumption (3),

$$ \begin{aligned} &\underset{\boldsymbol{x}}{\text{Minimize}} \qquad \| FD\boldsymbol{x} \|_{2}^{2} \\ &\mathrm{Subject \ to} \qquad (\boldsymbol{x})_{i} = c_{i},\forall i \in \cal{C}, \end{aligned} $$

((4))

where F is a diagonal matrix whose ith diagonal element is 1/f((D y) _i ). The details are described in [4]. While the problem (4) is a convex quadratic programming and can be solved exactly, [4] applies Lagrangian relaxation to reduce the computational cost. This paper provides an exact analytical solution of (4) which can be implemented simply.

Let us define H∈R ^{(2mn−m−n)×mn} by FD. Then, we define H ^∗∈R ^{(2mn−m−n)×(mn−p)} and $\acute {H^{*}} \in R^{(2mn-m-n)\times p}$ as submatrices of H composed of columns corresponding to unknown color pixels and given color pixels, respectively. Let x ^∗∈R ^mn−p and $\acute {\boldsymbol {x^{*}}} \in R^{p}$ denote unknown color vectors and given color vectors, respectively, where p denotes the number of given color pixels. Then the objective function of (4) is rewritten as follows,

$$ FD\boldsymbol{x}=H\boldsymbol{x}=H^{*}\boldsymbol{x^{*}}+\acute{H^{*}}\boldsymbol{\acute{x^{*}}}. $$

((5))

Because all entries of $\acute {\boldsymbol {x^{*}}}$ are given by c _i , that is, they satisfy the constraint of (4), the problem (4) is equal to the following nonconstrained convex quadratic programming,

$$ \underset{\boldsymbol{x^{*}}}{\text{Minimize}} \qquad \| H^{*}\boldsymbol{x^{*}} + \acute{H^{*}}\boldsymbol{\acute{c^{*}}} \|_{2}^{2}, $$

((6))

where vector $\boldsymbol {\acute {c^{*}}} \in R^{p}$ is a given vector consisting of c _i . If H ^{∗ T} H ^∗ is nonsingular, we obtain the solution of (6) as follows,

$$ \boldsymbol{x^{*}} = -\left({H^{*}}^{T}H^{*}\right)^{-1} {H^{*}}^{T} \acute{H^{*}}\boldsymbol{\acute{c^{*}}}. $$

((7))

Now we move onto discuss the singularity of H ^{∗ T} H ^∗. For an image $\boldsymbol {a} =\, [\!a_{1},a_{2},\dots,a_{\textit {mn}}]^{T} \in \boldsymbol {R}^{mn}$, the following lemma is provided.

Lemma 1.

It holds that H a=0 _{2m n−m−n} if and only if $a_{1}=a_{2}=\dots =a_{\textit {mn}}$.

Proof.

Because F is an invertible matrix, H a=0 _{2m n−m−n} if and only if D a=0 _{2m n−m−n} . Since D is the difference operator defined by (1), D a is a zero vector if and only if all pixels have the same values, that is, $a_{1}=a_{2}=\dots =a_{\textit {mn}}$.

Let a ⁽ⁱ⁾∈R ^mn−1 denotes a subvector of a and be composed by deleting an ith element of a, and H ⁽ⁱ⁾∈R ^{(2mn−m−n)×(mn−1)} is denoted as a submatrix of H and is composed by deleting the ith column of H. Then the Lemma 1 leads to the following lemma:

Lemma 2.

For all i∈{1,2,…,m n}, it holds that H ⁽ⁱ⁾ a ⁽ⁱ⁾=0 _{2m n−m−n} if and only if a ⁽ⁱ⁾=0 _{m n−1}.

Proof.

We have that

$$ H^{(i)}\boldsymbol{a}^{(i)}= H\boldsymbol{a} - H\acute{\boldsymbol{a}}= H(\boldsymbol{a} - \acute{\boldsymbol{a}}), $$

((8))

where $\acute {\boldsymbol {a}} =\, [\!0,\dots,0,a_{i},0,\dots,0]^{T} \in \boldsymbol {R}^{mn}$. Therefore, we obtain followings from Lemma 1,

$$ \begin{aligned} {}H^{(i)}\boldsymbol{a}^{(i)}=\boldsymbol{0}_{2mn-m-n} &\Leftrightarrow \!H(\boldsymbol{a} - \acute{\boldsymbol{a}})=\boldsymbol{0}_{2mn-m-n} \\ &\Leftrightarrow \!a_{1}\,=\,a_{2}\,=\,\dots\!=a_{i}\!-a_{i}\,=\,\dots\!=a_{mn} \\ &\Leftrightarrow \!a_{1}\,=\,a_{2}\!=\dots= 0 =\dots\!=a_{mn} \\ &\Leftrightarrow \boldsymbol{\!a}^{(i)}\! = \mathbf{0}_{mn-1}.\\[-14pt] \end{aligned} $$

((9))

Theorem 1.

H ^{∗ T} H ^∗ is a nonsingular matrix

Proof.

Lemma 2 guarantees that the subspace of the column space of H is linear independent, that is, the column space of H ^∗ is linear independent.

From this theorem, there exists the inverse of H ^{∗ T} H ^∗, and therefore we can obtain the exact solution of (4) by (7).

2.3 RPs extraction algorithm

This subsection proposes an algorithm to extract the RPs from an original image utilizing the error feedback between the original image and the colorized image. Let Y∈R ^m×n, C b∈R ^m×n and C r∈R ^m×n denote a given luminance image and two given chrominance images, respectively. First we provide the following RPs extraction scheme for a given color image, its luminance image, a given initial set of RPs and a small constant ε>0,

Step 1. Recover two chrominance images $\bar {Cb}$ and $\bar {Cr}$ using (7) with the luminance image and current RPs.

Step 2. Calculate the error between the original image and the colorized image obtained in Step 1 at each pixel as follows,

$$ (E)_{i,j}=|(Cb-\bar{Cb})_{i,j}|+|(Cr-\bar{Cr})_{i,j}|. $$

((10))

Step 3. Terminate if (E) _i,j ≤ε for all (i,j), else go to Step 4.

Step 4. Add the pixel with the largest error E _i,j to the set of RPs.

Step 5. Terminate if the number of iterations is larger than a given threshold, else go to Step 1.

Since this scheme extracts one pixel in one iteration, some pixels are selected appropriately as RPs by repeating Step 1–5. Although this scheme can steadily reduce the recovery error, it requires a lot of iterations to obtain an adequate amount of RPs and takes a lot of computing time for a large size image. In order to reduce the computing cost, this paper proposes an RP extraction algorithm using multiple resolution images.

Let I ₀= [ Y ₀ C b ₀ C r ₀]∈R ^m×3n and $I_{k}=\,[\!Y_{k} \ Cb_{k} \ Cr_{k}] \in {R}^{m_{k}\times 3n_{k}}\ (k=1,2,\cdots,N)$ denote the original image and kth reduced image, respectively, and N≥1 is a given constant. Y _k is obtained by simply multiple downsampling as follows,

$$ \begin{aligned} &{(Y_{k})}_{i,j} = {(Y_{k-1})}_{2i,2j}, \\ &i = 1, \cdots, \lfloor \frac{m_{k-1}}{2}\rfloor, j = 1, \cdots, \lfloor \frac{n_{k-1}}{2} \rfloor, \end{aligned} $$

((11))

where m _k and n _k denote horizontal and vertical sizes of the kth reduced image. Next, we consider a way to generate C b _k and C r _k . Since C b _k and C r _k are composed by the same method, this paper gives a way to make the reduced image C b _k . The chrominance images of the kth reduced image are generated so that the (k−1)th chrominance image C b _k−1 can be restored preciously using Y _k−1 and C b _k . Let $\acute {\mathbf {c}}_{k-1}^{*} \in R^{m_{k}n_{k}}$ and $\mathbf {c}_{k-1}^{\ast } \in R^{{m_{k-1}}{n_{k-1}}{-m_{k}}{n_{k}}}$ denote vectors formed by pixels of even numbered columns and rows, that is, they do not include pixels of odd numbered columns nor rows. Then, this paper proposes the chrominance image recovery problem of the kth reduced image as follows,

$$ {}\underset{\boldsymbol{{x}_{k}}}{\text{Minimize}} \ \| \boldsymbol{c}_{k-1}^{*} +\left({H^{*}}^{T}H^{*}\right)^{-1} \!{H^{*}}^{T}\! \acute{H^{*}}\boldsymbol{{x}}_{k} \|_{2}^{2} +\lambda\|\boldsymbol{\acute{c}}_{k-1}^{*}-\boldsymbol{{x}}_{k}\|^{2}_{2}, $$

((12))

where λ>0 is a given constant. Because the chrominance images C b _k−1 and C r _k−1 are recovered using (7) with Y _k−1, C b _k and C r _k in the colorization phase as written Section 2.4, this problem gives C b _k to achieve a good colorization of the (k−1)th image. However, problem (12) requires high computational cost to obtain the optimal solution since it takes high cost to calculate (H ^{∗ T} H ^∗)⁻¹. Therefore this paper provides its relaxed problem to obtain an approximate solution with low computational cost. Let J denote the first term of the objective function in (12). Then, we have that

$$ \begin{aligned} J&= \| \boldsymbol{c}_{k-1}^{*} +\left({H^{*}}^{T}H^{*}\right)^{-1} {H^{*}}^{T} \acute{H^{*}}\boldsymbol{{x}}_{k} \|_{2}^{2} \\ &= \|\left({H^{*}}^{T}H^{*}\right)^{-1} {H^{*}}^{T} \left(H^{*}\boldsymbol{c}_{k-1}^{*} +\acute{H^{*}}\boldsymbol{x}_{k}\right) \|_{2}^{2} \\ &\leq \|\left({H^{*}}^{T}H^{*}\right)^{-1} {H^{*}}^{T}\|_{F}^{2} \ \|H^{*}\boldsymbol{c}_{k-1}^{*} +\acute{H^{*}}\boldsymbol{x}_{k}\|_{2}^{2}. \end{aligned} $$

((13))

Since J is bounded by $\|H^{*}\boldsymbol {c}_{k-1}^{*} +\acute {H^{*}}\boldsymbol {x}_{k}\|_{2}^{2}$, we consider the following problem to reduce J instead of (12),

$$ \underset{\boldsymbol{{x}}_{k}}{\text{Minimize}} \quad \| H^{*}\boldsymbol{c}^{*}_{k-1} +\acute{H^{*}}\boldsymbol{{x}}_{k} \|_{2}^{2} + \lambda\|\boldsymbol{\acute{c}}^{*}_{k-1}-\boldsymbol{x}_{k}\|^{2}_{2}. $$

((14))

Utilizing (11) and (14) to obtain a set of multiple resolution images, this paper provides the RPs extraction method as shown in Algorithm ??, where $\bar {Cb}_{k}$ and $\bar {Cr}_{k}$ denote the colorized chrominance images of I _k . This algorithm extracts RPs from I _N to I _N−K in turn for given constant K≥1, and the even numbered pixels of I _k are used as initial RPs at each iteration. The performance of Algorithm ?? depends heavily on the value of K. If the value of K is large, lots of RPs are extracted, that is, the K determines the number of RPs, the calculation time and the volume of information to store RPs. The numerical examples in Section 3 show the effects of K.

Note that the chrominance values of these even numbered pixels are deleted in $\Omega _{k}^{RP}$ at the end of each iteration. They are restored as C b _k+1 and C r _k+1 in the previous iteration and are reused as RPs for recovering I _k . Therefore Algorithm ?? does not store them in $\Omega _{k}^{RP}$ to reduce the amount of information about RPs. All pixels of the Nth reduced image are members of $\Omega _{N}^{RP}$. Figure 3 shows outline of the algorithm.

Fig. 3

Outline of the proposed RP extraction algorithm using multiple resolution images

2.4 Color image recovery

This subsection provides the chrominance image recovery algorithm. Now we consider a way to recover an image I ₀= [ Y ₀ C b ₀ C r ₀]∈R ^m×3n from the stored RPs $\Omega _{k}^{RP} \ (k = N,\ldots N-K)$. First, I _N is completely obtained by $\Omega _{N}^{RP}$. Next, we focus onto recover the image I _k (k=N−1,N−2,…,0). Since $\Omega _{k}^{RP}$ includes RPs without the even numbered pixels, image I _k+1 is used as $\Omega _{k}^{RP}$ corresponding to the even numbered pixels. Then the color images I _k is restored by (7) using $\Omega _{k}^{RP}$ and I _k+1. Finally we obtain the colorization algorithm with multiple resolution images as shown in Algorithm ??, and Fig. 4 shows its outline.

Fig. 4

Outline of the proposed chrominance image recovery algorithm using multiple resolution images

The calculation cost of the colorization proposed in subsection 2.2 mostly depends on the calculating of inverse matrix (H ^{∗ T} H ^∗)⁻¹, and the size of (H ^{∗ T} H ^∗)⁻¹ depends on the number of unknown color pixels. The number of unknown color pixels in Algorithm ?? is equal to the following,

$$ \begin{aligned} &\sum_{k=0}^{N-1} \left(m_{k} \times n_{k} -m_{k+1} \times n_{k+1} - |\Omega_{k}^{RP}|\right) \\ &= m_{0} \times n_{0} -m_{N} \times n_{N} - \sum_{k=0}^{N-1} \left|\Omega_{k}^{RP}\right| \\ &= m_{0} \times n_{0} - \sum_{k=0}^{N} \left|\Omega_{k}^{RP}\right|. \end{aligned} $$

((15))

This implies that the calculation cost does not depend on N. Numerical examples in Section 3 show that the computing time is independent of N.

2.5 Volume of information of RPs

This subsection gives an encoding method of RPs.

Let us consider the case that an m×n 24-bit image is compressed with p RPs. If the information of RPs is composed of their coodinates and two chrominance values, p(2×8+⌈log₂ m n⌉) [bits] are required to represent all RPs because 2×8 [bits] and ⌈log₂ m n⌉ [bits] are required to represent their chrominance values and coordinates, respectively. The proposed algorithm extracts RPs based on multiple resolution images and requires less information to represent them.

We consider encoding of the coordinates of RPs $\Omega _{k}^{RP} \ (k=N,N-1,\ldots N-K)$. In Algorithm ??, $\Omega _{N}^{RP}$ is given as the reduced image I _N , the $\Omega _{N}^{RP}$ are given as the reduced image I _N , that is, all pixels of I _N are RPs. Therefore, the coordinates of $\Omega _{N}^{RP}$ are not required, and only the value of N is coded to represent $\Omega _{N}^{RP}$. Next, we consider coding of the coordinates of RPs in $\Omega _{k}^{RP} \ (k=N-1, \ldots,N-K)$. For m×n image, we store the coordinates of RPs as a rasterized vector of a binary matrix of size m×n whose entries are 1 if corresponding pixels are RPs and 0 if otherwise. Hence, this vector is sparse, and it is coded by run length encoding (RLE), which stores the length of consecutive 0s. Empirical results show that the length of consecutive 0s on the image Ω N−k R P is less than z _k defined by

$$\begin{array}{@{}rcl@{}} z_{N-k} = \frac{m_{N-k} n_{N-k} -m_{N-k+1} n_{N-k+1}}{t_{max}}. \end{array} $$

((16))

Therefore, this paper proposes to encode the coordinate by RLE per following bit,

$$\begin{array}{@{}rcl@{}} bitunit_{N-k} = \text{max}\left(\left\lceil \log_{2} (z_{N-k}) \right\rceil,2\right). \end{array} $$

((17))

Table 1 shows the coding table of b i t u n i t _N−k =3. This decoding method requires the run length data of $\Omega _{k}^{RP} \ (k = N-1, \ldots,N-K)$ and the values of N and t _max to decode the coordinates of RPs.

Table 1 Correspondence table (bit unit is 3)

3 Numerical examples

This section presents numerical examples to show the efficiency of the proposed algorithm. We use the test images as shown in Fig. 5. In order to evaluate the quality of image compression, we measure the differences between the original image and the recovered image using the peak signal to noise ratio (PNSR) and the structural similarity (SSIM) [16]. In order to calculate these evaluates, we use the MATLAB functions psnr and ssim. In all experiments we use ε=2 and λ=0.3, which are selected empirically from ε∈ [ 0,10] and λ∈ [ 0.1,1] to achieve the best performance. We use t _max =70 except for the second experiment.

Fig. 5

Test images (256 × 256): a milkdrop, b boat, c lenna, d pepper, e airplane, f couple, and g earth

First we examine the effect of the parameters N and K using uncompressed luminance images. Table 2 shows PSNR, the computing time of Algorithm ??, the computing time of Algorithm ?? and the volume of information to store RPs. PSNR is obtained by averaging those of Cb and Cr. As can be seen, the results of N≥5 are almost the same. Figure 6 shows the PSNR and the computing time of Algorithm ?? with N=5 for N−K∈{0,1,2,3,4}. We can see that the algorithm with (N,K)=(5,3) achieves the best tradeoff between calculation time and PSNR, and therefore we use (N,K)=(5,3) in the rest of this section.

Fig. 6

The value of N−K vs. PSNR and coding time for a milkdrop, b boat, c lenna, d pepper, e airplane, f couple, and g earth

Table 2 Numerical results for several (N,K)s

Next this paper shows the coding performance of the proposed algorithm comparing with the algorithm proposed in [9] and the random algorithm which selects RPs randomly. We chose λ=5 in (12), which is selected empirically from λ∈[1,10] to achieve the best performance. To evaluate the performance of the relaxed problem (14) in RPs extraction scheme, we also examine Algorithm ?? with (12) instead of (14). Because we cannot fix the number of RPs exactly in the algorithm proposed in [9], we adjust the parameter t _max of Algorithm ?? such that the number of RPs is nearly equal to that of [9]. The random algorithm selects RPs randomly and gives the same number of RPs as the proposed algorithm, and we use colorization algorithm (7). In order to see the performance of colorization based coding method, these algorithms use uncompressed luminance images. Because the algorithm proposed in [9] and the proposed algorithm with (12) use a large amount of memory and take a high computational cost, they cannot be applied to a whole image of the test images. Therefore, we use their partial images as shown in Fig. 7. Table 3 shows the number of RPs, the volume of information to restore the RPs, SSIM, PSNR, the computing time to extract the RPs (encoding time), and to colorize the images (decoding time). As can be seen, the proposed algorithm with (14) works faster and achieves better performance than the other algorithms.

Fig. 7

Partial test images (192 × 192): a milkdrop, b boat, c lenna, d pepper, e airplane, f couple, and g earth

Table 3 Numerical results of comparison with other colorization based image coding algorithms

Thirdly, the proposed algorithm is compared with the algorithm proposed in [12] using the test images as shown in Fig. 5. In [12], Lee et al. have proposed the colorization-based image coding algorithm, which achieves a high coding performance. In order to see the performance of colorization-based coding method, these algorithms use uncompressed luminance images. Since an arbitrary number of RPs can be used in Lee’s algorithm, we examine the algorithm in two ways by using (1) the same number of RPs as those of the proposed algorithm and (2) RPs whose volume of information to be represented is equal to that of the proposed algorithm.

Table 4 shows the number of RPs, the volume of information to restore the RPs, SSIM, PSNR, the computing time to extract the RPs (encoding time) and to colorize the images (decoding time). We can see that the proposed algorithm achieves similar performance to Lee’s algorithm and requires less computational.

Table 4 Numerical results of comparison with Lee’s algorithm

Finally, we compare the proposed algorithm with JPEG and JPEG2000 coding method. Luminance images are compressed using JPEG/JPEG2000 coding where the quality parameter (QP) is selected such that the volume of information is equal to about 4000 bytes.

Table 5 shows the volume of information [byte] to represent each color image, SSIM, PSNR and quality parameter (QP) for JPEG/JPEG2000, Figs. 8, 9 and 10 show the recovery images, and Fig. 11 shows zoomed images of Fig. 8. SSIM and PSNR are obtained by averaging those of red, green, and blue images. As can be seen, the proposed algorithm with JPEG2000 can compress color images the best of all while the qualities of compressed images are equal or better than other coding methods. The proposed algorithm requires 0.7–0.8 [s] for encoding and 0.2–0.3 [s] for decoding in MATLAB 2014a on a PC with an Intel Core i7 3.4 GHz CPU, 8 GB of RAM memory.

Fig. 8

Image results (milkdrop) of a JPEG, b JPEG2000, c proposed algorithm with JPEG, and d proposed algorithm with JPEG2000

Fig. 9

Image results (boat) of a JPEG, b JPEG2000, c proposed algorithm with JPEG, and d proposed algorithm with JPEG2000

Fig. 10

Image results (lenna) of a JPEG, b JPEG2000, c proposed algorithm with JPEG, and d proposed algorithm with JPEG2000

Fig. 11

Zoomed image results (milkdrop) of a original, b JPEG2000, and c proposed algorithm with JPEG2000

Table 5 Numerical results of comparison with JPEG and JPEG2000

4 Conclusions

This paper proposes a representative pixel (RP) extraction and colorization algorithm for the colorization-based digital image coding. In order to achieve low computing cost and high image coding performance, multiple reduced images are generated by colorization error minimizing method, and the RPs are extracted from these reduced images. Numerical examples show that the proposed algorithm can extract RPs and recover a color image fast and effectively comparing with other colorization-based algorithms, and numerical results show that the proposed algorithm achieves higher coding performance than JPEG/JPEG2000.