Demystifying the Asymptotic Behavior of Global Denoising

Abstract

In this work, we revisit the global denoising framework recently introduced by Talebi and Milanfar. We analyze the asymptotic behavior of its mean-squared error restoration performance in the oracle case when the image size tends to infinity. We introduce precise conditions on both the image and the global filter to ensure and quantify this convergence. We also make a clear distinction between two different levels of oracle that are used in that framework. By reformulating global denoising with the classical formalism of diagonal estimation, we conclude that the second-level oracle can be avoided by using Donoho and Johnstone’s theorem, whereas the first-level oracle is mostly required in the sequel. We also discuss open issues concerning the most challenging aspect, namely the extension of these results to the case where neither oracle is required.

Appendices

Appendix 1: Global Denoising for Gaussian Textures on Fourier Basis

Proposition 3

Let ${\mathbf {u}}= {\mathbf {h}} * {\mathbf {m}}$ be a Gaussian texture where $m_i \sim {\mathcal {N}}(0,\tau ^2)$ iid, and the kernel ${\mathbf {h}}\in \ell ^1 ({\mathbb {Z}}^2)$ has a smooth Fourier transform $\widehat{{\mathbf {h}}}\in L^1\left( \left[ -\frac{\pi }{2},\frac{\pi }{2}\right] ^2\right) \cap C^\infty$. Then, when choosing V as a Fourier or DCT basis, and $\lambda$ as the optimal oracle eigenvalues for that basis, then the MSE for global denoising is upper-bounded by

$$\begin{aligned} {{{\mathrm{MSE}}}{}}^*_{\mathrm{bound}} = \frac{\sigma ^2}{2N} \Vert {{\mathbf {b}}} \Vert _1 = \frac{1}{N^2} \frac{\sigma }{2} \sum _{k} |\widehat{{\mathbf {h}}_N}(k)| |\widehat{\mathrm {m}_{N}\!}\,(k)|=:A_N \end{aligned}$$

Asymptotically, we have a strictly positive MSE bound

$$\begin{aligned} {{\mathrm{MSE}}}{}^*_\mathrm{bound} = A_N \xrightarrow [N\rightarrow \infty ]{} A_\infty = \frac{\sigma \tau \Vert {\hat{{\mathbf {h}}}}\Vert _1}{\sqrt{2\pi }} > 0. \end{aligned}$$

Proof

Consider for simplicity images of square size $N=n^2$ with $n\in {\mathbb {Z}}^2$.

Now consider ${\mathbf {m}}^N$ the restriction of m to $I_N = [-\frac{N}{2},\frac{N}{2}^2) \cap {\mathbb {Z}}^2$.

In the truncated case, the convolution is understood in the periodic sense, so that we can write

$$\begin{aligned} \widehat{{\mathbf {u}}_N}(k) = \widehat{{\mathbf {h}}_N}(k) \cdot \widehat{{\mathbf {m}}_N}(k) \quad \forall k\in I_N \end{aligned}$$

with $\widehat{{\mathbf {h}}_N}(k) = {\hat{h}}(\frac{2 \pi k}{n})$.

Now if we decompose ${\mathbf {u}}_N$ in the Fourier basis $V=F^*$, then

$$\begin{aligned} {\mathbf {b}}^N_k = (F {\mathbf {u}}_N)_k = \widehat{{\mathbf {h}}_N}(k) \cdot \widehat{{\mathbf {m}}_N}(k) \end{aligned}$$

So the upper bound given in Eq. (14) for the optimal MSE is

$$\begin{aligned} {{\mathrm{MSE}}}{}^*_N \le \frac{1}{N^2} \frac{\sigma \Vert {\mathbf {b}}_N \Vert _1}{2} = \frac{1}{N^2} \frac{\sigma }{2} \sum _{k\in I_N} |\widehat{{\mathbf {h}}_N}(k)| |\widehat{{\mathbf {m}}_N}(k)|=:A_N \end{aligned}$$

In the asymptotic case when $N\rightarrow \infty$, a simple Riemann sum argument based on the regularity of ${\hat{h}}$ and on the known value of $\int _{\mathrm{d}I} |{\hat{m}}| = |\mathrm{d}I| {\mathbb {E}}(\widehat{m_N}(k)) = \tau \sqrt{\frac{2}{\pi }}|\mathrm{d}I|$ over a small interval (mean absolute deviation of a gaussian) leads to the conclusion that the upper bound tends to a strictly positive constant

$$\begin{aligned} {{\mathrm{MSE}}}{}^*_\infty \le \lim _{N\rightarrow \infty } A_N = \frac{\sigma \tau \Vert {\widehat{h}}\Vert _1}{\sqrt{2\pi }} = A_\infty > 0. \end{aligned}$$

$\square$

Fig. 12

Left column the decay of the $|b_j^N|$ for each size and the result of the model fitting (dotted lines) for the image simpson for the different bases (from top to bottom) DCT, wavelet and O-NLM. Right column the decay of the ${{\mathrm{MSE}}}^\star$ (blue), the upper bound from (14) (orange) and the fitted bound (yellow) (Color figure online)

Fig. 13

Left column the decay of the $|b_j^N|$ for each size and the result of the model fitting (dotted lines) for the image bricks for the different bases (from top to bottom) DCT, wavelet and O-NLM. Right column the decay of the ${{\mathrm{MSE}}}^\star$ (blue), the upper bound from (14) (orange) and the fitted bound (yellow) (Color figure online)

Fig. 14

Left column the decay of the $|b_j^N|$ for each size and the result of the model fitting (dotted lines) for the image sparse for the different bases (from top to bottom) DCT, wavelet and O-NLM. Right column the decay of the ${{\mathrm{MSE}}}^\star$ (blue), the upper bound from (14) (orange) and the fitted bound (yellow) (Color figure online)

Fig. 15

Left column the decay of the $|b_j^N|$ for each size and the result of the model fitting (dotted lines) for the image synthetic for the different bases (from top to bottom) DCT, wavelet and O-NLM. Right column the decay of the ${{\mathrm{MSE}}}^\star$ (blue), the upper bound from (14) (orange) and the fitted bound (yellow) (Color figure online)

Fig. 16

Left column the decay of the $|b_j^N|$ for each size and the result of the model fitting (dotted lines) for the image man for the different bases (from top to bottom) DCT, Wavelet, and O-NLM. Right column the decay of the ${{\mathrm{MSE}}}^\star$ (blue), the upper bound from (14) (orange) and the fitted bound (yellow) (Color figure online)

Appendix 2: An Oracle Filter that Provides Asymptotically Zero MSE

Proposition 4

Consider an oracle filter which consists in denoising ${\tilde{\mathbf {u}}}$ by averaging at pixel i all values ${\tilde{\mathbf {u}}}_j$ such that $|u_i - u_j| \le \varepsilon$ for a given threshold $\varepsilon >0$ and an infinite image ${\mathbf {U}}_{}$ bounded with values in [0, 1]. Then, the value ${{\mathrm{MSE}}}(\widehat{{\mathbf {u}}^N} | {{\mathbf {u}}^N})$ converges to a limit smaller than $\varepsilon ^2$.

Proof

Indeed, let $n = \lceil \frac{1}{\varepsilon } \rceil$, for $0\le k <n$ let $C_k = \{i \,{\in }\, \varOmega ;\; u_i^N \,{\in }\, \left[ \frac{k}{n}, \frac{k+1}{n} \right] \}$, and for every pixel i let $A_i^{\varepsilon } = \{j; \; |u_i^N-u_j^N|\le \varepsilon \}$. It is clear that if $i \in C_k$, then $C_k \subset A_i^{\varepsilon }$. We can write

$$\begin{aligned} {{\mathrm{MSE}}}(\widehat{{\mathbf {u}}^N} | {{\mathbf {u}}^N})= & {} \frac{1}{N} \sum _{k=0}^{n-1} \sum _{i\in C_k} {{\mathrm{{\mathbb {E}}}}}[|u_i^N - \widehat{u^N_i}|^2]\\= & {} \frac{1}{N} \sum _{k=0}^{n-1} \sum _{i\in C_k} {{\mathrm{{\mathbb {E}}}}}\left[ \left| u_i^N -\frac{1}{\#A_i^{\varepsilon }}\sum _{j\in A_i^{\varepsilon }} {\tilde{u}}^N_j\right| ^2\right] \\= & {} \frac{1}{N} \left[ \sum _{k=0}^{n-1} \sum _{i\in C_k} \left| u_i^N -\frac{1}{\#A_i^{\varepsilon }}\sum _{j\in A_i^{\varepsilon }} u_j^N\right| ^2\right. \\&+\left. \sum _{k=0}^{n-1} \sum _{i\in C_k} \frac{\sigma ^2}{\#A_i^{\varepsilon }}\right] \\\le & {} \varepsilon ^2 + \frac{\sigma ^2}{N} \sum _{k=0}^{n-1} \sum _{i\in C_k} \frac{1}{\#A_i^{\varepsilon }}\\\le & {} \varepsilon ^2 + \frac{\sigma ^2}{N} \sum _{k=0}^{n-1}\sum _{i\in C_k}\frac{1}{\#C_k} \le \varepsilon ^2 + \frac{n\sigma ^2}{N}. \end{aligned}$$

For a given value of $\varepsilon$, this term converges to $\varepsilon ^2$ when $N\,{\rightarrow }\, \infty$. $\square$

Appendix 3: Additionnal Experiments

Figures 12, 13, 14, 15 and 16 show the detailed asymptotic convergence results for the images in Fig. 7 and Table 1.