Abstract
In image restoration, we usually assume that the underlying image has a good sparse approximation under a certain system. Wavelet tight frame system has been proven to be such an efficient system to sparsely approximate piecewise smooth images. Thus, it has been widely used in many practical image restoration problems. However, images from different scenarios are so diverse that no static wavelet tight frame system can sparsely approximate all of them well. To overcome this, recently, Cai et. al. (Appl Comput Harmon Anal 37:89–105, 2014) proposed a method that derives a data-driven tight frame adapted to the specific input image, leading to a better sparse approximation. The data-driven tight frame has been applied successfully to image denoising and CT image reconstruction. In this paper, we extend this data-driven tight frame construction method to multi-channel images. We construct a discrete tight frame system for each channel and assume their sparse coefficients have a joint sparsity. The multi-channel data-driven tight frame construction scheme is applied to joint color and depth image reconstruction. Experimental results show that the proposed approach has a better performance than state-of-the-art joint color and depth image reconstruction approaches.
Similar content being viewed by others
Notes
The dataset can be downloaded at http://web.cecs.pdx.edu/~fliu/project/depth-enhance/Middlebury.htm.
References
Bao, C., Cai, J.-F., and Ji, H.: Fast sparsity-based orthogonal dictionary learning for image restoration, In: IEEE International Conference on Computer Vision, pp. 3384–3391, Sydney, (2013)
Bao, C., Ji, H., Shen, Z.: Convergence analysis for iterative data-driven tight frame construction scheme, Appl. Comput. Harmon. Anal. (2014). doi:10.1016/j.acha.2014.06.007
Cai, J.-F., Chan, R.H., Shen, L., Shen, Z.: Restoration of chopped and nodded images by framelets. SIAM J. Sci. Comput. 30, 1205–1227 (2008)
Cai, J.-F., Chan, R.H., Shen, L., Shen, Z.: Convergence analysis of tight framelet approach for missing data recovery. Adv. Comput. Math. 31, 87–113 (2009)
Cai, J.-F., Chan, R.H., Shen, Z.: A framelet-based image inpainting algorithm. Appl. Comput. Harmon. Anal. 24, 131–149 (2008)
Cai, J.-F., Dong, B., Osher, S., Shen, Z.: Image restoration: total variation, wavelet frames, and beyond. J. Amer. Math. Soc. 25, 1033–1089 (2012)
Cai, J.-F., Ji, H., Shen, Z., Ye, G.-B.: Data-driven tight frame construction and image denoising. Appl. Comput. Harmon. Anal. 37, 89–105 (2014)
Cai, J.-F., Osher, S., Shen, Z.: Linearized Bregman iterations for compressed sensing. Math. Comp. 78, 1515–1536 (2009)
Cai, J.-F., Osher, S., Shen, Z.: Linearized Bregman iterations for frame-based image deblurring. SIAM J. Imaging Sci. 2, 226–252 (2009)
Candès, E.J., Donoho, D.L.: New tight frames of curvelets and optimal representations of objects with piecewise \(C^2\) singularities. Comm. Pure Appl. Math. 57, 219–266 (2004)
Chan, R.H., Chan, T.F., Shen, L., Shen, Z.: Wavelet algorithms for high-resolution image reconstruction. SIAM J. Sci. Comput. 24, 1408–1432 (2003)
Chan, T.F., Shen, J.: Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods. Society for Industrial and Applied Mathematics, Philadelphia (2005)
Daubechies, I.: Ten Lectures on Wavelets, vol. 61 of CBMS-NSF Regional Conference Series in Applied Mathematics. Society for Industrial and Applied Mathematics, Philadelphia (1992)
Daubechies, I., Han, B., Ron, A., Shen, Z.: Framelets: MRA-based constructions of wavelet frames. Appl. Comput. Harmon. Anal. 14, 1–46 (2003)
Fornasier, M., Ramlau, R., Teschke, G.: The application of joint sparsity and total variation minimization algorithms to a real-life art restoration problem. Adv. Comput. Math. 31, 157–184 (2009)
Fornasier, M., Rauhut, H.: Recovery algorithms for vector-valued data with joint sparsity constraints. SIAM J. Numer. Anal. 46, 577–613 (2008)
Han, B., Kutyniok, G., Shen, Z.: Adaptive multiresolution analysis structures and shearlet systems. SIAM J. Numer. Anal. 49, 1921–1946 (2011)
Hui, T.-W., and Ngi Ngan, K.: Motion-Depth: Rgb-d Depth Map Enhancement with Motion and Depth in Complement, In: IEEE conference on Computer Vision and Pattern Recognition, pp. 3962–3969, Columbus (2014)
Jain, R., Kasturi, R., Schunck, B.G.: Machine Vision, vol. 5. McGraw-Hill, New York (1995)
Ji, H., Shen, Z., Xu, Y.: Wavelet frame based scene reconstruction from range data. J. Comput. Phys. 229, 2093–2108 (2010)
Kuster, C., Popa, T., Zach, C., Gotsman, C., Gross, M. H., Eisert, P., Hornegger, J., and Polthier, K.: Freecam: A Hybrid Camera System for Interactive Free-viewpoint Video. In: Proceedings of International Workshop on Vision, Modeling and Visualization, pp. 17–24 (2011)
Labate, D., Lim, W.-Q., Kutyniok, G., Weiss, G.: Sparse multidimensional representation using shearlets, In: Optics & Photonics 2005, International Society for Optics and Photonics, pp. 59140U–59140U, San Diego, (2005)
Le Pennec, E., Mallat, S.: Sparse geometric image representations with bandelets. IEEE Trans. Image Process. 14, 423–438 (2005)
Liang, J., Ma, J., Zhang, X.: Seismic data restoration via data-driven tight frame. Geophysics 79, V65–V74 (2014)
Lu, S., Ren, X., and Liu, F.: Depth enhancement via low-rank matrix completion, In: IEEE Conference on Computer Vision and Pattern Recognition, Columbus, (2014)
Maimone, A., Fuchs, H.: Encumbrance-free telepresence system with real-time 3d capture and display using commodity depth cameras, In: 10th IEEE international symposium on Mixed and Augmented Reality, pp. 137–146, Basel, (2011)
Mallat, S.: A wavelet tour of signal processing, Elsevier/Academic Press, Amsterdam, 3rd edn. The sparse way, With contributions from Gabriel Peyré (2009)
Richardt, C., Stoll, C., Dodgson, N. A., Seidel, H.-P., and Theobalt, C.: Coherent spatiotemporal filtering, upsampling and rendering of RGBZ videos, In: Proceedings of Eurographics, 31 (2012)
Ron, A., Shen, Z.: Affine systems in \(L_2({{\bf R^d}})\): the analysis of the analysis operator. J. Funct. Anal. 148, 408–447 (1997)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)
Seok Lee H., and Mu Lee, K.: Simultaneous super-resolution of depth and images using a single camera, In: The IEEE Conference on Computer Vision and Pattern Recognition, Sydney, (2013)
Shekhar, S., Patel, V.M., Nasrabadi, N.M., Chellappa, R., Joint sparsity-based robust multimodal biometrics recognition, In: Computer Vision-ECCV 2012 Workshops and Demonstrations. Springer, pp. 365–374 (2012)
Shen, Z.: Wavelet frames and image restorations, In: Proceedings of the International Congress of Mathematicians, Volume IV, pp. 2834–2863. Hindustan Book Agency, New Delhi (2010)
Yang, X., Kim, S., and Xing, E. P.: Heterogeneous multitask learning with joint sparsity constraints, In: Advances in neural information processing systems, pp. 2151–2159, Vancouver, B.C., (2009)
Zhou, W., Cai, J.-F., Gao, H.: Adaptive tight frame based medical image reconstruction: a proof-of-concept study for computed tomography. Inverse Prob. (2013). doi:10.1088/0266-5611/29/12/125006
Author information
Authors and Affiliations
Corresponding author
Additional information
Jian-Feng Cai is partially supported by the National Natural Science Foundation of USA (No. DMS 1418737).
Rights and permissions
About this article
Cite this article
Wang, J., Cai, JF. Data-Driven Tight Frame for Multi-channel Images and Its Application to Joint Color-Depth Image Reconstruction. J. Oper. Res. Soc. China 3, 99–115 (2015). https://doi.org/10.1007/s40305-015-0074-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40305-015-0074-2