Skip to main content
SpringerLink
Log in

Video restoration based on PatchMatch and reweighted low-rank matrix recovery

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, a new video restoration approach is proposed. By using a modified version of random PatchMatch algorithm, nearest-neighbor patches among the video frames can be grouped quickly and accurately. Then the video restoration problem can be boiled down to a low-rank matrix recovery problem, which is able to separate sparse errors from matrices that possess potential low-rank structures. Furthermore, the reweighted low-rank matrix model is used to improve the performance of video restoration by enhancing the sparsity of the sparse matrix and the low-rank property of the low-rank matrix. Experimental results show that our system achieves good performance in denosing of joint multi-frames and inpainting in the presence of small damaged areas.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Barnes C, Shechtman E, Finkelstein A, Goldman DB (2009) PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans Graphics-TOG 28:24

    Google Scholar 

  2. Bertalmio M, SaPiro G, Caselles V, Ballester C (2000) Image inpainting. Proceedings of ACM SIGGRAPH. ACM Press, New York, pp 417–424

    Google Scholar 

  3. Bertalmio M, Vese L, Sapiro G, Osher S (2003) Simultaneous structure and texture image inpainting. Image Proc, IEEE Trans 12:882–889

    Article  Google Scholar 

  4. Candès EJ, Li XD, Ma Y, Wright J (2011) Robust principal component analysis? J ACM 58:1–37

    Article  MathSciNet  MATH  Google Scholar 

  5. Candes EJ, Wakin MB, Boyd SP (2008) Enhancing sparsity by reweighted ℓ 1 minimization. J Fourier Anal Appl 14:877–905

    Article  MathSciNet  MATH  Google Scholar 

  6. Cen YG, Wang FF, Zhao RZ, Cui LH, Cen LH, Miao ZJ, Cen Y (2013) Tree-based backtracking orthogonal matching pursuit for sparse signal reconstruction. J Appl Math. doi:10.1155/2013/864132, Article ID 864132

    MathSciNet  Google Scholar 

  7. Cen YG, Zhao RZ, Miao ZJ, Cen LH, Cui LH (2014) A new approach of conditions on δ 2s (Φ) for s-sparse recovery. SCIENCE CHINA Inf Sci 57:1–7

    Article  Google Scholar 

  8. Chan TF, Shen J (2001) Nontexture inpainting by curvature-driven diffusions. J Vis Commun Image Represent 12:436–449

    Article  Google Scholar 

  9. Dabov K, Foi A, Egiazarian K (2007) Video denoising by sparse 3D transform-domain collaborative filtering. Proc. 15th European Signal Processing Conference 1: 7

  10. Efros AA, Leung TK (1999) Texture synthesis by non-parametric sampling. Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference on 2: 1033–1038

  11. Hou J, Bian Z P, Chau L P (2014) Restoring corrupted motion capture data via jointly low-rank matrix completion. Multimedia and Expo (ICME), IEEE International Conference on: 1–6

  12. Ji H, Huang SB, Shen ZW, Xu YH (2011) Robust video restoration by joint sparse and low rank matrix approximation. SIAM J Imaging Sci 4:1122–1142

    Article  MathSciNet  MATH  Google Scholar 

  13. Lin Z, Chen M, Ma Y (2010) The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055

  14. Liu B, Zaccarin A (1993) New fast algorithms for the estimation of block motion vectors. Circ Sys Video Tech, IEEE Trans 3:148–157

    Article  Google Scholar 

  15. Mahmoudi M, Sapiro G (2005) Fast image and video denoising via non-local means of similar neighborhoods. IEEE Signal Proc Lett 12:839–842

    Article  Google Scholar 

  16. Peng YG, Ganesh A, Wright J, Xu WL, Ma Y (2010) RASL: robust alignment by sparse and low-rank decomposition for linearly correlated images. Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). San Francisco, CA: 763–770

  17. Peng Y, Suo J, Dai Q, Xu WL, Lu S (2014) Robust image restoration via reweighted low-rank matrix recovery. MultiMedia Modeling 8325:315–326

    Article  Google Scholar 

  18. Toh KC, Yun S (2010) An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pacific J Optimization 6(615–640):15

    MathSciNet  MATH  Google Scholar 

  19. Wang HY, Zhao RZ, Cen YG (2014) Rank adaptive atomic decomposition for low-rank matrix completion and its application on image recovery. Neurocomputing 145:374–380. doi:10.1016/j.neucom.2014.05.021

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grant Nos. 61272028, 61473317, 61273274); Fundamental Research Funds for the Central Universities of China (Grant No. 2013JBZ003); Specialized Research Fund for the Doctoral Program of Higher Education of China (Grant No. 20120009110008); Program for New Century Excellent Talents in University (Grant No. NCET-12-0768); National Key Technology R&D Program of China (Grant No. 2012BAH01F03); National Basic Research (973) Program of China (Grant No. 2011CB302203); The National High Technology Research and Development Program (863) of China (Grant No. 2014AA015202); Program for Changjiang Scholars and Innovative Research Team in University (Grant no. IRT201206).

Author information

Authors and Affiliations

  1. School of Computer and Information Technology, Beijing Jiaotong University, Beijing, 100044, China

    Bo-Hua Xu, Yi-Gang Cen, Rui-Zhen Zhao & Zhen-Jiang Miao

  2. Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing, 100044, China

    Bo-Hua Xu, Yi-Gang Cen, Rui-Zhen Zhao & Zhen-Jiang Miao

  3. Huawei International Pte. Ltd, Singapore, 117674, Singapore

    Zhe Wei

  4. School of Information Engineering, Minzu University of China, Beijing, 100081, China

    Yi Cen

Authors
  1. Bo-Hua Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi-Gang Cen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhe Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Cen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rui-Zhen Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhen-Jiang Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi-Gang Cen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, BH., Cen, YG., Wei, Z. et al. Video restoration based on PatchMatch and reweighted low-rank matrix recovery. Multimed Tools Appl 75, 2681–2696 (2016). https://doi.org/10.1007/s11042-015-2545-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2545-1

Keywords

Search

Navigation