Skip to main content
SpringerLink
Log in

Light Field Angular Super-Resolution via Dense Correspondence Field Reconstruction

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 Workshops (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13802))

Included in the following conference series:

  • 1704 Accesses

Abstract

Light field (LF) angular super-resolution (SR) aims at reconstructing a densely sampled LF from a sparsely sampled one. To achieve accurate angular SR, it is important but challenging to incorporate the complementary information among input views, especially when dealing with large disparities. In this paper, we propose to reconstruct dense correspondence field among different views for LF angular SR. According to the LF geometry structure, we first capture correspondences along the horizontal and vertical axes of input views with a global receptive field. We then incorporate the linear structure prior among angular viewpoints to reconstruct a dense correspondence field. With the reconstructed dense correspondence field, the relationship between each target view and the input views is constructed. Next, we develop a view projection approach to project input views to the target positions. Moreover, a projection loss is introduced to preserve the LF parallax structure. Extensive experiments demonstrate that our proposed network can recover accurate details and preserve LF parallax structure. Comparative results show the advantage of our method over state-of-the-art methods on synthetic and real-world datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Shin, C., Jeon, H.G., Yoon, Y., So Kweon, I., Joo Kim, S.: EPINET: a fully-convolutional neural network using epipolar geometry for depth from light field images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4748–4757 (2018)

    Google Scholar 

  2. Wang, Y., Wang, L., Liang, Z., Yang, J., An, W., Guo, Y.: Occlusion-aware cost constructor for light field depth estimation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 19809–19818 (CVPR) (2022)

    Google Scholar 

  3. Chao, W., Wang, X., Wang, Y., Chang, L., Duan, F.: Learning sub-pixel disparity distribution for light field depth estimation. arXiv preprint (2022)

    Google Scholar 

  4. Wang, Y., Yang, J., Guo, Y., Xiao, C., An, W.: Selective light field refocusing for camera arrays using bokeh rendering and superresolution. IEEE Signal Process. Lett. 26(1), 204–208 (2018)

    Article  Google Scholar 

  5. Viganò, N., Gil, P.M., Herzog, C., de la Rochefoucauld, O., van Liere, R., Batenburg, K.J.: Advanced light-field refocusing through tomographic modeling of the photographed scene. Opt. Express 27(6), 7834–7856 (2019)

    Article  Google Scholar 

  6. Jayaweera, S.S., Edussooriya, C.U., Wijenayake, C., Agathoklis, P., Bruton, L.T.: Multi-volumetric refocusing of light fields. IEEE Signal Process. Lett. 28, 31–35 (2020)

    Article  Google Scholar 

  7. Wang, Y., Wang, L., Yang, J., An, W., Yu, J., Guo, Y.: Spatial-angular interaction for light field image super-resolution. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12368, pp. 290–308. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58592-1_18

    Chapter  Google Scholar 

  8. Cheng, Z., Xiong, Z., Chen, C., Liu, D., Zha, Z.J.: Light field super-resolution with zero-shot learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 10010–10019 (2021)

    Google Scholar 

  9. Wang, Y., Liang, Z., Wang, L., Yang, J., An, W., Guo, Y.: Learning a degradation-adaptive network for light field image super-resolution. arXiv preprint arXiv:2206.06214 (2022)

  10. Zhu, H., Guo, M., Li, H., Wang, Q., Robles-Kelly, A.: Revisiting spatio-angular trade-off in light field cameras and extended applications in super-resolution. IEEE Trans. Visual Comput. Graphics 27(6), 3019–3033 (2019)

    Article  Google Scholar 

  11. Shi, L., Hassanieh, H., Davis, A., Katabi, D., Durand, F.: Light field reconstruction using sparsity in the continuous Fourier domain. ACM Trans. Graph. 34(1), 1–13 (2014)

    Article  Google Scholar 

  12. Vagharshakyan, S., Bregovic, R., Gotchev, A.: Light field reconstruction using shearlet transform. IEEE Trans. Pattern Anal. Mach. Intell. 40(1), 133–147 (2017)

    Article  Google Scholar 

  13. Wu, G., et al.: Light field image processing: an overview. IEEE J. Sel. Top. Signal Process. 11(7), 926–954 (2017)

    Article  Google Scholar 

  14. Wang, Y., Liu, F., Wang, Z., Hou, G., Sun, Z., Tan, T.: End-to-end view synthesis for light field imaging with pseudo 4DCNN. In: European Conference on Computer Vision, pp. 333–348 (2018)

    Google Scholar 

  15. Wu, G., Wang, Y., Liu, Y., Fang, L., Chai, T.: Spatial-angular attention network for light field reconstruction. IEEE Trans. Image Process. 30, 8999–9013 (2021)

    Article  Google Scholar 

  16. Meng, N., So, H.K.H., Sun, X., Lam, E.: High-dimensional dense residual convolutional neural network for light field reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. 43, 873–886 (2019)

    Article  Google Scholar 

  17. Meng, N., Li, K., Liu, J., Lam, E.Y.: Light field view synthesis via aperture disparity and warping confidence map. IEEE Trans. Image Process. 30, 3908–3921 (2021)

    Article  Google Scholar 

  18. Kalantari, N.K., Wang, T.C., Ramamoorthi, R.: Learning-based view synthesis for light field cameras. ACM Trans. Graph. 35(6), 1–10 (2016)

    Article  Google Scholar 

  19. Jin, J., Hou, J., Yuan, H., Kwong, S.: Learning light field angular super-resolution via a geometry-aware network. In: AAAI Conference on Artificial Intelligence (2020)

    Google Scholar 

  20. Shi, J., Jiang, X., Guillemot, C.: Learning fused pixel and feature-based view reconstructions for light fields. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2555–2564 (2020)

    Google Scholar 

  21. Wu, G., Liu, Y., Fang, L., Chai, T.: Revisiting light field rendering with deep anti-aliasing neural network. IEEE Trans. Pattern Anal. Mach. Intell. 44, 5430–5444 (2021)

    Google Scholar 

  22. Wang, Y., Wang, L., Wu, G., Yang, J., An, W., Yu, J., Guo, Y.: Disentangling light fields for super-resolution and disparity estimation. IEEE Trans. Pattern Anal. Mach. Intell. 45, 425–443 (2022)

    Article  Google Scholar 

  23. Long, J., Ning, Z., Darrell, T.: Do convnets learn correspondence? Adv. Neural Inf. Process. Syst. 27, 1601–1609 (2014)

    Google Scholar 

  24. Wang, Y., Liu, F., Zhang, K., Hou, G., Sun, Z., Tan, T.: LFNet: a novel bidirectional recurrent convolutional neural network for light-field image super-resolution. IEEE Trans. Image Process. 27(9), 4274–4286 (2018)

    Article  MathSciNet  Google Scholar 

  25. Yoon, Y., Jeon, H.G., Yoo, D., Lee, J.Y., Kweon, I.S.: Light-field image super-resolution using convolutional neural network. IEEE Signal Process. Lett. 24(6), 848–852 (2017)

    Article  Google Scholar 

  26. Zhang, S., Lin, Y., Sheng, H.: Residual networks for light field image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 11046–11055 (2019)

    Google Scholar 

  27. Meng, N., Wu, X., Liu, J., Lam, E.Y.: High-order residual network for light field super-resolution. In: AAAI Conference on Artificial Intelligence (2020)

    Google Scholar 

  28. Jin, J., Hou, J., Chen, J., Kwong, S.: Light field spatial super-resolution via deep combinatorial geometry embedding and structural consistency regularization. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2260–2269 (2020)

    Google Scholar 

  29. Yoon, Y., Jeon, H.G., Yoo, D., Lee, J.Y., So Kweon, I.: Learning a deep convolutional network for light-field image super-resolution. In: IEEE International Conference on Computer Vision Workshops, pp. 24–32 (2015)

    Google Scholar 

  30. Wing Fung Yeung, H., Hou, J., Chen, J., Ying Chung, Y., Chen, X.: Fast light field reconstruction with deep coarse-to-fine modeling of spatial-angular clues. In: European Conference on Computer Vision, pp. 137–152 (2018)

    Google Scholar 

  31. Wu, G., Zhao, M., Wang, L., Dai, Q., Chai, T., Liu, Y.: Light field reconstruction using deep convolutional network on EPI. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 6319–6327 (2017)

    Google Scholar 

  32. Liu, D., Huang, Y., Wu, Q., Ma, R., An, P.: Multi-angular epipolar geometry based light field angular reconstruction network. IEEE Trans. Comput. Imaging 6, 1507–1522 (2020)

    Article  MathSciNet  Google Scholar 

  33. Wang, Y., Liu, F., Zhang, K., Wang, Z., Sun, Z., Tan, T.: High-fidelity view synthesis for light field imaging with extended pseudo 4DCNN. IEEE Trans. Comput. Imaging 6, 830–842 (2020)

    Article  Google Scholar 

  34. Wanner, S., Goldluecke, B.: Variational light field analysis for disparity estimation and super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 606–619 (2014)

    Article  Google Scholar 

  35. Wu, G., Liu, Y., Dai, Q., Chai, T.: Learning sheared epi structure for light field reconstruction. IEEE Trans. Image Process. 28(7), 3261–3273 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  36. Mildenhall, B., et al.: Local light field fusion: practical view synthesis with prescriptive sampling guidelines. ACM Trans. Graph. 38(4), 1–14 (2019)

    Article  Google Scholar 

  37. Jin, J., Hou, J., Chen, J., Zeng, H., Kwong, S., Yu, J.: Deep coarse-to-fine dense light field reconstruction with flexible sampling and geometry-aware fusion. IEEE Trans. Pattern Anal. Mach. Intell. 44, 1819–1836 (2020)

    Article  Google Scholar 

  38. Levoy, M., Hanrahan, P.: Light field rendering. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 31–42 (1996)

    Google Scholar 

  39. Wang, L., et al.: Learning parallax attention for stereo image super-resolution. In: IEEE Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  40. Honauer, K., Johannsen, O., Kondermann, D., Goldluecke, B.: A dataset and evaluation methodology for depth estimation on 4D light fields. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10113, pp. 19–34. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54187-7_2

    Chapter  Google Scholar 

  41. Wanner, S., Meister, S., Goldluecke, B.: Datasets and benchmarks for densely sampled 4D light fields. In: Vision, Modelling and Visualization, pp. 225–226. Citeseer (2013)

    Google Scholar 

  42. Raj, A.S., Lowney, M., Shah, R., Wetzstein, G.: Stanford lytro light field archive (2016)

    Google Scholar 

  43. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning and Representation (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National University of Defense Technology, Changsha, China

    Yu Mo, Yingqian Wang, Longguang Wang, Jungang Yang & Wei An

Authors
  1. Yu Mo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yingqian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Longguang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jungang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei An
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingqian Wang .

Editor information

Editors and Affiliations

  1. IBM Research - MIT-IBM Watson AI Lab, Massachusetts, USA

    Leonid Karlinsky

  2. Technion – Israel Institute of Technology, Haifa, Israel

    Tomer Michaeli

  3. Kyoto University, Kyoto, Japan

    Ko Nishino

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mo, Y., Wang, Y., Wang, L., Yang, J., An, W. (2023). Light Field Angular Super-Resolution via Dense Correspondence Field Reconstruction. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13802. Springer, Cham. https://doi.org/10.1007/978-3-031-25063-7_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25063-7_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25062-0

  • Online ISBN: 978-3-031-25063-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation