Kshitij Marwah
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Light field photography has gained a significant research interest in the last two decades; today, commercial light field cameras are widely available. Nevertheless, most existing acquisition approaches either multiplex a low-resolution light field into a single 2D sensor image or require multiple photographs to be taken for acquiring a high-resolution light field. We propose a compressive light field camera architecture that allows for higher-resolution light fields to be recovered than previously possible from a single image. The proposed architecture comprises three key components: light field atoms as a sparse representation of natural light fields, an optical design that allows for capturing optimized 2D light field projections, and robust sparse reconstruction methods to recover a 4D light field from a single coded 2D projection. In addition, we demonstrate a variety of other applications for light field atoms and sparse coding, including 4D light field compression and denoising.
Supplemental Material
Available for Download
zip
a46-marwah.zip (77.5 MB)
Supplemental material.
- Adelson, E., and Wang, J. 1992. Single Lens Stereo with a Plenoptic Camera. IEEE Trans. PAMI 14, 2, 99--106. Google Scholar
Digital Library
- Ashok, A., and Neifeld, M. A. 2010. Compressive Light Field Imaging. In Proc. SPIE 7690, 76900Q.Google Scholar
- Babacan, S., Ansorge, R., Luessi, M., Mataran, P., Molina, R., and Katsaggelos, A. 2012. Compressive Light Field Sensing. IEEE Trans. Im. Proc. 21, 12, 4746--4757.Google ScholarDigital Library
- Bishop, T., Zanetti, S., and Favaro, P. 2009. Light-Field Superresolution. In Proc. ICCP, 1--9.Google Scholar
- Candès, E., and Wakin, M. B. 2008. An Introduction to Compressive Sampling. IEEE Signal Processing 25, 2, 21--30.Google ScholarCross Ref
- Candès, E. J., Eldar, Y. C., Needell, D., and Randall, P. 2011. Compressed Sensing with Coherent and Redundant Dictionaries. Appl. and Comp. Harmonic Analysis 31, 1, 59--73.Google ScholarCross Ref
- Chen, S. S., Donoho, D. L., Michael, and Saunders, A. 1998. Atomic Decomposition by Basis Pursuit. SIAM J. on Scientific Computing 20, 33--61. Google ScholarDigital Library
- Donoho, D. 2006. Compressed Sensing. IEEE Trans. Inform. Theory 52, 4, 1289--1306. Google ScholarDigital Library
- Duarte-Carvajalino, J., and Sapiro, G. 2009. Learning to sense sparse signals: Simultaneous Sensing Matrix and Sparsifying Dictionary Optimization. IEEE Trans. Im. Proc. 18, 7, 1395--1408. Google ScholarDigital Library
- Elad, M., and Aharon, M. 2006. Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries. IEEE Trans. Im. Proc. 15, 12, 3736--3745. Google ScholarDigital Library
- Feigin, M., Feldman, D., and Sochen, N. A. 2012. From High Definition Image to Low Space Optimization. In Scale Space and Var. Methods in Comp. Vision, vol. 6667, 459--470. Google ScholarDigital Library
- Fergus, R., Torralba, A., and Freeman, W. T. 2006. Random Lens Imaging. Tech. Rep. TR-2006-058, MIT.Google Scholar
- Georgiev, T., and Lumsdaine, A. 2006. Spatio-angular Resolution Tradeoffs in Integral Photography. Proc. EGSR, 263--272. Google ScholarDigital Library
- Gortler, S., Grzeszczuk, R., Szelinski, R., and Cohen, M. 1996. The Lumigraph. In Proc. ACM SIGGRAPH, 43--54. Google ScholarDigital Library
- Heide, F., Wetzstein, G., Raskar, R., and Heidrich, W. 2013. Adaptive Image Synthesis for Compressive Displays. ACM Trans. Graph. (SIGGRAPH) 32, 4, 1--11. Google ScholarDigital Library
- Hitomi, Y., Gu, J., Gupta, M., Mitsunaga, T., and Nayar, S. K. 2011. Video from a Single Coded Exposure Photograph using a Learned Over-Complete Dictionary. In Proc. IEEE ICCV. Google ScholarDigital Library
- Ives, H., 1903. Parallax Stereogram and Process of Making Same. US patent 725,567.Google Scholar
- Kamal, M., Golbabaee, M., and Vandergheynst, P. 2012. Light Field Compressive Sensing in Camera Arrays. In Proc. ICASSP, 5413--5416.Google Scholar
- Lanman, D., Raskar, R., Agrawal, A., and Taubin, G. 2008. Shield Fields: Modeling and Capturing 3D Occluders. ACM Trans. Graph. (SIGGRAPH Asia) 27, 5, 131. Google ScholarDigital Library
- Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., and Raskar, R. 2011. Polarization Fields: Dynamic Light Field Display using Multi-Layer LCDs. ACM Trans. Graph. (SIGGRAPH Asia) 30, 1--9. Google ScholarDigital Library
- Levin, A., Freeman, W. T., and Durand, F. 2008. Understanding Camera Trade-Offs through a Bayesian Analysis of Light Field Projections. In Proc. ECCV, 88--101. Google ScholarDigital Library
- Levin, A., Hasinoff, S. W., Green, P., Durand, F., and Freeman, W. T. 2009. 4D Frequency Analysis of Computational Cameras for Depth of Field Extension. ACM Trans. Graph. (SIGGRAPH) 28, 3, 97. Google ScholarDigital Library
- Levoy, M., and Hanrahan, P. 1996. Light Field Rendering. In Proc. ACM SIGGRAPH, 31--42. Google ScholarDigital Library
- Liang, C.-K., Lin, T.-H., Wong, B.-Y., Liu, C., and Chen, H. H. 2008. Programmable Aperture Photography: Multiplexed Light Field Acquisition. ACM Trans. Graph. (SIGGRAPH) 27, 3, 1--10. Google ScholarDigital Library
- Lippmann, G. 1908. La Photographie Intégrale. Academie des Sciences 146, 446--451.Google Scholar
- Lumsdaine, A., and Georgiev, T. 2009. The Focused Plenoptic Camera. In Proc. ICCP, 1--8.Google Scholar
- Mairal, J., Bach, F., Ponce, G., and Sapiro, G. 2009. Online Dictionary Learning For Sparse Coding. In International Conference on Machine Learning. Google ScholarDigital Library
- Marcia, R. F., and Willett, R. M. 2008. Compressive coded aperture video reconstruction. In EUSIPCO.Google Scholar
- Natarajan, B. K. 1995. Sparse Approximate Solutions to Linear Systems. SIAM J. Computing 24, 227--234. Google ScholarDigital Library
- Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., and Hanrahan, P. 2005. Light Field Photography with a Hand-Held Plenoptic Camera. Tech. rep., Stanford University.Google Scholar
- Ng, R. 2005. Fourier Slice Photography. ACM Trans. Graph. (SIGGRAPH) 24, 3, 735--744. Google ScholarDigital Library
- Park, J. Y., and Wakin, M. B. 2012. A geometric approach to multi-view compressive imaging. EURASIP Journal on Advances in Signal Processing 37.Google Scholar
- Peers, P., Mahajan, D. K., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., and Debevec, P. 2009. Compressive Light Transport Sensing. ACM Trans. Graph. 28, 3. Google ScholarDigital Library
- Perwass, C., and Wietzke, L. 2012. Single Lens 3D-Camera with Extended Depth-of-Field. In Proc. SPIE 8291, 29--36.Google Scholar
- Reddy, D., Veeraraghavan, A., and Chellappa, R. 2011. P2C2: Programmable Pixel Compressive Camera for High Speed Imaging. In Proc. IEEE CVPR, 329--336. Google ScholarDigital Library
- Sen, P., and Darabi, S. 2009. Compressive Dual Photography. Computer Graphics Forum 28, 609--618.Google ScholarCross Ref
- Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., and Tumblin, J. 2007. Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocussing. ACM Trans. Graph. (SIGGRAPH) 26, 3, 69. Google ScholarDigital Library
- Wakin, M. B., Laska, J. N., Duarte, M. F., Baron, D., Sarvotham, S., Takhar, D., Kelly, K. F., and Baraniuk, R. G. 2006. Compressive imaging for video representation and coding. In Picture Coding Symposium.Google Scholar
- Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic Image Synthesis for Attenuation-based Light Field and High Dynamic Range Displays. ACM Trans. Graph. (SIGGRAPH). Google ScholarDigital Library
- Wetzstein, G., Ihrke, I., and Heidrich, W. 2012. On Plenoptic Multiplexing and Reconstruction. IJCV, 1--16. Google ScholarDigital Library
- Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting. ACM Trans. Graph. (SIGGRAPH) 31, 1--11. Google ScholarDigital Library
- Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Horowitz, M., and Levoy, M. 2005. High Performance Imaging using Large Camera Arrays. ACM Trans. Graph. (SIGGRAPH) 24, 3, 765--776. Google ScholarDigital Library
- Xu, Z., and Lam, E. Y. 2012. A High-resolution Lightfield Camera with Dual-mask Design. In Proc. SPIE 8500, 85000U.Google Scholar
- Yang, A., Ganesh, A., Sastry, S., and Ma, Y. 2010. Fast L1-Minimization Algorithms and An Application in Robust Face Recognition: A Review. Tech. rep., UC Berkeley.Google Scholar
- Yang, J., Wang, Z., Lin, Z., Cohen, S., and Huang, T. 2012. Coupled Dictionary Training for Image Super-Resolution. IEEE Trans. Im. Proc. 21, 8, 3467--3478.Google ScholarDigital Library
Index Terms
- Compressive light field photography using overcomplete dictionaries and optimized projections
Recommendations
Image compressive sensing via Truncated Schatten-p Norm regularization
Low-rank property as a useful image prior has attracted much attention in image processing communities. Recently, a nonlocal low-rank regularization (NLR) approach toward exploiting low-rank property has shown the state-of-the-art performance in ...
Compressive light field photography
SIGGRAPH '12: ACM SIGGRAPH 2012 TalksLight field cameras, e.g. [Lytro 2012; Veeraraghavan et al. 2007], have ushered a new direction in photography allowing consumers to synthesize photographs with novel viewpoints or varying focus after the actual recording. Unfortunately, current light ...
Compressive sensing via nonlocal low-rank tensor regularization
The aim of Compressing sensing (CS) is to acquire an original signal, when it is sampled at a lower rate than Nyquist rate previously. In the framework of CS, the original signal is often assumed to be sparse and correlated in some domain. Recently, ...
Comments