ABSTRACT
Event cameras offer the advantages of low latency, high temporal resolution and HDR compared to conventional cameras. Due to the asynchronous and sparse nature of events, many existing algorithms cannot be directly applied, necessitating the reconstruction of intensity frames. However, existing reconstruction methods often result in artifacts and edge blurring due to noise and event accumulation. In this paper, we argue that the key to event-based image reconstruction is to enhance the edge information of objects and restore the artifacts in the reconstructed images. To explain, edge information is one of the most important features in the event stream, providing information on the shape and contour of objects. Considering the extraordinary capabilities of Denoising Diffusion Probabilistic Models (DDPMs) in image generation, reconstruction, and restoration, we propose a new framework which incorporate it into the reconstruction pipeline to obtain high-quality results which effectively remove artifacts and blur in reconstructed images. Specifically, we first extract edge information from the event stream using the proposed event-based denoising method. It employs the contrast maximization framework to remove noise from the event stream and extract clear object edge information. And then, the edge information is further adopted to our diffusion model, which is used to enhance the edges of objects in the reconstructed images, thus improving the restoration effect. Experimental results show that our method achieves significant improvements in the mean squared error (MSE), the structural similarity (SSIM), and the perceptual similarity (LPIPS) metrics, with average improvements of 40%, 15%, and 25%, respectively, compared to previous state-of-the-art models, and has good generalization performance.
- Patrick Bardow, Andrew J. Davison, and Stefan Leutenegger. 2016. Simultaneous Optical Flow and Intensity Estimation from an Event Camera. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, 884--892. https://doi.org/10.1109/CVPR.2016.102Google ScholarCross Ref
- Anthony Bisulco, Fernando Cladera Ojeda, Volkan Isler, and Daniel D. Lee. 2021. Fast Motion Understanding with Spatiotemporal Neural Networks and Dynamic Vision Sensors. In IEEE International Conference on Robotics and Automation, ICRA 2021, Xi'an, China, May 30 - June 5, 2021. IEEE, 14098--14104. https://doi.org/10.1109/ICRA48506.2021.9561290Google ScholarDigital Library
- Christian Brandli, Raphael Berner, Minhao Yang, Shih-Chii Liu, and Tobi Delbruck. 2014. A 240× 180 130 db 3 μs latency global shutter spatiotemporal vision sensor. IEEE Journal of Solid-State Circuits, Vol. 49, 10 (2014), 2333--2341.Google ScholarCross Ref
- Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, and William Chan. 2021. WaveGrad: Estimating Gradients for Waveform Generation. (2021). https://openreview.net/forum?id=NsMLjcFaO8OGoogle Scholar
- Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye. 2022. Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. IEEE, 12403--12412. https://doi.org/10.1109/CVPR52688.2022.01209Google ScholarCross Ref
- Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. 2023. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).Google ScholarDigital Library
- Grady Daniels, Tyler Maunu, and Paul Hand. 2021. Score-based Generative Neural Networks for Large-Scale Optimal Transport. (2021), 12955--12965. https://proceedings.neurips.cc/paper/2021/hash/6c2e49911b68d315555d5b3eb0dd45bf-Abstract.htmlGoogle Scholar
- Prafulla Dhariwal and Alexander Quinn Nichol. 2021. Diffusion Models Beat GANs on Image Synthesis. (2021), 8780--8794. https://proceedings.neurips.cc/paper/2021/hash/49ad23d1ec9fa4bd8d77d02681df5cfa-Abstract.htmlGoogle Scholar
- Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, Jörg Conradt, Kostas Daniilidis, et al. 2022. Event-based vision: A survey. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 44, 1 (2022), 154--180. https://doi.org/10.1109/TPAMI.2020.3008413Google ScholarDigital Library
- Guillermo Gallego, Henri Rebecq, and Davide Scaramuzza. 2018. A Unifying Contrast Maximization Framework for Event Cameras, With Applications to Motion, Depth, and Optical Flow Estimation. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. Computer Vision Foundation / IEEE Computer Society, 3867--3876. https://doi.org/10.1109/CVPR.2018.00407Google ScholarCross Ref
- Daniel Gehrig, Henri Rebecq, Guillermo Gallego, and Davide Scaramuzza. 2018. Asynchronous, Photometric Feature Tracking Using Events and Frames. In Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XII (Lecture Notes in Computer Science, Vol. 11216), Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (Eds.). Springer, 766--781. https://doi.org/10.1007/978-3-030-01258-8_46Google ScholarDigital Library
- Rafael C Gonzales and Paul Wintz. 1987. Digital image processing. Addison-Wesley Longman Publishing Co., Inc.Google Scholar
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM, Vol. 63, 11 (2020), 139--144.Google ScholarDigital Library
- Matan Haroush, Itay Hubara, Elad Hoffer, and Daniel Soudry. 2020. The Knowledge Within: Methods for Data-Free Model Compression. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. Computer Vision Foundation / IEEE, 8491--8499. https://doi.org/10.1109/CVPR42600.2020.00852Google ScholarCross Ref
- Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han. 2018. AMC: AutoML for Model Compression and Acceleration on Mobile Devices. In Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part VII (Lecture Notes in Computer Science, Vol. 11211), Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (Eds.). Springer, 815--832. https://doi.org/10.1007/978-3-030-01234-2_48Google ScholarDigital Library
- Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. (2020). https://proceedings.neurips.cc/paper/2020/hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.htmlGoogle Scholar
- S. Mohammad Mostafavi I., Jonghyun Choi, and Kuk-Jin Yoon. 2020. Learning to Super Resolve Intensity Images From Events. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. Computer Vision Foundation / IEEE, 2765--2773. https://doi.org/10.1109/CVPR42600.2020.00284Google ScholarCross Ref
- Zhuangyi Jiang, Pengfei Xia, Kai Huang, Walter Stechele, Guang Chen, Zhenshan Bing, and Alois C. Knoll. 2019. Mixed Frame-/Event-Driven Fast Pedestrian Detection. In International Conference on Robotics and Automation, ICRA 2019, Montreal, QC, Canada, May 20-24, 2019. IEEE, 8332--8338. https://doi.org/10.1109/ICRA.2019.8793924Google ScholarDigital Library
- Alex Kendall and Roberto Cipolla. 2017. Geometric Loss Functions for Camera Pose Regression with Deep Learning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017. IEEE Computer Society, 6555--6564. https://doi.org/10.1109/CVPR.2017.694Google ScholarCross Ref
- Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. 2022. DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. IEEE, 2416--2425. https://doi.org/10.1109/CVPR52688.2022.00246Google ScholarCross Ref
- Hanme Kim, Ankur Handa, Ryad Benosman, Sio-Hoi Ieng, and Andrew J. Davison. 2014. Simultaneous Mosaicing and Tracking with an Event Camera. (2014). http://www.bmva.org/bmvc/2014/papers/paper066/index.htmlGoogle Scholar
- Hanme Kim, Stefan Leutenegger, and Andrew J. Davison. 2016. Real-Time 3D Reconstruction and 6-DoF Tracking with an Event Camera. In Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VI (Lecture Notes in Computer Science, Vol. 9910), Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer, 349--364. https://doi.org/10.1007/978-3-319-46466-4_21Google ScholarCross Ref
- Jianing Li, Siwei Dong, Zhaofei Yu, Yonghong Tian, and Tiejun Huang. 2019. Event-Based Vision Enhanced: A Joint Detection Framework in Autonomous Driving. In IEEE International Conference on Multimedia and Expo, ICME 2019, Shanghai, China, July 8-12, 2019. IEEE, 1396--1401. https://doi.org/10.1109/ICME.2019.00242Google ScholarCross Ref
- Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V (Lecture Notes in Computer Science, Vol. 8693), David J. Fleet, Tomá s Pajdla, Bernt Schiele, and Tinne Tuytelaars (Eds.). Springer, 740--755. https://doi.org/10.1007/978-3-319-10602-1_48Google ScholarCross Ref
- Andreas Lugmayr, Martin Danelljan, André s Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. 2022. RePaint: Inpainting using Denoising Diffusion Probabilistic Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. IEEE, 11451--11461. https://doi.org/10.1109/CVPR52688.2022.01117Google ScholarCross Ref
- Hengyuan Ma, Li Zhang, Xiatian Zhu, and Jianfeng Feng. 2022. Accelerating Score-Based Generative Models with Preconditioned Diffusion Sampling. In Computer Vision - ECCV 2022 - 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXIII (Lecture Notes in Computer Science, Vol. 13683), Shai Avidan, Gabriel J. Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (Eds.). Springer, 1--16. https://doi.org/10.1007/978-3-031-20050-2_1Google ScholarDigital Library
- Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. 2022. SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. https://openreview.net/forum?id=aBsCjcPu_tEGoogle Scholar
- Anton Mitrokhin, Cornelia Fermüller, Chethan Parameshwara, and Yiannis Aloimonos. 2018. Event-based moving object detection and tracking. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018, Madrid, Spain, October 1-5, 2018. IEEE, 1--9. https://doi.org/10.1109/IROS.2018.8593805Google ScholarDigital Library
- Elias Mueggler, Henri Rebecq, Guillermo Gallego, Tobi Delbrück, and Davide Scaramuzza. 2017. The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM. Int. J. Robotics Res., Vol. 36, 2 (2017), 142--149. https://doi.org/10.1177/0278364917691115Google ScholarDigital Library
- Gottfried Munda, Christian Reinbacher, and Thomas Pock. 2018. Real-Time Intensity-Image Reconstruction for Event Cameras Using Manifold Regularisation. Int. J. Comput. Vis., Vol. 126, 12 (2018), 1381--1393. https://doi.org/10.1007/s11263-018-1106-2Google ScholarDigital Library
- John A. Nelder and R. Mead. 1965. A Simplex Method for Function Minimization. Comput. J., Vol. 7, 4 (1965), 308--313. https://doi.org/10.1093/comjnl/7.4.308Google ScholarCross Ref
- Anh Nguyen, Thanh-Toan Do, Darwin G. Caldwell, and Nikos G. Tsagarakis. 2019. Real-Time 6DOF Pose Relocalization for Event Cameras With Stacked Spatial LS™ Networks. In IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE, 1638--1645. https://doi.org/10.1109/CVPRW.2019.00207Google ScholarCross Ref
- Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. 2022. GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models., Vol. 162 (2022), 16784--16804. https://proceedings.mlr.press/v162/nichol22a.htmlGoogle Scholar
- Henri Rebecq, Daniel Gehrig, and Davide Scaramuzza. 2018a. ESIM: an open event camera simulator. In Conference on robot learning. PMLR, 969--982.Google Scholar
- Henri Rebecq, Daniel Gehrig, and Davide Scaramuzza. 2018b. ESIM: an Open Event Camera Simulator. In 2nd Annual Conference on Robot Learning, CoRL 2018, Zürich, Switzerland, 29-31 October 2018, Proceedings (Proceedings of Machine Learning Research, Vol. 87). PMLR, 969--982. http://proceedings.mlr.press/v87/rebecq18a.htmlGoogle Scholar
- Henri Rebecq, Timo Horstschaefer, Guillermo Gallego, and Davide Scaramuzza. 2017. EVO: A Geometric Approach to Event-Based 6-DOF Parallel Tracking and Mapping in Real Time. IEEE Robotics Autom. Lett., Vol. 2, 2 (2017), 593--600. https://doi.org/10.1109/LRA.2016.2645143Google ScholarCross Ref
- Henri Rebecq, René Ranftl, Vladlen Koltun, and Davide Scaramuzza. 2021. High Speed and High Dynamic Range Video with an Event Camera. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 43, 6 (2021), 1964--1980. https://doi.org/10.1109/TPAMI.2019.2963386Google ScholarCross Ref
- Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis with Latent Diffusion Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022. IEEE, 10674--10685. https://doi.org/10.1109/CVPR52688.2022.01042Google ScholarCross Ref
- Chitwan Saharia, William Chan, Huiwen Chang, Chris A. Lee, Jonathan Ho, Tim Salimans, David J. Fleet, and Mohammad Norouzi. 2022. Palette: Image-to-Image Diffusion Models. In SIGGRAPH '22: Special Interest Group on Computer Graphics and Interactive Techniques Conference, Vancouver, BC, Canada, August 7 - 11, 2022, Munkhtsetseg Nandigjav, Niloy J. Mitra, and Aaron Hertzmann (Eds.). ACM, 15:1--15:10. https://doi.org/10.1145/3528233.3530757Google ScholarDigital Library
- Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, and Mohammad Norouzi. 2023. Image Super-Resolution via Iterative Refinement. IEEE Trans. Pattern Anal. Mach. Intell., Vol. 45, 4 (2023), 4713--4726. https://doi.org/10.1109/TPAMI.2022.3204461Google ScholarDigital Library
- Nikolaus Salvatore and Justin Fletcher. 2022. Learned Event-based Visual Perception for Improved Space Object Detection. In IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022, Waikoloa, HI, USA, January 3-8, 2022. IEEE, 3301--3310. https://doi.org/10.1109/WACV51458.2022.00336Google ScholarCross Ref
- Nitin J. Sanket, Chethan M. Parameshwara, Chahat Deep Singh, Ashwin V. Kuruttukulam, Cornelia Fermüller, Davide Scaramuzza, and Yiannis Aloimonos. 2020. EVDodgeNet: Deep Dynamic Obstacle Dodging with Event Cameras. In 2020 IEEE International Conference on Robotics and Automation, ICRA 2020, Paris, France, May 31 - August 31, 2020. IEEE, 10651--10657. https://doi.org/10.1109/ICRA40945.2020.9196877Google ScholarCross Ref
- Cedric Scheerlinck, Nick Barnes, and Robert E. Mahony. 2018. Continuous-Time Intensity Estimation Using Event Cameras. In Computer Vision - ACCV 2018 - 14th Asian Conference on Computer Vision, Perth, Australia, December 2-6, 2018, Revised Selected Papers, Part V (Lecture Notes in Computer Science, Vol. 11365), C. V. Jawahar, Hongdong Li, Greg Mori, and Konrad Schindler (Eds.). Springer, 308--324. https://doi.org/10.1007/978-3-030-20873-8_20Google ScholarCross Ref
- Cedric Scheerlinck, Henri Rebecq, Daniel Gehrig, Nick Barnes, Robert E. Mahony, and Davide Scaramuzza. 2020. Fast Image Reconstruction with an Event Camera. In IEEE Winter Conference on Applications of Computer Vision, WACV 2020, Snowmass Village, CO, USA, March 1-5, 2020. IEEE, 156--163. https://doi.org/10.1109/WACV45572.2020.9093366Google ScholarCross Ref
- Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. Deep Unsupervised Learning using Nonequilibrium Thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015 (JMLR Workshop and Conference Proceedings, Vol. 37), Francis R. Bach and David M. Blei (Eds.). JMLR.org, 2256--2265. http://proceedings.mlr.press/v37/sohl-dickstein15.htmlGoogle Scholar
- Jiaming Song, Chenlin Meng, and Stefano Ermon. 2021a. Denoising Diffusion Implicit Models. (2021). https://openreview.net/forum?id=St1giarCHLPGoogle Scholar
- Yang Song and Stefano Ermon. 2019. Generative Modeling by Estimating Gradients of the Data Distribution. (2019), 11895--11907. https://proceedings.neurips.cc/paper/2019/hash/3001ef257407d5a371a96dcd947c7d93-Abstract.htmlGoogle Scholar
- Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2021b. Score-Based Generative Modeling through Stochastic Differential Equations. (2021). https://openreview.net/forum?id=PxTIG12RRHSGoogle Scholar
- Timo Stoffregen, Cedric Scheerlinck, Davide Scaramuzza, Tom Drummond, Nick Barnes, Lindsay Kleeman, and Robert E. Mahony. 2020. Reducing the Sim-to-Real Gap for Event Cameras. In Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVII (Lecture Notes in Computer Science, Vol. 12372), Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer, 534--549. https://doi.org/10.1007/978-3-030-58583-9_32Google ScholarDigital Library
- Bishan Wang, Jingwei He, Lei Yu, Gui-Song Xia, and Wen Yang. 2020a. Event Enhanced High-Quality Image Recovery. In Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIII (Lecture Notes in Computer Science, Vol. 12358), Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer, 155--171. https://doi.org/10.1007/978-3-030-58601-0_10Google ScholarDigital Library
- Lin Wang, S. Mohammad Mostafavi I., Yo-Sung Ho, and Kuk-Jin Yoon. 2019. Event-Based High Dynamic Range Image and Very High Frame Rate Video Generation Using Conditional Generative Adversarial Networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE, 10081--10090. https://doi.org/10.1109/CVPR.2019.01032Google ScholarCross Ref
- Lin Wang, Tae-Kyun Kim, and Kuk-Jin Yoon. 2020b. EventSR: From Asynchronous Events to Image Reconstruction, Restoration, and Super-Resolution via End-to-End Adversarial Learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. Computer Vision Foundation / IEEE, 8312--8322. https://doi.org/10.1109/CVPR42600.2020.00834Google ScholarCross Ref
- Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., Vol. 13, 4 (2004), 600--612. https://doi.org/10.1109/TIP.2003.819861Google ScholarDigital Library
- Wenming Weng, Yueyi Zhang, and Zhiwei Xiong. 2021. Event-based Video Reconstruction Using Transformer. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021. IEEE, 2543--2552. https://doi.org/10.1109/ICCV48922.2021.00256Google ScholarCross Ref
- Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018. Computer Vision Foundation / IEEE Computer Society, 586--595. https://doi.org/10.1109/CVPR.2018.00068Google ScholarCross Ref
- Yanfu Zhang, Shangqian Gao, and Heng Huang. 2021. Exploration and Estimation for Model Compression. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021. IEEE, 477--486. https://doi.org/10.1109/ICCV48922.2021.00054Google ScholarCross Ref
- Alex Zihao Zhu, Dinesh Thakur, Tolga Özaslan, Bernd Pfrommer, Vijay Kumar, and Kostas Daniilidis. 2018. The Multivehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception. IEEE Robotics Autom. Lett., Vol. 3, 3 (2018), 2032--2039. https://doi.org/10.1109/LRA.2018.2800793Google ScholarCross Ref
- Alex Zihao Zhu, Liangzhe Yuan, Kenneth Chaney, and Kostas Daniilidis. 2019. Live Demonstration: Unsupervised Event-Based Learning of Optical Flow, Depth and Egomotion. In IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16--20, 2019. Computer Vision Foundation / IEEE, 1694. https://doi.org/10.1109/CVPRW.2019.00216Google ScholarCross Ref
- Yunhao Zou, Yinqiang Zheng, Tsuyoshi Takatani, and Ying Fu. 2021. Learning To Reconstruct High Speed and High Dynamic Range Videos From Events. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021. Computer Vision Foundation / IEEE, 2024--2033. https://doi.org/10.1109/CVPR46437.2021.00206Google ScholarCross Ref
Index Terms
- Event-Diffusion: Event-Based Image Reconstruction and Restoration with Diffusion Models
Recommendations
Fine-Grained Video Deblurring with Event Camera
MultiMedia ModelingAbstractDespite CNN-based deblurring models have shown their superiority on solving motion blurs, how to restore photorealistic images from severe motion blurs remains an ill-posed problem due to the loss of temporal information and textures. Video ...
Event-guided Video Clip Generation from Blurry Images
MM '22: Proceedings of the 30th ACM International Conference on MultimediaDynamic and active pixel vision sensors (DAVIS) can simultaneously produce streams of asynchronous events captured by the dynamic vision sensor (DVS) and intensity frames from the active pixel sensor (APS). Event sequences show high temporal resolution ...
Event-guided low light image enhancement via a dual branch GAN
AbstractIn the low light conditions, images are corrupted by low contrast and severe noise, but event cameras capture event streams with clear edge structures. Therefore, we propose an Event-Guided Low Light Image Enhancement method using a dual branch ...
Highlights- A dataset includes low-light images and corresponding events, and normal light images.
- A network consists of a Gradient Reconstruction Branch and a Low-Light Enhancement Branch.
- The Gradient Reconstruction Branch is used to ...
Comments