17 January 2024 Event camera object recognition using spatiotemporal event time surface and reward-modulated spike-timing-dependent plasticity learning rule
Qian Zhou, Guimeng Zhang
Author Affiliations +
Abstract

Event cameras record moving objects with asynchronous event streams. It remains a challenge to make full use of the spatiotemporal information of event streams to extract high-quality features and to make event camera object recognition. We propose an event-based event camera object recognition system, which includes a denoising module and an object recognition module. The denoising module removes noise events using spatiotemporal information of the events. The object recognition module extracts primary spatiotemporal features based on the event time surface prototypes obtained by clustering, then further extracts complex diagnostic features, and makes object recognition using spiking neural networks with reward-modulated spike-timing-dependent plasticity learning rule. Experimental results demonstrate that our system has better performance than baseline methods on five popular event camera datasets, especially on datasets with rich spatiotemporal dynamics. Our denoising method improves the noise robustness of our event camera object recognition system on high-speed moving datasets that are greatly affected by noises. Moreover, our method has much better recognition ability than baseline methods when using short input event streams. Our method is very beneficial for developing event-based event camera object recognition algorithm when event streams are short or noises are serious.

© 2024 SPIE and IS&T
Qian Zhou and Guimeng Zhang "Event camera object recognition using spatiotemporal event time surface and reward-modulated spike-timing-dependent plasticity learning rule," Journal of Electronic Imaging 33(1), 013017 (17 January 2024). https://doi.org/10.1117/1.JEI.33.1.013017
Received: 26 June 2023; Accepted: 29 December 2023; Published: 17 January 2024
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Prototyping

Feature extraction

Cameras

Denoising

Object recognition

Neurons

Tunable filters

Back to Top