Enhancing Feature Representation for Anomaly Detection via Local-and-Global Temporal Relations and a Multi-stage Memory

Abstract

Weakly supervised video anomaly detection is a challenging task because frame-level labels are not accessible at the training time. Effectively tackling this task necessitates models to learn discriminative feature representation. To address this challenge, we propose a multi-stage memory-augmented feature discrimination learning (MMFDL) method. The first stage obtains the preliminary abnormal probabilities of clip features. In the second stage, an easy normal pattern memory (ENPM) are proposed to store normal patterns with low abnormal probabilities. In the last stage, we bring clip features with high abnormal probabilities in normal videos close to ENPM and away from the clip features with high probabilities of being abnormal in abnormal videos to make models learn more discriminative features for anomaly detection. Furthermore, we propose a local-and-global temporal relations modeling (LGTRM) module to enhance clip features by aggregating local and global contexts. Our LGTRM module can be divided into two subnetworks: DW-Net and TF-Net. DW-Net integrates the current clip feature with its adjacent clip features to capture local-range temporal dependencies. TF-Net utilizes the multi-head self-attention mechanism of the transformer to capture global-range temporal dependencies. Experiments on two datasets demonstrate that our method outperforms state-of-the-art approaches. The code is available at https://github.com/xuanli01/PRCV347.

This work was supported in part by the National Key Research and Development Program of China under Grant 2020AAA0106502, in part by the Natural Science Foundation of China under Grant 62073105, in part by the Natural Science Foundation of Heilongjiang Province of China under Grant ZD2022F002, and in part by the Heilongjiang Touyan Innovation Team Program.