Anomaly detection is a key issue in public security. Its accuracy is essential to identify abnormalities and take corresponding actions to ensure the safety of relevant objects, which have a broad application space. The traditional anomaly detection method based on deep learning has too strong generalization ability. At the same time, it lacks recognition ability because it only uses normal data for training. To this end, we propose an anomaly detection model based on few-shot learning, guided by memory modules and trained by a large number of normal samples combined with a small number of observed abnormalities. We introduce memory modules to record normal features, which has the function of updating and reading. When training modules, feature compactness, and separateness loss are utilized, it successfully weakens the strong generalization of CNN and improves the memory module to identify normal samples. Then, based on the few-shot learning approach, we learn a more compact normal data distribution and expand the margin between normal and anomalous events to improve the discriminant ability. Many experimental results demonstrate that our method is practical and feasible, and its performance is better than the existing detection methods.