Abstract
Domestic violence has increased globally as the COVID-19 pandemic combines with economic and social stresses. Some works have used traditional feature extractors to identify features from sound signals to detect physical violence. However, these extractors have not performed well at recognizing physical violence in audio. Besides, the use of Machine Learning is limited by the trade-off between collecting more data while keeping users privacy. Federated Learning (FL) is a technique that allows the creation of client-server networks, in which anonymized training result can be uploaded to a central model, responsible for aggregating and keeping the model up to date, and then distribute the updated model to the client nodes. In this paper, we proposed a FL approach to the violence detection problem in audio signals. The framework was evaluated on a newly proposed synthetic dataset, in which audio signals are represented as mel-spectrograms images, augmented with violence extracts. Thereby, it treats it as a problem of image classification using pre-trained Convolutional Neural Networks (CNN). Inception v3, MobileNet v2, ResNet152 v2 and VGG-16 architectures were evaluated, with the MobileNet architecture presenting the best performance, in terms of accuracy (71.9%), with a loss of 3.6% when compared to the non-FL setting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Beutel, D.J., Topal, T., Mathur, A., Qiu, X., Parcollet, T., Lane, N.D.: Flower: a friendly federated learning research framework. arXiv preprint arXiv:2007.14390 (2020)
Choi, K., Fazekas, G., Sandler, M.: Automatic tagging using deep convolutional neural networks (2016)
Durães, D., Marcondes, F.S., Gonçalves, F., Fonseca, J., Machado, J., Novais, P.: Detection violent behaviors: a survey. In: Novais, P., Vercelli, G., Larriba-Pey, J.L., Herrera, F., Chamoso, P. (eds.) ISAmI 2020. AISC, vol. 1239, pp. 106–116. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-58356-9_11
Friedman, M.: The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Am. Stat. Assoc. 32(200), 675–701 (1937). https://doi.org/10.1080/01621459.1937.10503522. https://www.tandfonline.com/doi/abs/10.1080/01621459.1937.10503522
Fukushima, K.: Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36(4), 193–202 (1980). https://doi.org/10.1007/BF00344251
Gu, B., Xu, A., Huo, Z., Deng, C., Huang, H.: Privacy-preserving asynchronous vertical federated learning algorithms for multiparty collaborative learning. IEEE Trans. Neural Netw. Learn. Syst. 33, 1–13 (2021). https://doi.org/10.1109/TNNLS.2021.3072238
Hard, A., et al.: Training keyword spotting models on non-iid data with federated learning (2020). https://arxiv.org/abs/2005.10406
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015). http://arxiv.org/abs/1512.03385
Hossain, M.S., Muhammad, G.: Emotion recognition using deep learning approach from audio–visual emotional big data. Inf. Fusion 49, 69–78 (2019). https://doi.org/10.1016/j.inffus.2018.09.008, https://www.sciencedirect.com/science/article/pii/S1566253517307066
Hu, R., Guo, Y., Gong, Y.: Concentrated differentially private federated learning with performance analysis. IEEE Open J. Comput. Soc. 2, 276–289 (2021). https://doi.org/10.1109/OJCS.2021.3099108
Volkmann, J., Stevens, S.S., Newman, E.B.: A scale for the measurement of the psychological magnitude pitch. J. Acoust. Soc. Am. 8, 208 (1937). https://doi.org/10.1121/1.1901999
Khan, A., Sohail, A., Zahoora, U., Qureshi, A.S.: A survey of the recent architectures of deep convolutional neural networks. Artif. Intell. Rev. 53(8), 5455–5516 (2020). https://doi.org/10.1007/s10462-020-09825-6. http://arxiv.org/abs/1901.06032
Kong, Q., Cao, Y., Iqbal, T., Wang, Y., Wang, W., Plumbley, M.D.: PANNs: large-scale pretrained audio neural networks for audio pattern recognition. IEEE/ACM Trans. Audio Speech Lang. Process. 28, 2880–2894 (2020). https://doi.org/10.1109/TASLP.2020.3030497. https://ieeexplore.ieee.org/document/9229505/
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017). https://doi.org/10.1145/3065386
Lacerda, T. B., Miranda, P., Camara, A., Furtado, A.P.C.: Deep learning and mel-spectrograms for physical violence detection in audio. In: The 18th National Meeting on Artificial and Computational Intelligence, pp. 268–279 (2021). https://sol.sbc.org.br/index.php/eniac/article/view/18259/18093
Lee, J., Park, J., Kim, K.L., Nam, J.: Sample-level deep convolutional neural networks for music auto-tagging using raw waveforms (2017)
Lu, Y., Huang, X., Dai, Y., Maharjan, S., Zhang, Y.: Differentially private asynchronous federated learning for mobile edge computing in urban informatics. IEEE Trans. Ind. Inf. 16(3), 2134–2143 (2020). https://doi.org/10.1109/TII.2019.2942179
McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.y.: Communication-efficient learning of deep networks from decentralized data. In: Singh, A., Zhu, J. (eds.) Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, vol. 54, pp. 1273–1282. PMLR (2017). https://proceedings.mlr.press/v54/mcmahan17a.html
Nations, U.: Policy brief: the impact of covid-19 on women (2020). https://www.un.org/sexualviolenceinconflict/wp-content/uploads/2020/06/report/policy-brief-the-impact-of-covid-19-on-women/policy-brief-the-impact-of-covid-19-on-women-en-1.pdf
Nayyar, R.K., Nair, S., Patil, O., Pawar, R., Lolage, A.: Content-based auto-tagging of audios using deep learning. In: 2017 International Conference on Big Data, IoT and Data Science (BID), pp. 30–36 (2017). https://doi.org/10.1109/BID.2017.8336569
Organization, W.H.: Violence against women (2021). https://www.who.int/news-room/fact-sheets/detail/violence-against-women
Organization, W.H.: Violence against women prevalence estimates, 2018: global, regional and national prevalence estimates for intimate partner violence against women and global and regional prevalence estimates for non-partner sexual violence against women (2021). https://www.who.int/publications/i/item/9789240022256
Paul, S., Sengupta, P., Mishra, S.: Flaps: Federated learning and privately scaling. In: 2020 IEEE 17th International Conference on Mobile Ad Hoc and Sensor Systems (MASS), pp. 13–19 (2020). https://doi.org/10.1109/MASS50613.2020.00011
Purwins, H., Li, B., Virtanen, T., Schluter, J., Chang, S.Y., Sainath, T.: Deep learning for audio signal processing. IEEE J. Sel. Topics Signal Process. 13(2), 206–219 (2019). https://doi.org/10.1109/jstsp.2019.2908700
Ramzan, M., et al.: A review on state-of-the-art violence detection techniques. IEEE Access 7, 107560–107575 (2019). https://doi.org/10.1109/ACCESS.2019.2932114
Rouas, J.L., Louradour, J., Ambellouis, S.: Audio events detection in public transport vehicle. In: 2006 IEEE Intelligent Transportation Systems Conference, pp. 733–738. IEEE (2006). https://doi.org/10.1109/ITSC.2006.1706829. http://ieeexplore.ieee.org/document/1706829/
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv2: inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4510–4520. IEEE (2018). https://doi.org/10.1109/CVPR.2018.00474. https://ieeexplore.ieee.org/document/8578572/
Santos, F.: In-car violence detection based on the audio signal. In: Yin, H., et al. (eds.) IDEAL 2021. LNCS, vol. 13113, pp. 437–445. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-91608-4_43
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015). https://arxiv.org/abs/1409.1556
Souto, H., Mello, R., Furtado, A.: An acoustic scene classification approach involving domestic violence using machine learning. In: Anais do ENIAC, pp. 705–716 (2019). https://doi.org/10.5753/eniac.2019.9327. https://sol.sbc.org.br/index.php/eniac/article/view/9327
Szegedy, C., et al.: Going deeper with convolutions (2014)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision (2015). http://arxiv.org/abs/1512.00567
Triastcyn, A., Faltings, B.: Federated learning with bayesian differential privacy. In: 2019 IEEE International Conference on Big Data (Big Data), pp. 2587–2596 (2019). https://doi.org/10.1109/BigData47090.2019.9005465
Tripathi, G., Singh, K.V.D.K.: Violence recognition using convolutional neural network: a survey. J. Intell. Fuzzy Syst. 39, 7931–7952 (2020). https://doi.org/10.3233/JIFS-201400. https://content.iospress.com/articles/journal-of-intelligent-and-fuzzy-systems/ifs201400
Wilcoxon, F.: Individual comparisons by ranking methods. Biometr. Bull. 1(6), 80–83 (1945). http://www.jstor.org/stable/3001968
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
de S. Silva, V.E., Lacerda, T.B., Miranda, P., Câmara, A., Chagas, A.R.C., Furtado, A.P.C. (2023). Federated Learning and Mel-Spectrograms for Physical Violence Detection in Audio. In: Naldi, M.C., Bianchi, R.A.C. (eds) Intelligent Systems. BRACIS 2023. Lecture Notes in Computer Science(), vol 14197. Springer, Cham. https://doi.org/10.1007/978-3-031-45392-2_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-45392-2_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45391-5
Online ISBN: 978-3-031-45392-2
eBook Packages: Computer ScienceComputer Science (R0)