Abstract
In recent years, deep neural networks have gained significant popularity in real-time semantic segmentation tasks, particularly in the domain of autonomous driving. However, these networks are susceptible to adversarial examples, which pose a serious threat to the safety of autonomous driving systems. Existing adversarial attacks on semantic segmentation models primarily focus on the digital space and lack validation in real-world scenarios, or they generate meaningless and visually unnatural examples. To address this gap, we propose a method called Adversarial Poster (ADV-POST), which generates physically plausible adversarial patches to preserve semantic information and visual naturalness by adding small-scale noise to posters. Specifically, we introduce a dynamic regularization method that balances the effectiveness and intensity of the generated patches. Moreover, we conduct comprehensive evaluations of the attack effectiveness in both digital and physical environments. Our experimental results demonstrate the successful misguidedness of real-time semantic segmentation models in the context of autonomous driving, resulting in inaccurate semantic segmentation results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arnab, A., Miksik, O., Torr, P.H.: On the robustness of semantic segmentation models to adversarial attacks. In: Proceedings of CVPR (2018)
Athalye, A., Engstrom, L., Ilyas, A., Kwok, K.: Synthesizing robust adversarial examples. In: Proceedings of ICML (2018)
Brown, T.B., Mané, D., Roy, A., Abadi, M., Gilmer, J.: Adversarial patch. arXiv preprint arXiv:1712.09665 (2017)
Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: Proceedings of SP (2017)
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. (2017)
Chen, Z., Wang, C., Crandall, D.: Semantically stealthy adversarial attacks against segmentation models. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (2022)
Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of CVPR (2016)
Eykholt, K., et al.: Robust physical-world attacks on deep learning visual classification. In: Proceedings of CVPR (2018)
Fischer, V., Kumar, M.C., Metzen, J.H., Brox, T.: Adversarial examples for semantic image segmentation. arXiv preprint arXiv:1703.01101 (2017)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014)
Hendrik Metzen, J., Chaithanya Kumar, M., Brox, T., Fischer, V.: Universal adversarial perturbations against semantic image segmentation. In: Proceedings of ICCV (2017)
Hong, Y., Pan, H., Sun, W., Jia, Y.: Deep dual-resolution networks for real-time and accurate semantic segmentation of road scenes. arXiv preprint arXiv:2101.06085 (2021)
Hu, Y.C.T., Kung, B.H., Tan, D.S., Chen, J.C., Hua, K.L., Cheng, W.H.: Naturalistic physical adversarial patch for object detectors. In: Proceedings of ICCV (2021)
Hu, Z., Huang, S., Zhu, X., Sun, F., Zhang, B., Hu, X.: Adversarial texture for fooling person detectors in the physical world. In: Proceedings of CVPR (2022)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Kong, Z., Guo, J., Li, A., Liu, C.: Physgan: generating physical-world-resilient adversarial examples for autonomous driving. In: Proceedings of CVPR (2020)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of CVPR (2015)
Moosavi-Dezfooli, S.M., Fawzi, A., Fawzi, O., Frossard, P.: Universal adversarial perturbations. In: Proceedings of CVPR (2017)
Moosavi-Dezfooli, S.M., Fawzi, A., Frossard, P.: Deepfool: a simple and accurate method to fool deep neural networks. In: Proceedings of CVPR (2016)
Nakka, K.K., Salzmann, M.: Indirect local attacks for context-aware semantic segmentation networks. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12350, pp. 611–628. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58558-7_36
Nesti, F., Rossolini, G., Nair, S., Biondi, A., Buttazzo, G.: Evaluating the robustness of semantic segmentation for autonomous driving against real-world adversarial patch attacks. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (2022)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Rony, J., Pesquet, J.C., Ben Ayed, I.: Proximal splitting adversarial attack for semantic segmentation. In: Proceedings of CVPR (2023)
Sharif, M., Bhagavatula, S., Bauer, L., Reiter, M.K.: Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (2016)
Shen, G., Mao, C., Yang, J., Ray, B.: Advspade: realistic unrestricted attacks for semantic segmentation. arXiv preprint arXiv:1910.02354 (2019)
Strong, D., Chan, T.: Edge-preserving and scale-dependent properties of total variation regularization. Inverse problems (2003)
Szegedy, C., et al.: Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013)
Xie, C., Wang, J., Zhang, Z., Zhou, Y., Xie, L., Yuille, A.: Adversarial examples for semantic segmentation and object detection. In: Proceedings of ICCV (2017)
Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: Bisenet: bilateral segmentation network for real-time semantic segmentation. In: Proceedings of ECCV (2018)
Zheng, S., et al.: Conditional random fields as recurrent neural networks. In: Proceedings of ICCV (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Deng, H. et al. (2024). ADV-POST: Physically Realistic Adversarial Poster for Attacking Semantic Segmentation Models in Autonomous Driving. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1967. Springer, Singapore. https://doi.org/10.1007/978-981-99-8178-6_27
Download citation
DOI: https://doi.org/10.1007/978-981-99-8178-6_27
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8177-9
Online ISBN: 978-981-99-8178-6
eBook Packages: Computer ScienceComputer Science (R0)