Abstract
Within the realm of deep learning, the interpretability of Convolutional Neural Networks (CNNs), particularly in the context of image classification tasks, remains a formidable challenge. To this end, we present a neurosymbolic framework, NeSyFOLD-G that generates a symbolic rule-set using the last layer kernels of the CNN to make its underlying knowledge interpretable. We find groups of similar kernels in the CNN (kernel-grouping) using the cosine-similarity score between the feature maps generated by various kernels. Once such kernel groups are found, we binarize each kernel group’s output and use it to generate a rule-set using a Rule Based Machine Learning (RBML) algorithm called FOLD-SE-M. We present a novel kernel grouping algorithm and show that grouping similar kernels leads to a significant reduction in the size of the rule-set generated by FOLD-SE-M, consequently, improving the interpretability. The rule-set can be viewed as a stratified Answer Set Program wherein each predicate’s truth value depends on a kernel group in the CNN. Each predicate in the rule-set is mapped to a semantic concept using a novel semantic labeling algorithm that utilizes a few semantic segmentation masks of the images used for training. The last layers of the CNN can then be replaced by this rule-set to obtain the NeSy-G model which can then be used for the image classification task. The goal-directed ASP system s(CASP) can be used to obtain the justification of any prediction made using the NeSy-G model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arias, J., Carro, M., Salazar, E., Marple, K., Gupta, G.: Constraint answer set programming without grounding. Theory Pract. Logic Program. 18(3–4), 337–354 (2018)
Bologna, G., Fossati, S.: A two-step rule-extraction technique for a CNN. Electronics 9(6), 990 (2020)
Chen, C., Li, O., Tao, D., Barnett, A., Rudin, C., Su, J.K.: This looks like that: deep learning for interpretable image recognition. In: Advances in Neural Information Processing Systems 32 (2019)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Denil, M., Demiraj, A., de Freitas, N.: Extraction of salient sentences from labelled documents (2014). https://doi.org/10.48550/ARXIV.1412.6815. https://arxiv.org/abs/1412.6815
Evans, R., Grefenstette, E.: Learning explanatory rules from noisy data. J. Artif. Intell. Res. 61, 1–64 (2018)
Ferreira, J., de Sousa Ribeiro, M., Gonçalves, R., Leite, J.: Looking inside the black-box: logic-based explanations for neural networks. In: Proceedings of the 19th International Conference on Principles of Knowledge Representation and Reasoning, pp. 432–442, August 2022. https://doi.org/10.24963/kr.2022/45
Gelfond, M., Kahl, Y.: Knowledge Representation, Reasoning, and the Design of Intelligent Agents: The Answer-Set Programming Approach. Cambridge University Press, Cambridge (2014)
Kanagaraj, N., Hicks, D., Goyal, A., Mishra Tiwari, S., Singh, G.: Deep learning using computer vision in self driving cars for lane and traffic sign detection. Int. J. Syst. Assur. Eng. Manag. 12, May 2021. https://doi.org/10.1007/s13198-021-01127-6
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014). https://doi.org/10.48550/ARXIV.1412.6980. https://arxiv.org/abs/1412.6980
Kirillov, A., et al.: Segment anything (2023)
Ko, B., Kwak, S.: Survey of computer vision-based natural disaster warning systems. Opt. Eng. 51, 0901 (2012). https://doi.org/10.1117/1.OE.51.7.070901
Lage, I., et al.: Human evaluation of models built for interpretability. In: Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, vol. 7, pp. 59–67 (2019)
Law, M., Russo, A., Broda, K.: The ILASP system for inductive learning of answer set programs. arXiv:2005.00904 (2020)
LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989). https://doi.org/10.1162/neco.1989.1.4.541
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Muggleton, S.: Inductive logic programming. New Gen. Comput. 8(4), 295–318 (1991)
Padalkar, P., Wang, H., Gupta, G.: NeSyFOLD: neurosymbolic framework for interpretable image classification (2023). https://arxiv.org/abs/2301.12667
Qi, Z., Khorram, S., Fuxin, L.: Embedding deep networks into visual explanations. Artif. Intell. 292, 103435 (2021). https://doi.org/10.1016/j.artint.2020.103435
Ray, O.: Nonmonotonic abductive inductive learning. J. Appl. Logic 7(3), 329–340 (2009). https://doi.org/10.1016/j.jal.2008.10.007. https://www.sciencedirect.com/science/article/pii/S1570868308000682. Special Issue: Abduction and Induction in Artificial Intelligence
Reiter, R.: A logic for default reasoning. Artif. Intell. 13(1), 81–132 (1980). https://doi.org/10.1016/0004-3702(80)90014-4. https://www.sciencedirect.com/science/article/pii/0004370280900144. Special Issue on Non-Monotonic Logic
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)
Sen, P., de Carvalho, B.W., Riegel, R., Gray, A.: Neuro-symbolic inductive logic programming with logical neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 8212–8219 (2022)
Shindo, H., Nishino, M., Yamamoto, A.: Differentiable inductive logic programming for structured examples. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 5034–5041 (2021)
Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps (2013). https://doi.org/10.48550/ARXIV.1312.6034. https://arxiv.org/abs/1312.6034
Stallkamp, J., Schlipsing, M., Salmen, J., Igel, C.: Man vs. computer: benchmarking machine learning algorithms for traffic sign recognition. Neural Netw. 32, 323–332 (2012). https://doi.org/10.1016/j.neunet.2012.02.016. https://www.sciencedirect.com/science/article/pii/S0893608012000457. Selected Papers from IJCNN 2011
Sun, W., Zheng, B., Qian, W.: Computer aided lung cancer diagnosis with deep learning algorithms. In: SPIE Medical Imaging (2016)
Townsend, J., Kasioumis, T., Inakoshi, H.: ERIC: extracting relations inferred from convolutions. In: Ishikawa, H., Liu, C.L., Pajdla, T., Shi, J. (eds.) ACCV 2020. LNIP, vol. 12624, pp. 206–222. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-69535-4_13
Townsend, J., Kudla, M., Raszkowska, A., Kasiousmis, T.: On the explainability of convolutional layers for multi-class problems. In: Combining Learning and Reasoning: Programming Languages, Formalisms, and Representations (2022). https://openreview.net/forum?id=jgVpiERy8Q8
Wang, H., Gupta, G.: FOLD-SE: an efficient rule-based machine learning algorithm with scalable explainability (2022). https://doi.org/10.48550/ARXIV.2208.07912
Wang, X., Zhang, X., Cao, Y., Wang, W., Shen, C., Huang, T.: SegGPT: towards segmenting everything in context. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1130–1140 (2023)
Xie, N., Sarker, M.K., Doran, D., Hitzler, P., Raymer, M.: Relating input concepts to convolutional neural network decisions (2017). https://doi.org/10.48550/ARXIV.1711.08006
Yang, Y., Kim, S., Joo, J.: Explaining deep convolutional neural networks via latent visual-semantic filter attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8333–8343 (2022)
Yang, Z., Ishay, A., Lee, J.: NeurASP: embracing neural networks into answer set programming. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020 (2021)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53
Zhang, Q., Cao, R., Shi, F., Wu, Y.N., Zhu, S.C.: Interpreting CNN knowledge via an explanatory graph. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Zhang, Q., Cao, R., Wu, Y.N., Zhu, S.C.: Growing interpretable part graphs on convnets via multi-shot learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929 (2016)
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2017)
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., Torralba, A.: Scene parsing through ADE20K dataset. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5122–5130 (2017). https://doi.org/10.1109/CVPR.2017.544
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Padalkar, P., Wang, H., Gupta, G. (2023). Using Logic Programming and Kernel-Grouping for Improving Interpretability of Convolutional Neural Networks. In: Gebser, M., Sergey, I. (eds) Practical Aspects of Declarative Languages. PADL 2024. Lecture Notes in Computer Science, vol 14512. Springer, Cham. https://doi.org/10.1007/978-3-031-52038-9_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-52038-9_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-52037-2
Online ISBN: 978-3-031-52038-9
eBook Packages: Computer ScienceComputer Science (R0)