Skip to main content
SpringerLink
Log in

Reidentifying Asian Elephants from Ear Images Using a Cascade of Convolutional Neural Networks and Explaining with GradCAM

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Zoologists visually recognise each Asian elephant (Elephas maximus), mainly based on their ear patterns. Towards automating this process, the existing methods on African elephants are less instrumental for Asian elephants, because the nick patterns are rare. This paper presents a cascade of Convolutional Neural Networks for uniquely detecting Asian elephants with two steps: (1) an elephant-ear localisation step at a species level, and (2) an ear-patch classification step at an individual level. First, a YOLO CNN with pre-trained weights on ImageNet is retrained with manually cropped elephant ears to localise them in the colour image. Second, these cropped ear patches are learnt by a CNN to classify each elephant by the Zoologist’s labelling; Xception outperformed VGG16, ResNet50, InceptionV3, and AlexNet in this second step on 56 elephants. Xception produced a top-1 accuracy of 88% and top-5 accuracy of 99.27% for reidentification as the best performance. Discriminative regions of elephant ears were visually explained by GradCAM on Xception reidentification classifier.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Adopted from https://www.programmersought.com/article/38933052023/

Fig. 3
Fig. 4

Adopted from https://towardsdatascience.com/understanding-and-coding-a-resnet-in-keras-446d7ff84d33

Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. bbc. Sri Lanka elephants: ’record number’ of deaths in 2019. 2020. URL https://www.bbc.com/news/world-asia-51076898. Accessed 5 Oct 2021

  2. Adadi A, Berrada M. Peeking inside the black-box: a survey on explainable artificial intelligence (xai). IEEE access. 2018;6:52138–60.

    Article  Google Scholar 

  3. Andrew W, Greatwood C, Burghardt T. Visual localisation and individual identification of holstein friesian cattle via deep learning. In: Proceedings of the IEEE International Conference on Computer Vision Workshops. 2017; 2850–2859.

  4. Ardovini A, Cinque L, Sangineto E. Identifying elephant photos by multi-curve matching. Pattern Recogn. 2008;41:1867–77. https://doi.org/10.1016/j.patcog.2007.11.010.

    Article  MATH  Google Scholar 

  5. Arzoumanian Z, Holmberg J, Norman B. An astronomical pattern-matching algorithm for computer- aided identification of whale sharks rhincodon typus. J Appl Ecol. 2005;42(6):999–1011.

    Article  Google Scholar 

  6. Azhar MAHB, Hoque S, Deravi F. Automatic identification of wildlife using local binary patterns. In: IET Conference on Image Processing (IPR 2012), 2012; 1–6, DOI https://doi.org/10.1049/cp.2012.0454

  7. Bentley JL. Multidimensional binary search trees used for associative searching. Commun ACM. 1975;18(9):509–17.

    Article  Google Scholar 

  8. Beugeling T, Branzan-Albu A. Computer vision-based identification of individual turtles using characteristic patterns of their plastrons. In: 2014 Canadian Conference on Computer and Robot Vision, IEEE. 2014; pp 203–210.

  9. Bolei Z, Khosla A, Lapedriza A, Oliva A, Torralba A. Object detectors emerge in deep scenecnns. 2015.

  10. Bolger DT, Morrison TA, Vance B, Lee D, Farid H. A computer-assisted system for photographic mark—recapture analysis. Methods Ecol Evol. 2012;3(5):813–22.

    Article  Google Scholar 

  11. Brust CA, Burghardt T, Groenenberg M, Kading C, Kuhl HS, Manguette ML, Denzler J. Towards automated visual monitoring of individual gorillas in the wild. In: Proceedings of the IEEE International Conference on Computer Vision Workshops. 2017; 2820–2830.

  12. Burghardt T, Campbell N. Generic phase curl localisation for an individual identification of turing-patterned animals. Visual Observation and Analysis of Animal and Insect Behavior. 2010; 17–21. https://www.semanticscholar.org/paper/Generic-Phase-Curl-Localisation-for-an-Individual-Burghardt-Campbell/d74023affded114ffe86b6fe90e3f200ecec114e.

  13. Canny J. A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell PAMI. 1986;8(6):679–98. https://doi.org/10.1109/TPAMI.1986.4767851.

    Article  Google Scholar 

  14. Chang CC, Lin CJ. Libsvm: a library for support vector ma- chines. ACM Trans Intel Syst Technol (TIST). 2011;2(3):1–27.

    Article  Google Scholar 

  15. Chatfield K, Simonyan K, Vedaldi A, Zisserman A. Return of the devil in the details: Delving deep into convolutional nets. 2014, arXiv preprint arXiv:14053531.

  16. Chollet F. Xception: Deep learning with depthwise separable convolu- tions. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017; 00: 1800–1807, DOI: https://doi.org/10.1109/CVPR.2017.195

  17. Cortes C, Vapnik V. Support-vector networks. Mach Learn. 1995;20(3):273–97.

    MATH  Google Scholar 

  18. Dosovitskiy A, Brox T. Inverting visual representations with convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016; 4829–4837.

  19. Freytag A, Rodner E, Simon M, Loos A, Ku¨hl HS, Denzler. Chimpanzee faces in the wild: Log- euclidean cnns for predicting identities and attributes of primates. In: German Conference on Pattern Recognition, Springer. 2016; 51–63

  20. Groth EJ. A pattern-matching algorithm for two-dimensional coordinate lists. Astron J. 1986;91:1244–8.

    Article  Google Scholar 

  21. Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, Pedreschi D. A survey of methods for explaining black box models. ACM Comput Surv (CSUR). 2018;51(5):1–42.

    Article  Google Scholar 

  22. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27–30, 2016. 2016; 770–778.

  23. Hiby L, Lovell P. Computer aided matching of natural markings: a prototype system for grey seals. Rep Int Whal Comm. 1990;12:57–61.

    Google Scholar 

  24. Hiby L, Lovell P, Patil N, Kumar NS, Gopalaswamy AM, Karanth KU. A tiger cannot change its stripes: using a three-dimensional model to match images of living tigers and tiger skins. Biol Lett. 2009;5(3):383–6.

    Article  Google Scholar 

  25. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ICML'15. Lille: JMLR; 2015; p. 448–56. https://doi.org/10.5555/3045118.3045167.

  26. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T. Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on Multimedia. 2014; 675–678.

  27. Keras T. Keras documentation: Keras Applications. 2021. URL https://keras.io/api/applications/. Accessed 5 Oct 2021

  28. Konovalov DA, Hillcoat S, Williams G, Birtles RA, Gardiner N, Curnock MI. Individual minke whale recognition using deep learning convolutional neural networks. J Geosci Environ Prot. 2018;6:25–36.

    Google Scholar 

  29. Korschens M, Denzler J. Elpephants: a fine-grained dataset for elephant re-identification. In: 2019 IEEE/CVF International Conference on Computer Vision Workshops, ICCV Workshops 2019, Seoul, Korea (South), October 27–28, 2019, IEEE. 2019; 263–270. DOI https://doi.org/10.1109/ICCVW.2019.00035

  30. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ, editors. Advances in neural information processing systems 25. Curran Associates Inc; 2012. p. 1097–105.

    Google Scholar 

  31. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 2017;60(6):84–90. https://doi.org/10.1145/3065386.

    Article  Google Scholar 

  32. Kumar S, Singh SK. Visual animal biometrics: survey. IET Biom. 2017;6(3):139–56. https://doi.org/10.1049/iet-bmt.2016.0017.

    Article  Google Scholar 

  33. Lahiri M, Tantipathananandh C, Warungu R, Rubenstein DI, Berger-Wolf TY. Biometric animal databases from field photographs: identification of individual zebra in the wild. In: Proceedings of the 1st ACM international conference on multimedia retrieval. 2011; 1–8.

  34. Lin TY, Goyal P, Girshick R, He K, Doll´ar P. Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision. 2017; 2980–2988.

  35. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015; 3431–3440.

  36. Lowe G. Sift-the scale invariant feature transform. Int J. 2004;2(91–110):2.

    Google Scholar 

  37. Mahendran A, Vedaldi A. Understanding deep image representations by inverting them. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015; 5188–5196.

  38. Miele V, Dussert G, Spataro B, Chamaill´e-Jammes S, Allain´e D, Bonenfant C. Revisiting giraffe photo-identification using deep learning and network analysis. bioRxiv. 2020.

  39. Murdoch WJ, Singh C, Kumbier K, Abbasi-Asl R, Yu B. Definitions, methods, and applications in interpretable machine learning. Proc Natl Acad Sci. 2019;116(44):22071–80.

    Article  MathSciNet  Google Scholar 

  40. Nguyen A, Clune J, Bengio Y, Dosovitskiy A, Yosinski J. Plug & play generative networks: Conditional iterative generation of images in latent space. In: Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition. 2017;4467–4477.

  41. Olah C, Mordvintsev A, Schubert L. Feature visualization. Distill. 2017. https://doi.org/10.23915/distill.00007.

    Article  Google Scholar 

  42. Pearson K. LIII. on lines and planes of closest fit to systems of points in space. Lond Edinb Dublin Philos Mag J Sci. 1901;2(11):559–72.

    Article  Google Scholar 

  43. Redmon J. Darknet: open source neural networks in c. 2013. http://pjreddie.com/darknet/. Accessed 5 Oct 2021

  44. Redmon J, Divvala S, Gir- shick R, Farhadi A. You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016; 779–788.

  45. Ren S, He K, Girshick R, Sun J. Faster r-cnn: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst. 2015;28:91–9.

    Google Scholar 

  46. Ribeiro MT, Singh S, Guestrin C” why should i trust you?” explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016;1135–1144.

  47. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In: 2017 IEEE International Conference on Computer Vision (ICCV). 2017; 618–626, DOI https://doi.org/10.1109/ICCV.2017.74

  48. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations. 2015. https://www.robots.ox.ac.uk/~vgg/publications/2015/Simonyan15/.

  49. Springenberg J, Dosovitskiy A, Brox T, Riedmiller M. Striving for simplicity: the all convolutional net. In: ICLR (workshop track). 2015. URL http://lmb.informatik.uni-freiburg.de/Publications/2015/DB15a. Accessed 5 Oct 2021

  50. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: computer vision and pattern recognition (CVPR). 2015. URL http://arxiv.org/abs/1409.4842. Accessed 5 Oct 2021

  51. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016; 2818–2826.

  52. Tzutalin (2021) Labelimg is a graphical image annotation tool and label object bounding boxes in images. URL https://github.com/tzutalin/labelImg. Accessed 5 Oct 2021

  53. Vidya TNC, Prasad D, Ghosh A. In- dividual identification in asian elephants. Gajah. 2014;40:3–17.

    Google Scholar 

  54. Vilone G, Longo L. Explainable artificial intelligence: a systematic review. arXiv preprint arXiv:200600093. 2020.

  55. Zeiler MD, Fergus R. Visualizing and understanding convo- lutional networks. In: European conference on computer vision, Springer. 2014; 818–833.

  56. Zhang Q, Cao R, Shi F, Wu YN, Zhu SC. Interpreting cnn knowledge via an explanatory graph. In: Thirty-Second AAAI Conference on Artificial Intelligence. 2018a.

  57. Zhang Q, Yang Y, Wu YN, Zhu SC. Interpreting cnns via decision trees. arXiv preprint arXiv:180200121. 2018b.

  58. Zhang QS, Zhu SC. Visual interpretability for deep learning: a survey. Front Inf Technol Electron Eng. 2018;19(1):27–39.

    Article  Google Scholar 

  59. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A. Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016; 2921–2929.

Download references

Acknowledgements

Special thank goes to Mr D. G Ashoka Ranjeewa and Dr Shermin De Silva for the elephant image dataset and annotations, mentioned in this paper as ele-raw, based on elephants at Udawalawe National Park of Sri Lanka. For the Elephant dataset, we would like to thank Dr Matthias K¨orschens. Special thanks go to Mr Chathura Suduwella, Mr Tharindu Wijethilake, Dr Kasun Karunanayake, Dr Kasun Gunawardena of University of Colombo School of Computing for their support throughout.

Author information

Authors and Affiliations

  1. SCORE Lab, University of Colombo School of Computing, Colombo 7, Colombo, Sri Lanka

    Mithsen De Silva, Prabhash Kumarasinghe, Kasun De Zoysa & Chamath Keppitiyagama

Authors
  1. Mithsen De Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prabhash Kumarasinghe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kasun De Zoysa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chamath Keppitiyagama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Prabhash Kumarasinghe.

Ethics declarations

Conflict of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

De Silva, M., Kumarasinghe, P., De Zoysa, K. et al. Reidentifying Asian Elephants from Ear Images Using a Cascade of Convolutional Neural Networks and Explaining with GradCAM. SN COMPUT. SCI. 3, 192 (2022). https://doi.org/10.1007/s42979-022-01057-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-022-01057-5

Keywords

Search

Navigation