Abstract
Pollen grain micrograph classification has multiple applications in medicine and biology. Automatic pollen grain image classification can alleviate the problems of manual categorisation such as subjectivity and time constraints. While a number of computer-based methods have been introduced in the literature to perform this task, classification performance needs to be improved for these methods to be useful in practice.
In this paper, we present an ensemble approach for pollen grain microscopic image classification into four categories: Corylus Avellana well-developed pollen grain, Corylus Avellana anomalous pollen grain, Alnus well-developed pollen grain, and non-pollen (debris) instances. In our approach, we develop a classification strategy that is based on fusion of four state-of-the-art fine-tuned convolutional neural networks, namely EfficientNetB0, EfficientNetB1, EfficientNetB2 and SeResNeXt-50 deep models. These models are trained with images of three fixed sizes (\(224 \times 224\), \(240 \times 240\), and \(260 \times 260\) pixels) and their prediction probability vectors are then fused in an ensemble method to form a final classification vector for a given pollen grain image.
Our proposed method is shown to yield excellent classification performance, obtaining an accuracy of 94.48% and a weighted F1-score of 94.54% on the ICPR 2020 Pollen Grain Classification Challenge training dataset based on five-fold cross-validation. Evaluated on the test set of the challenge, our approach achieves a very competitive performance in comparison to the top ranked approaches with an accuracy and weighted F1-score of 96.28% and 96.30%, respectively.
This research has received funding from the Austrian Research Promotion Agency (FFG), No. 872636. We thank Nvidia corporation for their generous GPU donation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
References
Battiato, S., Ortis, A., Trenta, F., Ascari, L., Politi, M., Siniscalco, C.: Detection and classification of pollen grain microscope images. In: Conference on Computer Vision and Pattern Recognition Workshops (2020)
Biedermann, T., Winther, L., Till, S.J., Panzner, P., Knulst, A., Valovirta, E.: Birch pollen allergy in Europe. Allergy 74(7), 1237–1248 (2019). https://doi.org/10.1111/all.13758
Brodersen, K.H., Ong, C.S., Stephan, K.E., Buhmann, J.M.: The balanced accuracy and its posterior distribution. In: International Conference on Pattern Recognition, pp. 3121–3124 (2010). https://doi.org/10.1109/ICPR.2010.764
Chica, M.: Authentication of bee pollen grains in bright-field microscopy by combining one-class classification techniques and image processing. Microsc. Res. Tech. 75(11), 1475–1485 (2012). https://doi.org/10.1002/jemt.22091
Chinchor, N.A., Sundheim, B.: Message understanding conference (MUC) tests of discourse processing. In: Spring Symposium on Empirical Methods in Discourse Interpretation and Generation, pp. 21–26 (1995)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995). https://doi.org/10.1007/BF00994018
Cruz, A.A.: Global Surveillance, Prevention and Control of Chronic Respiratory Diseases: A Comprehensive Approach. World Health Organization, Geneva (2007)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Computer Society Conference on Computer Vision and Pattern Recognition. vol. 1 (2005)
Daood, A., Ribeiro, E., Bush, M.: Pollen grain recognition using deep learning. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Porikli, F., Skaff, S., Entezari, A., Min, J., Iwai, D., Sadagic, A., Scheidegger, C., Isenberg, T. (eds.) Advances in Visual Computing, pp. 321–330. Springer International Publishing, Cham (2016). https://doi.org/10.1007/978-3-319-50835-1_30
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848
D’Amato, G., et al.: Allergenic pollen and pollen allergy in Europe. Allergy 62(9), 976–990 (2007). https://doi.org/10.1111/j.1398-9995.2007.01393.x
Fernandez-Delgado, M., Carrion, P., Cernadas, E., Galvez, J., Sa-Otero, P.: Improved classification of pollen texture images using SVM and MLP. In: International Conference on Visualization, Imaging and Image Processing. vol. 2 (2003)
Gardner, M.W., Dorling, S.: Artificial neural networks (the multilayer perceptron) - a review of applications in the atmospheric sciences. Atmos. Environ. 32(14–15), 2627–2636 (1998)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)
Goncalves, A.B., et al.: Feature extraction and machine learning for the classification of Brazilian Savannah pollen grains. PloS One 11(6), e0157044 (2016). https://doi.org/10.1371/journal.pone.0157044
Halbritter, H., et al.: Palynology: history and systematic aspects. Illustrated Pollen Terminology, pp. 3–21. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-71365-6_1
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Holt, K.A., Bennett, K.D.: Principles and methods for automated palynology. N. Phytol. 203(3), 735–742 (2014). https://doi.org/10.1111/nph.12848
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. Conf. Comput. Vis. Pattern Recogn. 1, 4700–4708 (2017)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: International Conference on Computer Vision, pp. 2980–2988 (2017)
Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)
Mahbod, A., Ellinger, I., Ecker, R., Smedby, Ö., Wang, C.: Breast cancer histological image classification using fine-tuned deep network fusion. In: Campilho, A., Karray, F., ter Haar Romeny, B. (eds.) ICIAR 2018. LNCS, vol. 10882, pp. 754–762. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93000-8_85
Mahbod, A., Schaefer, G., Ellinger, I., Ecker, R., Pitiot, A., Wang, C.: Fusing fine-tuned deep features for skin lesion classification. Computer. Med. Imaging Graph. 71, 19–29 (2019). https://doi.org/10.1016/j.compmedimag.2018.10.007
Mahbod, A., Schaefer, G., Wang, C., Dorffner, G., Ecker, R., Ellinger, I.: Transfer learning using a multi-scale and multi-network ensemble for skin lesion classification. Comput. Methods Programs Biomed. 193, p. 105475 (2020). https://doi.org/10.1016/j.cmpb.2020.105475
Mahbod, A., Schaefer, G., Wang, C., Ecker, R., Dorffner, G., Ellinger, I.: Investigating and exploiting image resolution for transfer learning-based skin lesion classification. In: 25th International Conference on Pattern Recognition (2020)
Mahbod, A., Tschandl, P., Langs, G., Ecker, R., Ellinger, I.: The effects of skin lesion segmentation on the performance of dermatoscopic image classification. Comput. Methods Programs Biomed. 197, 105725 (2020). https://doi.org/10.1016/j.cmpb.2020.105725
Menad, H., Ben-Naoum, F., Amine, A.: Deep convolutional neural network for pollen grains classification. In: 3rd Edition of the National Study Day on Research on Computer Sciences. CEUR Workshop Proceedings, vol. 2351 (2019)
Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)
Ranzato, M., Taylor, P., House, J., Flagan, R., LeCun, Y., Perona, P.: Automatic recognition of biological particles in microscopic images. Pattern Recogn. Lett. 28(1), 31–39 (2007)
Sevillano, V., Aznarte, J.L.: Improving classification of pollen grain images of the POLEN23E dataset through three different applications of deep learning convolutional neural networks. PloS One 13(9), e0201807 (2018). https://doi.org/10.1371/journal.pone.0201807
Sevillano, V., Holt, K., Aznarte, J.L.: Precise automatic classification of 46 different pollen types with convolutional neural networks. PLoS One 15(6), e0229751 (2020). https://doi.org/10.1371/journal.pone.0229751
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Stillman, E., Flenley, J.R.: The needs and prospects for automation in palynology. Quat. Sci. Rev. 15(1), 1–5 (1996)
Szegedy, C., et al.: Going deeper with convolutions. In: Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)
Tan, M., Le, Q.V.: EfficientNet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946 (2019)
Travieso, C.M., Briceño, J.C., Ticay-Rivas, J.R., Alonso, J.B.: Pollen classification based on contour features. In: International Conference on Intelligent Engineering Systems (2011). https://doi.org/10.1109/INES.2011.5954712
Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Conference on Computer Vision and Pattern Recognition, pp. 5987–5995 (2017)
Zhang, Z.: Deep-learning-based early detection of diabetic retinopathy on fundus photography using efficientnet. In: International Conference on Innovation in Artificial Intelligence, pp. 70–74 (2020). https://doi.org/10.1145/3390557.3394303
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Mahbod, A., Schaefer, G., Ecker, R., Ellinger, I. (2021). Pollen Grain Microscopic Image Classification Using an Ensemble of Fine-Tuned Deep Convolutional Neural Networks. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12661. Springer, Cham. https://doi.org/10.1007/978-3-030-68763-2_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-68763-2_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-68762-5
Online ISBN: 978-3-030-68763-2
eBook Packages: Computer ScienceComputer Science (R0)
