Skip to main content
SpringerLink
Log in

Detecting and counting pistachios based on deep learning

  • Original Article
  • Published:
Iran Journal of Computer Science Aims and scope Submit manuscript

Abstract

Pistachios are nutritious nuts that are sorted based on the shape of their shell into two categories: Open mouth and Closed mouth. The open-mouth pistachios are higher in price, value, and demand than the closed-mouth pistachios. Because of these differences, it is considerable for production companies to precisely count the number of each kind. This paper aims to propose a new system for counting the different types of pistachios with computer vision. We have introduced and shared a new data set of pistachios, including six videos with a total length of 167 s and 3927 labeled pistachios. Unlike many other works, our model counts pistachios in videos, not images. Counting objects in videos need assigning each object between the video frames so that each object be counted once. The main two challenges in our work are the existence of pistachios’ occlusion and deformation of pistachios in different frames, because open-mouth pistachios that move and roll on the transportation line may appear as closed mouth in some frames and open mouth in other frames. Our novel model first is trained on the RetinaNet object detector network using our data set to detect different types of pistachios in video frames. After gathering the detections, we apply them to a new counter algorithm based on a new tracker to assign pistachios in consecutive frames with high accuracy. Our model is able to assign pistachios that turn and change their appearance (e.g., open-mouth pistachios that look closed mouth) to each other so does not count them incorrectly. Our algorithm performs very fast and achieves good counting results. The computed accuracy of our algorithm on six videos (9486 frames) is 94.75%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

In this GitHub profile (https://github.com/mr7495/Pesteh-Set), we have shared our data set and all the codes that were used for preparing and labeling the data set.

Notes

  1. <This data set is shared in https://github.com/mr7495/Pesteh-Set>

References

  1. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: Large-scale machine learning on heterogeneous systems, (2015). Software available from tensorflow.org

  2. Berman, D.S., Buczak, A.L., Chavis, J.S., Corbett, C.L.: A survey of deep learning methods for cyber security. Information 10(4), 122 (2019)

    Article  Google Scholar 

  3. Burlock, C. D., Lemmons, G. E., Williams, D. W.: Apparatus for splitting closed shell pistachio nuts, Mar. 5 (1991). US Patent 4,996,917

  4. F. Chollet and Others. keras, (2015)

  5. Cohen, O., Linker, R., Naor, A.: Estimation of the number of apples in color images recorded in orchards. In International Conference on Computer and Computing Technologies in Agriculture, pages 630–642. Springer, (2010)

  6. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, (2009)

  7. Wikimedia Foundation. Retrieved from https://en.wikipedia.org/ wiki/Pistachio. Accessed 2 May 2020

  8. Feng, J., Liu, G., Wang, S., Zeng, L., Ren, W.: A novel 3d laser vision system for robotic apple harvesting. In 2012 Dallas, Texas, July 29-August 1, 2012, page 1. American Society of Agricultural and Biological Engineers, (2012)

  9. Food, D. Administration, et al. Qualified health claims: Letter of enforcement discretion–nuts and coronary heart disease. docket no. 02p-0505. Food and Drug Administration, Washington, DC, 2003

  10. Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Garcia-Rodriguez, J.: A review on deep learning techniques applied to semantic segmentation. arXiv preprint arXiv:1704.06857, (2017)

  11. Gongal, A., Amatya, S., Karkee, M., Zhang, Q., Lewis, K.: Sensors and systems for fruit detection and localization: a review. Comput. Electron. Agric. 116, 8–19 (2015)

    Article  Google Scholar 

  12. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, (2016)

  13. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K. Q.: Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4700–4708, (2017)

  14. Kashaninejad, M., Mortazavi, A., Safekordi, A., Tabil, L.: Some physical properties of pistachio (pistacia vera l.) nut and its kernel. J. Food Eng., 72(1):30–38, 2006

  15. Kiger, P. J.: Why pistachios are sold in their shells - unlike most nuts, Mar 2017 (accessed May 2 2020)

  16. Krizhevsky, A., Sutskever, I., Hinton, G. E.: Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, (2012)

  17. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  18. Lee, J.-G., Jun, S., Cho, Y.-W., Lee, H., Kim, G.B., Seo, J.B., Kim, N.: Deep learning in medical imaging: general overview. Korean J. Radiol. 18(4), 570–584 (2017)

    Article  Google Scholar 

  19. Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2117–2125, (2017)

  20. Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (2018)

  21. Linker, R., Cohen, O., Naor, A.: Determination of the number of green apples in rgb images recorded in orchards. Comput. Electron. Agric. 81, 45–57 (2012)

    Article  Google Scholar 

  22. Liu, X., Chen, S. W., Aditya, S., Sivakumar, N., Dcunha, S., Qu, C., Taylor, C. J., Das, J., Kumar, V.: Robust fruit counting: Combining deep learning, tracking, and structure from motion. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1045–1052. IEEE, (2018)

  23. Maskan, M., Karataş, Ş.: Fatty acid oxidation of pistachio nuts stored under various atmospheric conditions and different temperatures. J. Sci. Food Agric. 77(3), 334–340 (1998)

    Article  Google Scholar 

  24. Mureşan, H., Oltean, M.: Fruit recognition from images using deep learning. Acta Universitatis Sapientiae, Informatica 10(1), 26–42 (2018)

    Article  Google Scholar 

  25. OpenCV. Open source computer vision library, 2015

  26. Rahimzadeh, M., Attar, A.: A modified deep convolutional neural network for detecting covid-19 and pneumonia from chest x-ray images based on the concatenation of xception and resnet50v2. Informatics in Medicine Unlocked, page 100360, (2020)

  27. Rahimzadeh, M., Attar, A. et al.: Sperm detection and tracking in phase-contrast microscopy image sequences using deep learning and modified csr-dcf. arXiv preprint arXiv:2002.04034, (2020)

  28. Rahimzadeh,M., Attar, A., Sakhaei, S.M.: A fully automated deep learning-based network for detecting covid-19 from a new and large lung ct scan dataset. Biomedical Signal Processing and Control, p 102588 (2021)

  29. Rahnemoonfar, M., Sheppard, C.: Deep count: fruit counting based on deep simulated learning. Sensors 17(4), 905 (2017)

    Article  Google Scholar 

  30. Different types of iranian pistachios and products of pistachios, Jan 2020 (accessed May 2 2020)

  31. Safren, O., Alchanatis, V., Ostrovsky, V., Levi, O.: Detection of green apples in hyperspectral images of apple-tree foliage using machine vision. Trans ASABE 50(6), 2303–2313 (2007)

    Article  Google Scholar 

  32. Shrestha, A., Mahmood, A.: Review of deep learning algorithms and architectures. IEEE Access 7, 53040–53065 (2019)

    Article  Google Scholar 

  33. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, (2014)

  34. Slaughter, D.C., Harrell, R.C.: Discriminating fruit for robotic harvest using color in natural outdoor scenes. Trans ASAE 32(2), 757–0763 (1989)

    Article  Google Scholar 

  35. Stajnko, D., Lakota, M., Hočevar, M.: Estimation of number and diameter of apple fruits in an orchard during the growing season by thermal imaging. Comput. Electron. Agric. 42(1), 31–42 (2004)

    Article  Google Scholar 

  36. Wachs, J.P., Stern, H.I., Burks, T., Alchanatis, V.: Low and high-level visual feature-based apple detection from multi-modal images. Precision Agric. 11(6), 717–735 (2010)

    Article  Google Scholar 

  37. Whittaker, D., Miles, G., Mitchell, O., Gaultney, L.: Fruit location in a partially occluded image. Trans. ASAE 30(3), 591–0596 (1987)

    Article  Google Scholar 

  38. Pistachio, Apr 2020 (accessed May 2 2020)

  39. Woodruff, J. G. et al.: Tree nuts: production, processing, products. Number Ed. 2. AVI Publishing Co. Inc., (1979)

Download references

Acknowledgements

We wish to thank Mr.Navid Akhundi, who recorded our data set videos and shared them with us, and Fizyr, who implemented RetinaNet with Keras on GitHub. We also thank Colab server for providing free and powerful GPU and Google Drive for providing space for data hosting.

Author information

Authors and Affiliations

  1. School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran

    Mohammad Rahimzadeh

  2. Department of Electrical Engineering, Sharif University of Technology, Tehran, Iran

    Abolfazl Attar

Authors
  1. Mohammad Rahimzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abolfazl Attar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Rahimzadeh.

Ethics declarations

Code availability

In this GitHub profile (https://github.com/mr7495/Pistachio-Counting), we made the trained neural networks, the counting algorithm and all the codes that were used for training and validating the networks, public for researchers use.

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rahimzadeh, M., Attar, A. Detecting and counting pistachios based on deep learning. Iran J Comput Sci 5, 69–81 (2022). https://doi.org/10.1007/s42044-021-00090-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42044-021-00090-6

Keywords

Search

Navigation