Skip to main content
SpringerLink
Log in

Deep learning for fine-grained classification of jujube fruit in the natural environment

  • Original Paper
  • Published:
Journal of Food Measurement and Characterization Aims and scope Submit manuscript

Abstract

Jujube is a popular fruit with a long cultivation history and numerous varieties in China. It is necessary to develop an automatic visual identification system of jujube classification in the natural environment. However, practical success in this area is still limited. In this paper, we propose a deep convolutional neural network model for the fine-grained classification of jujube, which exploits a two-stream network to effectively learn discriminative features for each image from both shape level and fine-grained level simultaneously. Specifically, it can also learn the contrastive discrepancies from jujube image pairs. To further facilitate the research, we create a rich jujube image dataset in the natural environment. The dataset consists of more than 1700 images of 20 jujube varieties, and these images have a large degree of variations including angles, background and illumination conditions. The proposed model achieves an average accuracy of 84.16% on this dataset, which outperforms the other four models, including SVM, AlexNet, VGGNet-16 and ResNet-18. The feasibility of this method is demonstrated by the experiment results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. R.R. Wang, S.H. Ding, D.D. Zhao, Z.F. Wang, J.H. Wu, X.S. Hu, Effect of dehydration methods on antioxidant activities, phenolic contents, cyclic nucleotides, and volatiles of jujube fruits. Food Sci. Biotechnol. 25(1), 137–143 (2016)

    Article  Google Scholar 

  2. J.P. Chen, Z.G. Li, M. Maiwulanjiang, W.L. Zhang, J.Y.X. Zhan, C.T.W. Lam, K.Y. Zhu, P. Yao, R.C.Y. Choi, D.T.W. Lau, Chemical and biological assessment of Ziziphus jujuba Fruits from china: different geographical sources and developmental stages. J. Agric. Food Chem. 61(30), 7315–7324 (2013)

    Article  CAS  Google Scholar 

  3. H.Q. Lou, Y. Hu, B. Wang, H.F. Lu, Dried jujube classification using support vector machine based on fractal parameters and red, green and blue intensity. Int. J. Food Sci. Tech. 47(9), 1951–1957 (2012)

    Article  CAS  Google Scholar 

  4. J.W. Li, L.P. Fan, S.D. Ding, X.L. Ding, Nutritional composition of five cultivars of chinese jujube. Food Chem. 103(2), 454–460 (2007)

    Article  CAS  Google Scholar 

  5. L.Y. Song, J. Zheng, L. Zhang, S.J. Yan, W.J. Huang, J. He, P.Z. Liu, Phytochemical profiling and fingerprint analysis of chinese jujube (ziziphus jujuba mill.) leaves of 66 cultivars from xinjiang province. Molecules 24(24), 4528 (2019)

    Article  CAS  Google Scholar 

  6. X.Y. Sun, L.Y. Ma, G.Y. Li, Multi-vision attention networks for on-line red jujube grading. Chin. J. Electron. 28(6), 1108–1117 (2019)

    Article  Google Scholar 

  7. H.J. Ni, J.Q. Zhang, N.S. Zhao, C. Wang, S. Lv, F.J. Ren, X.X. Wang, Design on the winter jujubes harvesting and sorting device. Appl. Sci. 9(24), 5546 (2019)

    Article  Google Scholar 

  8. G.Y. Li, Y.L. Ren, L.Y. Ma, Recognition of wizened red jujube based on ununiformities of gradient distribution. Trans. Chin. Soc. Agric. Mach. 47(11), 213–218 (2016)

    Google Scholar 

  9. L.G. Wu, J.G. He, G.S. Liu, S.L. Wang, X.G. He, Detection of common defects on jujube using vis-nir and nir hyperspectral imaging. Postharvest Biol. Technol. 112, 134–142 (2016)

    Article  CAS  Google Scholar 

  10. K.Q. Yu, Y.R. Zhao, X.L. Li, Y.N. Shao, F.L. Zhu, Y. He, Identification of crack features in fresh jujube using vis/nir hyperspectral imaging combined with image processing. Comput. Electron. Agric. 103(103), 1–10 (2014)

    Article  Google Scholar 

  11. L. Geng, W.L. Xu, F. Zhang, Z.T. Xiao, Y.B. Liu, Dried jujube classification based on a double branch deep fusion convolution neural network. Food Sci. Technol. Res. 24(6), 1007–1015 (2018)

    Article  Google Scholar 

  12. X. Zhong, Y. Cao, Y. Zhang, C. Xiao, Research and implementation of the non-destructive sorting system for full surface information of jujube based on machine vision technology. Food Mach. 33(05), 114–118 (2017)

    Google Scholar 

  13. M. Wu, C. Li, X. Yi, Recognition of southern jujube varieties based on image recognition technology. J. Tarim Univ. 26(004), 105–110 (2014)

    Google Scholar 

  14. Y.T. Yang, S.J. Zhang, Y. He, Dynamic detection of fresh jujube based on elm and visible/near infrared spectra. Spectrosc. Spectr. Anal. 35(7), 1870–1874 (2015)

    CAS  Google Scholar 

  15. L. Liu, D. He, M. Li, X. Liu, J. Qu, Identification of xinjiang jujube varieties based on hyperspectral technique and machine learning. Chin. J. Lasers 47(11), 291–298 (2020)

    Google Scholar 

  16. G. Huang, Z. Liu, L. Van Der Maaten, K.Q. Weinberger, Densely connected convolutional networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition. (2017), pp. 4700–4708

  17. Y. Lecun, Y. Bengio, G.E. Hinton, Deep learning. Nature 521(7553), 436–444 (2015)

    Article  CAS  Google Scholar 

  18. J. Schmidhuber, Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  Google Scholar 

  19. W.B. Liu, Z.D. Wang, X.H. Liu, N.Y. Zeng, Y.R. Liu, F.E. Alsaadi, A survey of deep neural network architectures and their applications. Neurocomputing 234(234), 11–26 (2017)

    Google Scholar 

  20. A. Voulodimos, N. Doulamis, A. Doulamis, E. Protopapadakis, Deep learning for computer vision: a brief review. Comput. Intell. Neurosci. 2018, 1–13 (2018)

    Google Scholar 

  21. J.X. Gu, Z.H. Wang, J. Kuen, L. Ma, A. Shahroudy, B. Shuai, T. Liu, X. Wang, G. Wang, J. Cai et al., Recent advances in convolutional neural networks. Pattern Recogn. 77, 354–377 (2018)

    Article  Google Scholar 

  22. S.P. Mohanty, D.P. Hughes, M. Salathe, Using deep learning for imagebased plant disease detection. Front. Plant Sci. 7, 1419–1419 (2016)

    Article  Google Scholar 

  23. E. Shelhamer, J. Long, T. Darrell, Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017)

    Article  Google Scholar 

  24. L. Perez, J. Wang, The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621

  25. W. Rawat, Z. Wang, Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput. 29(9), 2352–2449 (2017)

    Article  Google Scholar 

  26. J.M. Ponce, A. Aquino, J.M. Andujar, Olive-fruit variety classification by means of image processing and convolutional neural networks. IEEE Access. 7, 147629–147641 (2019)

    Article  Google Scholar 

  27. M.O. Al-Shawwa, S.S. Abu-Naser, Classification of apple fruits by deep learning. Int. J. Acad. Eng. Res. 3(12), 1–7 (2020)

    Google Scholar 

  28. H. Altaheri, M. Alsulaiman, G. Muhammad, Date fruit classification for robotic harvesting in a natural environment using deep learning. IEEE Access. 7, 117115–117133 (2019)

    Article  Google Scholar 

  29. S. Naik, B. Patel, Machine vision based fruit classification and grading—a review. Int. J. Comput. Appl. 170(9), 22–34 (2017)

    Google Scholar 

  30. Y.D. Zhang, Z.C. Dong, X.Q. Chen, W.J. Jia, S. Du, K. Muhammad, S. Wang, Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation. Multimed. Tools Appl. 78(3), 3613–3632 (2019)

    Article  Google Scholar 

  31. F.J. Rodriguez, A.M. Garcia, P.J. Pardo, F. Chavez, R.M. Luquebaena, Study and classification of plum varieties using image analysis and deep learning techniques. Prog. Artif. Intell. 7(2), 119–127 (2018)

    Article  Google Scholar 

  32. T. Hu, H. Qi, Q. Huang, Y. Lu, See better before looking closer: weakly supervised data augmentation network for fine-grained visual classification. arXiv preprint arXiv:1901.09891

  33. Y. Chen, Y. Bai, W. Zhang, T. Mei, Destruction and construction learning for fine-grained image recognition. in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019), pp. 5157–5166

  34. H.L. Zheng, J.L. Fu, Z.J. Zha, J.B. Luo, Looking for the devil in the details: learning trilinear attention sampling network for fine-grained image recognition. in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019), pp. 5012–5021

  35. Y. Yu, Q. Jin, C.W. Chen, Ff-cmnet: a cnn-based model for fine-grained classification of car models based on feature fusion. in: 2018 IEEE International Conference on Multimedia and Expo. (2018), pp. 1–6

  36. M. Biglari, A. Soleimani, H. Hassanpour, A cascaded part-based system for fine-grained vehicle classification. IEEE Trans. Intell. Transp. Syst. 19(1), 273–283 (2017)

    Article  Google Scholar 

  37. J.L. Fu, H.L. Zheng, T. Mei, Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition. in: Proceedings of the IEEE conference on computer vision and pattern recognition (2017), pp. 4438–4446

  38. T.J. Xiao, Y.C. Xu, K.Y. Yang, J.X. Zhang, Y.X. Peng, Z. Zhang, The application of two-level attention models in deep convolutional neural network for fine-grained image classification, in: Proceedings of the IEEE conference on computer vision and pattern recognition (2015), pp. 842–850

  39. Y. Cui, Y. Song, C. Sun, A. Howard, S. Belongie, Large scale fine-grained categorization and domain-specific transfer learning, in: Proceedings of the IEEE conference on computer vision and pattern recognition (2018), pp. 4109–4118

  40. A. Kamilaris, F.X. Prenafeta-Boldu, Deep learning in agriculture: a survey. Comput. Electron. Agric. 147, 70–90 (2018)

    Article  Google Scholar 

  41. K. Simonyan, A. Zisserman, Very deep convolutional networks for largescale image recognition. arXiv preprint arXiv:abs/1409.1556

  42. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)

    Article  Google Scholar 

  43. A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)

    Article  Google Scholar 

  44. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, in: Proceedings of the IEEE conference on computer vision and pattern recognition (2015), pp. 1–9

  45. T.-Y. Lin, P. Goyal, R. Girshick, K. He, P. Dollar, Focal loss for dense object detection, in: Proceedings of the IEEE International Conference on Computer Vision (2017), pp. 2980–2988

  46. A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, A. Lerer, Automatic differentiation in pytorch.

  47. C. Li, P.K.-S. Tam, An iterative algorithm for minimum cross entropy thresholding. Pattern Recogn. Lett. 19(8), 771–776 (1998)

    Article  Google Scholar 

  48. Q. Xu, K. Yan, Y. Tian, Learning a repression network for precise vehicle search. arXiv preprint arXiv:abs/1708.02386

  49. J.A.K. Suykens, J. Vandewalle, Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)

    Article  Google Scholar 

  50. D. Tao, X. Li, X. Wu, S.J. Maybank, General tensor discriminant analysis and gabor features for gait recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(10), 1700–1715 (2007). https://doi.org/10.1109/TPAMI.2007.1096

    Article  PubMed  Google Scholar 

  51. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Imagenet: a large-scale hierarchical image database, in IEEE Conference on Computer Vision and Pattern Recognition (2009), pp. 248–255. https://doi.org/10.1109/CVPR.2009.5206848.

Download references

Funding

This work was supported by National Key Research and Development Program Project of China (Grant No. 2019YFD1001605) and the Postgraduate Innovation Funding Project of Hebei Province (Grant No. CXZZBS2020099).

Author information

Authors and Affiliations

  1. College of Information Science and Technology, Hebei Agricultural University, Baoding, 07100, China

    Xi Meng, Yingchun Yuan, Guifa Teng & Tianzhen Liu

  2. College of Information Engineering, Baoding University, Baoding, 07100, China

    Tianzhen Liu

Authors
  1. Xi Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yingchun Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guifa Teng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tianzhen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingchun Yuan.

Ethics declarations

Conflict of interest

We declare that we have no financial and personal relationships with other people or organizations that can inappropriately influence our work, there is no professional or other personal interest of any nature or kind in any product, service or company.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meng, X., Yuan, Y., Teng, G. et al. Deep learning for fine-grained classification of jujube fruit in the natural environment. Food Measure 15, 4150–4165 (2021). https://doi.org/10.1007/s11694-021-00990-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11694-021-00990-y

Keywords

Search

Navigation