Skip to main content
SpringerLink
Log in

YOLOv5s-BC: an improved YOLOv5s-based method for real-time apple detection

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The current apple detection algorithms fail to accurately differentiate obscured apples from pickable ones, thus leading to low accuracy in apple harvesting and a high rate of instances where apples are either mispicked or missed altogether. To address the issues associated with the existing algorithms, this study proposes an improved YOLOv5s-based method, named YOLOv5s-BC, for real-time apple detection, in which a series of modifications have been introduced. First, a coordinate attention block has been incorporated into the backbone module to construct a new backbone network. Second, the original concatenation operation has been replaced with a bi-directional feature pyramid network in the neck network. Finally, a new detection head has been added to the head module, enabling the detection of smaller and more distant targets within the field of view of the robot. The proposed YOLOv5s-BC model was compared to several target detection algorithms, including YOLOv5s, YOLOv4, YOLOv3, SSD, Faster R-CNN (ResNet50), and Faster R-CNN (VGG), with significant improvements of 4.6%, 3.6%, 20.48%, 23.22%, 15.27%, and 15.59% in mAP, respectively. The detection accuracy of the proposed model is also greatly enhanced over the original YOLOv5s model. The model boasts an average detection speed of 0.018 s per image, and the weight size is only 16.7 Mb with 4.7 Mb smaller than that of YOLOv8s, meeting the real-time requirements for the picking robot. Furthermore, according to the heat map, our proposed model can focus more on and learn the high-level features of the target apples, and recognize the smaller target apples better than the original YOLOv5s model. Then, in other apple orchard tests, the model can detect the pickable apples in real time and correctly, illustrating a decent generalization ability. It is noted that our model can provide technical support for the apple harvesting robot in terms of real-time target detection and harvesting sequence planning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Bao, W.X., Zhu, Z.Q., Hu, G.S., et al.: UAV remote sensing detection of tea leaf blight based on DDMA-YOLO. Comput. Electron. Agric. [J] 205, 17 (2023). https://doi.org/10.1016/j.compag.2023.107637

    Article  Google Scholar 

  2. Bochkovskiy, A., Wang, C.-Y., Mark Liao, H.-Y.: YOLOv4: Optimal Speed and Accuracy of Object Detection (2020). Arxiv [J]. arXiv:2004.10934

  3. Fountas, S., Mylonas, N., Malounas, I., et al.: Agricultural robotics for field operations. Sensors [J] 20(9), 27 (2020). https://doi.org/10.3390/s20092672

    Article  Google Scholar 

  4. Häni N., Roy P., Isler, V.: MinneApple Data [M] (2019)

  5. He, K.M., Gkioxari G., Dollar P., et al.: Mask R-CNN[C]. In: 16th IEEE International Conference on Computer Vision (ICCV). IEEE, Venice, Italy, pp. 2980–2988 (2017). https://doi.org/10.1109/iccv.2017.322

  6. Hou, Q., Zhou, D., Feng, J.: Coordinate attention for efficient mobile network design[C]. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13713–13722 (2021). https://doi.org/10.48550/arXiv.2103.02907

  7. Jia, W.K., Zhang, Y., Lian, J., et al.: Apple harvesting robot under information technology: a review. Int. J. Adv. Robot. Syst. [J] 17(3), 16 (2020). https://doi.org/10.1177/1729881420925310

    Article  Google Scholar 

  8. Li, K.S., Wang, J.C., Jalil, H., et al.: A fast and lightweight detection algorithm for passion fruit pests based on improved YOLOv5. Comput. Electron. Agric. [J] 204, 11 (2023). https://doi.org/10.1016/j.compag.2022.107534

    Article  Google Scholar 

  9. Liang, J.T., Chen, X., Liang, C.J., et al.: A detection approach for late-autumn shoots of litchi based on unmanned aerial vehicle (UAV) remote sensing. Comput. Electron. Agric. [J] 204, 10 (2023). https://doi.org/10.1016/j.compag.2022.107535

    Article  Google Scholar 

  10. Liu, W., Anguelov, D., Erhan, D., et al.: Ssd: single shot multibox detector[C]. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14. Springer, pp. 21–37 (2016). https://doi.org/10.1007/978-3-319-46448-0_2

  11. Lu, Y.Z., Young, S.: A survey of public datasets for computer vision tasks in precision agriculture. Comput. Electron. Agric. [J] 178, 13 (2020). https://doi.org/10.1016/j.compag.2020.105760

    Article  Google Scholar 

  12. Lv, J.D., Xu, H., Han, Y., et al.: A visual identification method for the apple growth forms in the orchard. Comput. Electron. Agric. [J] 197, 9 (2022). https://doi.org/10.1016/j.compag.2022.106954

    Article  Google Scholar 

  13. Qi, J.T., Liu, X.N., Liu, K., et al.: An improved YOLOv5 model based on visual attention mechanism: application to recognition of tomato virus disease. Comput. Electron. Agric. [J] 194, 12 (2022). https://doi.org/10.1016/j.compag.2022.106780

    Article  Google Scholar 

  14. Redmon, J., Divvala, S., Girshick, R., et al.: You only look once: unified, real-time object detection[C]. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Seattle, pp. 779–788 (2016). https://doi.org/10.1109/cvpr.2016.91

  15. Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: IEEE Conference on Computer Vision & Pattern Recognition [J], pp. 6517–6525 (2017). https://doi.org/10.1109/CVPR.2017.690

  16. Redmon, J., Farhadi, A.: YOLOv3: An Incremental Improvement. Arxiv [J] (2018). arXiv:1804.02767

  17. Ren, S.Q., He, K.M., Girshick, R., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. [J] 39(6), 1137–1149 (2017). https://doi.org/10.1109/tpami.2016.2577031

    Article  Google Scholar 

  18. Sun, L.J., Hu, G.R., Chen, C., et al.: Lightweight apple detection in complex orchards using YOLOV5-PRE. Horticulturae [J] 8(12), 15 (2022). https://doi.org/10.3390/horticulturae8121169

    Article  Google Scholar 

  19. Tan, M., Pang, R., Le, Q.V.: EfficientDet: scalable and efficient object detection. Arxiv [J], pp. 10778–10787 (2020). arXiv:1911.09070

  20. Ultralytics yolov5 [M]

  21. Wang, C.-Y., Liao, H.-Y. M., Wu ,Y.-H., et al.: CSPNet: A new backbone that can enhance learning capability of CNN[C]. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 390–391 (2020)

  22. Wu, F.Y., Duan, J.L., Ai, P.Y., et al.: Rachis detection and three-dimensional localization of cut off point for vision-based banana robot. Comput. Electron. Agric. [J] 198, 12 (2022). https://doi.org/10.1016/j.compag.2022.107079

    Article  Google Scholar 

  23. Xu, B., Cui, X., Ji, W., et al.: Apple grading method design and implementation for automatic grader based on improved YOLOv5. Agric. Basel [J] 13(1), 18 (2023). https://doi.org/10.3390/agriculture13010124

    Article  Google Scholar 

  24. Xu, Z.B., Huang, X.P., Huang, Y., et al.: A real-time zanthoxylum target detection method for an intelligent picking robot under a complex background, based on an improved YOLOv5s architecture. Sensors [J] 22(2), 15 (2022). https://doi.org/10.3390/s22020682

    Article  MathSciNet  Google Scholar 

  25. Yan, B., Fan, P., Lei, X.Y., et al.: A real-time apple targets detection method for picking robot based on improved YOLOv5. Remote Sens. [J] 13(9), 23 (2021). https://doi.org/10.3390/rs13091619

    Article  Google Scholar 

  26. Yao, J., Qi, J.M., Zhang, J., et al.: A real-time detection algorithm for kiwifruit defects based on YOLOv5. Electronics [J] 10(14), 13 (2021). https://doi.org/10.3390/electronics10141711

    Article  Google Scholar 

  27. Zhang, D.Y., Luo, H.S., Wang, D.Y., et al.: Assessment of the levels of damage caused by Fusarium head blight in wheat using an improved YoloV5 method. Comput. Electron. Agric. [J] 198, 16 (2022). https://doi.org/10.1016/j.compag.2022.107086

    Article  Google Scholar 

  28. Zhao, Y.S., Gong, L., Huang, Y.X., et al.: A review of key techniques of vision-based control for harvesting robot. Comput. Electron. Agric. [J] 127, 311–323 (2016). https://doi.org/10.1016/j.compag.2016.06.022

    Article  Google Scholar 

  29. Zheng, Z.H., Wang, P., Liu, W., et al.: Distance-IoU loss: faster and better learning for bounding box regression[C]. In: 34th AAAI Conference on Artificial Intelligence/32nd Innovative Applications of Artificial Intelligence Conference/10th AAAI Symposium on Educational Advances in Artificial Intelligence. Assoc Advancement Artificial Intelligence, New York, pp. 12993–13000 (2020). https://doi.org/10.48550/arXiv.1911.08287

  30. Zhou, H.Y., Wang, X., Au, W., et al.: Intelligent robots for fruit harvesting: recent developments and future challenges. Precis. Agric. [J] 23(5), 1856–1907 (2022). https://doi.org/10.1007/s11119-022-09913-3

    Article  Google Scholar 

  31. Zhu, X.K., Lyu, S.C., Wang, X., et al.: TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]. In: 18th IEEE/CVF International Conference on Computer Vision (ICCV). Electr Network: Ieee Computer Soc, pp. 2778–2788 (2021). https://doi.org/10.1109/iccvw54120.2021.00312

Download references

Acknowledgements

The present work is supported by the Start-up Funds from Wuhan University of Technology and the National Innovation and Entrepreneurship Training Program for College Students (S202310497143). Research participants (Jie Lin, Yu Pei, and Rongzhen Yang) are also appreciated.

Author information

Authors and Affiliations

  1. Hubei Digital Manufacturing Key Laboratory, School of Mechanical and Electronic Engineering, Wuhan University of Technology, Wuhan, 430070, China

    Jingfan Liu & Zhaobing Liu

Authors
  1. Jingfan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhaobing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Jingfan Liu: Methodology, Software, Investigation, Visualization, Writing – original draft, Writing – review & editing. Zhaobing Liu: Conceptualization, Methodology, Investigation, Visualization, Writing – original draft, Writing – review & editing, Supervision, Project administration.

Corresponding author

Correspondence to Zhaobing Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, J., Liu, Z. YOLOv5s-BC: an improved YOLOv5s-based method for real-time apple detection. J Real-Time Image Proc 21, 88 (2024). https://doi.org/10.1007/s11554-024-01473-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-024-01473-1

Keywords

Search

Navigation