Abstract
Pinus Radiata trees form pollen-producing catkins that can be harvested for pharmaceutical uses. Unmanned Aerial Vehicles (UAVs) may be well suited to the task of autonomously harvesting these catkins. We propose a method to reliably detect and track P. Radiata catkins in three dimensions that can be used for real-time guidance of a UAV. The method applies the YOLOv5 deep learning algorithm to detect catkins in the X-Y plane. A novel optimisation of the MeanShift algorithm is utilised to assist existing contour detection algorithms in segmenting individual catkins in the Z plane. A Kanade-Lucas-Tomasi tracker was used with RANSAC for accurate frame-to-frame tracking. The method achieved a Mean Average Precision of 0.87 on images taken at a commercial pine pollen farm. The method detected the depth of catkins at distances of up to 1200 mm to an accuracy of 2 mm, or 8 mm for occluded catkins. Detected catkins can be reliably tracked at speeds of 1ms. An average frame rate of 22 frames per second was achieved on an Intel i5 CPU, with the Meanshift optimisation performing up to 41 times faster than existing implementations. These results indicate that the proposed method could be used to successfully assist in the automated harvesting of P. Radiata catkins.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
van der Colff, M., Kimberley, M.O.: A national height-age model for Pinus radiata in New Zealand. NZ J. Forest. Sci. 43(1), 4 (2013). https://doi.org/10.1186/1179-5395-43-4
Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002). https://doi.org/10.1109/34.1000236
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981). https://doi.org/10.1145/358669.358692
Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004). ISBN 0521540518
Jocher, G., et al.: Marc, albinxavi, fatih, oleg, wanghaoyang0106: ultralytics/yolov5: v6.0 - YOLOv5n ‘Nano’ models, Roboflow integration, TensorFlow export, OpenCV DNN support (2021). https://doi.org/10.5281/zenodo.5563715
Kuznetsova, A., Maleva, T., Soloviev, V.: Detecting apples in orchards using YOLOv3 and YOLOv5 in general and close-up images. In: Han, M., Qin, S., Zhang, N. (eds.) ISNN 2020. LNCS, vol. 12557, pp. 233–243. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64221-1_20
Lin, G., Tang, Y., Zou, X., Xiong, J., Fang, Y.: Color-, depth-, and shape-based 3D fruit detection. Precis. Agric. 21(1), 1–17 (2019). https://doi.org/10.1007/s11119-019-09654-w
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Liu, T., Moore, A.W., Gray, A., Cardie, C.: New algorithms for efficient high-dimensional nonparametric classification. J. Mach. Learn. Res. 7(6) (2006)
Liu, X., et al.: Robust fruit counting: Combining deep learning, tracking, and structure from motion. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1045–1052 (2018). https://doi.org/10.1109/IROS.2018.8594239
Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision (IJCAI), vol. 81 (1981)
Luo, Z., Zhang, Y., Wang, K., Sun, L.: Detection of pine cones in natural environment using improved YOLOv4 deep learning algorithm. Comput. Intell. Neurosci. 2021, 1–12 (2021). https://doi.org/10.1155/2021/5601414
Onishi, Y., Yoshida, T., Kurita, H., Fukao, T., Arihara, H., Iwai, A.: An automated fruit harvesting robot by using deep learning. ROBOMECH J. 6(1) (2019). https://doi.org/10.1186/s40648-019-0141-2
Osco, L.P., et al.: A review on deep learning in UAV remote sensing. Int. J. Appl. Earth Obs. Geoinform. 102, 102456 (2021). https://doi.org/10.1016/j.jag.2021.102456. https://www.sciencedirect.com/science/article/pii/S030324342100163X
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979). https://doi.org/10.1109/TSMC.1979.4310076
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
New Zealand’s forests. https://www.mpi.govt.nz/forestry/new-zealand-forests-forest-industry/new-zealands-forests/
Roy, P., Isler, V.: Surveying apple orchards with a monocular vision system. In: 2016 IEEE International Conference on Automation Science and Engineering (CASE), pp. 916–921 (2016). https://doi.org/10.1109/COASE.2016.7743500
SepúLveda, D., Fernández, R., Navas, E., Armada, M., González-De-Santos, P.: Robotic aubergine harvesting using dual-arm manipulation. IEEE Access 8, 121889–121904 (2020). https://doi.org/10.1109/ACCESS.2020.3006919
Suzuki, S., Be, K.: Topological structural analysis of digitized binary images by border following. Comput. Vis. Graph. Image Process. 30(1), 32–46 (1985). https://doi.org/10.1016/0734-189X(85)90016-7. https://www.sciencedirect.com/science/article/pii/0734189X85900167
Williams, H., et al.: Improvements to and large-scale evaluation of a robotic kiwifruit harvester. J. Field Robot. 37(2), 187–201 (2019). https://doi.org/10.1002/rob.21890
Xiao, W., Zaforemska, A., Smigaj, M., Wang, Y., Gaulton, R.: Mean shift segmentation assessment for individual forest tree delineation from airborne lidar data. Remote Sens. 11(11) (2019). https://doi.org/10.3390/rs11111263. https://www.mdpi.com/2072-4292/11/11/1263
Yang, J., Rahardja, S., Fränti, P.: Mean-shift outlier detection and filtering. Pattern Recognit. 115, 107874 (2021). https://doi.org/10.1016/j.patcog.2021.107874. https://www.sciencedirect.com/science/article/pii/S0031320321000613
Zhang, W., et al.: Deep-learning-based in-field citrus fruit detection and tracking. Hortic. Res. 9 (2022). https://doi.org/10.1093/hr/uhac003
Acknowledgements
The research reported in this article was conducted as part of “Enabling unmanned aerial vehicles (drones) to use tools in complex dynamic environments UOCX2104”, which is funded by the New Zealand Ministry of Business, Innovation and Employment.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Song, E., Schofield, S., Green, R. (2023). Detection and Tracking of Pinus Radiata Catkins. In: Yan, W.Q., Nguyen, M., Stommel, M. (eds) Image and Vision Computing. IVCNZ 2022. Lecture Notes in Computer Science, vol 13836. Springer, Cham. https://doi.org/10.1007/978-3-031-25825-1_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-25825-1_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25824-4
Online ISBN: 978-3-031-25825-1
eBook Packages: Computer ScienceComputer Science (R0)