All Issue

2024 Vol.42, Issue 1 Preview Page

Research Article

Prediction of Fruit Maturity of ‘Mihwang’ Peach using Deep Learning

딥러닝 기술을 활용한 복숭아 ‘미황’의 성숙도 자동 분류

Sang Jun Lee1†
,
Mi Hee Shin2†
,
L. Sugandhi Hirushika Jayasooriya3
,
W.M. Upeksha Darshani Wijethunga3
,
Seul Ki Lee4
,
Jung Gun Cho4
,
Si Hyeong Jang4
,
Byoung-Kwan Cho1
,
Jin Gook Kim25*
이 상준1†
,
신 미희2†
,
수간디 히루시카3
,
우펙샤 달샤니3
,
이 슬기4
,
조 정건4
,
장 시형4
,
조 병관1
,
김 진국25*
1Department of Biosystems Machinery Engineering, Chungnam National University, Daejeon 34134, Korea
2Institute of Agriculture and Life Sciences, Gyeongsang National University, Jinju 52828, Korea
3Division of Applied Life Science, Gyeongsang National University, Jinju 52828, Korea
4Fruit Research Division, National Institute of Horticultural and Herbal Science, Wanju 55365, Korea
5Division of Horticultural Science, College of Agriculture and Life Science, Gyeongsang National University, Jinju 52828, Korea
1충남대학교 농업생명과학대학 바이오시스템 기계공학과
2경상국립대학교 농업생명과학연구원
3경상국립대학교대학원 응용생명과학부
4국립원예특작과학원 과수과
5경상국립대학교 농업생명과학대학 원예과학부
*Corresponding Author
†These authors contributed equally to this work.
28 February 2024. pp. 80-93
PDF XML Citation
Abstract

Peach must be delivered to market when at their proper ripeness, as its fruit quality declines quickly after harvest. Therefore, it is necessary to consider suitable ripeness for consumption and distribution. However, research on ripeness judgments for peaches in the orchard is scarce. This study used deep learning technology to develop a ripeness classification model for ‘Mihwang’ peaches. A dataset was prepared using 2,800 images, each taken from a peach orchard (outside dataset) and a laboratory (inside dataset) with the same fruit. The dataset was constructed based on the harvest date of the peaches and the peach apex’s skin color (a* value). It uses three classes, immature, ripe, and overripe, according to the classification criteria of the two datasets. The model was trained with a ratio of 7:2:1 of training data, validation data, and test data, and image data augmentation was carried out to improve the diversity of the data and to solve any imbalances. Among EfficientNet, YOLOv5, and Vision Transformer, the deep learning model recorded the best classification model performance on EfficientNet. Based on the classification model and performance evaluation index, the harvest-date-based classification model achieved the highest accuracy of 100%. The classification model based on the apex color a* value of peaches showed high accuracy with a minimum rate of 94.7% and a maximum rate of 98.2%. The peach ripeness classification model developed in this study can be used for determining the proper time for the mechanical harvesting of fruit from an orchard.

Keywords
EfficientNet
fruit firmness
robot harvester
Vision Transformer
YOLOv5

소비자에게 전달되는 복숭아는 숙도에 따라서 품질이 달라지기 때문에 섭취하기에 적합한 숙도를 고려하여 유통하는 과정이 필요하다. 또한, 숙도는 복숭아의 상품성 및 저장성에 영향을 미칠 수 있어 적합한 수확 시기를 선정하는 작업이 요구되지만, 현재 노지 과수 작목의 숙도 판별에 대한 국내 연구는 미미한 실정이다. 그렇기 때문에 본 연구에서는 딥러닝 객체 탐지 분류 모델을 활용하여 복숭아 ‘미황’에 대한 숙도 분류 모델을 개발하였다. 실험실 내부 및 야외에서 촬영된 각 2,800장의 이미지를 활용하여 데이터 셋을 구축하였고, 수확 날짜 및 복숭아 과정부(apex)의 색도 a* 값을 기준으로 하는 두 개의 데이터 셋으로 구성하여 각 셋의 구분 기준에 따라 미숙, 적숙 그리고 과숙 3개의 class로 분류하였다. Train : Validation : Test 데이터 셋은 7 : 2 : 1의 비율로 분류하였고 데이터의 다양성 향상 및 unbalance를 해결하기 위해 augmentation을 실시하였다. 딥러닝 모델은 EfficientNet, YOLOv5 그리고 Vision Transformer를 활용하였으며 EfficientNet에서 가장 우수한 분류 모델 성능을 기록하였다. 날짜 기준 분류 모델은 분류 모델 성능 평가 지표 기준 최저 및 최대 100%의 정확도를 달성하였고, 색도 a* 값 기준 분류 모델은 최저 94.7%, 최대 98.2%의 높은 정확도를 보였다. 본 연구에서 개발된 객체 탐지 기반 복숭아 숙도 분류 모델은 향후 노지 과수 작목의 숙도 분류를 통한 기계수확 적기 판정 작업에 활용될 수 있을 것으로 판단된다.

키워드
EfficientNet
과실경도
로봇수확
Vision Transformer
YOLOv5
References
1
Bae SM, Park JS, Ju CY, Lee DH (2022) Selective face de-identification scheme using multiple face recognition and classification techniques. IV International Conference on Advances in Computer Technology 2022, Inf Sci Commun (CTISC): IEEE, Suzhou, China, pp 1-8. doi:10.1109/CTISC54888.2022.9849764 10.1109/CTISC54888.2022.9849764
2
Baek Y, Sul S, Cho YY (2023) Estimation of days after transplanting using an artificial intelligence CNN (Convolutional Neural Network) model in a closed-type plant factory. Hortic Sci Technol 41:81-90. doi:10.7235/HORT.20230008 10.7235/HORT.20230008
3
Bang JH, Park J, Park SW, Kim JY, Jung SH, Sim CB (2022) A system for determining the growth stage of fruit tree using a deep learning-based object detection model. Smart Mdeia J 11:9-18. doi:10.30693/SMJ.2022.11.4.9 10.30693/SMJ.2022.11.4.9
4
Buslaev A, Iglovikov VI, Khvedchenya E, Parinov A, Druzhinin M, Kalinin AA (2020) Albumentations: fast and flexible image augmentations. Information 11:125. doi:10.3390/info11020125 10.3390/info11020125
5
Dosovitskiy A, Beyer A, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, et al. (2020) An image is worth 16x16 words: Transformers for image recognition at scale. Cornell University, New York, USA. doi:10.48550/arXiv.2010.11929 10.48550/arXiv.2010.11929
6
Jung, DH, Cho, YY (2022) A prediction model for Hallabong tangor product prices using LSTM (Long Short-term Memory) network. Hortic Sci Technol 40:571-577. doi:10.7235/HORT.20220051 10.7235/HORT.20220051
7
Kim I, Lee MG, Jeon Y (2021) Comparative analysis of defect detection using YOLO of deep learning. J KSMTE 30:514-519. doi:10.7735/ksmte.2021.30.6.513 10.7735/ksmte.2021.30.6.513
8
Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60:84-90. doi:10.1145/3065386 10.1145/3065386
9
LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1:541-551. doi:10.1162/neco.1989.1.4.541 10.1162/neco.1989.1.4.541
10
Lee HJ, Lee WS, Choi IH, Lee CK (2020) Detection model of fruit epidermal defects using YOLOv3: A case of peach. Information System Review 22:113-124. doi:10.14329/isr.2020.22.1.113 10.14329/isr.2020.22.1.113
11
Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: Hierarchical vision transformer using shifted windows. IEEE/CVF International Conference on Computer Vision 2021, pp 10012-10022. doi:10.1109/ICCV48922.2021.00986 10.1109/ICCV48922.2021.0098634576156PMC8471952
12
Loshchilov I, Hutter F (2017) Decoupled weight decay regularization. arXiv preprint. doi:10.48550/arXiv.1711.05101 10.48550/arXiv.1711.05101
13
Nam DS, Moon T, Lee JW, Son JE (2019) Estimating transpiration rates of hydroponically-grown paprika via an artificial neural network using aerial and root-zone environments and growth factors in greenhouses. Hortic Environ Biotechnol 60:913-923. doi:10.1007/s13580-019-00183-z 10.1007/s13580-019-00183-z
14
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, et al. (2019) Pytorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process Syst, Vol 32. doi:10.48550/arXiv.1912.01703 10.48550/arXiv.1912.01703
15
Reza AM (2004) Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. J VLSI Signal Process Syst Signal Image Video Technol 38:35-44. doi:10.1023/B:VLSI.0000028532.53893.82 10.1023/B:VLSI.0000028532.53893.82
16
Seo D, Kim KC, Lee M, Kwon KD, Kim G (2021) Research on tomato flowers and fruits object detection model in greenhouse environment using deep learning J Korean Inst Commu Infor Sci 46:2072-2077. doi:10.7840/kics.2021.46.11.2072 10.7840/kics.2021.46.11.2072
17
Shaha M, Pawar M (2018) Transfer learning for image classification. II International Conference on Electronics 2018, Commun. Aerosp. Technol (ICECA), 2018: IEEE, pp 656-660. doi:10.1109/ICECA.2018.8474802 10.1109/ICECA.2018.8474802
18
Shin MH, Jang KE, Lee SK, Cho JG, Song SJ, Kim JG (2022) Grading of harvested 'Mihwang' peach maturity with convolutional neural network. J Bio-Envir Con 31:270-278. doi:10.12791/KSBEC.2022.31.4.270 10.12791/KSBEC.2022.31.4.270
19
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Cornell University, New York, USA. arXiv:1409.1556. doi:10.48550/arXiv.1409.1556 10.48550/arXiv.1409.1556
20
Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. IEEE Conference on Computer Vision and Pattern Recognition, MA, USA, pp 1-9. doi:10.1109/CVPR.2015.7298594 10.1109/CVPR.2015.7298594
21
Tan M, Le Q (2019) EfficientNet: Rethinking model scaling for convolutional neural networks. International Conference on Machine Learning. PMLR, California, USA, pp 6105-6114. doi:10.48550/arXiv.1905.11946 10.48550/arXiv.1905.11946
22
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. Advances in Neural Information Processing Systems 30, California, USA, doi:10.48550/arXiv.1706.03762 10.48550/arXiv.1706.03762
23
Wang CY, Liao HYM, Wu YH, Chen PY, Hsieh JW, Yeh IH (2020) CSPNet: A new backbone that can enhance learning capability of CNN. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, WA, USA, pp 390-391. doi:10.1109/CVPRW50498.2020.00203 10.1109/CVPRW50498.2020.00203
Information
  • Publisher :KOREAN SOCIETY FOR HORTICULTURAL SCIENCE
  • Publisher(Ko) :원예과학기술지
  • Journal Title :Horticultural Science and Technology
  • Journal Title(Ko) :원예과학기술지
  • Volume : 42
  • No :1
  • Pages :80-93
  • Received Date : 2023-03-30
  • Revised Date : 2023-05-10
  • Accepted Date : 2023-06-21
  • DOI :https://doi.org/10.7235/HORT.20240007
Journal Informaiton Horticultural Science and Technology Horticultural Science and Technology
Online Submission
  • scopus
  • NRF
  • KOFST
  • crossref crossmark
  • crossref cited-by
  • crosscheck
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close