Skip to main content

Advertisement

SpringerLink
Log in

A shadow detection and removal method for fruit recognition in natural environments

  • Published:
Precision Agriculture Aims and scope Submit manuscript

Abstract

Effective shadow detection and shadow removal can improve the performance of fruit recognition in natural environments and provide technical support for agricultural intelligence. In this study, a superpixel segmentation method was used to divide an image into multiple small regions. Based on the superpixel segmentation results, the shadow regions and the shadowless regions of the orchard images under natural light were compared and studied. Seven shadow saliency features (SSF) were explored and analyzed for shadow detection. The SSF were used to enhance the shadow characteristics. Then, the genetic algorithm (GA) was used to optimize the parameters, and support vector machine recursive feature elimination (SVM-RFE) was used to determine the best feature combination for shadow detection. According to the best feature combination, the support vector machine (SVM) algorithm was used to determine whether each segment of the superpixel segmentation results belonged to the shadow region. Shadow removal was carried out on each detected shadow region, and a natural light image after shadow removal was obtained. Finally, the accuracy of shadow detection was tested. The experimental results showed that the average accuracy of the shadow detection algorithm in this study was 91.91%. As a result, the precision and recall for fruits recognition after shadow removal generally improved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Adankon, M. M., & Cheriet, M. (2009). Support vector machine. In S. Z. Li & A. Jain (Eds.), Encyclopedia of biometrics (pp. 1303–1308). Boston, MA, USA: Springer.

    Chapter  Google Scholar 

  • Baba, M., Mukunoki, M., & Asada, N. (2004). Shadow removal from a real image based on shadow density. In Proceedings of the ACM SIGGRAPH (pp. 8–12). New York, USA: ACM.

  • Chang, C.-C., & Lin, C.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology,2(3), 1–27. https://doi.org/10.1145/1961189.1961199.

    Article  Google Scholar 

  • Comaniciu, D., & Meer, P. (2002). Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence,24(5), 603–619. https://doi.org/10.1109/34.1000236.

    Article  Google Scholar 

  • Finlayson, G. D., Drew, M. S., & Lu, C. (2004). Intrinsic images by entropy minimization. In T. Pajdla & J. Matas (Eds.), Computer vision (ECCV) (pp. 582–595). Berlin, Heidelberg, Germany: Springer.

    Google Scholar 

  • Finlayson, G. D., Drew, M. S., & Lu, C. (2009). Entropy minimization for shadow removal. International Journal of Computer Vision,85(1), 35–57. https://doi.org/10.1007/s11263-009-0243-z.

    Article  Google Scholar 

  • Finlayson, G., Fredembach, C., & Drew, M. S. (2007). Detecting illumination in images. In 2007 IEEE 11th international conference on computer vision (ICCV) (pp. 1–8). Los Alamitos, CA, USA: IEEE.

  • Finlayson, G. D., Hordley, S. D., & Drew, M. S. (2002). Removing shadows from images. In A. Heyden, G. Sparr, M. Nielsen, & P. Johansen (Eds.), Computer vision (ECCV) (pp. 823–836). Berlin, Heidelberg, Germany: Springer.

    Google Scholar 

  • Finlayson, G. D., Hordley, S. D., Lu, C., & Drew, M. S. (2006). On the removal of shadows from images. IEEE Transactions on Pattern Analysis and Machine Intelligence,28(1), 59–68. https://doi.org/10.1109/tpami.2006.18.

    Article  PubMed  Google Scholar 

  • Freeman, W. T., Pasztor, E. C., & Carmichael, O. T. (2000). Learning low-level vision. International Journal of Computer Vision,40(1), 25–47. https://doi.org/10.1023/A:1026501619075.

    Article  Google Scholar 

  • Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc.

    Google Scholar 

  • Gong, H., & Cosker, D. (2017). User-assisted image shadow removal. Image and Vision Computing,62, 19–27. https://doi.org/10.1016/j.imavis.2017.04.001.

    Article  Google Scholar 

  • Gonzalez, R. C., & Woods, R. E. (2007). Digital image processing (3rd ed.). Upper Saddle River, NJ, USA: Prentice-Hall Inc.

    Google Scholar 

  • Gonzalez, R. C., Woods, R. E., & Eddins, S. L. (2003). Digital image processing using MATLAB. Upper Saddle River, NJ, USA: Prentice-Hall Inc.

    Google Scholar 

  • Guo, R. Q., Dai, Q. Y., & Hoiem, D. (2013). Paired regions for shadow detection and removal. IEEE Transactions on Pattern Analysis and Machine Intelligence,35(12), 2956–2967. https://doi.org/10.1109/tpami.2012.214.

    Article  PubMed  Google Scholar 

  • Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning,46(1–3), 389–422. https://doi.org/10.1023/a:1012487302797.

    Article  Google Scholar 

  • Hertog, W., Llenas, A., & Carreras, J. (2015). Optimizing indoor illumination quality and energy efficiency using a spectrally tunable lighting system to augment natural daylight. Optics Express,23(24), 1564–1574. https://doi.org/10.1364/oe.23.01564.

    Article  CAS  Google Scholar 

  • Jobson, D. J., Rahman, Z., & Woodell, G. A. (1997). A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing,6(7), 965–976. https://doi.org/10.1109/83.597272.

    Article  CAS  PubMed  Google Scholar 

  • Khan, S. H., Bennamoun, M., Sohel, F., & Togneri, R. (2016). Automatic shadow detection and removal from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence,38(3), 431–446. https://doi.org/10.1109/tpami.2015.2462355.

    Article  PubMed  Google Scholar 

  • Lalonde, J. F., Efros, A. A., & Narasimhan, S. G. (2010). Detecting ground shadows in outdoor consumer photographs. In K. Daniilidis, P. Maragos, & N. Paragios (Eds.), Computer vision (ECCV) (Vol. 6312, Lecture Notes in Computer Science, pp. 322–335). Berlin, Heidelberg, Germany: Springer.

  • Lei, T., Jia, X. H., Zhang, Y. N., He, L. F., Meng, H. Y., & Nandi, A. K. (2018). Significantly fast and robust fuzzy c-means clustering algorithm based on morphological reconstruction and membership filtering. IEEE Transactions on Fuzzy Systems,26(5), 3027–3041. https://doi.org/10.1109/tfuzz.2018.2796074.

    Article  Google Scholar 

  • Levine, M. D., & Bhattacharyya, J. (2005). Removing shadows. Pattern Recognition Letters,26(3), 251–265. https://doi.org/10.1016/j.patrec.2004.10.021.

    Article  Google Scholar 

  • MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth berkeley symposium on mathematical statistics and probability (pp. 281–297). Berkeley, CA, USA: University of California Press.

  • Matsushita, Y., Nishino, K., Ikeuchi, K., & Sakauchi, M. (2004). Illumination normalization with time-dependent intrinsic images for video surveillance. IEEE Transactions on Pattern Analysis and Machine Intelligence,26(10), 1336–1347. https://doi.org/10.1109/tpami.2004.86.

    Article  PubMed  Google Scholar 

  • Mo, N., Zhu, R. X., Yan, L., & Zhao, Z. (2018). Deshadowing of urban airborne imagery based on object-oriented automatic shadow detection and regional matching compensation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,11(2), 585–605. https://doi.org/10.1109/jstars.2017.2787116.

    Article  Google Scholar 

  • Otsu, N. (1979). Threshold selection method from gray-level histograms. IEEE Transactions on Systems Man and Cybernetics,9(1), 62–66. https://doi.org/10.1109/tsmc.1979.4310076.

    Article  Google Scholar 

  • Phong, B. T. (1975). Illumination for computer generated pictures. Communications of the ACM,18(6), 311–317. https://doi.org/10.1145/360825.360839.

    Article  Google Scholar 

  • Shen, L., & Yeo, C. (2011). Intrinsic images decomposition using a local and global sparse representation of reflectance. In 2011 IEEE conference on computer vision and pattern recognition (CVPR) (pp. 697–704). Los Alamitos, CA, USA: IEEE.

  • Shen, L., Yeo, C., & Hua, B.-S. (2013). Intrinsic image decomposition using a sparse representation of reflectance. IEEE Transactions on Pattern Analysis and Machine Intelligence,35(12), 2904–2915. https://doi.org/10.1109/tpami.2013.136.

    Article  PubMed  Google Scholar 

  • Suh, H. K., Hofstee, J. W., & van Henten, E. J. (2018). Improved vegetation segmentation with ground shadow removal using an HDR camera. Precision Agriculture,19(2), 218–237. https://doi.org/10.1007/s11119-017-9511-z.

    Article  Google Scholar 

  • Sun, J., Tian, J. D., Du, Y. K., & Tang, Y. D. (2009). Retinex theory-based shadow detection and removal in single outdoor image. Industrial Robot: An International Journal,36(3), 263–269. https://doi.org/10.1108/01439910910950531.

    Article  Google Scholar 

  • Wang, G., Wei, Y., & Qiao, S. (2018). Equation solving generalized inverses. In G. Wang, Y. Wei, & S. Qiao (Eds.), Generalized inverses: Theory and computations (pp. 1–64). Singapore: Springer.

    Chapter  Google Scholar 

  • Wang, S.-T., Yuan, Y.-Y., Zhu, C.-Y., Kong, D.-M., & Wang, Y.-T. (2019). Discrimination of polycyclic aromatic hydrocarbons based on fluorescence spectrometry coupled with CS-SVM. Measurement,139, 475–481. https://doi.org/10.1016/j.measurement.2019.01.087.

    Article  Google Scholar 

  • Weiss, Y. (2001). Deriving intrinsic images from image sequences. In Proceedings eighth IEEE international conference on computer vision (ICCV 2001) (pp. 68–75). Los Alamitos, CA, USA: IEEE. https://doi.org/10.1109/iccv.2001.937606.

  • Wu, T. P., & Tang, C. K. (2005). A Bayesian approach for shadow extraction from a single image. In Tenth IEEE international conference on computer vision (ICCV) (pp. 480–487). Los Alamitos, CA, USA: IEEE.

  • Zhu, J. J., Samuel, K. G. G., Masood, S. Z., & Tappen, M. F. (2010). Learning to recognize shadows in monochromatic natural images. In 2010 IEEE conference on computer vision and pattern recognition (pp. 223–230). Los Alamitos, CA, USA: IEEE.

Download references

Acknowledgements

This work is supported by the Natural Science Foundation of Guangdong (No. 2018A030313330), the Special Funds for the Cultivation of Guangdong College Students’ Scientific and Technological Innovation (“Climbing Program” Special Funds.) (pdjh2019a0073), the National Natural Science Foundation of China (Nos. 31201135, 31571568) and the Science and Technology Plan Project of Guangzhou (201802020032). The authors wish to thank the useful comments of the anonymous reviewers to this paper.

Author information

Authors and Affiliations

  1. College of Mathematics and Informatics, South China Agricultural University, Guangzhou, 510642, China

    Rongbin Bu, Juntao Xiong, Shumian Chen, Zhenhui Zheng, Wentao Guo, Zhengang Yang & Xiaoyun Lin

Authors
  1. Rongbin Bu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juntao Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shumian Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhenhui Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wentao Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhengang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xiaoyun Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

BR and XJ conceived and designed the experiments; BR designed the algorithm; BR, CS and ZZ performed the experiments and analyzed the data; BR, XJ, CS, ZZ, GW, YZ and LX wrote the paper.

Corresponding author

Correspondence to Juntao Xiong.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bu, R., Xiong, J., Chen, S. et al. A shadow detection and removal method for fruit recognition in natural environments. Precision Agric 21, 782–801 (2020). https://doi.org/10.1007/s11119-019-09695-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11119-019-09695-1

Keywords

Search

Navigation