Skip to main content
SpringerLink
Log in

A reinforced CenterNet scheme on position detection of acoustic levitated objects

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Position detection is essential for precise contactless manipulation as it can play an important role in analyzing the behavior patterns and motion regularities of levitated objects. However, traditional detection methods have several limitations, such as finite feature representation, low detection accuracy, and poor adaptability, notably for small targets. To address these issues, this paper proposes an effective detector called RFRM-CenterNet, which is the first attempt to detect the location of levitated objects based on a convolutional neural network. First, a receptive field refinement module consisting of multi-stage parallel dilated convolutional layers is designed to detect small levitated objects. Then, to increase the feature representation capability, a feature fusion network is designed, which employs a parallel fusion of ResNet50 and a receptive field refinement module. In addition, an experimental system is constructed, and a dataset is established to train the model and verify the performance of the proposed method. The experimental results show that RFRM-CenterNet has better performance in detecting levitated objects than other detection methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data availibility

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Brandt E (2001) Suspended by sound. Nature 413(6855):474–475

    Article  Google Scholar 

  2. Andrade MA, Marzo A, Adamowski JC (2020) Acoustic levitation in mid-air: recent advances, challenges, and future perspectives. Appl Phys Lett 116(25):250501

    Article  Google Scholar 

  3. Lu X, Twiefel J, Ma Z, Yu T, Wallaschek J, Fischer P (2021) Dynamic acoustic levitator based on subwavelength aperture control. Adv Sci 8(15):2100888

    Article  Google Scholar 

  4. Collins DJ, Ma Z, Han J, Ai Y (2017) Continuous micro-vortex-based nanoparticle manipulation via focused surface acoustic waves. Lab Chip 17(1):91–103

    Article  Google Scholar 

  5. Bruus H (2012) Acoustofluidics 7: the acoustic radiation force on small particles. Lab Chip 12(6):1014–1021

    Article  Google Scholar 

  6. Zang D, Yu Y, Chen Z, Li X, Wu H, Geng X (2017) Acoustic levitation of liquid drops: dynamics, manipulation and phase transitions. Adv Colloid Interface Sci 243:77–85

    Article  Google Scholar 

  7. Yang Y, Ma T, Zhang Q, Huang J, Hu Q, Li Y, Wang C, Zheng H (2022) 3d acoustic manipulation of living cells and organisms based on 2d array. IEEE Trans Biomed Eng. https://doi.org/10.1109/TBME.2022.3142774

    Article  Google Scholar 

  8. Foresti D, Nabavi M, Klingauf M, Ferrari A, Poulikakos D (2013) Acoustophoretic contactless transport and handling of matter in air. Proc Natl Acad Sci 110(31):12549–12554

    Article  Google Scholar 

  9. Laurell T, Petersson F, Nilsson A (2007) Chip integrated strategies for acoustic separation and manipulation of cells and particles. Chem Soc Rev 36(3):492–506

    Article  Google Scholar 

  10. Xu T, Xu L-P, Zhang X (2017) Ultrasound propulsion of micro-/nanomotors. Appl Mater Today 9:493–503

    Article  Google Scholar 

  11. Ahmed D, Ozcelik A, Bojanala N, Nama N, Upadhyay A, Chen Y, Hanna-Rose W, Huang TJ (2016) Rotational manipulation of single cells and organisms using acoustic waves. Nat Commun 7(1):1–11

    Article  Google Scholar 

  12. Andrade MA, Marzo A (2019) Numerical and experimental investigation of the stability of a drop in a single-axis acoustic levitator. Phys Fluids 31(11):117101

    Article  Google Scholar 

  13. Andrade MA, Polychronopoulos S, Memoli G, Marzo A (2019) Experimental investigation of the particle oscillation instability in a single-axis acoustic levitator. AIP Adv 9(3):035020

    Article  Google Scholar 

  14. Andrade MA, Buiochi F, Baer S, Esen C, Ostendorf A, Adamowski JC (2012) Experimental analysis of the particle oscillations in acoustic levitation. In: 2012 IEEE international ultrasonics symposium, pp 2006–2009. IEEE

  15. Watanabe A, Hasegawa K, Abe Y (2018) Contactless fluid manipulation in air: droplet coalescence and active mixing by acoustic levitation. Sci Rep 8(1):1–8

    Article  Google Scholar 

  16. Hasegawa K, Watanabe A, Abe Y (2019) Acoustic manipulation of droplets under reduced gravity. Sci Rep 9(1):1–8

    Article  Google Scholar 

  17. Hu Q, Ma T, Zhang Q, Wang J, Zheng H (2021) 3d acoustic tweezers using a 2d matrix array with time-multiplexed traps. IEEE Trans Ultrason Ferroelectr Freq Control 68(12):3646–3653

    Article  Google Scholar 

  18. Khalil IS, Mahdy D, El Sharkawy A, Moustafa RR, Tabak AF, Mitwally ME, Hesham S, Hamdi N, Klingner A, Mohamed A et al (2018) Mechanical rubbing of blood clots using helical robots under ultrasound guidance. IEEE Robot Autom Lett 3(2):1112–1119

    Article  Google Scholar 

  19. Andrade MA, Pérez N, Adamowski JC (2014) Experimental study of the oscillation of spheres in an acoustic levitator. J Acoust Soc Am 136(4):1518–1529

    Article  Google Scholar 

  20. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  21. Canayaz M, Şehribanoğlu S, Özdağ R, Demir M (2022) Covid-19 diagnosis on CT images with Bayes optimization-based deep neural networks and machine learning algorithms. Neural Comput Appl 34(7):5349–5365

    Article  Google Scholar 

  22. Liu J, Guo F, Zhang Y, Hou B, Zhou H (2022) Defect classification on limited labeled samples with multiscale feature fusion and semi-supervised learning. Appl Intell 52(7):8243–8258

    Article  Google Scholar 

  23. Baig MM, Awais MM, El-Alfy E-SM (2017) Adaboost-based artificial neural network learning. Neurocomputing 248:120–126

    Article  Google Scholar 

  24. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25

  25. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  26. Xu Y, Yu G, Wang Y, Wu X, Ma Y (2017) Car detection from low-altitude UAV imagery with the faster R-CNN. J Adv Transp. https://doi.org/10.1155/2017/2823617

    Article  Google Scholar 

  27. Shi T, Liu M, Niu Y, Yang Y, Huang Y (2020) Underwater targets detection and classification in complex scenes based on an improved yolov3 algorithm. J Electron Imaging 29(4):043013

    Article  Google Scholar 

  28. He D, Zou Z, Chen Y, Liu B, Yao X, Shan S (2021) Obstacle detection of rail transit based on deep learning. Measurement 176:109241

    Article  Google Scholar 

  29. Fan S, Zhu F, Chen S, Zhang H, Tian B, Lv Y, Wang F-Y (2021) Fii-centernet: an anchor-free detector with foreground attention for traffic object detection. IEEE Trans Veh Technol 70(1):121–132

    Article  Google Scholar 

  30. Zhou X, Wang D, Krähenbühl P (2019) Objects as points. arXiv preprint arXiv:1904.07850

  31. Hirayama R, Martinez Plasencia D, Masuda N, Subramanian S (2019) A volumetric display for visual, tactile and audio presentation using acoustic trapping. Nature 575(7782):320–323

    Article  Google Scholar 

  32. Marzo A, Barnes A, Drinkwater BW (2017) Tinylev: a multi-emitter single-axis acoustic levitator. Rev Sci Instrum 88(8):085105

    Article  Google Scholar 

  33. Fushimi T, Marzo A, Hill TL, Drinkwater BW (2018) Trajectory optimization of levitated particles in mid-air ultrasonic standing wave levitators. In: 2018 IEEE international ultrasonics symposium (IUS)

  34. Kremer AJ, Kilzer Petermann M (2018) Simultaneous measurement of surface tension and viscosity using freely decaying oscillations of acoustically levitated droplets. Rev Sci Instrum 89:015109

    Article  Google Scholar 

  35. Ju M, Luo J, Liu G, Luo H (2021) Istdet: an efficient end-to-end neural network for infrared small target detection. Infrared Phys Technol 114:103659

    Article  Google Scholar 

  36. Guo H, Yang X, Wang N, Gao X (2021) A CenterNet++ model for ship detection in SAR images. Pattern Recogn 112:107787

    Article  Google Scholar 

  37. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  38. Jiang Z, Ma Z, Wang Y, Shao X, Yu K, Jolfaei A (2021) Aggregated decentralized down-sampling-based ResNet for smart healthcare systems. Neural Comput Appl 1–13

  39. Strisciuglio N, Lopez-Antequera M, Petkov N (2020) Enhanced robustness of convolutional networks with a push-pull inhibition layer. Neural Comput Appl 32(24):17957–17971

    Article  Google Scholar 

  40. Dai Z, Yi J, Zhang Y, Zhou B, He L (2020) Fast and accurate cable detection using CNN. Appl Intell 50(12):4688–4707

    Article  Google Scholar 

  41. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122

  42. Law H, Deng J (2018) Cornernet: detecting objects as paired keypoints. In: Proceedings of the European conference on computer vision (ECCV), pp 734–750

  43. Chen C, Liu M-Y, Tuzel O, Xiao J (2016) R-CNN for small object detection. In: Asian conference on computer vision. Springer, pp 214–230

  44. Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1–48

    Article  Google Scholar 

  45. Sergey I, Christian S (2021) Batch normalization: accelerating deep network training by reducing internal covariate shift. arxiv 2015. arXiv preprint arXiv:1502.03167

  46. Jocher G, Stoken A, Borovec J, Chaurasia A, Changyu L, Laughing A, Hogan A, Hajek J, Diaconu L, Marc Y et al (2021) ultralytics/yolov5: v5. 0-yolov5-p6 1280 models aws supervise. ly and youtube integrations. Zenodo 11 (2021)

  47. Wang C-Y, Bochkovskiy A, Liao HYM (2022) Yolov7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696

  48. Marzo A, Seah SA, Drinkwater BW, Sahoo DR, Long B, Subramanian S (2015) Holographic acoustic elements for manipulation of levitated objects. Nat Commun 6(1):1–7

    Article  Google Scholar 

  49. Marzo A, Drinkwater BW (2019) Holographic acoustic tweezers. Proc Natl Acad Sci 116(1):84–89

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Xinbo Li, Ziyi Chen and Shuyuan Fan have contributed equally.

Authors and Affiliations

  1. College of Communication Engineering, Jilin University, 5372 Nanhu Rd, Changchun, 130022, Jilin, China

    Xinbo Li, Yingwei Wang, Liangxu Jiang, Ziyi Chen & Shuyuan Fan

Authors
  1. Xinbo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yingwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liangxu Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ziyi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shuyuan Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yingwei Wang or Liangxu Jiang.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (mp4 4066 KB)

Supplementary file 2 (mp4 3782 KB)

Supplementary file 3 (mp4 3395 KB)

Supplementary file 4 (mp4 2335 KB)

Supplementary file 5 (mp4 3389 KB)

Supplementary file 6 (mp4 6164 KB)

Supplementary file 7 (pdf 720 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Wang, Y., Jiang, L. et al. A reinforced CenterNet scheme on position detection of acoustic levitated objects. Neural Comput & Applic 35, 8987–9002 (2023). https://doi.org/10.1007/s00521-022-08140-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-08140-1

Keywords

Search

Navigation