Skip to main content
SpringerLink
Log in

Learning needle tip localization from digital subtraction in 2D ultrasound

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

This paper addresses localization of needles inserted both in-plane and out-of-plane in challenging ultrasound-guided interventions where the shaft and tip have low intensity. Our approach combines a novel digital subtraction scheme for enhancement of low-level intensity changes caused by tip movement in the ultrasound image and a state-of-the-art deep learning scheme for tip detection.

Methods

As the needle tip moves through tissue, it causes subtle spatiotemporal variations in intensity. Relying on these intensity changes, we formulate a foreground detection scheme for enhancing the tip from consecutive ultrasound frames. The tip is augmented by solving a spatial total variation regularization problem using the split Bregman method. Lastly, we filter irrelevant motion events with a deep learning-based end-to-end data-driven method that models the appearance of the needle tip in ultrasound images, resulting in needle tip detection.

Results

The detection model is trained and evaluated on an extensive ex vivo dataset collected with 17G and 22G needles inserted in-plane and out-of-plane in bovine, porcine and chicken phantoms. We use 5000 images extracted from 20 video sequences for training and 1000 images from 10 sequences for validation. The overall framework is evaluated on 700 images from 20 sequences not used in training and validation, and achieves a tip localization error of 0.72 ± 0.04 mm and an overall processing time of 0.094 s per frame (~ 10 frames per second).

Conclusion

The proposed method is faster and more accurate than state of the art and is resilient to spatiotemporal redundancies. The promising results demonstrate its potential for accurate needle localization in challenging ultrasound-guided interventions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Elsharkawy H, Babazade R, Kolli S, Kalagara H, Soliman ML (2016) The Infiniti plus ultrasound needle guidance system improves needle visualization during the placement of spinal anesthesia. Korean J Anesthesiol 69(4):417–419

    Article  PubMed  PubMed Central  Google Scholar 

  2. Lu H, Li J, Lu Q, Bharat S, Erkamp R, Chen B, Drysdale J, Vignon F, Jain A (2014) A new sensor technology for 2D ultrasound-guided needle tracking. MICCAI 17(Pt. 2):389–396

    PubMed  Google Scholar 

  3. Xia W, West S, Finlay M, Mari J, Ourselin S, David A, Desjardins A (2017) Looking beyond the imaging plane: 3D needle tracking with a linear array ultrasound probe. Sci Rep 7(1):3674

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Miura M, Takeyama K, Suzuki T (2014) Visibility of ultrasound-guided echogenic needle and its potential in clinical delivery of regional anesthesia. Tokai J Exp Clin Med 39(2):80–86

    PubMed  Google Scholar 

  5. Arif M, Moelker A, van Walsum T (2018) Needle Tip Visibility in 3D Ultrasound Images. Cardiovasc Interv Radiol 41(1):145–152

    Article  Google Scholar 

  6. Fevre MC, Vincent C, Picard J, Vighetti A, Chapuis C, Detavernier M, Allenet B, Payen JF, Bosson JL, Albaladejo P (2018) Reduced variability and execution time to reach a target with a needle GPS system: comparison between physicians, residents and nurse anaesthetists. Anaesth Crit Care Pain Med 37(1):55–60

    Article  PubMed  Google Scholar 

  7. Stolka PJ, Foroughi P, Rendina M, Weiss CR, Hager GD, Boctor EM (2014) Needle guidance using handheld stereo vision and projection for ultrasound-based interventions. MICCAI 17(Pt.2):684–691

    PubMed  Google Scholar 

  8. Priester AM, Natarajan S, Culjat MO (2013) Robotic ultrasound systems in medicine. IEEE EEE Trans Ultrason Ferroelectr Freq Control 60:507–523

    Article  Google Scholar 

  9. Ayvali E, Desai J (2014) Optical flow-based tracking of needles and needle-tip localization using circular hough transform in ultrasound images. Ann Biomed Eng 43(8):1828–1840

    Article  PubMed  PubMed Central  Google Scholar 

  10. Zhao Y, Cachard C, Liebgott H (2013) Automatic needle detection and tracking in 3D ultrasound using an ROI-based RANSAC and Kalman method. Ultrason Imaging 35(4):283–306

    Article  PubMed  Google Scholar 

  11. Hacihaliloglu I, Beigi P, Ng G, Rohling RN, Salcudean S, Abolmaesumi P (2015) Projection-based phase features for localization of a needle Tip in 2D curvilinear ultrasound. MICCAI 9349:347–354

    Google Scholar 

  12. Hatt CR, Ng G, Parthasarathy V (2015) Enhanced needle localization in ultrasound using beam steering and learning-based segmentation. Comput Med Imaging Graph 41:46–54

    Article  PubMed  Google Scholar 

  13. Mwikirize C, Nosher JL, Hacihaliloglu I (2018) Signal attenuation maps for needle enhancement and localization in 2D ultrasound. Int J CARS 13(3):363–374

    Article  Google Scholar 

  14. Beigi P, Rohling R, Salcudean S, Ng G (2017) CASPER: computer-aided segmentation of imperceptible motion-a learning-based tracking of an invisible needle in ultrasound. Int J CARS 12(11):1857–1866

    Article  Google Scholar 

  15. Beigi P, Rohling R, Salcudean SE, Ng GC (2016) Spectral analysis of the tremor motion for needle detection in curvilinear ultrasound via spatiotemporal linear sampling. Int J CARS 11(6):1183–1192

    Article  Google Scholar 

  16. Mwikirize C, Nosher JL, Hacihaliloglu I (2018) Convolution neural networks for real-time needle detection and localization in 2D ultrasound. Int J CARS 13(5):647–657

    Article  Google Scholar 

  17. Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng G, Korsten H, With P (2017) Improving needle detection in 3D ultrasound using orthogonal-plane convolutional networks. MICCAI 2:610–618

    Google Scholar 

  18. Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng G, Korsten H, With P (2018) Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks. Int J CARS 13(9):1321–1333

    Article  Google Scholar 

  19. Redmon J, Farhadi A (2016) Yolo9000: better, faster, stronger. arXiv:1612.08242

  20. Afonso M, Bioucas-Dias J, Figueiredo M (2010) Fast image recovery using variable splitting and constrained optimization. IEEE Trans Image Process 19(9):2345–2356

    Article  PubMed  Google Scholar 

  21. Chan S, Khoshabeh R, Gibson K, Gill P, Nguyen T (2011) An augmented lagrangian method for total variation video restoration. IEEE Trans Image Process 20(11):3097–3111

    Article  PubMed  Google Scholar 

  22. Goldstein T, Osher S (2009) The split Bregman method for L1-regularized problems. SIAM J Imaging Sci 2(2):323–343

    Article  Google Scholar 

  23. Fong D, Saunders M (2011) LSMR: an iterative algorithm for sparse least-squares problems. SIAM J Sci Comput 33(5):2950–2971

    Article  Google Scholar 

  24. Klambauer G, Unterthiner T, Mayr A, Hochreiter S (2017) Self-normalizing neural networks. arXiv:1706.02515v5

  25. Everingham M, Gool LV, Williams CKI, Winn J, Zisserman A (2010) The PASCAL visual object classes (VOC) challenge. Int J Comput Vis 88:303–338

    Article  Google Scholar 

Download references

Acknowledgements

This work was accomplished with funding support from the North American Spine Society 2017 young investigator award.

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, Rutgers University, Piscataway, NJ, 08854, USA

    Cosmas Mwikirize & Ilker Hacihaliloglu

  2. Department of Radiology, Rutgers Robert Wood Johnson Medical School, New Brunswick, NJ, 08901, USA

    John L. Nosher & Ilker Hacihaliloglu

Authors
  1. Cosmas Mwikirize
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John L. Nosher
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ilker Hacihaliloglu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cosmas Mwikirize.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This article does not contain patient data.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (MP4 51745 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mwikirize, C., Nosher, J.L. & Hacihaliloglu, I. Learning needle tip localization from digital subtraction in 2D ultrasound. Int J CARS 14, 1017–1026 (2019). https://doi.org/10.1007/s11548-019-01951-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-019-01951-z

Keywords

Search

Navigation