Skip to main content

Advertisement

SpringerLink
Log in

Non-rigid registration of 3D ultrasound for neurosurgery using automatic feature detection and matching

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

The brain undergoes significant structural change over the course of neurosurgery, including highly nonlinear deformation and resection. It can be informative to recover the spatial mapping between structures identified in preoperative surgical planning and the intraoperative state of the brain. We present a novel feature-based method for achieving robust, fully automatic deformable registration of intraoperative neurosurgical ultrasound images.

Methods

A sparse set of local image feature correspondences is first estimated between ultrasound image pairs, after which rigid, affine and thin-plate spline models are used to estimate dense mappings throughout the image. Correspondences are derived from 3D features, distinctive generic image patterns that are automatically extracted from 3D ultrasound images and characterized in terms of their geometry (i.e., location, scale, and orientation) and a descriptor of local image appearance. Feature correspondences between ultrasound images are achieved based on a nearest-neighbor descriptor matching and probabilistic voting model similar to the Hough transform.

Results

Experiments demonstrate our method on intraoperative ultrasound images acquired before and after opening of the dura mater, during resection and after resection in nine clinical cases. A total of 1620 automatically extracted 3D feature correspondences were manually validated by eleven experts and used to guide the registration. Then, using manually labeled corresponding landmarks in the pre- and post-resection ultrasound images, we show that our feature-based registration reduces the mean target registration error from an initial value of 3.3 to 1.5 mm.

Conclusions

This result demonstrates that the 3D features promise to offer a robust and accurate solution for 3D ultrasound registration and to correct for brain shift in image-guided neurosurgery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Bucholz RD, Smith KR, Laycock KA, McDurmont LL (2001) Three-dimensional localization: from image-guided surgery to information-guided therapy. Methods 25(2):186–200

    Article  CAS  PubMed  Google Scholar 

  2. Hill DG, Maurer CR, Maciunas RJ, Barwise JA, Fitzpatrick JM, Wang MY (1998) Measurement of intraoperative brain surface deformation under a craniotomy. Neurosurgery 43(3):514–528

    Article  CAS  PubMed  Google Scholar 

  3. Roberts D, Hartov A, Kennedy F, Miga M, Paulsen K (1998) Intraoperative brain shift and deformation: a quantitative analysis of cortical displacement in 28 cases. Neurosurgery 43:749–758

    Article  CAS  PubMed  Google Scholar 

  4. Letteboer MMJ, Willems PW, Viergever MA, Niessen WJ (2005) Brain shift estimation in image-guided neurosurgery using 3-D ultrasound. IEEE Trans Biomed Eng 52(2):268–276

    Article  PubMed  Google Scholar 

  5. Audette MA, Siddiqi K, Ferrie FP, Peters TM (2003) An integrated range-sensing, segmentation and registration framework for the characterization of intra-surgical brain deformations in image-guided surgery. Comput Vis Image Underst 89(2–3):226–251

    Article  Google Scholar 

  6. Marko NF, Weil RJ, Schroeder JL, Lang FF, Suki D, Sawaya RE (2014) Extent of resection of glioblastoma revisited: personalized survival modeling facilitates more accurate survival prediction and supports a maximum-safe-resection approach to surgery. J Clin Oncol 32(8):774

    Article  PubMed  PubMed Central  Google Scholar 

  7. Coburger J, Merkel A, Scherer M, Schwartz F, Gessler F, Roder C, Jungk C (2015) Low-grade glioma surgery in intraoperative magnetic resonance imaging: results of a multicenter retrospective assessment of the German Study Group for Intraoperative Magnetic Resonance Imaging. Neurosurgery 78(6):775–786

    Article  Google Scholar 

  8. Brown TJ, Brennan MC, Li M, Church EW, Brandmeir NJ, Rakszawski KL, Glantz M (2016) Association of the extent of resection with survival in glioblastoma: a systematic review and meta-analysis. JAMA Oncol 2(11):1460–1469

    Article  PubMed  PubMed Central  Google Scholar 

  9. Almeida JP, Chaichana KL, Rincon-Torroella J, Quinones-Hinojosa A (2015) The value of extent of resection of glioblastomas: clinical evidence and current approach. Curr Neurol Neurosci Rep 15(2):517

    Article  PubMed  Google Scholar 

  10. Hatiboglu MA, Weinberg JS, Suki D, Rao G, Prabhu SS, Shah K, Jackson E, Sawaya R (2009) Impact of intra-operative high-field magnetic resonance imaging guidance on glioma surgery: a prospective volumetric study. Neurosurgery 64:1073–1081

    Article  PubMed  Google Scholar 

  11. Claus EB, Horlacher A, Hsu L, Schwartz RB, Dello-Iacono D, Talos F, Jolesz FA, Black PM (2005) Survival rates in patients with low-grade glioma after intra-operative magnetic resonance image guidance. Cancer 103:1227–1233

    Article  PubMed  Google Scholar 

  12. Nabavi A, Black PM, Gering DT, Westin CF, Mehta V, Pergolizzi RS Jr, Ferrant M, Warfield SK, Hata N, Schwartz RB, Wells WM, Kikinis R, Jolesz F (2001) Serial intra-operative magnetic resonance imaging of brain shift. Neurosurgery 48:787–797

    CAS  PubMed  Google Scholar 

  13. Kuhnt D, Bauer MH, Nimsky C (2012) Brain shift compensation and neurosurgical image fusion using intraoperative MRI: current status and future challenges. Crit Rev Biomed Eng 40:175–185

    Article  PubMed  Google Scholar 

  14. Tirakotai D, Miller S, Heinze L, Benes L, Bertalanffy H, Sure U (2006) A novel platform for image-guided ultrasound. Neurosurgery 58:710–718

    Article  PubMed  Google Scholar 

  15. Zhou H, Rivaz H (2016) Registration of pre-and postresection ultrasound volumes with noncorresponding regions in neurosurgery. IEEE J Biomed Health Inform 20(5):1240–1249

    Article  PubMed  Google Scholar 

  16. Rivaz H, Collins DL (2015) Near real-time robust non-rigid registration of volumetric ultrasound images for neurosurgery. Ultrasound Med Biol 41(2):574–587

    Article  PubMed  Google Scholar 

  17. Mercier L, Araujo D, Haegelen C, Del Maestro RF, Petrecca K, Collins DL (2013) Registering pre- and post-resection 3-dimensional ultrasound for improved visualization of residual brain tumor. Ultrasound Med Biol 39(1):16–29

    Article  PubMed  Google Scholar 

  18. Gerard IJ, Kersten-Oertel M, Petrecca K, Sirhan D, Hall JA, Collins DL (2017) Brain shift in neuronavigation of brain tumors: a review. Med Image Anal 35:403–420

    Article  PubMed  Google Scholar 

  19. Pheiffer TS, Thompson RC, Rucker DC, Simpson AL, Miga MI (2014) Model-based correction of tissue compression for tracked ultrasound in soft tissue image-guided surgery. Ultrasound Med Biol 40(4):788–803

    Article  PubMed  PubMed Central  Google Scholar 

  20. Morin F, Chabanas M, Courtecuisse H, Payan Y (2017) Biomechanical modeling of brain soft tissues for medical applications. Biomechanics of living organs. Academic Press, Cambridge, pp 127–146

    Google Scholar 

  21. Morin F, Courtecuisse H, Reinertsen I, Lann FL, Palombi O, Payan Y, Chabanas M (2017) Brain-shift compensation using intraoperative ultrasound and constraint-based biomechanical simulation. Med Image Anal 40:133–153 ISSN 1361-8415

    Article  PubMed  Google Scholar 

  22. Luo M, Frisken SF, Weis JA, Clements LW, Unadkat P, Thompson RC, Golby AJ, Miga MI (2017) Validation of model-based brain shift correction in neurosurgery via intraoperative magnetic resonance imaging: preliminary results. In: Proceedings of SPIE 10135, medical imaging: image-guided procedures, robotic interventions, and modeling

  23. Rivaz H, Collins DL (2015) Deformable registration of preoperative MR, pre-resection ultrasound, and post-resection ultrasound images of neurosurgery. Int J Comput Assist Radiol Surg 10(7):1017–1028

    Article  PubMed  Google Scholar 

  24. Blumenthal T, Hartov A, Lunn K, Kennedy FE, Roberts DW, Paulsen KD (2005) Quantifying brain shift during neurosurgery using spatially tracked ultrasound. In: Proceedings of SPIE 5744, medical imaging: visualization, image-guided procedures, and display

  25. Mercier L, Fonov V, Haegelen C, Del Maestro RF, Petrecca K, Collins DL (2012) Comparing two approaches to rigid registration of three-dimensional ultrasound and magnetic resonance images for neurosurgery. Int J Comput Assist Radiol Surg 7(1):125–136

    Article  PubMed  Google Scholar 

  26. Poon T, Rohling R (2006) Three-dimensional extended field-of-view ultrasound. Ultrasound Med Biol 32:357–369

    Article  PubMed  Google Scholar 

  27. Schers J, Troccaz J, Daanen V, Fouard C, Plaskos C, Kilian P (2007) 3D/4D ultrasound registration of bone. In: Proceedings of the IEEE ultrasonics symposium, pp 2519–2522

  28. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE TPAMI 24(5):603–619

    Article  Google Scholar 

  29. Grimson WEL, Lozano-Perez T (1987) Localizing overlapping parts by searching the interpretive tree. IEEE TPAMI 9(4):469–482

    Article  CAS  Google Scholar 

  30. Beis JS, Lowe DG (1997) Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In: CVPR, pp 1000–1006

  31. Biederman I (1987) Recognition-by-components: a theory of human image understanding. Psychol Rev 2(94):115–147

    Article  Google Scholar 

  32. Pratikakis I (2003) Robust multiscale deformable registration of 3D ultrasound images. Int J Image Graph 3:547–565

    Article  Google Scholar 

  33. Cen F, Jiang Y, Zhang Z, Tsui HT, Lau TK, Xie H (2004) Robust registration of 3D-ultrasound images based on gabor filter and mean-shift. In: Method, pp 304–316

  34. Schneider RJ, Perrin DP, Vasilyev NV, Marx GR, Pedro J, Howe RD (2012) Real-time image-based rigid registration of three-dimensional ultrasound. Med Image Anal 16:402–414

    Article  PubMed  Google Scholar 

  35. Toews M, Wells WM III (2013) Efficient and robust model-to-image registration using 3D scale-invariant features. Med Image Anal 17(3):271–282

    Article  PubMed  Google Scholar 

  36. Ni D, Qu Y, Yang X, Chui YP, Wong TT, Ho SS, Heng PA (2008) Volumetric ultrasound panorama based on 3D SIFT. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 52-60

    Chapter  Google Scholar 

  37. Bersvendsen J, Toews M, Danudibroto A, Wells WM, Urheim S, Estépar RSJ, Samset E (2016) Robust spatio-temporal registration of 4D cardiac ultrasound sequences. In: Medical imaging 2016: ultrasonic imaging and tomography, vol. 9790. International society for optics and photonics, p 97900F

  38. Kadir T, Brady M (2001) Saliency, scale and image description. Int J Comput Vis 45(2):83–105

    Article  Google Scholar 

  39. Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell 27(10):1615–1630

    Article  PubMed  Google Scholar 

  40. Toews M, Wells WM (2009) Sift-rank: ordinal description for invariant feature correspondence. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009, pp 172–177

  41. Jenkinson M, Smith S (2001) A global optimisation method for robust affine registration of brain images. Med Image Anal 5(2):143–156

    Article  CAS  PubMed  Google Scholar 

  42. Bookstein FL (1989) Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans Pattern Anal Mach Intell 11(6):567–585

    Article  Google Scholar 

  43. Rohr K, Stiehl HS, Sprengel R, Beil W, Buzug TM, Weese J, Kuhn MH (1996) Point-based elastic registration of medical image data using approximating thin-plate splines. In: Visualization in biomedical computing. Springer, Berlin, pp 297–306

    Google Scholar 

  44. Tempany C, Jayender J, Kapur T, Bueno R, Golby A, Agar N, Jolesz FA (2015) Multimodal imaging for improved diagnosis and treatment of cancers. Cancer 121(6):817–827

    Article  PubMed  Google Scholar 

  45. Strong EB, Rafii A, Holhweg-Majert B, Fuller SC, Metzger MC (2008) Comparison of 3 optical navigation systems for computer-aided maxillofacial surgery. Arch Otolaryngol Head Neck Surg 134(10):1080–1084

    Article  PubMed  Google Scholar 

  46. Tokuda J, Fischer GS, Papademetris X, Yaniv Z, Ibanez L, Cheng P, Liu H, Blevins J, Arata J, Golby AJ, Kapur T, Pieper S, Burdette EC, Fichtinger G, Tempany CM, Hata N (2009) OpenIGTLink: an open network protocol for image-guided therapy environment. Int J Med Robot 5(4):423–34

    Article  PubMed  PubMed Central  Google Scholar 

  47. Lasso A, Heffter T, Rankin A, Pinter C, Ungi T, Fichtinger G (2014) PLUS: open-source toolkit for ultrasound-guided intervention systems. IEEE Trans Biomed Eng 61(10):2527–2537

    Article  PubMed  PubMed Central  Google Scholar 

  48. Gobbi DG, Peters TM (2002) Interactive intra-operative 3D ultrasound reconstruction and visualization. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 156–163

    Chapter  Google Scholar 

  49. Boisvert J, Gobbi D, Vikal S, Rohling R, Fichtinger G, Abolmaesumi P (2008) An open-source solution for interactive acquisition, processing and transfer of interventional ultrasound images. In: The MIDAS journal-systems and architectures for computer assisted interventions, p 70

  50. Selbekk T, Jakola AS, Solheim O, Johansen TF, Lindseth F, Reinertsen I, Unsgård G (2013) Ultrasound imaging in neurosurgery: approaches to minimize surgically induced image artefacts for improved resection control. Acta Neurochir 155(6):973–980. https://doi.org/10.1007/s00701-013-1647-7

    Article  PubMed  Google Scholar 

  51. Kikinis R, Pieper SD, Vosburgh K (2014) 3D Slicer: a platform for subject-specific image analysis, visualization, and clinical support. Intraoperative Imaging Image Guided Ther 3(19):277–289. ISBN: 978-1-4614-7656-6

  52. Jannin P, Fitzpatrick JM, Hawkes D, Pennec X, Shahidi R, Vannier M (2002) Validation of medical image processing in image-guided therapy. IEEE Trans Med Imaging 21(12):1445–9

    Article  PubMed  Google Scholar 

  53. Mercier L, Del Maestro RF, Petrecca K, Araujo D, Haegelen C, Collins DL (2012) Online database of clinical MR and ultrasound images of brain tumors. Med Phys 39(6Part1):3253–3261

    Article  PubMed  Google Scholar 

  54. Xiao Y, Fortin M, Unsgård G, Rivaz H, Reinertsen I (2017) REtroSpective Evaluation of Cerebral Tumors (RESECT): a clinical database of pre-operative MRI and intra-operative ultrasound in low-grade glioma surgeries. Med Phys 44(7):3875–3882

    Article  PubMed  Google Scholar 

Download references

Funding

This study is supported by the National Institute of Health Grants P41-EB015898-09, P41-EB015902 and R01-NS049251 and the Natural Sciences and Engineering Research Council of Canada, Discovery grant. The authors would like to acknowledge the financial support from the Portuguese Foundation for Science and Technology under the references PD/BD/105869/2014 and IDMEC/LAETA UID/EMS/50022/2013.

Author information

Authors and Affiliations

  1. Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, 75 Francis St., Boston, MA, 02115, USA

    Inês Machado, Jie Luo, Elizabeth George, Steve Pieper, Sarah Frisken & William Wells III

  2. IDMEC, Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais 1, 1049-001, Lisbon, Portugal

    Inês Machado, Pedro Teodoro & Jorge Martins

  3. École de Technologie Superieure, 1100 Notre-Dame St W, Montreal, QC, H3C 1K3, Canada

    Matthew Toews

  4. Graduate School of Frontier Sciences, University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa, Chiba, Japan

    Jie Luo

  5. Department of Neurosurgery, Brigham and Women’s Hospital, Harvard Medical School, 75 Francis St., Boston, MA, 02115, USA

    Prashin Unadkat, Walid Essayed & Alexandra Golby

  6. Department of Neurosurgery, CHLN, Hospital de Santa Maria, Avenida Professor Egas Moniz, 1649-035, Lisbon, Portugal

    Herculano Carvalho

  7. Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St, Cambridge, MA, 02139, USA

    Polina Golland & William Wells III

  8. Isomics, Inc., 55 Kirkland St, Cambridge, MA, 02138, USA

    Steve Pieper

Authors
  1. Inês Machado
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew Toews
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Prashin Unadkat
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Walid Essayed
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Elizabeth George
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Pedro Teodoro
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Herculano Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jorge Martins
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Polina Golland
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Steve Pieper
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Sarah Frisken
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Alexandra Golby
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. William Wells III
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Inês Machado.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Machado, I., Toews, M., Luo, J. et al. Non-rigid registration of 3D ultrasound for neurosurgery using automatic feature detection and matching. Int J CARS 13, 1525–1538 (2018). https://doi.org/10.1007/s11548-018-1786-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-018-1786-7

Keywords

Search

Navigation