Skip to main content
SpringerLink
Log in

A surgical robot with augmented reality visualization for stereoelectroencephalography electrode implantation

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Using existing stereoelectroencephalography (SEEG) electrode implantation surgical robot systems, it is difficult to intuitively validate registration accuracy and display the electrode entry points (EPs) and the anatomical structure around the electrode trajectories in the patient space to the surgeon. This paper proposes a prototype system that can realize video see-through augmented reality (VAR) and spatial augmented reality (SAR) for SEEG implantation. The system helps the surgeon quickly and intuitively confirm the registration accuracy, locate EPs and visualize the internal anatomical structure in the image space and patient space.

Methods

We designed and developed a projector-camera system (PCS) attached to the distal flange of a robot arm. First, system calibration is performed. Second, the PCS is used to obtain the point clouds of the surface of the patient’s head, which are utilized for patient-to-image registration. Finally, VAR is produced by merging the real-time video of the patient and the preoperative three-dimensional (3D) operational planning model. In addition, SAR is implemented by projecting the planning electrode trajectories and local anatomical structure onto the patient’s scalp.

Results

The error of registration, the electrode EPs and the target points are evaluated on a phantom. The fiducial registration error is \(0.25 \pm 0.23\) mm (max 1.22 mm), and the target registration error is \(0.62\pm 0.28\) mm (max 1.18 mm). The projection overlay error is \(0.75\pm 0.52\) mm, and the TP error after the pre-warped projection is \(0.82\pm 0.23\) mm. The TP error caused by a surgeon’s viewpoint deviation is also evaluated.

Conclusion

The presented system can help surgeons quickly verify registration accuracy during SEEG procedures and can provide accurate EP locations and internal structural information to the surgeon. With more intuitive surgical information, the surgeon may have more confidence and be able to perform surgeries with better outcomes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Cardinale F, Cossu M, Castana L, Casaceli G, Schiariti MP, Miserocchi A, Fuschillo D, Moscato A, Caborni C, Arnulfo G (2013) Stereoelectroencephalography: surgical methodology, safety, and stereotactic application accuracy in 500 procedures. Neurosurgery 72(3):353–366

    Article  PubMed  Google Scholar 

  2. Cossu M, Fuschillo D, Casaceli G, Pelliccia V, Castana L, Mai R, Francione S, Sartori I, Gozzo F, Nobili L (2015) Stereoelectroencephalography-guided radiofrequency thermocoagulation in the epileptogenic zone: a retrospective study on 89 cases. J Neurosurg 123(6):1358–1367

    Article  PubMed  Google Scholar 

  3. Nowell M, Rodionov R, Diehl B, Wehner T, Zombori G, Kinghorn J, Ourselin S, Duncan J, Miserocchi A, McEvoy A (2014) A novel method for implementation of frameless StereoEEG in epilepsy surgery. Neurosurgery 10(4):525

    Article  PubMed  PubMed Central  Google Scholar 

  4. Faria C, Erlhagen W, Rito M, De Momi E, Ferrigno G, Bicho E (2015) Review of robotic technology for stereotactic neurosurgery. IEEE Rev Biomed Eng 8:125–137

    Article  PubMed  Google Scholar 

  5. Tagaytayan R, Kelemen A, Sik-Lanyi C (2016) Augmented reality in neurosurgery. Arch Med Sci. doi:10.5114/aoms.2016.58690

  6. Besharati Tabrizi L, Mahvash M (2015) Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg 123(1):206–211

    Article  PubMed  Google Scholar 

  7. Meola A, Cutolo F, Carbone M, Cagnazzo F, Ferrari M, Ferrari V (2016) Augmented reality in neurosurgery: a systematic review. Neurosurg Rev. doi:10.1007/s10143-016-0732-9

  8. Kersten-Oertel M, Gerard I, Drouin S, Mok K, Sirhan D, Sinclair DS, Collins DL (2015) Augmented reality in neurovascular surgery: feasibility and first uses in the operating room. Int J Comput Assist Radiol Surg 10(11):1823–1836

    Article  PubMed  Google Scholar 

  9. González-Martínez J, Bulacio J, Thompson S, Gale J, Smithason S, Najm I, Bingaman W (2016) Technique, results, and complications related to robot-assisted stereoelectroencephalography. Neurosurgery 78(2):169–180

    Article  PubMed  Google Scholar 

  10. Watanabe E, Satoh M, Konno T, Hirai M, Yamaguchi T (2016) The trans-visible navigator: a see-through neuronavigation system using augmented reality. World Neurosurg 87:399–405

    Article  PubMed  Google Scholar 

  11. Kockro RA, Tsai YT, Ng I, Hwang P, Zhu C, Agusanto K, Hong LX, Serra L (2009) DEX - RAY: augmented reality neurosurgical navigation with a handheld video probe. Neurosurgery 65(4):795–808

    Article  PubMed  Google Scholar 

  12. Park H, Kang G-C, Lee M-H, Kim S-J, Park J-I (2005) Direct-projected augmented reality considering user’s viewpoint. In: Proceedings of international meeting on information display and exhibition, pp 748–751

  13. Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201

    Article  PubMed  Google Scholar 

  14. Hennersperger C, Manus J, Navab N (2016) Mobile laserprojection in computer assisted neurosurgery. In: International conference on medical imaging and virtual reality. Springer, pp 151–162

  15. Geng J (2011) Structured-light 3D surface imaging: a tutorial. Adv Opt Phot 3(2):128–160

    Article  CAS  Google Scholar 

  16. Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge

    Google Scholar 

  17. Moreno D, Taubin G (2012) Simple, accurate, and robust projector-camera calibration. In: 2012 Second international conference on 3D imaging, modeling, processing, visualization & transmission. IEEE, pp 464–471

  18. Tsai RY, Lenz RK (1989) A new technique for fully autonomous and efficient 3D robotics hand/eye calibration. IEEE Trans on Robot Autom 5(3):345–358

    Article  CAS  Google Scholar 

  19. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Patt Anal Mach Intell 22(11):1330–1334

    Article  Google Scholar 

  20. Besl PJ, McKay ND (1992) Method for registration of 3-D shapes. In: Robotics-DL tentative. International society for optics and photonics, pp 586–606

  21. Rusinkiewicz S, Levoy M (2001) Efficient variants of the ICP algorithm. In: 3-D Digital imaging and modeling. Proceedings. Third international conference on, 2001. IEEE, pp 145–152

  22. Sielhorst T, Bichlmeier C, Heining SM, Navab N (2006) Depth perception—a major issue in medical AR: evaluation study by twenty surgeons. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 364–372

  23. Cheng O, Guangzhi W, Quan Z, Wei K, Hui D (2005) Evaluating harris method in camera calibration. In: Engineering in medicine and biology society. IEEE-EMBS 2005. 27th annual international conference of the. IEEE, pp 6383–6386

  24. Zhu C, Liang X, Kockro R, Serra L (2004) Accuracy evaluation of an augmented reality enhanced surgical navigation system. In: International congress series. Elsevier, p 1329

  25. Roessler K, Sommer B, Merkel A, Rampp S, Gollwitzer S, Hamer HM, Buchfelder M (2016) A frameless stereotactic implantation technique for depth electrodes in refractory epilepsy using intraoperative magnetic resonance imaging. World Neurosurg 94:206–210

    Article  PubMed  Google Scholar 

  26. Vakharia VN, Sparks R, O’Keeffe AG, Rodionov R, Miserocchi A, McEvoy A, Ourselin S, Duncan J (2017) Accuracy of intracranial electrode placement for stereoelectroencephalography: A systematic review and meta-analysis. Epilepsia 58(6):921–932

  27. Krempien R, Hoppe H, Kahrs L, Daeuber S, Schorr O, Eggers G, Bischof M, Munter MW, Debus J, Harms W (2008) Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3D interstitial brachytherapy. Int J Radiat Oncol BiolPhys 70(3):944–952

    Article  Google Scholar 

Download references

Acknowledgements

The authors acknowledge the support of the National Natural Science Foundation of China (61361160417, 81271735) and the Ministry of Science and Technology of China (2016YFC0105800, 2017YFA0205904).

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, 100084, People’s Republic of China

    Bowei Zeng, Fanle Meng, Hui Ding & Guangzhi Wang

Authors
  1. Bowei Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fanle Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guangzhi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guangzhi Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This article does not contain patient data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeng, B., Meng, F., Ding, H. et al. A surgical robot with augmented reality visualization for stereoelectroencephalography electrode implantation. Int J CARS 12, 1355–1368 (2017). https://doi.org/10.1007/s11548-017-1634-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-017-1634-1

Keywords

Search

Navigation