Skip to main content
SpringerLink
Log in

Proof of Concept: Wearable Augmented Reality Video See-Through Display for Neuro-Endoscopy

  • Conference paper
  • First Online:
Book cover Augmented Reality, Virtual Reality, and Computer Graphics (AVR 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10851))

Abstract

In mini-invasive surgery and in endoscopic procedures, the surgeon operates without a direct visualization of the patient’s anatomy. In image-guided surgery, solutions based on wearable augmented reality (AR) represent the most promising ones. The authors describe the characteristics that an ideal Head Mounted Display (HMD) must have to guarantee safety and accuracy in AR-guided neurosurgical interventions and design the ideal virtual content for guiding crucial task in neuro endoscopic surgery. The selected sequence of AR content to obtain an effective guidance during surgery is tested in a Microsoft Hololens based app.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Inoue, D., Cho, B., Mori, M., Kikkawa, Y., Amano, T., Nakamizo, A., Yoshimoto, K., Mizoguchi, M., Tomikawa, M., Hong, J., Hashizume, M., Sasaki, T.: Preliminary study on the clinical application of augmented reality neuronavigation. J. Neurol. Surg. A Cent. Eur. Neurosurg. 74, 71–76 (2013)

    Article  Google Scholar 

  2. Kockro, R.A., Tsai, Y.T., Ng, I., Hwang, P., Zhu, C., Agusanto, K., Hong, L.X., Serra, L.: Dex-ray: augmented reality neurosurgical navigation with a handheld video probe. Neurosurgery 65, 795–807 (2009). discussion 807-798

    Article  Google Scholar 

  3. Schulz, M., Bohner, G., Knaus, H., Haberl, H., Thomale, U.-W.: Navigated endoscopic surgery for multiloculated hydrocephalus in children. J. Neurosurg. Pediatr. 5, 434–442 (2010)

    Article  Google Scholar 

  4. King, A.P., Edwards, P.J., Maurer Jr., C.R., de Cunha, D.A., Hawkes, D.J., Hill, D.L., Gaston, R.P., Fenlon, M.R., Strong, A.J., Chandler, C.L., Richards, A., Gleeson, M.J.: A system for microscope-assisted guided interventions. Stereotact. Funct. Neurosurg. 72, 107–111 (1999)

    Article  Google Scholar 

  5. Edwards, P.J., King, A.P., Maurer Jr., C.R., de Cunha, D.A., Hawkes, D.J., Hill, D.L., Gaston, R.P., Fenlon, M.R., Jusczyzck, A., Strong, A.J., Chandler, C.L., Gleeson, M.J.: Design and evaluation of a system for microscope-assisted guided interventions (MAGI). IEEE Trans. Med. Imaging 19, 1082–1093 (2000)

    Article  Google Scholar 

  6. Stadie, A.T., Reisch, R., Kockro, R.A., Fischer, G., Schwandt, E., Boor, S., Stoeter, P.: Minimally invasive cerebral cavernoma surgery using keyhole approaches - solutions for technique-related limitations. Minim. Invasive Neurosurg. 52, 9–16 (2009)

    Article  Google Scholar 

  7. Cabrilo, I., Bijlenga, P., Schaller, K.: Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations. Acta Neurochir. (Wien) 156, 1769–1774 (2014)

    Article  Google Scholar 

  8. Deng, W.W., Li, F., Wang, M.N., Song, Z.J.: Easy-to-Use augmented reality neuronavigation using a wireless tablet PC. Stereot. Funct. Neuros. 92, 17–24 (2014)

    Article  Google Scholar 

  9. Besharati Tabrizi, L., Mahvash, M.: Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J. Neurosurg. 123, 206–211 (2015)

    Article  Google Scholar 

  10. Citardi, M.J., Agbetoba, A., Bigcas, J.L., Luong, A.: Augmented reality for endoscopic sinus surgery with surgical navigation: a cadaver study. Int. Forum Allergy Rhinol. 6, 523–528 (2016)

    Article  Google Scholar 

  11. Cutolo, F., Meola, A., Carbone, M., Sinceri, S., Cagnazzo, F., Denaro, E., Esposito, N., Ferrari, M., Ferrari, V.: A new head-mounted display-based augmented reality system in neurosurgical oncology: a study on phantom. Comput. Assist. Surg. 22, 39–53 (2017)

    Article  Google Scholar 

  12. Kawamata, T., Iseki, H., Shibasaki, T., Hori, T.: Endoscopic augmented reality navigation system for endonasal transsphenoidal surgery to treat pituitary tumors: technical note. Neurosurgery 50, 1393–1397 (2002)

    Google Scholar 

  13. Meola, A., Cutolo, F., Carbone, M., Cagnazzo, F., Ferrari, M., Ferrari, V.: Augmented reality in neurosurgery: a systematic review. Neurosurg. Rev. 40, 537–548 (2017)

    Article  Google Scholar 

  14. Finger, T., Schaumann, A., Schulz, M., Thomale, U.W.: Augmented reality in intraventricular neuroendoscopy. Acta Neurochir. (Wien) 159, 1033–1041 (2017)

    Article  Google Scholar 

  15. Cutolo, F.: Augmented Reality in Image-Guided Surgery. In: Lee, N. (ed.) Encyclopedia of computer graphics and games, pp. 1–11. Springer, Cham (2017)

    Google Scholar 

  16. Rolland, J.P., Fuchs, H.: Optical versus video see-through head-mounted displays in medical visualization. Presence Teleoper. Virtual Environ. 9, 287–309 (2000)

    Article  Google Scholar 

  17. Cutolo, F., Fontana, U., Carbone, M., D’Amato, R., Ferrari, V.: Hybrid video/optical see-through HMD. Adjunct. In: Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality (Ismar-Adjunct), pp. 52–57 (2017)

    Google Scholar 

  18. www.vostars.eu

  19. Kersten-Oertel, M., Jannin, P., Collins, D.L.: The state of the art of visualization in mixed reality image guided surgery. Comput. Med. Imag. Grap. 37, 98–112 (2013)

    Article  Google Scholar 

  20. Bichlmeier, C., Wimme, F., Heining, S.M., Navab, N.: Contextual anatomic mimesis hybrid in-situ visualization method for improving multi-sensory depth perception in medical augmented reality. In: 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, 2007, ISMAR 2007, pp. 129–138 (2007)

    Google Scholar 

  21. Kersten-Oertel, M., Chen, S.J.S., Collins, D.L.: An evaluation of depth enhancing perceptual cues for vascular volume visualization in neurosurgery. IEEE Trans. Vis. Comput. Graph. 20, 391–403 (2014)

    Article  Google Scholar 

  22. Badiali, G., Ferrari, V., Cutolo, F., Freschi, C., Caramella, D., Bianchi, A., Marchetti, C.: Augmented reality as an aid in maxillofacial surgery: Validation of a wearable system allowing maxillary repositioning. J. Cranio. Maxill. Surg. 42, 1970–1976 (2014)

    Article  Google Scholar 

  23. Cutolo, F., Parchi, P.D., Ferrari, V.: Video see through AR head-mounted display for medical procedures. In: IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2014, pp. 393–396. IEEE (2014)

    Google Scholar 

  24. Parrini, S., Cutolo, F., Freschi, C., Ferrari, M., Ferrari, V.: Augmented reality system for freehand guide of magnetic endovascular devices. In: 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 490–493. IEEE (2014)

    Google Scholar 

  25. Ferrari, V., Viglialoro, R.M., Nicoli, P., Cutolo, F., Condino, S., Carbone, M., Siesto, M., Ferrari, M.: Augmented reality visualization of deformable tubular structures for surgical simulation. Int. J. Med. Robot. Comput. Assist. Surg. 12(2), 231–240 (2015)

    Article  Google Scholar 

  26. Cutolo, F., Badiali, G., Ferrari, V.: Human-PnP: ergonomic AR interaction paradigm for manual placement of rigid bodies. In: Linte, Cristian A., Yaniv, Z., Fallavollita, P. (eds.) AE-CAI 2015. LNCS, vol. 9365, pp. 50–60. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24601-7_6

    Chapter  Google Scholar 

  27. Evans, G., Miller, J., Pena, M.I., MacAllister, A., Winer, E.: Evaluating the Microsoft HoloLens through an augmented reality assembly application. Degrad. Environ. Sens. Process. Display 2017, 10197 (2017)

    Google Scholar 

  28. Ferrari, V., Carbone, M., Cappelli, C., Boni, L., Melfi, F., Ferrari, M., Mosca, F., Pietrabissa, A.: Value of multidetector computed tomography image segmentation for preoperative planning in general surgery. Surg. Endosc. 26, 616–626 (2012)

    Article  Google Scholar 

  29. https://www.vuforia.com/

  30. Badiali, G., Roncari, A., Bianchi, A., Taddei, F., Marchetti, C., Schileo, E.: Navigation in orthognathic surgery: 3D accuracy. Facial Plast. Surg. FPS 31, 463–473 (2015)

    Article  Google Scholar 

  31. Volonte, F., Pugin, F., Bucher, P., Sugimoto, M., Ratib, O., Morel, P.: Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J Hepatobiliary Pancreat. Sci. 18, 506–509 (2011)

    Article  Google Scholar 

  32. Zheng, G., Nolte, L.P.: Computer-assisted orthopedic surgery: current state and future perspective. Front. Surg. 2, 66 (2015)

    Article  Google Scholar 

  33. Luebbers, H.T., Messmer, P., Obwegeser, J.A., Zwahlen, R.A., Kikinis, R., Graetz, K.W., Matthews, F.: Comparison of different registration methods for surgical navigation in cranio-maxillofacial surgery. J. Cranio-Maxillo-Facial Surg. 36, 109–116 (2008). Official publication of the European Association for Cranio-Maxillo-Facial Surgery

    Article  Google Scholar 

  34. Condino, S., Calabro, E.M., Alberti, A., Parrini, S., Cioni, R., Berchiolli, R.N., Gesi, M., Ferrari, V., Ferrari, M.: Simultaneous tracking of catheters and guidewires: comparison to standard fluoroscopic guidance for arterial cannulation. Eur. J. Vasc. Endovasc. Surg. 47, 53–60 (2014). The official journal of the European Society for Vascular Surgery

    Article  Google Scholar 

  35. Parrini, S., Zhang, L., Condino, S., Ferrari, V., Caramella, D., Ferrari, M.: Automatic carotid centerline extraction from three-dimensional ultrasound Doppler images. In: 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014, pp. 5089–5092 (2014)

    Google Scholar 

  36. Condino, S., Ferrari, V., Freschi, C., Alberti, A., Berchiolli, R., Mosca, F., Ferrari, M.: Electromagnetic navigation platform for endovascular surgery: how to develop sensorized catheters and guidewires. Int. J. Med. Robot. + Comput. Assist. Surg. MRCAS 8, 300–310 (2012)

    Article  Google Scholar 

  37. Ukimura, O., Gill, I.S.: Image-fusion, augmented reality, and predictive surgical navigation. Urol. Clin. North Am. 36, 115–123, vii (2009)

    Article  Google Scholar 

  38. Lamata, P., Ali, W., Cano, A., Cornella, J., Declerck, J., Elle, O.J., Freudenthal, A., Furtado, H., Kalkofen, D., Naerum, E., Samset, E., Sánchez-Gonzalez, P., Sánchez-Margallo, F.M., Schmalstieg, D., Sette, M., Stüdeli, T., Sloten, J.V., Gómez, E.J.: Augmented Reality for Minimally Invasive Surgery: Overview and Some Recent Advances (2010)

    Google Scholar 

  39. Nicolau, S., Soler, L., Mutter, D., Marescaux, J.: Augmented reality in laparoscopic surgical oncology. Surg. Oncol. 20, 189–201 (2011)

    Article  Google Scholar 

  40. Rankin, T.M., Slepian, M.J., Armstrong, D.G.: Augmented reality in surgery. In: Latifi, R., Rhee, P., Gruessner, W.G.R. (eds.) Technological Advances in Surgery, Trauma and Critical Care, pp. 59–71. Springer, New York (2015)

    Chapter  Google Scholar 

  41. Ferrari, V., Viglialoro, R.M., Nicoli, P., Cutolo, F., Condino, S., Carbone, M., Siesto, M., Ferrari, M.: Augmented reality visualization of deformable tubular structures for surgical simulation. Int. J. Med. Rob. + Comput. Assist. Surg. MRCAS 12, 231–240 (2016)

    Article  Google Scholar 

  42. Viglialoro, R., Ferrari, V., Carbone, M.C.M., Condino, S., Porcelli, F., Puccio, F.D., Ferrari, M., Mosca, F.: A physical patient specific simulator for cholecystectomy training. In: CARS Proceedings of the 25th International Congress and Exhibition, Pisa, Italy, June 27–30 (2012)

    Google Scholar 

  43. Francesconi, M., Freschi, C., Sinceri, S., Carbone, M., Cappelli, C., Morelli, L., Ferrari, V., Ferrari, M.: New training methods based on mixed reality for interventional ultrasound: design and validation. In: Engineering in Medicine and Biology Society (EMBC), 2015, 37th Annual International Conference of the IEEE, pp. 5098–5101. IEEE (2015)

    Google Scholar 

  44. Freschi, C., Parrini, S., Dinelli, N., Ferrari, M., Ferrari, V.: Hybrid simulation using mixed reality for interventional ultrasound imaging training. Int. J. Comput. Assist. Radiol. Surg. 10(7), 1109–1115 (2014)

    Article  Google Scholar 

Download references

Acknowledgments

Funded BY THE HORIZON2020 Project VOSTARS, Project ID: 731974. Call: ICT-29-2016 - Photonics KET 2016.

Author information

Authors and Affiliations

  1. EndoCAS Center, Department of Translational Research and New Technologies in Medicine and Surgery, University of Pisa, Pisa, Italy

    Marina Carbone, Sara Condino, Fabrizio Cutolo, Rosanna Maria Viglialoro & Vincenzo Ferrari

  2. Information Engineering Department, University of Pisa, Pisa, Italy

    Marina Carbone, Fabrizio Cutolo & Vincenzo Ferrari

  3. Ear-Nose-Throat Department, Sankt Gertrauden-Krankenhaus GmbH, Berlin, Germany

    Oliver Kaschke

  4. Department of Neurosurgery with Pediatric Neurosurgery, Charité Universitätsmedizin, Berlin, Germany

    Ulrich W. Thomale

Authors
  1. Marina Carbone
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sara Condino
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabrizio Cutolo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rosanna Maria Viglialoro
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Oliver Kaschke
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ulrich W. Thomale
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Vincenzo Ferrari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Marina Carbone or Sara Condino .

Editor information

Editors and Affiliations

  1. University of Salento, Lecce, Italy

    Lucio Tommaso De Paolis

  2. University of Paris-Sud, Orsay, France

    Patrick Bourdot

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Carbone, M. et al. (2018). Proof of Concept: Wearable Augmented Reality Video See-Through Display for Neuro-Endoscopy. In: De Paolis, L., Bourdot, P. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2018. Lecture Notes in Computer Science(), vol 10851. Springer, Cham. https://doi.org/10.1007/978-3-319-95282-6_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-95282-6_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-95281-9

  • Online ISBN: 978-3-319-95282-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation