Abstract
Swept-Source Optical Coherence Tomography (SS-OCT) allows surgeons to perform certain ophthalmic procedures under the exclusive guidance of real-time volumetric optical coherence tomography (4D OCT). In such scenarios, surgeons are no longer limited to rigid views through an operating microscope. Instead, direct volume rendering (DVR) of 4D OCT enables surgical maneuvers to be performed from arbitrary viewpoints. While 4D OCT maximizes the use of the depth-resolved OCT data by displaying it from an oblique perspective, performing complex instrument maneuvers from such views places a higher mental demand on the surgeon. In this work, we propose an Intelligent Virtual B-scan Mirror (IVBM), a novel concept for surgical 4D OCT visualization to provide additional guidance for targeted instrument interactions. The IVBM integrates a virtual mirror into a selected cross-section of the OCT volume. This mirror acts intelligently by only being sensitive to voxels associated with surgical instruments. Furthermore, volume structures aligned with the IVBM are highlighted, while structures behind the IVBM are preserved through an adaptive opacity transfer function. Unlike previous perceptual OCT visualization concepts, which primarily address depth perception in axial OCT direction, this novel approach aids surgical interactions from arbitrary views. This paper presents the definition and implementation of an IVBM in a 4D OCT integrated microscope. Our user study in a virtual simulation environment confirms the benefits and provides insights into the interaction with the concept.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bichlmeier, C., Heining, S.M., Feuerstein, M., Navab, N.: The virtual mirror: a new interaction paradigm for augmented reality environments. IEEE Trans. Med. Imaging 28(9), 1498–1510 (2009)
Bichlmeier, C., Heining, S.M., Rustaee, M., Navab, N.: Laparoscopic virtual mirror for understanding vessel structure evaluation study by twelve surgeons. In: 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 125–128. IEEE (2007)
Bleicher, I.D., Jackson-Atogi, M., Viehland, C., Gabr, H., Izatt, J.A., Toth, C.A.: Depth-based, motion-stabilized colorization of microscope-integrated optical coherence tomography volumes for microscope-independent microsurgery. Transl. Vis. Sci. Technol. 7(6), 1–1 (2018)
Britten, A., et al.: Surgical microscope integrated MHz SS-OCT with live volumetric visualization. Biomed. Optics Express 14(2), 846–865 (2023)
Carrasco-Zevallos, O.M., Viehland, C., Keller, B., McNabb, R.P., Kuo, A.N., Izatt, J.A.: Constant linear velocity spiral scanning for near video rate 4d oct ophthalmic and surgical imaging with isotropic transverse sampling. Biomed. Optics Express 9(10), 5052 (2018)
Draelos, M., Keller, B., Toth, C., Kuo, A., Hauser, K., Izatt, J.: Teleoperating robots from arbitrary viewpoints in surgical contexts. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2549–2555 (2017)
Ehlers, J.P., et al.: Outcomes of intraoperative oct-assisted epiretinal membrane surgery from the pioneer study. Ophthalmol. Retina 2(4), 263–267 (2018)
Ehlers, J.P., et al.: The DISCOVER study 3-year results: feasibility and usefulness of microscope-integrated intraoperative oct during ophthalmic surgery. Ophthalmology 125(7), 1014–1027 (2018)
Falkner-Radler, C.I., Glittenberg, C., Gabriel, M., Binder, S.: Intrasurgical microscope-integrated spectral domain optical coherence tomography-assisted membrane peeling. Retina 35(10), 2100–2106 (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Jo, Y.J., Heo, D.W., Shin, Y.I., Kim, J.Y.: Diurnal variation of retina thickness measured with time domain and spectral domain optical coherence tomography in healthy subjects. Invest. Ophthalmol. Vis. Sci. 52(9), 6497–6500 (2011)
Kolb, J.P., et al.: Live video rate volumetric OCT imaging of the retina with multi-MHz a-scan rates. PLoS ONE 14(3), e0213144 (2019)
Li, N., Zhang, Z., Liu, C., Yang, Z., Fu, Y., Tian, F., Han, T., Fan, M.: vMirror: enhancing the interaction with occluded or distant objects in VR with virtual mirrors. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–11 (2021)
Matinfar, S., et al.: Surgical Soundtracks: towards automatic musical augmentation of surgical procedures. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10434, pp. 673–681. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66185-8_76
Navab, N., Feuerstein, M., Bichlmeier, C.: Laparoscopic virtual mirror new interaction paradigm for monitor based augmented reality. In: 2007 IEEE Virtual Reality Conference, pp. 43–50. IEEE (2007)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Roodaki, H., Filippatos, K., Eslami, A., Navab, N.: Introducing augmented reality to optical coherence tomography in ophthalmic microsurgery. In: 2015 IEEE International Symposium on Mixed and Augmented Reality, pp. 1–6 (2015)
Sommersperger, M., et al.: Surgical scene generation and adversarial networks for physics-based iOCT synthesis. Biomed. Optics Express 13(4), 2414–2430 (2022)
Tatzgern, M., Kalkofen, D., Grasset, R., Schmalstieg, D.: Embedded virtual views for augmented reality navigation. In: Proceedings of the International Symposium Mixed Augmented Reality-Workshop Visualization in Mixed Reality Environments, vol. 115, p. 123 (2011)
Viehland, C., et al.: Enhanced volumetric visualization for real time 4D intraoperative ophthalmic swept-source OCT. Biomed. Optics Express 7(5), 1815–1829 (2016)
Wang, J., Fallavollita, P., Wang, L., Kreiser, M., Navab, N.: Augmented reality during angiography: integration of a virtual mirror for improved 2D/3D visualization. In: 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 257–264. IEEE (2012)
Weiss, J., Eck, U., Nasseri, M.A., Maier, M., Eslami, A., Navab, N.: Layer-aware iOCT volume rendering for retinal surgery. In: Kozlíková, B., Linsen, L., Vázquez, P.P., Lawonn, K., Raidou, R.G. (eds.) Eurographics Workshop on Visual Computing for Biology and Medicine. The Eurographics Association (2019)
Weiss, J., Sommersperger, M., Nasseri, A., Eslami, A., Eck, U., Navab, N.: Processing-aware real-time rendering for optimized tissue visualization in intraoperative 4D OCT. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 267–276. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_26
Wilkins, J.R., et al.: Characterization of epiretinal membranes using optical coherence tomography. Ophthalmology 103(12), 2142–2151 (1996)
Acknowledgements
This work is partially supported and the data is provided by Carl Zeiss Meditec. The authors wish to thank SynthesEyes (https://syntheseyes.de) for providing the excellent simulation setup for the user study.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sommersperger, M. et al. (2023). Intelligent Virtual B-Scan Mirror (IVBM). In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14228. Springer, Cham. https://doi.org/10.1007/978-3-031-43996-4_40
Download citation
DOI: https://doi.org/10.1007/978-3-031-43996-4_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43995-7
Online ISBN: 978-3-031-43996-4
eBook Packages: Computer ScienceComputer Science (R0)
