Skip to main content

Advertisement

SpringerLink
Log in

Robust augmented reality guidance with fluorescent markers in laparoscopic surgery

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Laparoscopic interventions require the precise navigation of medical instruments through the patient’s body, while taking critical structures into account. Although numerous concepts have been proposed for displaying subsurface anatomical detail using augmented reality, clinical translation of these methods has suffered from a lack of robustness as well as from cumbersome integration into the clinical workflow. The purpose of this study was to investigate the feasibility of a new approach to intra-operative registration based on fluorescent markers.

Methods

The proposed approach to augmented reality visualization relies on metabolizable fluorescent markers that are attached to the target organ to guide a 2D/3D intra-operative registration algorithm. In an ex vivo porcine study, marker tracking performance is evaluated in the presence of smoke, blood, and tissue in the field of view of the endoscope.

Results

In contrast to state-of-the-art needle-shaped fiducial markers, the fluorescent markers can be reliably tracked when occluded by smoke, blood or tissue. This makes the new 2D/3D intra-operative registration approach considerably more robust than state-of-the-art marker-based methods.

Conclusion

As the concept can be smoothly integrated into the clinical workflow, its potential for application in clinical laparoscopy is high.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. This concept is part of patent application no. 15 193 169.8 at the European Patent Office and preliminary results have been presented at the German conference [25].

  2. cv::findContours.

  3. cv::minEnclosingCircle.

References

  1. Alander JT, Kaartinen I, Laakso A, Pätilä T, Spillmann T, Tuchin VV, Venermo M, Välisuo P (2012) A review of indocyanine green fluorescent imaging in surgery. J Biomed Imaging 2012:7

    Google Scholar 

  2. Baumhauer M, Feuerstein M, Meinzer HP, Rassweiler J (2008) Navigation in endoscopic soft tissue surgery: perspectives and limitations. J Endourol 22(4):751–766

    Article  PubMed  Google Scholar 

  3. Baumhauer M, Simpfendoerfer T, Schwarz R, Seitel M, Müller-Stich B, Gutt C, Rassweiler J, Meinzer HP, Wolf I (2007) Soft tissue navigation for laparoscopic prostatectomy: Evaluation of camera pose estimation for enhanced visualization. Proc. SPIE 6509, Medical Imaging 2007:650911. doi:10.1117/12.709655

  4. van den Berg NS, van Leeuwen FW, van der Poel HG (2012) Fluorescence guidance in urologic surgery. Curr Opin Urol 22(2):109–120

    Article  PubMed  Google Scholar 

  5. Cleary K, Peters TM (2010) Image-guided interventions: technology review and clinical applications. Annu Rev Biomed Eng 12:119–142

    Article  CAS  PubMed  Google Scholar 

  6. Dalvi A, Faria M, Pinto A (1986) Non-suture closure of wound using cyanoacrylate. J Postgrad Med 32(2):97

    CAS  PubMed  Google Scholar 

  7. Dementhon DF, Davis LS (1995) Model-based object pose in 25 lines of code. Int J Comput Vis 15(1–2):123–141

    Article  Google Scholar 

  8. Eder M, Schafer M, Bauder-Wust U, Hull WE, Wangler C, Mier W, Haberkorn U, Eisenhut M (2012) 68ga-complex lipophilicity and the targeting property of a urea-based PSMA inhibitor for pet imaging. Bioconjug Chem 23(4):688–697

    Article  CAS  PubMed  Google Scholar 

  9. Gioux S, Choi HS, Frangioni JV (2010) Image-guided surgery using invisible near-infrared light: fundamentals of clinical translation. Mol Imaging 9(5):237

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Glocker B, Sotiras A, Komodakis N, Paragios N (2011) Deformable medical image registration: setting the state of the art with discrete methods*. Annu Rev Biomed Eng 13:219–244

    Article  CAS  PubMed  Google Scholar 

  11. Itseez: Open source computer vision (2015). http://opencv.org

  12. Lepetit V, Moreno-Noguer F, Fua P (2009) Epnp: an accurate o (n) solution to the pnp problem. Int J Comput Vis 81(2):155–166

    Article  Google Scholar 

  13. Lu CP, Hager GD, Mjolsness E (2000) Fast and globally convergent pose estimation from video images. IEEE Trans Pattern Anal 22(6):610–622

    Article  Google Scholar 

  14. Maier-Hein L, Müller SA, Pianka F, Wörz S, Müller-Stich BP, Seitel A, Rohr K, Meinzer HP, Schmied BM, Wolf I (2008) Respiratory motion compensation for ct-guided interventions in the liver. Comput Aided Surg 13(3):125–138

    Article  PubMed  Google Scholar 

  15. Markelj P, Tomaževič D, Likar B, Pernuš F (2012) A review of 3d/2d registration methods for image-guided interventions. Med Image Anal 16(3):642–661

    Article  CAS  PubMed  Google Scholar 

  16. Müller M (2013) Augmented reality navigation for percutaneous nephrolithotomy on mobile devices. Ph.D. thesis, Universität Heidelberg. http://www.ub.uni-heidelberg.de/archiv/16102

  17. Nolden M, Zelzer S, Seitel A, Wald D, Müller M, Franz AM, Maleike D, Fangerau M, Baumhauer M, Maier-Hein L, Maier-Hein K, Meinzer HP, Wolf I (2013) The medical imaging interaction toolkit: challenges and advances. Int J Comput Assist Radiol Surg 8(4):607–620

    Article  PubMed  Google Scholar 

  18. Park H, Park JI (2005) Invisible marker based augmented reality system. Proc. SPIE 5960, Visual Communications and Image Processing 2005, 59601I. doi:10.1117/12.631416

  19. Pluim JP, Maintz JA, Viergever M (2003) Mutual-information-based registration of medical images: a survey. IEEE Trans Med Imaging 22(8):986–1004

    Article  PubMed  Google Scholar 

  20. Prahl S (2015) Optical absorption of hemoglobin. http://omlc.org/spectra/hemoglobin/

  21. Rucker DC, Wu Y, Clements LW, Ondrake JE, Pheiffer TS, Simpson AL, Jarnagin WR, Miga M (2014) A mechanics-based nonrigid registration method for liver surgery using sparse intraoperative data. IEEE Trans Med Imaging 33(1):147–158

    Article  PubMed  PubMed Central  Google Scholar 

  22. Schweighofer G, Pinz A (2006) Robust pose estimation from a planar target. IEEE Trans Pattern Anal 28(12):2024–2030

    Article  Google Scholar 

  23. Simpfendörfer T, Baumhauer M, Müller M, Gutt CN, Meinzer HP, Rassweiler JJ, Guven S, Teber D (2011) Augmented reality visualization during laparoscopic radical prostatectomy. J Endourol 25(12):1841–1845

    Article  PubMed  Google Scholar 

  24. Suwelack S, Röhl S, Bodenstedt S, Reichard D, Dillmann R, dos Santos T, Maier-Hein L, Wagner M, Wünscher J, Kenngott H, Müller BP, Speidel S (2014) Physics-based shape matching for intraoperative image guidance. Med Phys 41(11):111,901

    Article  Google Scholar 

  25. Wild E, Teber D, Schmid D, Simpfendörfer T, Müller M, Kenngott H, Maier-Hein L (2016) Robuste intraoperative Registrierung mit fluoreszierenden Markern für die computergestützte Laparoskopie. Bildverarbeitung für die Medizin 2016:74–79. doi:10.1007/978-3-662-49465-3_15

  26. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal 22(11):1330–1334

    Article  Google Scholar 

  27. Zheng Y, Kuang Y, Sugimoto S, Astrom K, Okutomi M (2013) Revisiting the pnp problem: A fast, general and optimal solution. In: 2013 IEEE international conference on computer vision (ICCV), pp 2344–2351

Download references

Author information

Authors and Affiliations

  1. Junior Group Computer-assisted Interventions, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, 69120, Heidelberg, Germany

    Esther Wild, Daniel Schmid, Michael Müller & Lena Maier-Hein

  2. Department of Urology, University of Heidelberg, Im Neuenheimer Feld 672, 69120, Heidelberg, Germany

    Dogu Teber & Tobias Simpfendörfer

  3. Faculty of Mechanics and Electronics, Heilbronn University, Heilbronn, Germany

    Daniel Schmid

  4. mbits imaging GmbH, Bergheimer Strasse 147, 69115, Heidelberg, Germany

    Michael Müller

  5. Division of Radiopharmaceutical Chemistry, German Cancer Research Center (DKFZ), Im Neuenheimer Feld 280, 69120, Heidelberg, Germany

    Ann-Christin Baranski & Klaus Kopka

  6. General, Visceral and Transplant Surgery, University of Heidelberg, Im Neuenheimer Feld 110, 69120, Heidelberg, Germany

    Hannes Kenngott

Authors
  1. Esther Wild
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dogu Teber
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Schmid
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tobias Simpfendörfer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ann-Christin Baranski
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hannes Kenngott
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Klaus Kopka
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Lena Maier-Hein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Esther Wild.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 5199 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wild, E., Teber, D., Schmid, D. et al. Robust augmented reality guidance with fluorescent markers in laparoscopic surgery. Int J CARS 11, 899–907 (2016). https://doi.org/10.1007/s11548-016-1385-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-016-1385-4

Keywords

Search

Navigation