Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

Harnessing non-destructive 3D pathology

Nature Biomedical Engineering volume 5, pages 203–218 (2021)Cite this article

Subjects

Abstract

High-throughput methods for slide-free three-dimensional (3D) pathological analyses of whole biopsies and surgical specimens offer the promise of modernizing traditional histology workflows and delivering improvements in diagnostic performance. Advanced optical methods now enable the interrogation of orders of magnitude more tissue than previously possible, where volumetric imaging allows for enhanced quantitative analyses of cell distributions and tissue structures that are prognostic and predictive. Non-destructive imaging processes can simplify laboratory workflows, potentially reducing costs, and can ensure that samples are available for subsequent molecular assays. However, the large size of the feature-rich datasets that they generate poses challenges for data management and computer-aided analysis. In this Perspective, we provide an overview of the imaging technologies that enable 3D pathology, and the computational tools—machine learning, in particular—for image processing and interpretation. We also discuss the integration of various other diagnostic modalities with 3D pathology, along with the challenges and opportunities for clinical adoption and regulatory approval.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Conventional pathology versus non-destructive 3D pathology.
Fig. 2: 3D pathology can outperform 2D pathology.
Fig. 3: Selected imaging methods for 3D pathology.
Fig. 4: Data processing and image-analysis workflows.
Fig. 5: Non-destructive 3D pathology of clinical specimens.
Fig. 6: Implementing 3D pathology in clinical practice.

Similar content being viewed by others

References

  1. Carroll, P. R. et al. NCCN guidelines insights: prostate cancer early detection, version 2.2016. J. Natl Compr. Canc. Netw. 14, 509–519 (2016).

    Article  PubMed  Google Scholar 

  2. Gradishar, W. J. et al. NCCN guidelines insights: breast cancer, version 1.2016. J. Natl Compr. Canc. Netw. 13, 1475–1485 (2015).

    Article  CAS  PubMed  Google Scholar 

  3. McKenney, J. K. et al. The potential impact of reproducibility of Gleason grading in men with early stage prostate cancer managed by active surveillance: a multi-institutional study. J. Urol. 186, 465–469 (2011).

    Article  PubMed  Google Scholar 

  4. Shah, R. B. et al. Diagnosis of Gleason pattern 5 prostate adenocarcinoma on core needle biopsy: an interobserver reproducibility study among urologic pathologists. Am. J. Surg. Pathol. 39, 1242–1249 (2015).

    Article  PubMed  Google Scholar 

  5. Zhou, M. et al. Diagnosis of ‘poorly formed glands’ Gleason pattern 4 prostatic adenocarcinoma on needle biopsy: an interobserver reproducibility study among urologic pathologists with recommendations. Am. J. Surg. Pathol. 39, 1331–1339 (2015).

    Article  PubMed  Google Scholar 

  6. Kweldam, C. F. et al. Gleason grade 4 prostate adenocarcinoma patterns: an interobserver agreement study among genitourinary pathologists. Histopathology 69, 441–449 (2016).

    Article  PubMed  Google Scholar 

  7. Epstein, J. I. et al. A contemporary prostate cancer grading system: a validated alternative to the Gleason score. Eur. Urol. 69, 428–435 (2016).

    Article  PubMed  Google Scholar 

  8. Welch, H. G. & Black, W. C. Overdiagnosis in cancer. J. Natl Cancer Inst. 102, 605–613 (2010).

    Article  PubMed  Google Scholar 

  9. Haffner, M. C., De Marzo, A. M., Yegnasubramanian, S., Epstein, J. I. & Carter, H. B. Diagnostic challenges of clonal heterogeneity in prostate cancer. J. Clin. Oncol. 33, e38–e40 (2015).

    Article  PubMed  Google Scholar 

  10. Meyers, D. E., Bryan, P. M., Banerji, S. & Morris, D. G. Targeting the PD-1/PD-L1 axis for the treatment of non-small-cell lung cancer. Curr. Oncol. 25, e324–e334 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Hersom, M. & Jørgensen, J. T. Companion and complementary diagnostics-focus on PD-L1 expression assays for PD-1/PD-L1 checkpoint inhibitors in non-small cell lung cancer. Ther. Drug Monit. 40, 9–16 (2018).

    Article  CAS  PubMed  Google Scholar 

  12. Patel, S. P. & Kurzrock, R. PD-L1 expression as a predictive biomarker in cancer immunotherapy. Mol. Cancer Ther. 14, 847–856 (2015).

    Article  CAS  PubMed  Google Scholar 

  13. Brunnström, H. et al. PD-L1 immunohistochemistry in clinical diagnostics of lung cancer: inter-pathologist variability is higher than assay variability. Mod. Pathol. 30, 1411–1421 (2017).

    Article  PubMed  Google Scholar 

  14. Makhlouf, H. et al. Toward improving practices for submission of diagnostic tissue blocks for National Cancer Institute clinical trials. Am. J. Clin. Pathol. 153, 149–155 (2020).

    Article  PubMed  Google Scholar 

  15. Olson, E., Levene, M. J. & Torres, R. Multiphoton microscopy with clearing for three dimensional histology of kidney biopsies. Biomed. Opt. Express 7, 3089–3096 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Paul, D., Cowan, A. E., Ge, S. & Pachter, J. S. Novel 3D analysis of claudin-5 reveals significant endothelial heterogeneity among CNS microvessels. Microvasc. Res. 86, 1–10 (2013).

    Article  CAS  PubMed  Google Scholar 

  17. Torres, R. et al. Three-dimensional morphology by multiphoton microscopy with clearing in a model of cisplatin-induced CKD. J. Am. Soc. Nephrol. 27, 1102–1112 (2016).

    Article  CAS  PubMed  Google Scholar 

  18. Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V. & Madabhushi, A. Artificial intelligence in digital pathology—new tools for diagnosis and precision oncology. Nat. Rev. Clin. Oncol. 16, 703–715 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  19. Glaser, A. K. et al. Light-sheet microscopy for slide-free non-destructive pathology of large clinical specimens. Nat. Biomed. Eng. 1, 0084 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Reder, N. P. et al. Open-top light-sheet microscopy image atlas of prostate core needle biopsies. Arch. Pathol. Lab. Med. 143, 1069–1075 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  21. Johnson, D. B. et al. Quantitative spatial profiling of PD-1/PD-L1 interaction and HLA-DR/IDO-1 predicts improved outcomes of anti-PD-1 therapies in metastatic melanoma. Clin. Cancer Res. 24, 5250–5260 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Kargl, J. et al. Neutrophils dominate the immune cell composition in non-small cell lung cancer. Nat. Commun. 8, 14381 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. He, G. et al. Peritumoural neutrophils negatively regulate adaptive immunity via the PD-L1/PD-1 signalling pathway in hepatocellular carcinoma. J. Exp. Clin. Cancer Res. 34, 141 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  24. Yuan, Y. Spatial heterogeneity in the tumor microenvironment. Cold Spring Harb. Perspect. Med. 6, a026583 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  25. Masugi, Y. et al. Characterization of spatial distribution of tumor-infiltrating CD8+ T cells refines their prognostic utility for pancreatic cancer survival. Mod. Pathol. 32, 1495–1507 (2019).

    Article  CAS  PubMed  Google Scholar 

  26. Heindl, A., Nawaz, S. & Yuan, Y. Mapping spatial heterogeneity in the tumor microenvironment: a new era for digital pathology. Lab. Investig. 95, 377–384 (2015).

    Article  PubMed  Google Scholar 

  27. Plaks, V., Kong, N. & Werb, Z. The cancer stem cell niche: how essential is the niche in regulating stemness of tumor cells? Cell Stem Cell 16, 225–238 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Guo, W. et al. Multi-genetic events collaboratively contribute to Pten-null leukaemia stem-cell formation. Nature 453, 529–533 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Adams, J. M. & Strasser, A. Is tumor growth sustained by rare cancer stem cells or dominant clones? Cancer Res. 68, 4018–4021 (2008).

    Article  CAS  PubMed  Google Scholar 

  30. Cavé, H. et al. Clinical significance of minimal residual disease in childhood acute lymphoblastic leukemia. N. Engl. J. Med. 339, 591–598 (1998).

    Article  PubMed  Google Scholar 

  31. van Dongen, J. J. et al. Prognostic value of minimal residual disease in acute lymphoblastic leukaemia in childhood. Lancet 352, 1731–1738 (1998).

    Article  PubMed  Google Scholar 

  32. Herman, C. M., Wilcox, G. E., Kattan, M. W., Scardino, P. T. & Wheeler, T. M. Lymphovascular invasion as a predictor of disease progression in prostate cancer. Am. J. Surg. Pathol. 24, 859–863 (2000).

    Article  CAS  PubMed  Google Scholar 

  33. Mohammed, R. A. A. et al. Improved methods of detection of lymphovascular invasion demonstrate that it is the predominant method of vascular invasion in breast cancer and has important clinical consequences. Am. J. Surg. Pathol. 31, 1825–1833 (2007).

    Article  PubMed  Google Scholar 

  34. Song, Y. J. et al. The role of lymphovascular invasion as a prognostic factor in patients with lymph node-positive operable invasive breast cancer. J. Breast Cancer 14, 198–203 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  35. Haffner, M. C. et al. Tracking the clonal origin of lethal prostate cancer. J. Clin. Investig. 123, 4918–4922 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Pribluda, A., de la Cruz, C. C. & Jackson, E. L. Intratumoral heterogeneity: from diversity comes resistance. Clin. Cancer Res. 21, 2916–2923 (2015).

    Article  CAS  PubMed  Google Scholar 

  37. Eyler, C. E. & Rich, J. N. Survival of the fittest: cancer stem cells in therapeutic resistance and angiogenesis. J. Clin. Oncol. 26, 2839–2845 (2008).

    Article  CAS  PubMed  Google Scholar 

  38. Brooks, M. D., Burness, M. L. & Wicha, M. S. Therapeutic implications of cellular heterogeneity and plasticity in breast cancer. Cell Stem Cell 17, 260–271 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Humphrey, P. A. Complete histologic serial sectioning of a prostate gland with adenocarcinoma. Am. J. Surg. Pathol. 17, 468–472 (1993).

    Article  CAS  PubMed  Google Scholar 

  40. McCormick, B. H. et al. Construction of anatomically correct models of mouse brain networks. Neurocomputing 58–60, 379–386 (2004).

    Article  Google Scholar 

  41. Li, A. et al. Micro-optical sectioning tomography to obtain a high-resolution atlas of the mouse brain. Science 330, 1404–1408 (2010).

    Article  CAS  PubMed  Google Scholar 

  42. van Royen, M. E. et al. Three-dimensional microscopic analysis of clinical prostate specimens. Histopathology 69, 985–992 (2016).

    Article  PubMed  Google Scholar 

  43. Tanaka, N. et al. Whole-tissue biopsy phenotyping of three-dimensional tumours reveals patterns of cancer heterogeneity. Nat. Biomed. Eng. 1, 796–806 (2017).

    Article  CAS  PubMed  Google Scholar 

  44. Tanaka, N. et al. Mapping of the three-dimensional lymphatic microvasculature in bladder tumours using light-sheet microscopy. Br. J. Cancer 118, 995–999 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Lee, S. S.-Y., Bindokas, V. P., Lingen, M. W. & Kron, S. J. Nondestructive, multiplex three-dimensional mapping of immune infiltrates in core needle biopsy. Lab. Invest. 99, 1400–1413 (2019).

    Article  CAS  PubMed  Google Scholar 

  46. Verhoef, E. I. et al. Three-dimensional analysis reveals two major architectural subgroups of prostate cancer growth patterns. Mod. Pathol. 32, 1032–1041 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  47. Pierce, M. C., Javier, D. J. & Richards-Kortum, R. Optical contrast agents and imaging systems for detection and diagnosis of cancer. Int. J. Cancer 123, 1979–1990 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Abeytunge, S., Li, Y., Larson, B., Toledo-Crow, R. & Rajadhyaksha, M. Rapid confocal imaging of large areas of excised tissue with strip mosaicing. J. Biomed. Opt. 16, 050504 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  49. Abeytunge, S. et al. Confocal microscopy with strip mosaicing for rapid imaging over large areas of excised tissue. J. Biomed. Opt. 18, 61227 (2013).

    Article  PubMed  Google Scholar 

  50. Tao, Y. K. et al. Assessment of breast pathologies using nonlinear microscopy. Proc. Natl Acad. Sci. USA 111, 15304–15309 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Ji, M. et al. Detection of human brain tumor infiltration with quantitative stimulated Raman scattering microscopy. Sci. Transl. Med 7, 309ra163 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  52. Orringer, D. A. et al. Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat. Biomed. Eng. 1, 0027 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  53. Abeytunge, S. et al. Evaluation of breast tissue with confocal strip-mosaicking microscopy: a test approach emulating pathology-like examination. J. Biomed. Opt. 22, 34002 (2017).

    Article  PubMed  Google Scholar 

  54. Boppart, S. A. et al. Label-free optical imaging technologies for rapid translation and use during intraoperative surgical and tumor margin assessment. J. Biomed. Opt. 23, 021104 (2017).

    Article  PubMed Central  Google Scholar 

  55. Yoshitake, T. et al. Rapid histopathological imaging of skin and breast cancer surgical specimens using immersion microscopy with ultraviolet surface excitation. Sci. Rep. 8, 4476 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  56. Chen, Y. et al. Rapid pathology of lumpectomy margins with open-top light-sheet (OTLS) microscopy. Biomed. Opt. Express 10, 1257–1272 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  57. Liu, J. T. C. et al. Micromirror-scanned dual-axis confocal microscope utilizing a gradient-index relay lens for image guidance during brain surgery. J. Biomed. Opt. 15, 026029 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  58. Sanai, N. et al. Intraoperative confocal microscopy in the visualization of 5-aminolevulinic acid fluorescence in low-grade gliomas. J. Neurosurg. 115, 740–748 (2011).

    Article  CAS  PubMed  Google Scholar 

  59. Nguyen, Q. T. & Tsien, R. Y. Fluorescence-guided surgery with live molecular navigation—a new cutting edge. Nat. Rev. Cancer 13, 653–662 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Wei, L., Roberts, D. W., Sanai, N. & Liu, J. T. C. Visualization technologies for 5-ALA-based fluorescence-guided surgeries. J. Neurooncol. 141, 495–505 (2019).

    Article  CAS  PubMed  Google Scholar 

  61. Wei, L., Fujita, Y., Sanai, N. & Liu, J. T. C. Toward quantitative neurosurgical guidance with high-resolution microscopy of 5-aminolevulinic acid-induced protoporphyrin IX. Front. Oncol. 9, 592 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  62. Thawani, R. et al. Radiomics and radiogenomics in lung cancer: a review for the clinician. Lung Cancer 115, 34–41 (2018).

    Article  PubMed  Google Scholar 

  63. Lambin, P. et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat. Rev. Clin. Oncol. 14, 749–762 (2017).

    Article  PubMed  Google Scholar 

  64. Richardson, D. S. & Lichtman, J. W. Clarifying tissue clearing. Cell 162, 246–257 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  65. Azaripour, A. et al. A survey of clearing techniques for 3D imaging of tissues with special reference to connective tissue. Prog. Histochem. Cytochem. 51, 9–23 (2016).

    Article  PubMed  Google Scholar 

  66. Berke, I. M., Miola, J. P., David, M. A., Smith, M. K. & Price, C. Seeing through musculoskeletal tissues: improving in situ imaging of bone and the lacunar canalicular system through optical clearing. PLoS ONE 11, e0150268 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  67. Jing, D. et al. Tissue clearing of both hard and soft tissue organs with the PEGASOS method. Cell Res. 28, 803–818 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  68. Hama, H. et al. ScaleS: an optical clearing palette for biological imaging. Nat. Neurosci. 18, 1518–1529 (2015).

    Article  CAS  PubMed  Google Scholar 

  69. Ke, M.-T., Fujimoto, S. & Imai, T. SeeDB: a simple and morphology-preserving optical clearing agent for neuronal circuit reconstruction. Nat. Neurosci. 16, 1154–1161 (2013).

    Article  CAS  PubMed  Google Scholar 

  70. Chung, K. et al. Structural and molecular interrogation of intact biological systems. Nature 497, 332–337 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  71. Susaki, E. A. et al. Whole-brain imaging with single-cell resolution using chemical cocktails and computational analysis. Cell 157, 726–739 (2014).

    Article  CAS  PubMed  Google Scholar 

  72. Kim, S.-Y. et al. Stochastic electrotransport selectively enhances the transport of highly electromobile molecules. Proc. Natl Acad. Sci. USA 112, E6274–E6283 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  73. Miyawaki, T. et al. Visualization and molecular characterization of whole-brain vascular networks with capillary resolution. Nat. Commun. 11, 1104 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  74. Renier, N. et al. iDISCO: a simple, rapid method to immunolabel large tissue samples for volume imaging. Cell 159, 896–910 (2014).

    Article  CAS  PubMed  Google Scholar 

  75. Klingberg, A. et al. Fully automated evaluation of total glomerular number and capillary tuft size in nephritic kidneys using lightsheet microscopy. J. Am. Soc. Nephrol. 28, 452–459 (2017).

    Article  CAS  PubMed  Google Scholar 

  76. Silvestri, L., Costantini, I., Sacconi, L. & Pavone, F. S. Clearing of fixed tissue: a review from a microscopist’s perspective. J. Biomed. Opt. 21, 081205 (2016).

    Article  PubMed  Google Scholar 

  77. Glaser, A. K. et al. Multi-immersion open-top light-sheet microscope for high-throughput imaging of cleared tissues. Nat. Commun. 10, 2781 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  78. Elfer, K. N. et al. DRAQ5 and eosin (‘D&E’) as an analog to hematoxylin and eosin for rapid fluorescence histology of fresh tissues. PLoS ONE 11, e0165530 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  79. Mao, C. et al. Feature-rich covalent stains for super-resolution and cleared tissue fluorescence microscopy. Sci. Adv. 6, eaba4542 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  80. Chowdary, D. et al. Prognostic gene expression signatures can be measured in tissues collected in RNAlater preservative. J. Mol. Diagn. 8, 31–39 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  81. Mutter, G. L. et al. Comparison of frozen and RNALater solid tissue storage methods for use in RNA expression microarrays. BMC Genomics 5, 88 (2004).

    Article  PubMed  PubMed Central  Google Scholar 

  82. Ergin, B. et al. Proteomic analysis of PAXgene-fixed tissues. J. Proteome Res. 9, 5188–5196 (2010).

    Article  CAS  PubMed  Google Scholar 

  83. Urban, C. et al. PAXgene fixation enables comprehensive metabolomic and proteomic analyses of tissue specimens by MALDI MSI. Biochim. Biophys. Acta 1862, 51–60 (2018).

    Article  CAS  Google Scholar 

  84. Park, Y.-G. et al. Protection of tissue physicochemical properties using polyfunctional crosslinkers. Nat. Biotechnol. 37, 73–83 (2018).

    Article  Google Scholar 

  85. González-García, I., Solé, R. V. & Costa, J. Metapopulation dynamics and spatial heterogeneity in cancer. Proc. Natl Acad. Sci. USA 99, 13085–13089 (2002).

    Article  PubMed  PubMed Central  Google Scholar 

  86. Chung, K. & Deisseroth, K. CLARITY for mapping the nervous system. Nat. Methods 10, 508–513 (2013).

    Article  CAS  PubMed  Google Scholar 

  87. Susaki, E. A. et al. Advanced CUBIC protocols for whole-brain and whole-body clearing and imaging. Nat. Protoc. 10, 1709–1727 (2015).

    Article  CAS  PubMed  Google Scholar 

  88. Tainaka, K. et al. Whole-body imaging with single-cell resolution by tissue decolorization. Cell 159, 911–924 (2014).

    Article  CAS  PubMed  Google Scholar 

  89. Yoshitake, T. et al. Direct comparison between confocal and multiphoton microscopy for rapid histopathological evaluation of unfixed human breast tissue. J. Biomed. Opt. 21, 126021 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  90. Tu, H. et al. Stain-free histopathology by programmable supercontinuum pulses. Nat. Photon. 10, 534–540 (2016).

    Article  CAS  Google Scholar 

  91. Helmchen, F. & Denk, W. Deep tissue two-photon microscopy. Nat. Methods 2, 932–940 (2005).

    Article  CAS  PubMed  Google Scholar 

  92. Uhlén, P. & Tanaka, N. Improved pathological examination of tumors with 3D light-sheet microscopy. Trends Cancer 4, 337–341 (2018).

    Article  PubMed  Google Scholar 

  93. Pouli, D. et al. Imaging mitochondrial dynamics in human skin reveals depth-dependent hypoxia and malignant potential for diagnosis. Sci. Transl. Med. 8, 367ra169 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  94. Baugh, L. M. et al. Non-destructive two-photon excited fluorescence imaging identifies early nodules in calcific aortic-valve disease. Nat. Biomed. Eng. 1, 914–924 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  95. Skala, M. C. et al. In vivo multiphoton microscopy of NADH and FAD redox states, fluorescence lifetimes, and cellular morphology in precancerous epithelia. Proc. Natl Acad. Sci. USA 104, 19494–19499 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  96. You, S. et al. Label-free visualization and characterization of extracellular vesicles in breast cancer. Proc. Natl Acad. Sci. USA 116, 24012–24018 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  97. Xylas, J., Alt-Holland, A., Garlick, J., Hunter, M. & Georgakoudi, I. Intrinsic optical biomarkers associated with the invasive potential of tumor cells in engineered tissue models. Biomed. Opt. Express 1, 1387–1400 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  98. Conklin, M. W. et al. Aligned collagen is a prognostic signature for survival in human breast carcinoma. Am. J. Pathol. 178, 1221–1232 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  99. Campagnola, P. J. & Loew, L. M. Second-harmonic imaging microscopy for visualizing biomolecular arrays in cells, tissues and organisms. Nat. Biotechnol. 21, 1356–1360 (2003).

    Article  CAS  PubMed  Google Scholar 

  100. Freudiger, C. W. et al. Label-free biomedical imaging with high sensitivity by stimulated Raman scattering microscopy. Science 322, 1857–1861 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  101. Saar, B. G. et al. Video-rate molecular imaging in vivo with stimulated Raman scattering. Science 330, 1368–1370 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  102. Fischer, M. C., Wilson, J. W., Robles, F. E. & Warren, W. S. Invited review article: pump-probe microscopy. Rev. Sci. Instrum. 87, 031101 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  103. Matthews, T. E., Piletic, I. R., Selim, M. A., Simpson, M. J. & Warren, W. S. Pump-probe imaging differentiates melanoma from melanocytic nevi. Sci. Transl. Med 3, 71ra15 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  104. Giacomelli, M. G. et al. Multiscale nonlinear microscopy and widefield white light imaging enables rapid histological imaging of surgical specimen margins. Biomed. Opt. Express 9, 2457–2475 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  105. Nakano, A. Spinning-disk confocal microscopy—a cutting-edge tool for imaging of membrane traffic. Cell Struct. Funct. 27, 349–355 (2002).

    Article  PubMed  Google Scholar 

  106. Tanaami, T. et al. High-speed 1-frame/ms scanning confocal microscope with a microlens and Nipkow disks. Appl. Opt. 41, 4704–4708 (2002).

    Article  PubMed  Google Scholar 

  107. Cheng, L.-C. et al. Spatiotemporal focusing-based widefield multiphoton microscopy for fast optical sectioning. Opt. Express 20, 8939–8948 (2012).

    Article  PubMed  Google Scholar 

  108. Oron, D., Tal, E. & Silberberg, Y. Scanningless depth-resolved microscopy. Opt. Express 13, 1468–1476 (2005).

    Article  PubMed  Google Scholar 

  109. Zhang, T. et al. Kilohertz two-photon brain imaging in awake mice. Nat. Methods 16, 1119–1122 (2019).

    Article  CAS  PubMed  Google Scholar 

  110. Bewersdorf, J., Pick, R. & Hell, S. W. Multifocal multiphoton microscopy. Opt. Lett. 23, 655–657 (1998).

    Article  CAS  PubMed  Google Scholar 

  111. Bahlmann, K. et al. Multifocal multiphoton microscopy (MMM) at a frame rate beyond 600 Hz. Opt. Express 15, 10991–10998 (2007).

    Article  PubMed  Google Scholar 

  112. Dodt, H.-U. et al. Ultramicroscopy: three-dimensional visualization of neuronal networks in the whole mouse brain. Nat. Methods 4, 331–336 (2007).

    Article  CAS  PubMed  Google Scholar 

  113. Fahrbach, F. O., Simon, P. & Rohrbach, A. Microscopy with self-reconstructing beams. Nat. Photon. 4, 780–785 (2010).

    Article  CAS  Google Scholar 

  114. Huisken, J., Swoger, J., Del Bene, F., Wittbrodt, J. & Stelzer, E. H. K. Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science 305, 1007–1009 (2004).

    Article  CAS  PubMed  Google Scholar 

  115. Keller, P. J. et al. Fast, high-contrast imaging of animal development with scanned light sheet-based structured-illumination microscopy. Nat. Methods 7, 637–642 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  116. Keller, P. J., Schmidt, A. D., Wittbrodt, J. & Stelzer, E. H. K. Reconstruction of zebrafish early embryonic development by scanned light sheet microscopy. Science 322, 1065–1069 (2008).

    Article  CAS  PubMed  Google Scholar 

  117. Planchon, T. A. et al. Rapid three-dimensional isotropic imaging of living cells using Bessel beam plane illumination. Nat. Methods 8, 417–423 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  118. Power, R. M. & Huisken, J. A guide to light-sheet fluorescence microscopy for multiscale imaging. Nat. Methods 14, 360–373 (2017).

    Article  CAS  PubMed  Google Scholar 

  119. Tomer, R. et al. SPED light sheet microscopy: fast mapping of biological system structure and function. Cell 163, 1796–1806 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  120. Wu, Y. et al. Spatially isotropic four-dimensional imaging with dual-view plane illumination microscopy. Nat. Biotechnol. 31, 1032–1038 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  121. Scherf, N. & Huisken, J. The smart and gentle microscope. Nat. Biotechnol. 33, 815–818 (2015).

    Article  CAS  PubMed  Google Scholar 

  122. Fahrbach, F. O., Gurchenkov, V., Alessandri, K., Nassoy, P. & Rohrbach, A. Light-sheet microscopy in thick media using scanned Bessel beams and two-photon fluorescence excitation. Opt. Express 21, 13824–13839 (2013).

    Article  CAS  PubMed  Google Scholar 

  123. Tomer, R., Ye, L., Hsueh, B. & Deisseroth, K. Advanced CLARITY for rapid and high-resolution imaging of intact tissues. Nat. Protoc. 9, 1682–1697 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  124. Marx, V. Microscopy: seeing through tissue. Nat. Methods 11, 1209–1214 (2014).

    Article  CAS  PubMed  Google Scholar 

  125. Wu, Y. et al. Inverted selective plane illumination microscopy (iSPIM) enables coupled cell identity lineaging and neurodevelopmental imaging in Caenorhabditis elegans. Proc. Natl Acad. Sci. USA 108, 17708–17713 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  126. Migliori, B. et al. Light sheet theta microscopy for rapid high-resolution imaging of large biological samples. BMC Biol. 16, 57–19 (2018).

    Article  Google Scholar 

  127. Mcgorty, R. et al. Open-top selective plane illumination microscope for conventionally mounted specimens. Opt. Express 23, 16142–16153 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  128. Voigt, F. F. et al. The mesoSPIM initiative: open-source light-sheet microscopes for imaging cleared tissue. Nat. Methods 16, 1105–1108 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  129. Chakraborty, T. et al. Light-sheet microscopy of cleared tissues with isotropic, subcellular resolution. Nat. Methods 16, 1109–1113 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  130. Bouchard, M. B. et al. Swept confocally-aligned planar excitation (SCAPE) microscopy for high speed volumetric imaging of behaving organisms. Nat. Photon. 9, 113–119 (2015).

    Article  CAS  Google Scholar 

  131. Voleti, V. et al. Real-time volumetric microscopy of in vivo dynamics and large-scale samples with SCAPE 2.0. Nat. Methods 16, 1054–1062 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  132. Yang, B. et al. Epi-illumination SPIM for volumetric imaging with high spatial-temporal resolution. Nat. Methods 16, 501–504 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  133. Strnad, P. et al. Inverted light-sheet microscope for imaging mouse pre-implantation development. Nat. Methods 13, 139–142 (2016).

    Article  CAS  PubMed  Google Scholar 

  134. Barner, L. A., Glaser, A. K., True, L. D., Reder, N. P. & Liu, J. T. C. Solid immersion meniscus lens (SIMlens) for open-top light-sheet microscopy. Opt. Lett. 44, 4451–4454 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  135. Dunsby, C. Optically sectioned imaging by oblique plane microscopy. Opt. Express 16, 20306–20316 (2008).

    Article  CAS  PubMed  Google Scholar 

  136. Millett-Sikking, A. & York, A. AndrewGYork/high_na_single_objective_lightsheet: work-in-progress. Zenodo https://doi.org/10.5281/zenodo.3376243 (2019).

  137. Kumar, M., Kishore, S., Nasenbeny, J., McLean, D. L. & Kozorovitskiy, Y. Integrated one- and two-photon scanned oblique plane illumination (SOPi) microscopy for rapid volumetric imaging. Opt. Express 26, 13027–13041 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  138. Sapoznik, E. et al. A versatile oblique plane microscope for large-scale and high-resolution imaging of subcellular dynamics. eLife 9, 279 (2020).

    Article  Google Scholar 

  139. Bishop, K. W., Glaser, A. K. & Liu, J. T. C. Performance tradeoffs for single- and dual-objective open-top light-sheet microscope designs: a simulation-based analysis. Biomed. Opt. Express 11, 4627–4650 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  140. Glaser, A. K., Bishop, K. W., Barner, L. A., Serafin, R. B. & Liu, J. T. C. A hybrid open-top light-sheet microscope for multi-scale imaging of cleared tissues. Preprint at bioRxiv https://doi.org/10.1101/2020.05.06.081745 (2020).

  141. Barner, L. A., Glaser, A. K., Huang, H., True, L. D. & Liu, J. T. C. Multi-resolution open-top light-sheet microscopy to enable efficient 3D pathology workflows. Biomed. Opt. Express 11, 6605–6619 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  142. Bria, A., Bernaschi, M., Guarrasi, M. & Iannello, G. Exploiting multi-level parallelism for stitching very large microscopy images. Front. Neuroinform. 13, 41 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  143. Bria, A. & Iannello, G. TeraStitcher—a tool for fast automatic 3D-stitching of teravoxel-sized microscopy images. BMC Bioinformatics 13, 316 (2012).

    Article  PubMed  PubMed Central  Google Scholar 

  144. Hörl, D. et al. BigStitcher: reconstructing high-resolution image datasets of cleared and expanded samples. Nat. Methods 16, 870–874 (2019).

    Article  PubMed  Google Scholar 

  145. Amat, F. et al. Efficient processing and analysis of large-scale light-sheet microscopy data. Nat. Protoc. 10, 1679–1696 (2015).

    Article  CAS  PubMed  Google Scholar 

  146. Balazs, B., Deschamps, J., Albert, M., Ries, J. & Hufnagel, L. A real-time compression library for microscopy images. Preprint at bioRxiv https://doi.org/10.1101/164624 (2017).

  147. Stefansson, H. N. et al. Wavelet compression of three-dimensional time-lapse biological image data. Microsc. Microanal. 11, 9–17 (2005).

    Article  CAS  PubMed  Google Scholar 

  148. Giacomelli, M. G. et al. Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging. PLoS ONE 11, e0159337 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  149. Serafin, R., Xie, W., Glaser, A. K. & Liu, J. T. C. FalseColor-Python: a rapid intensity-leveling and digital-staining package for fluorescence-based slide-free digital pathology. PLoS ONE 15, e0233198 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  150. Schindelin, J. et al. Fiji: an open-source platform for biological-image analysis. Nat. Methods 9, 676–682 (2012).

    Article  CAS  PubMed  Google Scholar 

  151. Schneider, C. A., Rasband, W. S. & Eliceiri, K. W. NIH Image to ImageJ: 25 years of image analysis. Nat. Methods 9, 671–675 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  152. Rueden, C. T. et al. ImageJ2: ImageJ for the next generation of scientific image data. BMC Bioinformatics 18, 529 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  153. Edelstein, A., Amodaj, N., Hoover, K., Vale, R. & Stuurman, N. Computer control of microscopes using manager. Curr. Protoc. Mol. Biol. 92, 14.20.1–14.20.17 (2010).

    Article  Google Scholar 

  154. Pietzsch, T., Saalfeld, S., Preibisch, S. & Tomancak, P. BigDataViewer: visualization and processing for large image data sets. Nat. Methods 12, 481–483 (2015).

    Article  CAS  PubMed  Google Scholar 

  155. Pitrone, P. G. et al. OpenSPIM: an open-access light-sheet microscopy platform. Nat. Methods 10, 598–599 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  156. Marx, V. Microscopy: OpenSPIM 2.0. Nat. Methods 13, 979–982 (2016).

    Article  CAS  PubMed  Google Scholar 

  157. Carpenter, A. E., Kamentsky, L. & Eliceiri, K. W. A call for bioimaging software usability. Nat. Methods 9, 666–670 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  158. Cardona, A. & Tomancak, P. Current challenges in open-source bioimage informatics. Nat. Methods 9, 661–665 (2012).

    Article  CAS  PubMed  Google Scholar 

  159. Ghaznavi, F., Evans, A., Madabhushi, A. & Feldman, M. Digital imaging in pathology: whole-slide imaging and beyond. Annu. Rev. Pathol. 8, 331–359 (2013).

    Article  CAS  PubMed  Google Scholar 

  160. Niazi, M. K. K., Parwani, A. V. & Gurcan, M. N. Digital pathology and artificial intelligence. Lancet Oncol. 20, e253–e261 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  161. Pantanowitz, L. et al. Review of the current state of whole slide imaging in pathology. J. Pathol. Inform. 2, 36 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  162. Vaidya, P. et al. CT derived radiomic score for predicting the added benefit of adjuvant chemotherapy following surgery in stage I, II resectable non-small cell lung cancer: a retrospective multi-cohort study for outcome prediction. Lancet Digit. Health 2, e116–e128 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  163. Madabhushi, A. & Lee, G. Image analysis and machine learning in digital pathology: challenges and opportunities. Med. Image Anal. 33, 170–175 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  164. Cheplygina, V., de Bruijne, M. & Pluim, J. P. W. Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Med. Image Anal. 54, 280–296 (2019).

    Article  PubMed  Google Scholar 

  165. Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J. N., Wu, Z. & Ding, X. Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation. Med. Image Anal. 63, 101693 (2020).

    Article  PubMed  Google Scholar 

  166. He, K., Fan, H., Wu, Y., Xie, S. & Girshick, R. Momentum contrast for unsupervised visual representation learning. Preprint at https://arxiv.org/abs/1911.05722 (2019).

  167. Khan, A. M., Rajpoot, N., Treanor, D. & Magee, D. A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution. IEEE Trans. Biomed. Eng. 61, 1729–1738 (2014).

    Article  PubMed  Google Scholar 

  168. Belthangady, C. & Royer, L. A. Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nat. Methods 16, 1215–1225 (2019).

    Article  CAS  PubMed  Google Scholar 

  169. Bhargava, H. K. et al. Computationally derived image signature of stromal morphology is prognostic of prostate cancer recurrence following prostatectomy in African American patients. Clin. Cancer Res. 26, https://doi.org/10.1158/1078-0432.CCR-19-2659 (2020).

  170. Chandramouli, S. et al. Computer extracted features from initial H&E tissue biopsies predict disease progression for prostate cancer patients on active surveillance. Cancers 12, 2708 (2020).

    Article  CAS  PubMed Central  Google Scholar 

  171. Christiansen, E. M. et al. In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173, 792–803 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  172. Ounkomol, C., Seshamani, S., Maleckar, M. M., Collman, F. & Johnson, G. R. Label-free prediction of three-dimensional fluorescence images from transmitted-light microscopy. Nat. Methods 15, 917–920 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  173. Kiemen, A. et al. In situ characterization of the 3D microanatomy of the pancreas and pancreatic cancer at single cell resolution. Preprint at bioRxiv https://doi.org/10.1101/2020.12.08.416909 (2020).

  174. Maier, A. K. et al. Learning with known operators reduces maximum training error bounds. Nat. Mach. Intell. 1, 373–380 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  175. Maier, A., Syben, C., Lasser, T. & Riess, C. A gentle introduction to deep learning in medical image processing. Z. Med. Phys. 29, 86–101 (2019).

    Article  PubMed  Google Scholar 

  176. Dou, Q. et al. 3D deeply supervised network for automated segmentation of volumetric medical images. Med. Image Anal. 41, 40–54 (2017).

    Article  PubMed  Google Scholar 

  177. Martel, A. L. et al. An image analysis resource for cancer research: PIIP—Pathology Image Informatics Platform for visualization, analysis, and management. Cancer Res. 77, e83–e86 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  178. Bankhead, P. et al. QuPath: open source software for digital pathology image analysis. Sci. Rep. 7, 16878 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  179. Rivenson, Y. et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat. Biomed. Eng. 3, 466–477 (2019).

    Article  CAS  PubMed  Google Scholar 

  180. Jackson, C. R., Sriharan, A. & Vaickus, L. J. A machine learning algorithm for simulating immunohistochemistry: development of SOX10 virtual IHC and evaluation on primarily melanocytic neoplasms. Mod. Pathol. 33, 1638–1648 (2020).

    Article  PubMed  Google Scholar 

  181. Janowczyk, A., Zuo, R., Gilmore, H., Feldman, M. & Madabhushi, A. HistoQC: an open-source quality control tool for digital pathology slides. JCO Clin. Cancer Inform. https://doi.org/10.1200/CCI.18.00157 (2019).

  182. Leo, P. et al. Stable and discriminating features are predictive of cancer presence and Gleason grade in radical prostatectomy specimens: a multi-site study. Sci. Rep. 8, 14918 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  183. Liu, J. et al. An integrated TCGA Pan-Cancer Clinical Data Resource to drive high-quality survival outcome analytics. Cell 173, 400–416 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  184. Paik, S. et al. Gene expression and benefit of chemotherapy in women with node-negative, estrogen receptor-positive breast cancer. J. Clin. Oncol. 24, 3726–3734 (2006).

    Article  CAS  PubMed  Google Scholar 

  185. Paik, S. et al. A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer. N. Engl. J. Med 351, 2817–2826 (2004).

    Article  CAS  PubMed  Google Scholar 

  186. Bast, R. C. & Hortobagyi, G. N. Individualized care for patients with cancer—a work in progress. N. Engl. J. Med. 351, 2865–2867 (2004).

    Article  CAS  PubMed  Google Scholar 

  187. Engel, K. B., Vaught, J. & Moore, H. M. National Cancer Institute biospecimen evidence-based practices: a novel approach to pre-analytical standardization. Biopreserv. Biobank. 12, 148–150 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  188. Sparano, J. A. et al. Prospective validation of a 21-gene expression assay in breast cancer. N. Engl. J. Med. 373, 2005–2014 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  189. Allen, T. C. Food and Drug Administration approval of laboratory tests. Arch. Pathol. Lab. Med. 137, 13–18 (2013).

    Article  CAS  PubMed  Google Scholar 

  190. Evans, A. J. et al. US Food and Drug Administration approval of whole slide imaging for primary diagnosis: a key milestone is reached and new questions are raised. Arch. Pathol. Lab. Med. 142, 1383–1387 (2018).

    Article  PubMed  Google Scholar 

  191. Mukhopadhyay, S. et al. Whole slide imaging versus microscopy for primary diagnosis in surgical pathology: a multicenter blinded randomized noninferiority study of 1992 cases (pivotal study). Am. J. Surg. Pathol. 42, 39–52 (2018).

    Article  PubMed  Google Scholar 

  192. Joly, Y. et al. Regulatory approval for new pharmacogenomic tests: a comparative overview. Food Drug Law J. 66, 1–24 (2011).

    PubMed  Google Scholar 

  193. D’Angelo, R. et al. Facing the inevitable: being prepared for regulatory requirements for laboratory developed tests. Am. J. Clin. Pathol. 149, 484–498 (2018).

    Article  PubMed  Google Scholar 

  194. Madabhushi, A., Feldman, M. D. & Leo, P. Deep-learning approaches for Gleason grading of prostate biopsies. Lancet Oncol. 21, 187–189 (2020).

    Article  PubMed  Google Scholar 

  195. King, C. R. & Long, J. P. Prostate biopsy grading errors: a sampling problem? Int J. Cancer 90, 326–330 (2000).

    Article  CAS  PubMed  Google Scholar 

  196. Ruijter, E., van Leenders, G., Miller, G., Debruyne, F. & van de Kaa, C. Errors in histological grading by prostatic needle biopsy specimens: frequency and predisposing factors. J. Pathol. 192, 229–233 (2000).

    Article  CAS  PubMed  Google Scholar 

  197. Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1301–1309 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  198. Chen, D. S. & Mellman, I. Elements of cancer immunity and the cancer-immune set point. Nature 541, 321–330 (2017).

    Article  CAS  PubMed  Google Scholar 

  199. Rizvi, N. A. et al. Mutational landscape determines sensitivity to PD-1 blockade in non-small cell lung cancer. Science 348, 124–128 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  200. Viaud, S. et al. The intestinal microbiota modulates the anticancer immune effects of cyclophosphamide. Science 342, 971–976 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  201. Vétizou, M. et al. Anticancer immunotherapy by CTLA-4 blockade relies on the gut microbiota. Science 350, 1079–1084 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  202. Herbst, R. S. et al. Predictive correlates of response to the anti-PD-L1 antibody MPDL3280A in cancer patients. Nature 515, 563–567 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  203. Gajewski, T. F., Schreiber, H. & Fu, Y.-X. Innate and adaptive immune cells in the tumor microenvironment. Nat. Immunol. 14, 1014–1022 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  204. Corredor, G. et al. Spatial architecture and arrangement of tumor-infiltrating lymphocytes for predicting likelihood of recurrence in early-stage non-small cell lung cancer. Clin. Cancer Res. 25, 1526–1534 (2019).

    Article  CAS  PubMed  Google Scholar 

  205. Annecchino, L. A. et al. Robotic automation of in vivo two-photon targeted whole-cell patch-clamp electrophysiology. Neuron 95, 1048–1055 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  206. Long, B., Li, L., Knoblich, U., Zeng, H. & Peng, H. 3D image-guided automatic pipette positioning for single cell experiments in vivo. Sci. Rep. 5, 18426 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  207. Suk, H.-J. et al. Closed-loop real-time imaging enables fully automated cell-targeted patch-clamp neural recording in vivo. Neuron 95, 1037–1047 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  208. Li, L. et al. Co-registration of ex vivo surgical histopathology and in vivo T2 weighted MRI of the prostate via multi-scale spectral embedding representation. Sci. Rep. 7, 8717 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  209. Rusu, M. et al. Co-registration of pre-operative CT with ex vivo surgically excised ground glass nodules to define spatial extent of invasive adenocarcinoma on in vivo imaging: a proof-of-concept study. Eur. Radiol. 27, 4209–4217 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  210. Rusu, M. et al. Prostatome: a combined anatomical and disease based MRI atlas of the prostate. Med. Phys. 41, 072301 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  211. Antunes, J. et al. Coregistration of preoperative MRI with ex vivo mesorectal pathology specimens to spatially map post-treatment changes in rectal cancer onto in vivo imaging: preliminary findings. Acad. Radiol. 25, 833–841 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  212. Schillaci, O. et al. Combining diagnostic imaging and pathology for improving diagnosis and prognosis of cancer. Contrast Media Mol. Imaging 2019, 9429761 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  213. Grönroos, T. J. et al. Hypoxia, blood flow and metabolism in squamous-cell carcinoma of the head and neck: correlations between multiple immunohistochemical parameters and PET. BMC Cancer 14, 876 (2014).

    Article  PubMed  PubMed Central  Google Scholar 

  214. Surov, A., Meyer, H. J. & Wienke, A. Standardized uptake values derived from 18F-FDG PET may predict lung cancer microvessel density and expression of KI 67, VEGF, and HIF-1α but not expression of cyclin D1, PCNA, EGFR, PD L1, and p53. Contrast Media Mol. Imaging 2018, 9257929 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  215. Bensch, F. et al. 89Zr-atezolizumab imaging as a non-invasive approach to assess clinical response to PD-L1 blockade in cancer. Nat. Med. 24, 1852–1858 (2018).

    Article  CAS  PubMed  Google Scholar 

  216. Gibson, E. et al. Registration of prostate histology images to ex vivo MR images via strand-shaped fiducials. J. Magn. Reson. Imaging 36, 1402–1412 (2012).

    Article  PubMed  Google Scholar 

  217. Mori, K. From macro-scale to micro-scale computational anatomy: a perspective on the next 20 years. Med. Image Anal. 33, 159–164 (2016).

    Article  PubMed  Google Scholar 

  218. O'Keefe, E. B., Meltzer, J. P. & Bethea,T. N. Health disparities and cancer: racial disparities in cancer mortality in the United States, 2000–2010. Front. Public Health 3, 51 (2015).

    PubMed  PubMed Central  Google Scholar 

  219. Zhou, M. et al. Non-small cell lung cancer radiogenomics map identifies relationships between molecular and imaging phenotypes with prognostic implications. Radiology 286, 307–315 (2018).

    Article  PubMed  Google Scholar 

  220. Vaidya, P. et al. RaPtomics: integrating radiomic and pathomic features for predicting recurrence in early stage lung cancer. In Proc. SPIE 10581, Medical Imaging 2018: Digital Pathology (eds Gurcan, M. N. & Tomaszewski, J. E.) 105810M (International Society for Optics and Photonics, 2018).

  221. Savage, R. S. & Yuan, Y. Predicting chemoinsensitivity in breast cancer with ’omics/digital pathology data fusion. R. Soc. Open Sci. 3, 140501 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  222. Pinker, K., Chin, J., Melsaether, A. N., Morris, E. A. & Moy, L. Precision medicine and radiogenomics in breast cancer: new approaches toward diagnosis and treatment. Radiology 287, 732–747 (2018).

    Article  PubMed  Google Scholar 

  223. Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl Acad. Sci. USA 115, E2970–E2979 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  224. Penzias, G. et al. Identifying the morphologic basis for radiomic features in distinguishing different Gleason grades of prostate cancer on MRI: preliminary findings. PLoS ONE 13, e0200730 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  225. Dainis, A. M. & Ashley, E. A. Cardiovascular precision medicine in the genomics era. JACC Basic Transl. Sci. 3, 313–326 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  226. Kai, C., Uchiyama, Y., Shiraishi, J., Fujita, H. & Doi, K. Computer-aided diagnosis with radiogenomics: analysis of the relationship between genotype and morphological changes of the brain magnetic resonance images. Radiol. Phys. Technol. 11, 265–273 (2018).

    Article  PubMed  Google Scholar 

  227. Montalto, M. C. An industry perspective: an update on the adoption of whole slide imaging. J. Pathol. Inform. 7, 18 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We acknowledge research grants from the Department of Defense (DoD) Prostate Cancer Research Program through W81XWH-18-10358 (J.T.C.L. and L.D.T.), W81XWH-19-1-0589 (N.P.R.), W81XWH-20-1-0851(A.M. and J.T.C.L.) and W81XWH-15-1-0558 (A.M.); the DoD Breast Cancer Research Program W81XWH-19-1-0668 (A.M.); the DoD Lung Cancer Research Program W81XWH-18-1-0440 (A.M.); the DoD Peer Reviewed Cancer Research Program W81XWH-16-1-0329 (A.M.); the Ohio Third Frontier Technology Validation Fund (A.M.); the Wallace H. Coulter Foundation Program in the Department of Biomedical Engineering and The Clinical and Translational Science Award Program at Case Western Reserve University (A.M.); the Prostate Cancer Foundation (N.P.R.); and the National Science Foundation 1934292 HDR: I-DIRSE-FW (J.T.C.L.). We also acknowledge grants from the National Institutes of Health (NIH) National Cancer Institute (NCI) through K99CA240681 (A.K.G.), R01CA175391 (J.T.C.L.), R01CA244170 (J.T.C.L.), R01CA199996 (K.W.E.), U24CA199374 (A.M.), R01CA202752 (A.M.), R01CA208236 (A.M.), R01CA216579 (A.M.), R01CA220581 (A.M.), R01CA249992 (A.M.), U01CA248226 (A.M.), U54CA254566 (A.M.) and U01CA239055 (A.M.); from the NIH National Institute of Biomedical Imaging and Bioengineering (NIBIB) through R43EB028736 (A.M.); from the NIH National Heart, Lung and Blood Institute (NHLBI) through R01HL151277 (A.M.); from the NIH National Institute of General Medical Sciences (NIGMS) through P41GM135019 (K.W.E.); from the NIH National Center for Research Resources through C06RR12463 (A.M.); from the US Department of Veterans Affairs IBX004121A (A.M.); from the Institute for Prostate Cancer Research at the University of Washington (L.D.T.), and from the National Cancer Institute Breast Cancer SPORE/Safeway Foundation at the Fred Hutchinson Cancer Research Center (J.T.C.L). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, the US Department of Veterans Affairs, the Department of Defense or the United States Government.

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, University of Washington, Seattle, WA, USA

    Jonathan T. C. Liu, Adam K. Glaser & Nicholas P. Reder

  2. Department of Laboratory Medicine and Pathology, University of Washington, Seattle, WA, USA

    Jonathan T. C. Liu, Lawrence D. True & Nicholas P. Reder

  3. Department of Bioengineering, University of Washington, Seattle, WA, USA

    Jonathan T. C. Liu

  4. Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USA

    Kaustav Bera & Anant Madabhushi

  5. Department of Medical Physics, University of Wisconsin, Madison, WI, USA

    Kevin W. Eliceiri

  6. Department of Biomedical Engineering, University of Wisconsin, Madison, WI, USA

    Kevin W. Eliceiri

  7. Morgridge Institute for Research, Madison, WI, USA

    Kevin W. Eliceiri

  8. Louis Stokes Cleveland Veterans Administration Medical Center, Cleveland, OH, USA

    Anant Madabhushi

Authors
  1. Jonathan T. C. Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adam K. Glaser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kaustav Bera
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lawrence D. True
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nicholas P. Reder
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kevin W. Eliceiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Anant Madabhushi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors discussed and wrote the manuscript. J.T.C.L. and A.K.G. drew the figures. J.T.C.L. coordinated the effort and led the writing.

Corresponding authors

Correspondence to Jonathan T. C. Liu, Kevin W. Eliceiri or Anant Madabhushi.

Ethics declarations

Competing interests

J.T.C.L., A.K.G., N.P.R. and L.D.T. are co-founders and shareholders of Lightspeed Microscopy, of which J.T.C.L. and N.P.R. are board members and N.P.R. is the chief executive officer. Technology developed by J.T.C.L., A.K.G., N.P.R. and L.D.T. at the University of Washington has been licensed by Lightspeed Microscopy. K.W.E. is a co-founder and shareholder of OnLume, and a scientific advisory consultant for Bruker Corporation and Elephas Corporation. A.M. is an equity holder in Elucid Bioimaging and Inspirata, has served as a scientific advisory board member for Inspirata, AstraZeneca, Bristol Myers-Squibb, Aiforia and Merck, has had sponsored research agreements with Philips and Inspirata, has technology that is licensed to Elucid Bioimaging and Inspirata, and is involved in a NIH U24 grant with PathCore Inc. and in three different R01 grants with Inspirata Inc. A.M. has sponsored research projects from AstraZeneca, Bristol Myers-Squibb and Boehringer-Ingelheim.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, J.T.C., Glaser, A.K., Bera, K. et al. Harnessing non-destructive 3D pathology. Nat Biomed Eng 5, 203–218 (2021). https://doi.org/10.1038/s41551-020-00681-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41551-020-00681-x

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing