A review of snapshot multidimensional optical imaging: Measuring photon tags in parallel
Section snippets
Introduction to multidimensional imaging
When performing optical measurement with a limited photon budget, it is important to assure that each detected photon provides as much information as possible. Conventional optical imaging systems generally capture light with just two characteristics (), measuring its intensity in a 2D () lattice. However, this throws away much of the information content actually carried by a photon. This information can be written in nine dimensions as (): the spatial coordinates (
General acquisition schemes and advantages of parallel measurement in multidimensional imaging
To acquire a multidimensional datacube, a system must be able to differentiate photons with different characteristics. The most intuitive approach is to successively apply a variety of filters to the incident light and let photons with only desired characteristics pass through at each stage (Fig. 1a). Unfortunately, this results in a severe loss in optical throughput. By contrast, if an approach directs, rather than filters, photons with different tags towards distinct pixels on an FPA, the
Snapshot spectral imaging ()
Rather than simply capturing two-dimensional intensity images like a monochromatic camera or measuring spectra like a spectrometer, a spectral imager acquires entire 3D datacubes (, , ) for multivariate analysis, providing structural, molecular, and functional information about the sample with unprecedented detail [28], [29]. Using the conceptual framework in Fig. 2, snapshot spectral imagers can be divided into two categories. In the direct-measurement category, representative techniques
Discussions and outlook
In this review, we categorized snapshot multidimensional imagers based on their acquisition strategies and reconstruction strategies, and we discussed their state-of-the-art implementations in spectral imaging, plenoptic imaging, volumetric imaging, temporal imaging, and polarization imaging. Compared with their scanning-based counterparts, snapshot imagers have a remarkable advantage in optical throughput. The more datacube dimensions a snapshot imager measures, the greater the advantage in
Acknowledgments
The authors thank Professor James Ballard for close reading of the manuscript. This work was supported in part by National Institutes of Health grants DP1 EB016986 (NIH Director’s Pioneer Award) and R01 CA186567 (NIH Director’s Transformative Research Award). L.V.W. has a financial interest in Microphotoacoustics, Inc. and Endra, Inc., which, however, did not support this work.
References (251)
Basic principles of integral field spectroscopy
New Astron. Rev.
(2006)- et al.
Single-cell viability assessment with a novel spectro-imaging system
J. Biotechnol.
(2002) - et al.
Computed tomography-based spectral imaging for fluorescence microscopy
Biophys. J.
(2001) - et al.
Cellular image analysis and imaging by flow cytometry
Clin. Lab. Med.
(2007) - et al.
Hyperspectral confocal microscope
Appl. Opt.
(2006) - et al.
Design, construction, characterization, and application of a hyperspectral microarray scanner
Appl. Opt.
(2004) - et al.
Hyperspectral imaging system using acousto-optic tunable filter for flow cytometry applications
Cytometry A
(2006) - Kodak, KAF-50100 IMAGE SENSOR, in: Kodak Products,...
- et al.
Computational cameras: Convergence of optics and processing
IEEE Trans. Image Process.
(2011) - et al.
Computational plenoptic imaging
Comput. Graph. Forum
(2011)
Compressed sensing for practical optical imaging systems: a tutorial
Opt. Eng.
Compressive sensing demystified
Opt. Photonics News
Ultra-precision machining systems; an enabling technology for perfect surfaces
Field Guide to Optical Fabrication
Single disperser design for coded aperture snapshot spectral imaging
Appl. Opt.
Video rate spectral imaging using a coded aperture snapshot spectral imager
Opt. Express
Image mapping spectrometry: calibration and characterization
Opt. Eng.
Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy
Opt. Express
Compact Image Slicing Spectrometer (ISS) for hyperspectral fluorescence microscopy
Opt. Express
Multi-Dimensional Imaging
The luminosity of spectrometers with prisms, gratings, or Fabry–Perot etalons
J. Opt. Soc. Am.
Snapshot advantage: a review of the light collection improvement for parallel high-dimensional measurement systems
Opt. Eng.
Review of passive imaging polarimetry for remote sensing applications
Appl. Opt.
American National Standard for Safe Use of Lasers
Depth-resolved image mapping spectrometer (IMS) with structured illumination
Opt. Express
Light field microscopy
Sequentially timed all-optical mapping photography (STAMP)
Nature Photon.
Single-shot compressed ultrafast photography at one hundred billion frames per second
Nature
Optical hyperspectral imaging in microscopy and spectroscopy–a review of data acquisition
J. Biophotonics
Medical hyperspectral imaging: a review
J. Biomed. Opt.
Snapshot hyperspectral imaging: the hyperpixel array camera
Hyperspectral imaging systems
Distinguishing modes of cell death using imagestream (TM) multispectral imaging cytometry
Cytometry A
Generalization of the Lyot filter and its application to snapshot spectral imaging
Opt. Express
Compact real-time birefringent imaging spectrometer
Opt. Express
Compact snapshot birefringent imaging Fourier transform spectrometer
White-light Sagnac interferometer for snapshot multispectral imaging
Appl. Opt.
Large-image-format computed tomography imaging spectrometer for fluorescence microscopy
Opt. Express
Original image slicer designed for integral field spectroscopy with the near-infrared spectrograph for the James webb space telescope
Opt. Eng.
Development of image mappers for hyperspectral biomedical imaging applications
Appl. Opt.
Correction of vignetting and distortion errors induced by two-axis light beam steering
Opt. Eng.
Real-time snapshot hyperspectral imaging endoscope
J. Biomed. Opt.
Real-time hyperspectral fluorescence imaging of pancreatic -cell dynamics with the image mapping spectrometer
J. Cell Sci.
Snapshot image-mapping spectrometer for hyperspectral fluorescence microscopy
Opt. Photonics News
Quantitative comparison between full-spectrum and filter-based imaging in hyperspectral fluorescence microscopy
J. Microsc.
Snapshot hyperspectral retinal camera with the Image Mapping Spectrometer (IMS)
Biomed. Opt. Express
Cited by (131)
Lenslet-array-based snapshot hyperspectral imaging polarimeter using opposite spectral modulation
2024, Optics and Lasers in EngineeringTrapezoidal dual-function splitter under second Bragg incidence
2024, Optics CommunicationsCurrent advances and future perspectives of image fusion: A comprehensive review
2023, Information FusionCitation Excerpt :This model comprises three parts: feature extraction, fusion, and reconstruction, which resolve the LD and feature representation issues. Photons carry a prosperity of information, which can be represented by nine-dimensional data (x, y, z, θ, φ, λ, t, ψ, χ), where (x, y, z) represents the spatial dimensions, (θ, φ,) represents the propagation angle of light, λ represents the wavelength, t represents the emission time of photons, (ψ, χ) represents the polarization state of light and ellipticity [218]. Imaging technology is a technology for recording target photon information.
Pixelated polarizers array based coded aperture snapshot spectropolarimetric imaging with side information
2022, Optics CommunicationsMultidimensional photodetection of light fields based on metasurfaces or two-dimensional materials
2024, Applied Physics Letters