Synthetic Wavelength Holography: Snapshot Non-Line-of-Sight Imaging with High-Resolution and Wide Field of View

,


Introduction
There are numerous instances of imaging within the physical sciences wherein an opaque barrier (such as a wall) or a scattering medium (such as fog or tissue) impedes direct view of the object.Over the years, many attempts [1 -13] have been made to non-invasively recover images of objects obscured from direct view.These techniques are collectively referred to as 'Non Line-of-Sight Imaging' (NLoS) in our work.The problem is enjoying renewed attention due to emerging applications in autonomous navigation, planetary exploration, industrial inspection, and early-warning systems for first-responders [14][15][16][17][18][19][20][21][22][23].
Broadly speaking, current approaches to NLoS imaging circumvent the effect of scatter in one of two geometries: discrete scattering events distributed across multiple interfaces such as walls, and continuous scattering within a volume such as fog or tissue.Herein, we introduce a holographic approach -"Synthetic Wavelength Holography" (SWH) -that advances the state-of-the-art in NLoS imaging by exploiting spectral correlations in scattered light to see through any scattering geometry.
We make specific use of the observation that coherent light at two closely spaced wavelengths λ 1 ,λ 2 traversing near identical geometric paths in a scattering medium, preserves phase information at scales exceeding a 'Synthetic Wave-length' (SWL) Λ >> λ 1 ,λ 2 [24,25].We provide experi-Fig. 1. Imaging objects obscured from direct view using 'Synthetic Wavelength Holography'(SWL).The approach combines four key attributes highlighted in the following NLoS application scenarios: In each example, a scattering surface or medium is used to indirectly illuminate, and intercept light scattered by the hidden objects.a) A small probing area allows to inspect defects in tightly confined spaces, e.g., in running aircraft engines.b) A large FoV allows to measure/detect hidden objects without previous knowledge of their position as, e.g., important when navigating in degraded visual environments.c) High spatial resolution allows for the measurement of small structures, such as non-invasive imaging of brain vessels through the skull.d) High temporal resolution allows to image objects in motion, e.g., to discern cardiac arrhythmia through the chest.The combination of these attributes in a single approach is unprecedented in the current state of the art.
which cannot be larger than the FoV.
The wide disparity in the FoV and resolution of current NLoS Imaging schemes greatly limits their utility.We demonstrate how to recover a truly holographic description of the obscured scene with a high spatial resolution, over a wide field of regard.Our SWH approach exploits spectral correlations in scattered light at optical wavelengths λ 1 ,λ 2 to assemble a hologram of the obscured objects at the SWL . The approach combines a unique set of capabilities in providing a degree of versatility that is unmatched by competing NLoS approaches: • Small Probing Area (see Fig. 1a): ToF based NLoS schemes use probing areas ∼ 1m × 1m.SWH provides the ability to image obscured objects in tightly confined spaces by simultaneously illuminating and observing a small area (58mm × 58mm in our experiments).Potential applications include inline defect detection and inspection in heavy machinery such as running turbines, and endoscopic imaging applications.
• Large Field of View (see Fig. 1b): ME-based approaches produce highly restricted FOVs (< 2 ○ for drywall), while SWH provides the ability to recover holograms of obscured objects over a hemispherical FoV that far exceeds the limited angular extent of the memory effect.Potential applications include the design of early-warning systems in automotive sensing and planetary exploration.
• High Spatial Resolution (see Fig. 1c): ToF-based approaches produce low spatial resolutions (∼ cm).SWH provides the ability to resolve small features on obscured objects (up to < 1mm in our experiments) without requiring prior knowledge of the scattering geometry or attributes of the scattering medium such as the transmission matrix [32,33].Potential applications include non-invasive imaging of blood vessels through tissue.
• High Temporal Resolution (see Fig.   λ2) is subject to multiple scattering processes in or at the scatterer (which could be wall, tissue, fog,...) and the rough object surface.The introduced maximal pathlength variation Ψmax leads to a complete randomization of E(λ1), E(λ2) when arriving at the detector.However, computational mixing of the speckled fields E(λ1) ⋅ E * (λ2) = E(Λ), yields a complex-valued hologram of the object at a 'Synthetic Wavelength' (SWL) . The object is reconstructed by backpropagating E(Λ) with the SWL Λ. b) and c) Schematic setups for NLoS imaging around corners (b) and NLoS imaging through scatterers (c) with the SWH principle: The sample beam illuminates a spot on the wall/scatterer (the 'Virtual Source' VS), which scatters light towards the obscured object.A small fraction of the light incident on the object is scattered back to the wall/scatterer where it hits the 'Virtual Detector' (VD).The VD is imaged by the camera, meaning that the synthetic hologram is captured at the VD surface.
For closely spaced optical wavelength λ 1 and λ 2 , the SWL is orders of magnitude larger than λ 1 ,λ 2 so that the computationally recovered field E(Λ) is robust to the deleterious effects of scattering.An image of the hidden object can be retrieved by numerical backpropagation of E(Λ) at the SWL Λ, as illustrated in Fig. 2a.
The strikingly simple and computationally inexpensive strategy described above can significantly improve the visual acuity of imaging systems confounded by scatter.The computational immunity to scatter afforded by the existence of spectral correlations, relies only on the wave nature of light.As a consequence, the principles underlying SWH can be readily extended to other wave phenomena such as radio waves and acoustic waves (ultrasound).

NLoS Imaging Techniques 'Looking around corners'.
We use the scene arrangement depicted in Figure 2b where D is the diameter of the probing area at the VD, and  in Eq. 1, suggests that the resolution may be indefinitely improved by reducing the SWL Λ.This however is not the case.
The supplementary material includes a derivation for the specific case of 'looking around corners'.

341
The SWH approach, however, is not without limitations, 342 chief of which is the inability to recover phase information ceeding the SWL may be unambiguously recovered if the largest wavefront error Ψ max introduced during light transport through the scattering scene fulfills a Rayleigh Quarter Wavelength criterion (RQWR [58]) for the SWL, i.e., The wavefront error Ψ max represents the worst-case spread in the physical lengths of scattered light paths that share a common source location, object location and detector pixel.
For surface scattering processes, the spread in path lengths is limited by 2σ h , where σ h represents the RMS surface roughness (see Eq. 40 in supplementary material).This is demonstrated experimentally in Fig. 4b-e The principal distinction between scattering at discrete interfaces such as walls and continuous scattering through a volume, lies in the scale of wavefront error Ψ max .For instance, the typical wavefront error Ψ max for 'imaging around corners' is less than 1 millimeter, several centimeters for imaging through tissue, and many meters for long-range imaging through fog.It is expected that diffusive scatter over long propagation distances will severely limit the maximal achievable resolution.In the specific case of imaging through fog, we anticipate that the time-gating ability of FMCW Li-DAR [59] may be combined with SWH to see farther with a higher resolution than is otherwise possible.In the case of imaging through tissue, it is anticipated that ultrasound focusing aids may be combined with SWH to see farther into the brain with a higher resolution than is currently possible.
The notion of SWH as demonstrated in this paper has a broad range of applications including imaging through obscurants, fog, smoke, tissue, bone, face detection around corners, or 'pseudo-endoscopic' defect detection of mechanical assemblies during operation.These applications largely restrict attention to optical carrier frequencies.However, the true potential of our approach can be unlocked by transferring the notion to other wave phenomena.
The synthetic hologram E(Λ) is assembled by calculating: The synthetic phase map is eventually recovered from the     However, the speckle-artifacts increase due to the decorrelation of the two optical elds at λ1 and λ2.k)p) Reconstruction of an obscured point source for three different SWLs.k)-m) Phase of the synthetic holograms at the VD surface.n)-p) Reconstruction of the 'synthetic diffraction disc'.As in classical optics, the disc size varies linearly with the wavelength (in this case the SWL).The experimental value is close to the theoretical expectation.For p), the point source is reconstructed with sub-mm precision.q) Image of the targets used in the experiments of this paper: Two characters 'N' and 'U' with dimensions 15mm×20mm (plus black mountings).
1d): ToF-based 120 approaches require point-wise raster-scanning, while 121 SWH provides the ability to recover holograms of the 122 obscured object in a snapshot fashion using conven-123 tional focal plane array (FPA) technology.This al-124 lows to resolve object motion such as sensing cardiac 125 arrhythmia's through the chest, and remotely sensing 126 surface motion of small planetary bodies [34].127 The mathematical principles underlying SWH are by no 128 means restricted to the applications discussed above.The 129 approach can be adapted to other NLoS imaging tasks in-130 cluding imaging through deep turbulence, fog and turbid wa-131 ters, imaging through optically opaque barriers such as the 132 skull, or face identification around corners.The idea of utiliz-133 ing wavelength diversity to alleviate the effects of unwanted 134 aberrations in the detection of electromagnetic signals has 135 potential applications that go far beyond the original scope of 136 NLoS imaging.We conclude this paper by discussing bene-137 fits of applying the SWH principle in a diverse set of appli-138 cation areas such as medical ultrasound, synchrotron X-ray 139 diffraction imaging, and radio astronomy.

141Fig. 2 .
Fig. 2. Schematics of SWH for NLoS imaging around corners and NLoS imaging through scattering media.a) SWH image formation: A CW-tunable laser illuminates the scene at two slightly different wavelength λ1 and λ2.Each field E(λ1), E(λ2) is subject to multiple scattering processes in or at the scatterer (which could be wall, tissue, fog,...) and the rough object surface.The introduced maximal pathlength variation Ψmax leads to a complete randomization of E(λ1), E(λ2) when arriving at the detector.However, computational mixing of the speckled fields E(λ1) ⋅ E * (λ2) = E(Λ), yields a complex-valued hologram of the object at a 'Synthetic Wavelength' (SWL) Λ = Figures 3a-e illustrate the phase of the computationally as-

214z
is the propagation distance to the object.The ratio D 2z 215 defines the numerical aperture of the computational NLoS-216 imager.In the present example, a lateral resolution limit of 217 δx = 800µm is reached for a SWL of Λ = 280µm.Details of 218 the experiment are included in the methods section.219 The expression for the resolving power of SWH disclosed 220 Willomitzer et al. | Synthetic Wavelength Holography

Fig. 3 .
Fig. 3. Experimental results for imaging around corners with SWH.a)-j) Imaging the character 'N' (∼ 15mm × 20mm) at five different SWLs.a)-e) The phase of the synthetic holograms at the VD surface.f)-j) Respective reconstructions.The resolution of the reconstructions increases with decreasing SWL.However, the speckle-artifacts increase due to the decorrelation of the two optical fields at λ1 and λ2.k)-p) Reconstruction of an obscured point source for three different SWLs.k)-m) Phase of the synthetic holograms at the VD surface.n)-p) Reconstruction of the 'synthetic diffraction disc'.As in classical optics, the disc size varies linearly with the wavelength (in this case the SWL).The experimental value is close to the theoretical expectation.For p), the point source is reconstructed with sub-mm precision.q) Image of the targets used in the experiments of this paper: Two characters 'N' and 'U' with dimensions ∼ 15mm × 20mm (plus black mountings).

Fig. 4 .Fig. 5 .324
Fig. 4. Experimental results for imaging through scattering media with SWH.A schematic of the experimental setup is available in Fig. 2c.a) Scatterers obscuring the object: A 220 grit ground glass diffuser and a milky white plastic plate of ∼ 4mm thickness, both placed ∼ 1cm over a checker pattern to demonstrate the degradation in visibility.b)-e) Reconstructions of measurements taken through the ground glass diffuser.f)-i) Reconstructions of measurements taken through the milky plastic plate.The character can be reconstructed with impressive quality.The larger OPD in the plastic plate leads to greater decorrelation if the SWL is decreased.observationare best understood by recognizing that the visibility of ballistic light paths decays exponentially with the propagation distance through a scattering volume (in accordance with Beer's law [47]).

343Fig. 6 .
Fig. 6.Experimental Results for Wavefront sensing through scatterers with SWH.The data for this experiment where captured by the authors of [48] without the intention to be used for our approach.Nevertheless, our SHW reconstruction mechanism is able to recover residual phase variations in speckled wavefronts emerging from volumetric scattering samples.a) and b) Scattered (speckled) phasemaps after the volumetric scattering sample with thickness L = 720µm and scattering mean free path ℓs = 90µm for two different wavelengths λ1 = 690.00nmand λ1 = 690.23nm.c) Calculated synthetic phase map for Λ = 2.1mm.d)-f) Same experiment with scatterer of different thickness (L = 1080µm) and optical wavelengths λ1 = 690.23nmand λ1 = 690.46nm.
. Using knowledge of the surface roughness of the 220 grit diffuser and the experimental geometry, we estimate Ψ max to be 65µm.Speckle artifacts are observed when Λ approaches 4Ψ max , consistent with the RQWR.The supplementary material puts forth mathematical arguments supporting the existence of RQWR (Eq.2) for a single realization of a scattering surface (see Section 1.6).The analysis shows how the RQWR fundamentally limits performance of a large class of NLoS imagers, including ToF techniques.Furthermore, the analysis may be generalized to include volumetric scatter by adopting a diffusive approach to light propagation[45].It is observed that the spread in path lengths as determined by the ratio of the squared thickness of the medium L 2 to the transport mean free path ℓ * , plays a role analogous to the RMS roughness σ h of scattering surfaces.
For instance, we envision the possibility of adapting the SWH principle to ultrasound imaging of biological features embedded deep within layers of tissue.Another example is coherent X-ray diffraction imaging of specimens embedded in thick, inhomogeneous samples.In both the examples, SWH has the potential to decouple the resolution of the reconstruction (de-termined by the 'Synthetic Frequency') from the penetra-405 tion depth (determined by the carrier frequency).We also 406 envision the use of SWH in repurposing radio antenna ar-407 rays (e.g., the VLA) for space-based astronomical imaging at 408 microwave and radio frequencies through dense atmosphere, 409 and possibly below the surface of a planet for remote geolog-410 ical exploration.Using photonic mixers driven by continu-411 ous wave laser sources, it may be possible to simultaneously 412 probe both optical reflectance and spectroscopic information 413 of specimens by sensing THz synthetic wavelengths with op-414 tical detection techniques coupled together with direct sens-415 ing of THz electromagnetic signals.Moreover, we believe 416 that the SWH concept has huge potential for Material Sci-417 ence, may it be to see deeper through materials or for the pre-418 cise analysis of inhomogeneous or multi-layered structures.419 There is much to be gained from exploiting spectral corre-420 lations in coherent light transport.Examining the SWH ap-421 proach through the lens of Gabor holography provides newer 422 insights into its operation and scope.Gabor originally con-423 ceived holography as a two-step process that involved record-424 ing an electron wave hologram and subsequently replaying it 425 via optical diffraction.SWH can be interpreted as a general-426 ization of Gabor's original analysis/synthesis technique, with 427 an additional computation step.We hope that this novel view 428 of SWH will help usher a diverse array of new research di-429 rections, in much the same manner as the invention of holog-430 raphy did decades ago.ject motion between measurements, and time-varying fluctu-535 ations in the environmental conditions.Increased robustness 536 to these fluctuations is afforded by the Superheterodyne In-537 terferometer design, wherein light from both lasers is used 538 to simultaneously illuminate the target and scene.A possible 539 realization is shown in Fig. 4b of the supplementary material: 540 each laser beam is split into two arms, where one of which is 541 independently modulated with an AOM.The RF drive fre-542 quencies for AOMs 1A and 1B (see Fig. 4 b in the supple-543 mentary material) are identically set to ν AOM 1 , but include 544 a phase offset ∆ϕ AOM that is user controlled.Light leaving 545 the two AOMs is combined and modulated with a third AOM 546 (frequency ν AOM 2 ), which produces the desired modulation 547 frequency ν m = ν AOM 1 − ν AOM 2 = 3kHz.The expression 548 for the I-and Q-images (In-Phase and Quadrature) recorded 549 by the LI-FPA after locking in at ν m are: 550

552 interferograms recorded with three or more phase shifts 553 ∆ϕReference beam injection with reduced radiometric 570 losses.
AOM introduced between measurements.It should be 554 emphasized that the use of two tunable lasers is also not a pre-555 requisite for this approach.Identical results can be achieved 556 with one fixed and one tuned laser, or similar combinations 557 discussed above.The principal benefit of the Superhetero-558 dyne approach is the robustness to environmental fluctuations 559 and object motion.However, it requires an additional AOM 560 and fiber splitters that significantly reduce the available out-561 put power compared to the Dual Wavelength Heterodyne In-562 terferometer discussed previously.The loss of power presents 563 light throughput challenges for NLoS experiments that are in-564 trinsically light starved.565 In practice, there exists a trade-off between light through-566 put and robustness to environmental fluctuations, which de-567 pends on a multiple factors including stand-off distance, re-568 flectivity of the involved surfaces, and laser power.569 The reference beam required for interferometric 571 sensing of the speckle fields at the optical wavelengths is di-572 rected towards the lock-in FPA.In one possible embodiment, 573 a lensed fiber needle (WT&T Inc.) positioned in the front 574 focal plane of the imaging optic (see Fig. 4 f in the sup-575 plementary material) produces a near planar reference beam 576 on the FPA.The use of a lensed fiber provides two distinct 577 advantages over a beam-splitter: (1) the imaging optic can 578 be directly threaded to the camera (eliminates the need for 579 inserting beam splitter between optic and sensor) and easily

583tive
NLoS imaging ('looking around corners').The ex-584 perimental apparatus schematically displayed in Fig. 2b, and 585 shown in Fig. 4 c of the supplementary material is used to 586 demonstrate the ability of SWH to discern objects obscured 587 from view, in this case a cutout of the character 'N' with 588 dimensions ∼ 20mm × 15mm.The size of the object was 589 deliberately chosen to be smaller than the typical size of a 590 resolution cell (∼ 2cm) in competing wide-field ToF-based 591 approaches.The disadvantage when using a small object is 592 that it emits less light than the background.The problem is 593 additionally compounded by the limited laser power in the 594 object arm (about 30mW ).In an effort to bypass these engi-595 neering limitations, we glued a thin sheet of silver foil to the 596 sandblasted (280 grit) surface of the object 'N' and repeated 597 the process for the VS surface.An image of object 'N' under 598 ambient light can be seen in Fig. 3q.The fields reflected by 599 these materials are fully developed speckle patterns.The VD 600 wall surface is constructed from a standard dry-wall panel 601 that has been painted white (Beer Eggshell paint).

Figures Figure 1
Figures

Table 1 .
Light loss at combination of reference and sample arm: Lensed fiber needle vs. conventional 50/50 beam splitter