Fast tracking of hidden objects with single-pixel detectors

A fast tracking system is demonstrated that is based on single laser illu- mination and a few single-pixel single photon avalanche diode (SPAD) detectors that improves on the previous tracking of non-line-of-sight motion by a factor of 300 in laser power. With an average illumination power of only 2 mW, a 15 cm ×15 cm object located up to 2.5 m away is tracked whilst outside of the direct line-of-sight and moving at approximately 6 cm/s.

Introduction: The past decade has seen an increase in research devoted to visualising and locating objects hidden from view [1][2][3][4][5][6][7][8][9][10][11]. The work of Kirmani et al. [6] showed that a streak camera is able to detect indirectly scattered light from around a corner. Light imaging, detection and ranging (LIDAR)-based techniques [1,3,5,11,12] have been implemented to reconstruct the 3D structure of an occluded object from the temporal information retained in diffusely scattered light arriving back from a hidden scene. Also, using time-of-flight sensing [2,5,9,10] and intensity imaging [7], it is possible to track the motion of an object moving outside of the direct line-of-sight.
In this Letter, we present a fast tracking system that enables the detection of moving objects outside of the direct line-of-sight using only 2 mW of laser power. Our active imaging system uses three single-pixel single photon avalanche diode (SPAD) detectors and a single-pulsed laser to interrogate a 'room' with a hidden object. In a first experiment, we demonstrate that we can accurately locate the hidden object. In a second experiment, we demonstrate that we can accurately track the motion of the hidden object moving in real time. Data is collected and processed, and the position is updated as the object moves. On the basis of these results, we claim the feasibility of single-photon counting (SPC) technology for fast detection and location of moving objects such as humans or vehicles potentially in real time.
Experimental setup: The 'room' in our experiments is a purpose-built box measuring 102 × 102 × 76 cm 3 . Fig. 1 shows a schematic representation of the room as seen from the outside. Optical access is provided by a 28 × 12 cm 2 open window in the front wall (semi-transparent in this figure but completely opaque in the experiments). The target object is a 15 × 15 cm 2 screen, chosen to mimic a broad (extended) scattering object, that we angle at 45°to the rear wall and move along a designated ground track. The ground track traces out the shape of the letter 'E' and is outside of the field of view of our system. The transceiver comprises a pulsed laser diode (Picoquant LDH-P-780, 780 nm peak wavelength, 80 MHz repetition frequency, 2 mW average power), a time-correlated SPC [13] (TCSPC) module (Picoquant HydraHarp 400), three silicon SPAD detectors (Excelitas SPCM-AQRH) and the associated light collection optics. A train of light pulses from the laser diode propagates through the window situated 55 cm from our system, onto a white surface at the back of the room (157 cm away from the transceiver), as illustrated in Fig. 1. The pulses incident on the rear wall of the room scatter and continue to propagate approximately as an isotropic spherical wavefront (Fig. 2a). Some of this light reaches the hidden object (screen) and is scattered back again towards the rear wall. Three discrete positions on the wall are imaged to the SPADs. The returning light is coupled into each SPAD through 1 in diameter collection optics and a 105 μm diameter core fibre. The TCSPC module measures the photon arrival times (64 ps time binning) for the signal returning to each detector and a histogram is built up in 1 s of acquisition time over 80 million laser pulses. We use this temporal information between the laser sync signal and each SPAD detector signal to reconstruct the position of the hidden object. Without any loss of generality, we can simply define the point of reference (origin of the Cartesian coordinate system) in our system to be the centre of the rightmost pixel point on the rear wall. Position retrieval: We use the histogram position of the peak of the first-return signal of the laser as our reference 'start' time for each pixel and define the timing of all events relative to this. We then window the first-return signal out of the histogram in our target-position retrieval. In our current method, we pre-acquire and subtract a background signal with the object removed to suppress unwanted signals and isolate the temporal evolution of the target signal. Alternatively, we could consider the median background detection or change detection approaches discussed in [2,9,[14][15][16].
For each detector pixel i, we apply a Savitzky-Golay filter to the windowed temporal histogram to smooth out the background-subtracted signal (inset in Fig. 2b). We then locate the position of the highest peak in the return signal corresponding to our scattering source of interest. To retrieve the target position, we use an extension of the approach outlined in [2]. In summary, we generate a probability distribution for each of the single-pixel elements corresponding to where c is the speed of light, r i = (x i , y i , z i ) is the pixel position, r l = (x l , y l , z l ) is the laser position, r o = (x o , y o ) is the position of the hidden target object and t i is the photon time-of-flight. The dominant uncertainty s in the retrieval is related to the resolution (256 × 128 pixels) of the discretised search area (3 × 1.5 m 2 ). To obtain the location, we take the product of the probability density functions (PDFs) P i (r o ), which all overlap at the hidden object.
Results: In a first experiment, we place our target object at 11 distinct positions. We acquire data for 1 s for each position, and then based on the collected data, determine its position. Fig. 3 shows the joint probability densities that we retrieve for the hidden object for each measurement. These are overlaid with the ground truth. The agreement between the PDFs and the object's true positions show that our system is able to locate a stationary object of dimensions 15 × 15 cm 2 situated up to 2.5 m outside of the direct line-of-sight, and determine its position with an average precision of +7.0 cm in x and +6.2 cm in y.  In a second experiment, we investigate the parameters required to track a moving hidden object. We move the object continuously around the hidden scene at an average speed of 5.7 cm s −1 for 30 s and track its motion. Data is acquired in 1 s time slots. At this integration time, we are able to perform data acquisition followed by target-position retrieval approximately every 1.5 s, i.e. 1 s acquisition followed by 0.5 s computational retrieval. The result is a discrete set of the object's average position during each data collection period. We show this in Fig. 4. The reconstructed PDFs are in good agreement with the object's motion, bearing in mind that our object has a width over which it scatters.  With discretised sampling, a more continuous trace may be obtained by interpolating between the individual reconstructed positions. This interpolation problem between two successive observed positions can be interpreted as a transport problem [17]. Indeed, at each time instant the reconstructed position represents the PDF of the object position, which is evaluated on a uniform spatial grid, i.e. the search area pixels. Consequently, the interpolation is carried out by inferring a smooth mapping from a given frame or position to the next frame, and transforming one PDF into another. To estimate this mapping, we use the method proposed in [18] and select an arbitrary number of temporal steps between the images to be interpolated. The resulting discretised dynamic optimal transport problem, which reduces to a convex optimisation problem, can then be solved using, e.g. proximal splitting methods [19]. Here, we use the Douglas-Rachford splitting scheme [20], studied in [18], where the cost function involved in the minimisation problem is the L 2 -Wasserstein distance (see [18] for further details of the algorithm). The result of this interpolation is shown in Fig. 5. This last result is interesting mainly in view of the fact that though our setup is not designed to reconstruct the full 3D structure of the hidden scene, we may use this approach in real-life situations (e.g. in surveillance applications) to infer some information regarding the layout of the room from position measurements of the moving object alone. For example, with prior knowledge of the nature of the moving object (e. g. a human being), the interpolation obtained after monitoring our room for a period of time shows that it is possible to determine the hypothetical layout of the room with a level of accuracy based on the movement of the hidden object.
Conclusion: Our method builds on the previous work using an SPAD array [2,21,22,[23][24][25][26], and by employing a single laser and multiple SPAD detectors we eliminate the need to scan, thus reducing the data acquisition time. Working at similar scales to the SPAD camera in [2], our three single-pixel system already shows many advantages. The flexibility of single-pixel detection provides an increased field of view that allows us to detect and simultaneously track with better precision at longer range. The use of single-pixel detectors also has the advantage of higher detection efficiency. We have thus reduced the integration time required for each target-position retrieval to 1 s, and can update the position of the hidden object every 1.5 s. In particular, the required average laser power is significantly reduced to just 2 mW.
The scope of the experiments reported in this Letter is not to reconstruct an image of the scene hidden around an obstacle. Nevertheless, we show that we can gain some information from the immediate surroundings of a hidden moving object by performing measurements of its movement alone. This adds valuable information when remotely assessing a hidden space. The ability to perform fast non-line-of-sight motion tracking and infer information about the hidden scene using experimental components that can be made both compact and portable, takes us one step closer to developing a real-time solution that is usable for real-life scenarios. Future work will aim to take this to long distance.