Adaptive regional single-pixel imaging based on the Fourier slice theorem

: Single-pixel imaging (SPI) is a novel method for capturing high-quality 2D images of scenes using a non-spatially-resolved detector. While implementing conventional SPI, a huge number of illuminated patterns are projected onto the object to reconstruct a sharp image. For a situation in which the object occupies part of the illuminated region, we propose an adaptive regional SPI method (ARSI) to decrease the number of projected patterns. In the ARSI scheme, the object region is adaptively located based on the Fourier slice theorem. Then, the illuminated patterns are projected only onto the object region to facilitate imaging efficiency. Experiments demonstrate that the proposed ARSI method can achieve sharp image reconstruction with a substantial reduction in pattern number, thereby improving imaging efficiency.


Introduction
Single-pixel imaging (SPI) is a novel method for capturing 2D images using a single-pixel detector without spatial resolution. In SPI, known illuminated patterns are projected onto an object, and the intensity of the reflected light is measured using a single-pixel detector [1]. Then, the image can be reconstructed by calculating the correlation between projected patterns and recorded signals [2]. SPI has been applied to many fields, including 3D imaging [3][4][5], terahertz imaging [6,7], and multispectral imaging [8], due to its excellent anti-noise and high signal-to-noise ratio (SNR) characteristics.
Despite these excellent characteristics, SPI is required to project large amounts of patterns to reconstruct a sharp image. We divide SPI into two categories based on the orthogonality of illuminated patterns: SPI using patterns generated from non-orthogonal bases and SPI using patterns generated from orthogonal bases. The first SPI method typically uses random patterns [9], which actually comprise an overcomplete set. In this method, a huge number of patterns should be projected to reconstruct a sharp image because of the non-orthogonality of the patterns [1]. Sun [3] used 1 million random patterns to reconstruct an image with a resolution of 256 × 192. Hence, the numbers of used patterns are considerably higher than the resolution of the reconstructed image.
SPI that uses patterns from complete orthogonal bases includes SPI based on Fourier transform [10], discrete cosine transform [11], and Hadamard transform [12]. The number of patterns from a complete orthogonal basis is less than that from an overcomplete non-orthogonal basis. Thus, the use of patterns generated from a complete orthogonal basis can reduce the projection counts of SPI compared with the former SPI method. Recently, the Fourier SPI (FSI) method [10], which uses Fourier basis patterns for illumination, is proposed to achieve high-quality 2D imaging. However, data are obtained by sequence for all existing SPI methods, whereas millions of data are acquired simultaneously using conventional digital cameras. The time consumption of SPI is proportional to the total number of patterns given a certain rate of pattern projection, thereby more patterns require more time for projection.
A large number of projected patterns create a bottleneck in the application of SPI. The compressive sampling [13] method can be used to decrease the number of projected patterns, whereas the signal-to-noise ratio (SNR) of the reconstructed image will be reduced. Bian et al. [14] developed an efficient SPI technique (eSPI) to reduce the number of patterns. This technique captures only the most informative data in spatial frequency bands.
The number of projected patterns is reduced if the resolution of the image is decreased because the number of projected patterns for SPI is proportional to the spatial resolution of the reconstructed image. In fact, the object to be imaged typically takes up a partial region in the reconstructed image [1,3]. If SPI is conducted only on the object region, then the number of projected patterns and the time consumption will be significantly decreased without degrading image quality. This finding motivates us to develop a method for locating the object region in a scene and adaptively conduct SPI on the object region to reduce the number of projected patterns.
In this study, we propose an adaptive regional single-pixel imaging (ARSI) method to facilitate the efficiency of SPI. The proposed method applies the Fourier slice theorem [15] to adaptively locate the object region and then conducts SPI only on the object region. Compared with the conventional SPI method that images the whole scene, the proposed method significantly reduces the number of patterns, and thus, increases imaging efficiency.
The rest of this paper is organized as follows. The principles of the proposed method are explained in Section 2. The measurement system and the experimental results are discussed in Section 3. Lastly, the conclusions drawn from the study are presented in Section 4.

Principles
The proposed ARSI method is suitable for the situation in which an object occupies a partial area in the scene region. Unlike the conventional SPI method that images the whole scene [ Fig.  1(a)], ARSI adaptively conducts SPI on the object region after locating the object region [ Fig.  1(b)]. ARSI can reduce the number of illuminated patterns and time consumption via adaptive regional imaging. The framework for ARSI comprises the following steps.
 Step 1. Adaptively locating the object in the scene. First, we project vertical and horizontal 2D sinusoidal patterns to obtain slices of the Fourier coefficients of the scene region. Then, the object region is located based on the Fourier slice theorem [15] and the edge detection algorithm [16].
 Step 2. Conducting SPI only on the located object region. After the object region is located, the image of the object can be reconstructed by conducting any SPI method on the object region. In this study, FSI is used to obtain the object image.
 Step 3. Placing the reconstructed image on the object location in the scene. The reconstructed image is placed on the object location in the scene using the calculated object region location to generate a complete image.

Adaptively locating the object region in the scene based on the Fourier slice theorem
In accordance with the Fourier slice theorem, the 1D Fourier transform of the projection is a slice of the 2D Fourier transform of the image along the same orientation as the projection [15]. Projections p(x) and p(y) can be obtained using where IFT represents the 1D inverse Fourier transform, F(u,v) denotes the 2D Fourier coefficients of the image, and F(u,0) and F(0,v) are the slices of the Fourier coefficients. After F(u,0) and F(0,v) are acquired, projections p(x) and p(y) can be calculated based on the Fourier slice theorem. The method for acquiring F(u,0) and F(0,v) is introduced in the next part.
On the projection line of the scene, the mutation phenomenon occurs on the positions where the object edges are located because the grayscale distribution differs between the object and the background. The edges of the object region are located by using the edge detection algorithm [16] on projections p(x) and p(y); the edges are shown as points surrounded by a red circle in Figs. 2(b) and 2(c). The object region is located by the red rectangle shown in Fig. 2(a). To guarantee a correct object location, the object should have a high contrast with the background which has a uniform grayscale distribution. Let the object region be Ω a . Then, we have where x 1 and x 2 represent the edges of the object region along the x-axis, and y 1 and y 2 represent the edges of the object region along the y-axis.

Acquisition of the slice of the Fourier coefficients
We adopt the FSI method to acquire the required Fourier coefficients. As shown in Fig. 3, vertical and horizontal 2D sinusoidal patterns, i.e., PV φ and PH φ , are projected onto the scene to obtain the slice of the Fourier coefficients F(u,0) and F(0,v). The patterns of PV φ and PH φ are generated using Eqs. (4) and (5) as follows: where (x,y) represents the 2D Cartesian coordinates in the scene, (u,v) represents the spatial frequency, a represents the DC component, b denotes the contrast of the pattern, and φ denotes the initial phase. The response of the single-pixel detector to the illuminated sinusoidal patterns can be expressed by Eqs. (6) and (7) as follows: where RV φ and RH φ respectively represent the response of the single-pixel detector to the reflected light from the vertical and horizontal 2D sinusoidal patterns, k is a coefficient related to the sensitivity and location of the single-pixel detector, R n is the response of the single-pixel detector to environmental light, and Ω represents the scene region. The light reflected from four patterns, which has the same frequency and π/2 phase shifting, is collected individually by the single-pixel detector. In accordance with the FSI approach, the slice of the Fourier coefficients can be calculated using Eqs. (8) and (9) as follows: Moreover, p(x) and p(x) are calculated using Eqs. (1) and (2). Then, the object region Ω a can be located using the edge detection algorithm on p(x) and p(y).

Image reconstruction
To reconstruct the object image, we conduct SPI on the located object region. Any type of SPI method is suitable for reconstructing images. Similar to the conventional SPI method, the illuminated patterns with the same size as the object region are projected onto the object region. Once the single-pixel detector accomplishes data acquisition, the object image can be reconstructed using the corresponding reconstruction method of SPI. In this study, we conduct FSI on the object region, and the object image is reconstructed by performing 2D inverse Fourier transform on the acquired coefficients. Let the reconstructed object image be f o (x,y).
To acquire a complete image, a blank image with the same resolution as the scene image is generated as follows: Then, the complete image can be obtained by where (M,N) is the resolution of the object region, (x 1 ,y 1 ) is the location of the object region (Fig. 4).

Number of projected patterns
When conducting SPI on the whole illuminated region, the count of the SPI projected patterns, C, is proportional to the resolution of the reconstructed image and can be expressed as The resolution of the object region can be expressed as where k 1 and k 2 are the scale factors that are equal to or greater than one. The total number of projected patterns of ARSI, C a , is where C L represents the number of illuminated patterns while locating the object region, C S is the number of illuminated patterns while conducting SPI on the object region. C L and C S can be calculated using Eqs. (15) and (16) as follows: 1 .
In most cases, C L is sufficiently small compared with C S , therefore the time consumption of ARSI is mainly determined by the number of illuminated patterns C S . Through the adaptive projection, the number of illuminated patterns C S can be greatly reduced and the efficiency of SPI can be significantly improved.

Experiments
The structure of our experiment setup is shown in Fig. 5. To produce high-contrast structured patterns, a digital projector is selected as the SLM. The used digital projector contains a light source with a wavelength of 455 nm. An illuminated pattern is projected onto the scene by the digital projector every 0.1 s. The reflected light is collected by a lens, and detected by the single-pixel detector, which is actually a photodiode. The 455 nm bandpass filter is fixed in front of the single-pixel detector. The collected analog signal is converted into a digital signal by an analog-to-digital converter (ADC) and processed using a computer. The object is placed 0.5 m away from the lens of the digital projector. Fig. 5. Experimental setup. The digital projector projects illuminated patterns onto the object, which is located 0.5 m away from the experimental system. The reflected light is collected by a lens and detected by a photodiode. An ADC transfers the detected signal to a computer. The image is reconstructed using the obtained data.
We choose the FSI approach as the SPI method to reconstruct the object image given its anti-noise and high-quality imaging characteristics. To acquire a coefficient, four patterns with a π/2 phase that shifts in the same frequency are projected onto the scene by the digital projector and the reflected light is detected sequentially by the single-pixel detector. The use of a four-step phase-shifting technology [10] can eliminate the effect of environment illumination and enhance measurement accuracy.

Experiment on object location via ARSI
To verify the environmental adaptability of the proposed location method, the object location experiments with dark background, colored background and non-background were conducted respectively.
Firstly, the white toy bear with dark background, as shown in Fig. 6(a), was located. The resolution of scene image is 320 × 200. After projecting vertical patterns and horizontal patterns, the projection of the scene image in both directions was obtained; the result is shown in Figs. 6(b) and 6(c). For this certain resolution of the reconstructed image, the number of projected patterns for object location is invariable. In the projection line [blue lines shown in Figs. 6(b) and 6(c)], the positions where the mutation phenomenon occurs are the edges of the object region. The edges of the object are located by the red circle in Figs. 6(b) and 6(c) by using the edge detection algorithm. The object region is obtained as follows: The located object region, which is illuminated by the digital projector, is shown in Fig.  6(d). The illuminated region is larger than the real object region to guarantee that the whole object region can be reconstructed. The second experiment result with colored background is shown in Fig. 7. The resolution of scene image is 320 × 200. By applying proposed location method, object location is obtained as Eq. (18). In this experiment, the background of the object has a uniform grayscale distribution. As shown in Figs. 7(b) and 7(c), the projection lines on the non-object region have a gentle fluctuation. Whereas the object still can be located correctly by the proposed method. The third experimental result with non-background is shown in Fig. 8, and the located object region is represented by Eq. (19). As Fig. 8(a) shows, only the light reflected from object can be collected by the detector. The projection lines [Figs. 8(b) and 8(c)] on the non-object region is near 0, since few light was reflected from non-object region. And the ambient light was greatly eliminated by bandpass filter. After location, a white board was posed behind the object to better reveal the located object region. The proposed ARSI method has an excellent environmental adaptivity, which has been proved by the above experiments. The proposed method can be widely used in the situation where the object has a background with uniform grayscale distribution or non-background, since the object edges are judged by the mutation on the projection lines of scene image.

Experiment on image reconstruction
After locating the object region, FSI was conducted on the located object region to acquire the coefficients of the image in the Fourier domain. The object image was obtained by conducting inverse 2D Fourier transform on the coefficients.
We first acquired the whole coefficients in Fourier domain to reconstruct the image. Figure  9(a) shows the final reconstructed image with a resolution of 320 × 200 via ARSI, whereas the resolution of the located object region is 112 × 90. During ARSI, 20160 patterns were projected to obtain the whole Fourier coefficients and reconstruct the object image. For comparison, we conducted normal FSI on the whole scene. The reconstructed result is shown in Fig. 9(b). A total of 128000 patterns were used to obtain the whole coefficients in the Fourier domain. As shown that obtained 6.3 times hig projected num of normal FSI Furthermo same number We condu ARSI and nor ARSI: The ARSI method projects 14104 patterns to acquire all the Fourier coefficients of the object, whereas the normal FSI method uses 128000 patterns. The experimental result is presented in Figs. 13(a) and 13(b). The CNR values of the two images are nearly close. In Fig.  13(c), the grayscale distributions of the two results indicate that image contrast remains stable; ARSI used considerably fewer patterns. The comparison of the experimental results using the two methods is presented in Fig. 14.
In the case where the same number of patterns is projected onto the object, CNR and the contrast of the image reconstructed via ARSI are higher than those of the image reconstructed via normal FSI.
The aforementioned experiments illustrate that the number of illuminated patterns is significantly reduced in ARSI to reconstruct an image sharper than that derived from the conventional SPI method. The proposed ARSI can be applied to any situation in which the object region is smaller than the whole illuminated area. The efficiency of SPI can be significantly improved by adaptively locating the object region.

Discussions and conclusions
We adaptively conduct SPI on the located object region to facilitate imaging efficiency. The proposed ARSI method exhibits the advantages of less projected patterns and higher-quality reconstructed image compared with the conventional SPI method.
The core of the proposed ARSI is locating the object region via the Fourier slice theorem. In practice, only low-frequency vertical and horizontal Fourier patterns are required to project onto the scene, which can smooth the acquired projection line of the scene image and reduce measurement numbers for region location.
Any SPI method is suitable to perform on the located region. Nature images are sparse in the Fourier domain; hence, the FSI method can reconstruct high-quality images with a part of low-frequency coefficients via advanced sampling methods [14].
The positions of detector and digital projector are not overlapped in our experimental set-up, which might cause visible shadows in the reconstructed image, since the object can block the reflected light from some special region. The shadows are small to have little effect on the obtained projection lines, and thus won't affect the location correctness.
Our proposed method also has limitations. Firstly, the object is required to have a background with uniform grayscale distribution or non-background to guarantee a correct location. Secondly, the proposed ARSI is not applicable for multiple objects in the scene for now, because the location algorithms for one object and separate objects are different. In our future work, the location method for multiple objects will be investigated.
ARSI is experimentally capable of producing high-quality images of an object with less number of measurements than the conventional SPI method. The proposed method may be widely used in areas where the object only takes up a partial region of the scene to increase SPI efficiency. Moreover, if the detector used in the SPI system only observes part of the illuminated region, the proposed method could be applied to reduce the projected pattern number, because the unobserved region can be deemed as the dark background.