Hadamard single-pixel imaging versus Fourier single-pixel imaging

Single-pixel imaging is an innovative imaging scheme and has received increasing attentions in recent years. It is applicable to imaging at non-visible wavelengths and imaging under low light conditions. However, single-pixel imaging has once encountered problems of low reconstruction quality and long data-acquisition time. This situation has been changed thanks to the developments of Hadamard single-pixel imaging (HSI) and Fourier single-pixel imaging (FSI). Both techniques are able to achieve high-quality and efficient imaging, remarkably improving the applicability of single-pixel imaging scheme. In this paper, we compare the performances of HSI and FSI with theoretical analysis and experiments. The results show that FSI is more efficient than HSI while HSI is more noise-robust than FSI. Our work may provide a guideline for researchers to choose suitable single-pixel imaging technique for their applications.


Introduction
Contemporary single-pixel imaging originates from ghost imaging [1][2][3][4][5][6][7][8][9][10]. Ghost imaging was initially considered as a quantum effect [1] but later Bennink et. al. demonstrated that it can be implemented with a classical source [2]. Computational ghost imaging [3][4][5][6][7][8][9][10] allows one to capture a scene using a single-pixel (or bucket) detector. Objects to be imaged are under spatially and temporally varying illuminations. The illumination patterns are typically generated using a spatial light modulator (SLM). The single-pixel detector is used to collect the corresponding light signal for each illumination pattern. The desired image is computationally reconstructed by correlating the illumination patterns with the detected signals.
Single-pixel imaging techniques [11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29] do not need to use any pixelated detector for light signals detection. This advantage brings single-pixel imaging a potential capability of solving some challenges in conventional imaging. For example, single-pixel imaging scheme allows one to build a low-cost imaging system that can work at non-visible wavelengths. It should be noted that pixelated detectors are commonly expensive or even unavailable at most non-visible wavelengths. In comparison with pixelated detectors, single-pixel imaging may also have the advantage of imaging at low light conditions. This is because single-pixel detectors with large active area are easier to fabricate and more sensitive to light.
Inherited from ghost imaging, single-pixel imaging was initially based on a statistical model, which can be evidenced by the use of random patterns for illumination. The random patterns form an overcomplete non-orthogonal basis. Consequently, it requires a great number of measurements (much larger than pixel counts) and long data-acquisition time for recording signals. Even with so many measurements, the quality of images reconstructed by single-pixel imaging is hardly comparable with the quality of images obtained by conventional 2D-detectorbased imaging. Long data acquisition time and low-quality reconstruction limit the applicability of this innovative imaging scheme. Such a situation existed till the emergence of single-pixel imaging techniques based on a deterministic model.
Hadamard single-pixel imaging (HSI) [12][13][14][15][16][17][18][19][20][21][22] and Fourier single-pixel imaging (FSI) [23-27] are two single-pixel imaging techniques that use a deterministic model. Both techniques employ deterministic basis patterns for illumination --HSI uses Hadamard basis patterns for illumination while FSI uses Fourier basis patterns. The use of basis patterns for illumination brings two advantages. The first advantage is perfect image reconstruction. The basis patterns form a complete orthogonal set and the use of basis patterns for illumination allows one to acquire the spatial information of object image in a transformation domain. When is fully sampled in the transformation domain, the image can be losslessly reconstructed by the corresponding inverse transform. This feature solves the problem of low-quality reconstruction in ghost imaging. The second advantage is measurement reduction. Natural images give a sparse representation in either Hadamard or Fourier domain, allowing one to reconstruct a sharp image with under-sampled data. This feature solves the problem of long acquisition time. As such, HSI and FSI well tackle the problems that are inherited from ghost imaging.
In this paper, we theoretically and experimentally compare these two techniques, in terms of principles, imaging efficiency, noise robustness, and etc. This comparison shows the commons and the differences between HSI and FSI. The comparison also presents the advantages and disadvantages of the both techniques. HSI is based on Hadamard transform [30]. HSI acquires the Hadamard spectrum of the object image and reconstructs the object image by applying an inverse Hadamard transform. Hadamard spectrum is composed by a group of Hadamard coefficients. Each coefficient corresponds to a unique Hadamard basis pattern. To obtain a Hadamard coefficient, one can project the corresponding Hadamard basis pattern(s) onto the object and use a single-pixel detector to measure the resultant light intensity. The single-pixel light intensity measurement is mathematically equivalent to the inner product between the Hadamard basis pattern(s) and the object. As such, the Hadamard spectrum can be reconstructed based on the single-pixel measurements.

Principle of HSI and FSI
Differential HSI is an embodiment of HSI, allowing each Hadamard coefficient   , H u v to be acquired in a manner of differential measurement. Differential HSI is conducive to depression of noise. As illustrated in Figure 1, to acquire a coefficient   , H u v , differential HSI takes two measurements. The one coefficient is acquired by projecting a Hadamard basis pattern   H , P x y and the other coefficient is by its inverse FSI is based on Fourier transform. FSI acquires the Fourier spectrum the object image and reconstructs the object image by applying an inverse Fourier transform. Fourier spectrum is composed by a group of Fourier coefficients. Each coefficient corresponds to a unique Fourier basis pattern. To obtain a Fourier coefficient, one can project the corresponding Fourier basis pattern(s) onto the object and use the single-pixel detector to measure the inner product between Fourier basis patterns and the object. As such, the Fourier spectrum can be reconstructed from the single-pixel measurements. Fourier basis patterns are also known as sinusoidal patterns or fringe patterns. Fourier transform was proposed in the late 1800s by Joseph Fourier and has been widely used in a number of fields. It allows any signal to be decomposed into a set of orthogonal sinusoidal waveforms of different frequencies. For example, images as twodimensional signals are allowed to be broken down into a combination of sinusoidal intensity A Fourier basis pattern   F , P x y can be obtained by applying an inverse Fourier transform to a delta function 4-step FSI and 3-step FSI are two embodiments of FSI, using the 4-step phase-shifting formula (Eq. (10)) and the 3-step phase-shifting formula (Eq. (11)), respectively. Both embodiments allow each Fourier coefficient   , F u v to be acquired in a manner of differential measurements. 4-step FSI and 3-step FSI acquire each Fourier coefficient with 4 and 3 measurements, respectively. Please note that Fourier coefficients are complex-valued. 4-step FSI allows one to acquire each complex-valued Fourier coefficient   Also note that The number of Fourier coefficients is the same as that of image pixels ( MN  ). With the prior knowledge that the Fourier spectrum of any real-valued image is conjugated symmetric, fully sampling an MN  -pixel image using 4-step FSI takes 2 MN  measurements. It can be seen that the 4-step FSI is essentially a differential method of measurement. FSI can also be conducted by employing the 3-step phase-shifting formula. The 3-step FSI acquires each complex-valued Fourier coefficient with 3 measurements, FSI is also a differential method of measurement, but in an asymmetric form. As the 4-step FSI does, the 3-step FSI has the property of noise suppression, but its performance is not as good as the 4-step FSI. It is because the 4-step FSI is a differential method of measurement in a symmetric manner while the 3-step is in an asymmetric manner. Fully sampling an MN  -pixel image using 3-step FSI takes 1.5 MN  measurements. In comparison with 4-step FSI, 3-step reduces 25 % measurements.
FSI and HSI are single-pixel imaging techniques based on basis scan and theoretically allow perfect reconstruction for any images in noiseless situations.

Basis patterns generation
The core of single-pixel imaging is to employ active illumination to acquire the spatial information of a target object. Instead of using random patterns, basis-scanning single-pixel imaging techniques use deterministic basis patterns for illumination. Figure 1 shows the comparison between the Hadamard and Fourier basis patterns. The difference can be summarized as follows: 1) Hadamard basis patterns are binary and mosaics look-alike while Fourier basis patterns are grayscale and fringes look-alike; 2) Hadamard basis patterns only have horizontal and vertical features while Fourier basis patterns have horizontal, vertical, and oblique features; 3) Fourier basis patterns are strictly periodical while Hadamard basis is not.
The applicability and the performance of these basis scanning single-pixel imaging techniques rely on the effectiveness and efficiency of basis patterns generation. Thus, it is necessary to discuss the methods of basis patterns generation for HSI and FSI. Hadamard basis patterns are binary (black-and-white), which makes HSI naturally suitable for single-pixel imaging systems based on a digital micro-mirror device (DMD). As DMD is a binary device, HSI can benefit from the high-speed binary illumination ability given by a DMD. Additionally, binary Hadamard basis patterns would not lead to quantization errors or gamma (nonlinear) distortion. However, to our best knowledge, Hadamard transform does not carry clear physical meaning, which makes Hadamard basis patterns almost impossible to be generated by a physical means. The applicability of HSI relies on the use of SLM. In other words, HSI would likely have difficulties when SLMs are unavailable at certain spectral regions.
On the other hand, Fourier basis patterns are naturally grayscale. It leads to the fact that FSI is not able to take the benefit of a high-speed SLM, such as DMD. DMDs work much slower in the grayscale mode. For example, a state-of-art DMD can display ~20,000 binary patterns per second but can only display ~250 8-bit grayscale patterns per second. Thus, FSI suffers from slower illumination rate and therefore longer data-acquisition time than HSI. In addition, using any digital device to generate Fourier basis patterns would cause quantization errors. When the number of quantization levels is too low, the resulting quantization errors would lead to pronounced image quality degradation. Recently, Zhang et al. proposed binary FSI [25], a workaround that uses binary Fourier basis patterns for illumination. Binary Fourier basis patterns are generated by upsampling and then dithering the grayscale Fourier basis patterns. The shortcoming of this approach is at the expense of reduced spatial resolution. Fortunately, Fourier transform is a natural operator and has physical meaning, which enables Fourier basis pattern to be generated by some physical means. For example, an ideal thin lens can be used as a Fourier transform engine with which the Fourier transform of an object image can be obtained at the back focal plane of a thin lens. Even without a thin lens, the Fourier transform of an object image can be approximately obtained by the far-field diffraction pattern, which is subject to Fraunhofer diffraction. Thus, one can generate Fourier basis patterns by using the interference of two plane waves, which adds applicability to FSI, especially for the cases that SLMs are expensive or even not available.
In short, FSI is more flexible and variant than HSI in terms of illumination patterns generation while HSI can benefit much more from the high-speed binary DMDs.

Robustness to noise
Both HSI and FSI are robust to dark noise and read-out noise in principle, because, as evident by Eq. (1) and Eq. (7), Hadamard transform and Fourier transform are global transformation. Global transformation has a property that each point (coefficient) in the transformation domain is a weighted sum of all points in the spatial domain and each point (image pixel) in the spatial domain is also a weighted sum of all points in the transformation domain. In other words, global transformation implies that each pixel in a reconstructed image is contributed by all measurements and such an averaging process allows evening out errors in measurement.
In terms of quantization errors, HSI outperforms FSI. It is because Fourier basis patterns continuously vary in space and magnitude. Thus, quantization errors will be caused when using digital devices (such as DMD or Liquid Crystal on Silicon (LCoS)) to generate Fourier basis patterns. Such errors would degenerate the quality of final reconstruction. On the other hand, Hadamard basis patterns are naturally in a discrete manner. Digital devices are able to generate error-free Hadamard basis patterns.

Efficiency
We refer efficient single-pixel imaging to a technique that allows one to reconstruct a sharp image with a small number of measurements. Additionally, highly efficient single-pixel imaging enables time-lapse imaging. Since the throughput of a single-pixel imaging system is inherently limited by the readout rate of the single-pixel detector, it is of critical importance to improving the efficiency of single-pixel imaging techniques.
The efficiency of a basis scanning single-pixel imaging technique depends on how well the utilized transformation concentrates the image energy. If a transformation is able to highly concentrate the image energy within a waveband (i.e., a small number of coefficients have large magnitudes), one can simply measure those large-magnitude coefficients and omit the rest to improve efficiency.
For natural images, energy is usually uniformly distributed in the spatial domain. Both Hadamard transform and Fourier transform have the ability to concentrate the image energy near the origin of their transformation domain. Thus, both HSI and FSI are able to reconstruct an imaging under Nyquist conditions by sampling coefficients with large magnitude. As will be quantitatively demonstrated in the following experimental comparison section, for natural images, Fourier transform gives a more condensed representation than Hadamard transform does. In other words, FSI outperforms HSI in terms of energy concentration.
Moreover, the physical meaning of Fourier transform gives it the capability of characterizing the impulse-response of some optical system. Optical transfer function (OTF), defined as the Fourier transform of the point spread function, shows how different spatial frequencies are handled by the system. For most optical systems, the OTF is an equivalent lowpass spatial filter. As a result, the spatial information that an imaging system can collect is lowpass filtered.
It can be seen that the characteristics of natural images and optical imaging system concentrate the image energy at the lower frequencies range of the Fourier domain. As such, it leads to rapid convergence in terms of reconstruction quality for FSI to sample along the increment of spatial frequency.
In some diffraction-limited situations such as microscopy, the achievable spatial resolution is determined by the cut-off spatial frequency of the optical system. In this case, the optical system acts as a low pass filter with a finite ability to resolve details. In mathematical terms, a diffracted-limited system sets a spatial cutoff frequency 0 f in the Fourier domain. Only the spatial information that falls into the circular low-frequency band with a radius of 0 f can be acquired through the system. Thus, FSI using a circular sampling path is the most effective approach for a diffraction-limited system in principle. If the spatial cutoff frequency of the system is given, FSI allows one to sample the circular low-frequency band only. Even if the spatial cutoff frequency is not given, FSI allows one to sample the Fourier domain along a circular path until the sampled energy is lower than a certain threshold. Therefore, lower spatial frequency components are typically of much larger magnitude than higher spatial frequency components, as shown in Fig. 1. This prior knowledge implies a simple but efficient sampling strategy that only low-frequency components are needed to be sampled in the data-acquisition process.

Numerical simulations
We firstly compare the performance of both techniques through numerical simulations using MATLAB. In the presented simulations, we focus on the quality of reconstructed images by using both techniques, especially when the images are undersampled. It should be emphasized that the throughput of single-pixel imaging systems is, in general, lower than conventional imaging systems and undersampling is a straightforward approach to reduce the number of measurements. Undersampling refers to acquire only the low-frequency coefficients and omit the high-frequency ones, given the prior knowledge that most energy of natural images is concentrated in low-frequency bands. Since sampling strategies may change the quality of the reconstructed images, we use three different sampling strategies for HSI and FSI without losing generality. The sampling strategies refer to the path along which coefficients are to be acquired. The sampling strategies in comparison are shown in Fig. 2. We note that the origin for Fourier spectra is located at the center of the pictures while the origin for Hadamard spectra is located at the left-top corner of the pictures. Therefore, Fourier spiral path is equivalent to Hadamard square path; Fourier diamond path is equivalent to Hadamard square path; Fourier circular path is equivalent to Hadamard circular path. Three different and characterized input images are used in the comparison, the 1951 USAF test chart pattern, 'Siemens star' target pattern, and the Lena image. The 1951 USAF test chart pattern is a binary pattern consisting of groups of three bars with dimensions from big to small. The bars are along vertical and horizontal directions. The 'Siemens star' target pattern is also a binary pattern. It provides multiple contrast measurements from a wide range of spatial frequencies. Different from the USAF 1951 test chart pattern, the 'Siemens star' target pattern has many oblique features. The Lena image is one of the most widely used natural image in imaging system tests and has multiple gray levels. We use peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and power ratio to quantitatively evaluate the quality of reconstructed images. Power ratio is a quantity which can evaluate the energy concentration ability, defined as power of the acquired spectrum to that of the complete spectrum ratio. Please note that, for Tables 1-3, the data in blue are the best results for HSI, the data in red are the best results for FSI, and the data in bold are the overall best results.  As the results shown in Fig. 3 and Table 1, FSI with the diamond path gives the best reconstruction for the USAF 1951 test chart pattern. With the diamond path, Fourier coefficients on both spatial frequency axes are acquired preferentially. Those Fourier coefficients corresponding to horizontal and vertical fringe patterns highly correlate with the features of the resolution target (that is, horizontal and vertical bars). Thus, FSI with diamond sampling strategy achieves better reconstruction than the others for the same sampling ratio.  As the results shown in Fig. 4 and Table 2, FSI with the circular path gives the best reconstruction for the 'Simens star' image. The results by HSI, as they present, are with mosaic artifacts, especially for the oblique features. It is because, Hadamard basis patterns are mosaic look-alike, but lack of oblique features.  As the results shown in Fig. 5 and Table 3, FSI outperforms HSI for the 'Lena' image which is a natural image. However, both techniques introduce observable artifacts when sampling ratio is too low. HSI introduces mosaic artifacts while FSI introduces ringing artifacts.
Without loss of generality, we test all images in the USC-SIPI image database [31]. The database consists of 632 images and all the images are categorized into four groups-textures, aerials, miscellaneous, and sequences. Similarly, for every single image, we evaluate reconstruction quality of the six sampling strategies using PSNR, SSIM, and power ratio. The statistical comparison results are derived by counting the overall best for each image. The results are shown in Fig. 6. Based on the comparison results, it is found that FSI has better performance than HSI in terms of reconstruction quality in the situations of undersampling. It is also found that for natural images reconstruction circular path is the best sampling strategy for FSI and zig-zag path is the best sampling strategy for HSI. These simulations demonstrate that quality of sub-Nyquist sampled images depends on the energy concentration ability of the utilized transformation and the sampling strategy. For natural images, Fourier transform has better energy concentration ability than Hadamard transform, because the correlation between a Fourier basis pattern and a natural image is larger than that between a Hadamard pattern and a natural image. Or in other words, Fourier basis patterns are more similar to the natural images than Hadamard basis patterns. Thus, FSI outperforms HSI in terms of reconstruction quality under sub-Nyquist sampling conditions.
We further compare the both techniques in terms of robustness to noise. To simulate cases of different noise levels, we add white Gaussian noise to the raw data (that is, measurements D ) resulting in different SNRs. The results are shown in Fig. 7. As the figure shows, HSI is more robust to FSI. It turns out that there's a tradeoff between energy concentration and robustness against noise.

Experiments
We further compare HSI and FSI with experimental data. The first experiment is single-pixel photography using a commercial digital projector (Acer K750) for spatial light illumination. The experimental set-up is shown in Fig. 8. The projector switches pattern every 0.2 seconds. A photodiode (HAMAMATSU S1227-1010BR) is used as a single-pixel detector. The photodiode is driven by a customized amplifier circuit. The resultant electric signals are delivered to a data acquisition board (National Instruments USB-6343 (BNC)). The digitalized data is finally collected by the computer. The methods in comparison are differential HSI and 4-step FSI. The results are shown in Fig. 9. The resolution of the reconstructed images is 256×256 pixels. As the figure shows, the FSI presents more clear and sharp reconstructions than HSI does in the case of undersampling. The advantage of FSI is relatively observable for the sampling ratio lower than 10%. The results in this experiment coincide with those derived in the numerical simulations. The second experiment is fast single-pixel imaging. We employ a DLP development kit to achieve high-speed illumination. The DLP development kit is equipped with a 0.7-inch DMD. The DMD has 1024×768 micro mirrors, each of which is 13.6×13.6 m 2 in size. The light source is a 3-watt white LED. The DMD operates at 2,000 Hz, allowing 2,000 binary patterns projection per second. The backscattered light is detected using a photomultipliers tube (Thorlabs PMM01). The resultant electronic signals are transferred to the computer via a data acquisition board [National Instruments USB-6343 (BNC)]. The object to be imaged is some stationery and a piece of A4 paper printed an enlarged 1951 USAF resolution test pattern. The object is under illumination by basis patterns. The methods in comparison are differential HSI 3-step binary FSI, and 4-step binary FSI. The reason why we use binary FSI in this experiment for comparison is that the original FSI is not able to directly use a high-speed DMD for highspeed imaging. It is because the original FSI uses grayscale patterns for illumination, while even an edge-cutting DMD can only display ~250 8-bit grayscale patterns per second, which is a shortcoming of the original FSI. The resolution of illumination patterns is 256×256 pixels. For binary FSI, the patterns are upsampled using 'bicubic' interpolation so that the resolution of the images becomes 512×512 pixels, twice of the original. Floyd-Steinberg dithering is then applied to the upsampled Fourier basis patterns. For HSI, the Hadamard basis patterns are upsampled to be 512×512 pixels using 'nearest' interpolation. The comparison results are presented in Fig. 11. Based on the comparison results, binary FSI outperforms HSI in terms of reconstruction quality in the case of a small number of measurements. In the case of 1,000 measurements, heavy ringing artifacts present in the results of FSI while HSI gives a mosaic reconstruction. 3step FSI is the only technique that can reconstruct the horizontal lines on the pencil holder, showing its advantage in extremely undersampled cases. In the case of 4,000 measurements, 3step binary FSI gives better reconstruction, which is evidenced by that all five pencils in the holder on the right become distinguishable. In the case of 8,000, the results by both FSI techniques are satisfactory, except that there still ringing artifacts on the background while the result by HSI is of mosaic, especially for the oblique structures such as the pencil in the left pencil holder. HSI outperforms FSI in terms of uniform pattern reconstruction. The background in the reconstruction by HSI is free of artifacts or noises. In the case of 30,000 measurements, the result by HSI looks as good as those by FSI. In the case of 98,304, the result by HSI is even better than those by FSI in terms of noise level. The noise on the background of the results by FSI becomes noticeable, which is due to the quantization errors.
In short, in the case of a small number of measurements, FSI outperforms HSI for FSI has better energy concentration ability than HSI. In the case of a large number of measurements, HSI has better reconstruction quality than binary FSI. It is because binary FSI introduces quantization errors when generates binary Fourier basis patterns using dithering.
The third experiment is microscopy where we use a digital light projector (Texas Instruments LightCrafter Display 4710 evaluation module) along with a tube lens and an objective lens (Olympus 10X objective (NA = 0.4)) for illumination patterns generation. The experiment set-up is shown in Fig. 12. The DMD of the projector consists of 1920  1080 micro mirrors whose pitch is 5.4 microns. A Si amplified photodetector (Thorlabs PDA-100A) is used as a single-pixel detector that collects the transmitted light through the object. The model of the data acquisition board used in this experiment is National Instruments USB-6363 (BNC). The object to be imaged is a USAF1951 resolution target (Ready Optics #2015A). Please note that to make full use of the area the DMD, we use 4×4-pixel binning in this experiment. The resolution of illuminations patterns is 256×256 pixels and each pattern uses 1024×1024 mirrors for illumination. The illumination patterns switch every 0.2 seconds.  As the results shown, FSI presents clearer and sharper reconstruction than HSI in the case of undersampling, especially when the sampling ratio is under 10%. For instance, the digits and the bars of the 6 th . group become distinguishable for the results by FSI. For the full-sampled cases, HSI presents as good results as FSI does.

Conclusion
We present a systematic comparison between the HSI and the FSI in principle and experiments. The overall comparison is summarized in Table 4. According to the principle analysis and the experimental results, we conclude that the FSI is more efficient than the HSI as Fourier transform better concentrates image energy than Hadamard transform does. Additionally, the 3-step FSI allows measurement reduction in the case of differential measurement. Thus, for applications where efficiency is concerned, the FSI is the primary selection. We also conclude that the HSI has better noise robustness than the FSI and is perfectly suitable for DMD-based single-pixel imaging systems. Therefore, for applications where image quality or accuracy is concerned, the HSI is the primary selection.