Near video-rate linear Stokes imaging with single-pixel detectors

In this work we demonstrate a polarization sensitive computational imaging system based on a digital micro-mirror device (DMD) and several single-pixel photodetectors. By taking advantage of computational imaging techniques, the light measured by each single-pixel detector can reconstruct a 2D image for a specific linear polarization state. Using the rapid 22 kHz ?> frame-rate of the DMD to continuously project a series of spatially orthogonal illumination patterns, near video-rate reconstructions can be achieved. In addition we extend this approach to provide full-colour images through a process of sequential colour selection (RGB). Taking the difference between photodetector signals from orthogonal linear polarization states, we obtain images corresponding to the linear Stokes parameters. We apply this rapid polarization sensitive imaging system to inert and biological material. Since the spatial information in the images reconstructed by this approach are determined by the projection system, rather than the detectors, the approach provides perfect pixel registration between the various polarization selective images and associated Stokes parameters. Furthermore, the use of single-pixel detectors and the large operational bandwidth afforded by DMDʼs means that the approach can readily be extended for imaging at wavelengths where detector arrays are unavailable or limited.


Introduction
Single-pixel detectors can be used to produce 2D images of objects using a time-varying structured illumination and an appropriate computer algorithm [1][2][3][4][5][6][7][8][9][10][11][12]. The single-pixel photodetector is used to measure the overlap between the projected illumination pattern and the object. The number of patterns required is comparable to the number of target pixels in the resulting image. The long acquisition times and low resolution images produced with this approach have arguably inhibited widespread application when compared to conventional imaging systems utilising detector arrays. These limitations are directly related to the available hardware and computer processing power. Therefore, recent advances in micro-electromechanical systems technology and ever faster computation provide a rapidly changing platform on which computational imaging may offer alternative imaging solutions, particularly at wavelengths where a detector array is expensive or even unobtainable.
In this work we use a high-speed digital micro-mirror device (DMD) to provide structured illumination of the object and single-pixel detectors to measure the polarization components of the backscattered light. However, these same optical components could be reordered to instead image a uniformly illuminated object onto a DMD, where the spatially filtered light is measured by single-pixel detectors [13,14]this latter approach is commonly termed a single-pixel camera [15,16]. It should be noted that the masking approach is vulnerable, in terms of polarization sensitivity, to interference from potentially non-polarizing preserving optical components, such as the DMD itself. The structured illumination approach does not suffer this vulnerability since the linear illumination is set after projection, and the returned light only propagates through polarization preserving optical components before detection.
In this work we demonstrate a colour computational imaging system which has been made polarization sensitive and has the ability to reconstruct the linear Stokes parameters simultaneously. This system displays every pattern followed by its inverse, allowing a differential signal to be obtained by taking the difference between the two corresponding measurements made by the photodetectors. This differential acquisition method [17,18] improves image quality even in the presence of background light sources. Our main goal for this system was to validate that it could indeed reconstruct images containing polarization information for transmissive and reflective objects in 'real-time', and that simultaneous Stokes parameter image reconstruction can be achieved using differential signals, as opposed to differential images as is the current method for CCD based polarization imaging systems. Furthermore the system can be adapted to operate at a 'slow' pattern projection rate of ∼1440 Hz, which is effective when temporal information is not required in the scene. This allows standard low-cost photodetectors to be implemented in the detection process which produces superior results when given many samples to average over. Conversely when temporal information is desired, the system can be operated at a 'fast' pattern projection rate of up to ∼22 kHz. This allows near video-rate acquisition of the scene, however in this mode of operation photomultiplier tubes (PMTʼs) have been employed in order to compensate for low light levels resulting from reduced sampling time. The choice of pattern projection rate in this system therefore offers flexibility depending on the desired application.

Experimental setup
The experimental configuration used to produce polarization sensitive reconstructions is shown in figure 1. The digital light projector consists of a DMD, and four coloured LEDs, (red, green, blue and near-IR). The DMD spatially modulates the light of the illumination LEDs to project binary patterns onto an object. A linear polarizing filter positioned after the projection lens restricts the projected light to one polarization state. The object shown in figure 1(a) is a stressed clear plastic sheet with a triangular portion removed, where some of the light has transmitted through the plastic and is subsequently reflected from a white diffuse screen, thus enabling the birefringence effects due to internal stress to be observed. The four photodetectors result in four sets of signals for each differential pattern and is akin to a four CCD Stokes imaging system [19]. The patterns applied to the DMD and corresponding measured signals are used for image reconstruction utilising an efficient computer algorithm.

Image reconstruction
There are many possible sampling approaches that could be adopted when utilising the apparatus described in 2. The use of pseudo-random patterns have been adopted in many experiments [10,12], but because each pattern is not necessarily unique this introduces redundancy and often requires many more patterns than the number of pixels in the image. An efficient approach is to ensure that the patterns belong to an orthogonal basis, thereby removing spatial sampling redundancy and reducing acquisition times.

Hadamard patterns
A common sampling approach makes use of the Hadamard basis, from which each pattern can be derived and where the number of patterns is the same as the number of pixels to be sampled. The basic building block of any Hadamard matrix, with a dimension length of 2, is For any matrix of order 2 k there exists such a Hadamard [20], defined as which is known as a Sylvester constructed Hadamard matrix. In this way the Kronecker product, ⊗ − H H 2 2 k 1, can produce any square Hadamard matrix and hence the number of elements in a row equates to the number of rows of the matrix. Each pattern that illuminates the object, P x y ( , ), corresponds to one row of the large matrix that has been reshaped and upsampled onto the DMD. Taking into account computational overhead and pattern display rate of the DMD, we choose to construct multiple Hadamard arrays H 96 as a basis for operating our system in the 'slow' mode and H 128 for the 'fast' mode (supplementary video available at stacks.iop.org/jopt/ 17/025705/mmedia). The projected pattern illuminates the object and each single-pixel photodetector measures a portion of light, θ D , corresponding to the overlap of the object and the pattern for a particular linear polarization state, defined here as where θ T x y ( , ) is the transmission function (or reflection function) for a given object when the polarizing filter is in an orientation θ. With the photodetectors in reflection, we acquire four different signals for each pattern corresponding to backscattered light polarized at 0°,90°,+45°,−45°with respect to the linear polarization of the incident illumination (defined as°0 ).

Stokes parameter image reconstruction
The first stage of the image reconstruction process is correlating measured signals from each detector with its associated illumination pattern. We define is the resulting 2D image reconstruction for polarization θ. After the complete basis set of M Hadamard patterns has been displayed, a fully-sampled image is reconstructed for each photodetector. Figure 2 shows the analysis of a 'typical' single run of the system to quantify the strength of the signals versus the calculated noise on each of the detectors. The noise is calculated as the mean difference between two sets of signals whilst the scene remains static. In this experiment it is shown that only a fraction of the total patterns provide significant contribution (above the noise floor) to the final reconstruction. Utilising only the significant fraction of patterns would enable a form of compressed sensing for the following frame, provided the scene remains static, however this is not applicable in real-time to dynamic scenes since the significant pattern set would change.
The Stokes parameter image corresponding to the total intensity, is obtained from summing the images corresponding to the orthogonal polarization states, defined as One method of calculating images representing the linear Stokes parameters is by subtracting two images from each other, corresponding to orthogonal polarization states, defined as 2 45 45 Alternatively the Stokes images can be obtained by utilising the difference in the signals measured by detectors sensitive to orthogonal polarization states, such that In this experiment reconstruction of Stokes parameter images is performed utilising equations (8) and (9), which was found to reduce the computational load, enabling continuous operation in real-time. In addition, we can calculate the degree of polarization (DoP) from , ,  Reconstructed images of a 'corner cube' obtained by the polarization sensitive single pixel camera prototype for various incident linear polarization illumination. D1-D4 columns represent the reconstruction from each individual detectors. S 0 , S 1 and S 2 columns represent the first three Stokes parameter images obtained through combinations of the signals as described in the main text. The angle of polarization column, AoP, shows the effect of a multi-faced surface where each mirror 'segment' was at a different angle than its neighboring segment, with respect to the linear polarization illumination angle. Figure 3 shows a complete set of results obtained from the system when observing a 'corner cube' optical component, utilising a modified experimental setup as shown in figure 1(b). A corner cube was chosen because it is essentially a set of mirrors which have different angles with respect to the incident linear illumination angle. The results allow a qualitative characterisation of the systems response to a multifaced object, and may allow direct comparison to previously reported Stokes parameter imaging utilising detector arrays. We observe differences in relative intensity for the different mirror faces as expected. Single-pixel cameras operating in a projection or masking mode, tend to suffer when a portion of the scene exhibits high specular reflectivity, since this can result in overwhelming signals measured for certain patterns and not others, degrading the final reconstruction. In this instance however, the entire object exhibits specular reflection and . Stokes parameter images S 0 , S 1 and S 2 reconstructed in colour (RGB), for which the object is a stressed clear plastic sheet which has had a triangular portion removed. The birefringence effects which are wavelength dependent occur within the plastic. The angle (in degrees) of the projected linear polarization state is indicated on the left hand side and relative to an arbitrarily defined°0 state. Only one wavelength channel is acquired at a time, thus sequential illumination is utilized to provide RGB colour. The scale of each colour channel is separately normalized to 255 leading to conventional 24 bit colour depth images. indeed this resulted in high-signals measured independent of pattern choice, and led to high-quality images to be reconstructed (supplementary videos available at stacks. iop.org/jopt/17/025705/mmedia).

Observation of wavelength dependent birefringence effects
The Stokes parameter images, S 0 , S 1 and S 2 , are acquired simultaneously for a particular setting of the incident polarization. The object, shown in figure 4 is a stressed clear plastic sheet having a triangular section removed. The centre of each image indicates a point of stress where we observe colour dependent intensity variations in S 1 and S 2 that are not seen in S 0 , indicating birefringent effects and hence validating the polarization sensitivity of the system. The results in figure 4 have been obtained whilst operating the system in the 'slow' mode, with a pattern projection rate of ∼1440 Hz. The lower display rate, and thus increased sampling time, allows the use of low-cost, wide-band photodiodes. The choice of resolution (96 × 96 pixels) was chosen to provide enough detail in the images to reveal birefringence effect whilst also minimising the fully-sampled acquisition time of approximately 15 s per image. Figure 5 is the deconstructed colour planes which form the RGB full colour images shown in figure 4. The specific width and other features present in objects which show birefringent properties are highly dependent on wavelength, thus for proper analysis and interpretation of the results shown in figure 4, the individual colour planes are given which were recovered from the sequential illumination process.

Stokes parameter imaging of diffuse material
Orthogonal polarization imaging is a commonly used technique for investigating the non-reflective components in diffuse objects. Therefore in a subsequent experiment the inert object was replaced with a living object, in this case a small cactus plant, which was chosen for the distinct differences between the sharp 'needles' and soft 'flesh'. The results are shown in figure 6. As with figure 4 the first column, representing S 0 (reflected intensity) does not change as a consequence of different incident polarization. Whereas analysis of S 1 and S 2 reveal differences as a result of the incident illumination, in particular the 'needles' appear to exhibit enhanced contrast compared to the 'flesh' for different incident polarization. Note that°90 and°0 images of S 1 are the inverse of each other, similarly with°45 and −°45 images of S 2 . When the incident illumination is at an angle of°45 with respect to the angle of the PBS, the corresponding photodetectors will measure a similar light intensity leading to an image exhibiting mostly uniform noise, due to the normalisation process used here. We also note that these noisy images appear red, which is believed to be the consequence of the inherent differences in spectral response of the photodetectors. The final column represents the recovered DoLP information, the invariance with illumination polarization angle verifies the system can recover DoLP accurately.

Investigation of dynamic birefringence effects
To acquire Stokes parameter images from dynamic scenes requires utilising faster pattern projection rates or utilising fewer patterns from the set via compressed sensing techniques. The latter presents a non-trivial computational reconstruction algorithm, this increases computational load which can make it difficult to achieve near video framerates and 'real-time' reconstruction, thus it was beyond the scope of this work, although this approach is currently under investigation as a posible method of improving the technique. Instead, the pattern projection rate was increased to ∼22 kHz and the resolution of the patterns limited to 128 × 128 pixels. Along with differential pattern projection this leads to a maximum video frame rate of ∼0.7 Hz. Importantly however, when operating the system in 'fast' mode results in less light received by detectors and hence lower signal-to-noise, leading to poor image quality. To compensate for reduced light levels the photodetectors were replaced with PMTʼs. Figure 7 shows the results acquired from this system in one colour channel while the scene exhibited dynamic behaviour. For clarity only S 1 is displayed here. The object described in figure 4 was used for this investigation. In figure 4(a) the plastic was maintained under constant stress while the linear polarizing filter in the illumination path was rotated by hand. We observe that the intensity variations in each image rotate in accordance with the linear polarizer, verifying that the Stokes imaging system is capable of producing images at near video-rates. In figure 7(b) the linear polarizing filter was fixed while the stress applied to the plastic was gradually increased. We note that the contrast between the intensity variations in each image increases as the applied stress to the plastic also increases, further validating the near video-rate capabilities of the system.

Conclusions
In conclusion we have demonstrated a polarization sensitive colour imaging system utilising single-pixel detectors that can be used to reconstruct 128 × 128 pixel images at near video- rates. We have demonstrated the use of adding and subtracting signals acquired by four single-pixel detectors, each sensitive to orthogonal polarization states, to produce images representing the linear Stokes parameters, whilst illuminating an object with fixed linear polarization structured light field. A simple iterative reconstruction algorithm was used to produce images with no requirement for post processing or compressed sensing. The spatial resolution in the images is determined not by the position of the detectors but instead by the DMD, leading to 'alignment-free' Stokes image recovery, compared to sensor-based polarimetry systems.
The performance of the system was tested whilst operating at 'slow' and 'fast' display rates of structured illumination, indicating that when measuring diffusely reflected light from objects, low-light levels can result in poor image reconstruction, thereby demanding the use of more sensitive detectors such as PMTʼs. We have applied the system to probe birefringence effects in stressed plastic and to reveal reflective properties of a cactus plant, observing contrast enhancement in the orthogonal polarized plane as expected. When observing scenes containing dynamic behaviour the 'fast' projector display rate enabled near video-rate recovery of linear Stokes parameter images in real-time, by utilising a simple iterative algorithm. Furthermore utilising a modified experimental setup we have performed qualitative comparison of a multi-faced object (a corner cube) exhibiting high specular reflectivity further demonstrating the capability of the system for recovering Stokes parameter images and AoP.
The use of additional detectors, PBS' and an achromatic quarter wave plate would enable images representing the full Stokes parameters to be recovered, when the scene is appropriately illuminated at various wavelengths. If it is possible to overcome the increased computational load and related challenges, the use of compressed sensing techniques would allow faster acquisition times and higher resolution images to still be recovered in real-time, which is the subject of ongoing investigations. We anticipate the use of singlepixel detectors with similar systems to have importance for providing polarimetric imaging at wavelengths where detector arrays are unavailable or limited.