Noise creates polarization artefacts

The accuracy of calculations of both the degree and angle of polarization depend strongly on the noise in the measurements used. The noise in the measurements recorded by both camera based systems and spectrometers can lead to significant artefacts and incorrect conclusions about high degrees of polarization when in fact none exist. Three approaches are taken in this work: firstly, the absolute error introduced as a function of the signal to noise ratio for polarization measurements is quantified in detail. An important finding here is the reason for why several studies incorrectly suggest that black (low reflectivity) objects are highly polarized. The high degree of polarization is only an artefact of the noise in the calculation. Secondly, several simple steps to avoid such errors are suggested. Thirdly, if these points can not be followed, two methods are presented for mitigating the effects of noise: a maximum likelihood estimation method and a new denoising algorithm to best calculate the degree of polarization of natural polarization information.


Introduction
Polarization information is abundant in both terrestrial and aquatic environments, with many animals using this channel of information for a wide range of visual tasks including navigation [1][2][3], communication [4][5][6][7] and visual contrast enhancement [8][9][10]. However, visualising the polarization of light in the natural environment and understanding the ecological and behavioural relevance has proved challenging. Humans are effectively blind to the polarization of light [11], relying on artificial visualisation techniques and human mathematics and language to create a representation of the polarization of light. This is analogous to the behavioural importance of ultraviolet light signals and photoreception which has been difficult to understand [12,13].
There are three main sources of polarization in nature: scattering in the atmosphere and underwater, reflections from flat (dielectric) surfaces and structures that act as biological polarizers. In general, the polarization of light describes how waves of light travel through space. Three quantities are used to characterise this description: 1. the angle of polarization (AoP); the average angle at which the electric fields of waves of light in a beam oscillate. The AoP varies between 0 and 180 degrees. 2. The degree (or percentage) of polariza-tion (DoP); the distribution of the orientations of the waves in a beam around the AoP. This can also be considered as the ratio of the polarized component of the beam to the unpolarized component and is a quanti ty that varies between 0 and 1 (0% and 100%). 3. The ellipticity; the circular component of the beam, which ranges from −1 (left-handed circular) to 0 (linear polarization) to 1 (right-handed circular). When circular polarization is not measured, the degree of linear polarization DoLP is used instead of the DoP. The two quantities are equal when there is no circular polarization (elipticity = 0), which is often the case as circular polarization is rare in nature. For most of the rest of this paper DoLP is the quantity of interest.
The most broad-field sources of polarization in both the terrestrial and aquatic environments are generated by Rayleigh scattering [14]. The sky has a characteristic polarization pattern as light from the sun undergoes multiple Rayleigh scattering events with sub-wavelength diameter molecules in the upper atmosphere. The degree and angle of this scattered light depend on the scattering angle and the initial trajectory of the ray. Consequently, the skylight polarization pattern varies predictably both across the sky and with the time of day. In the aquatic environment, a similar scattering mechanism produces polarized 'veiling light' with the highest degree in the direction perpendicular to the predominant direction of downwelling light [15]. Reflections from surfaces such as water also produce partially polarized light [16]. The DoP of the reflected beam is governed by the angle of incidence and the refractive indices of the two materials, with the maximum DoP occurring at Brewster's angle. Consequently, the DoP of an object or body such as water will depend on the viewing angle of the observer. Polarization upon reflection is most dominant in terrestrial environments, where the mismatch in refractive indices of air and an object is greater than in aquatic environments.
Biological optical structures in the skin or carapace of animals can also affect the polarization of reflected light For example, some species of stomatopod crustacean manipulate the DoP and AoP of the reflected light from their antennal scales [17] and first maxillipeds [18] through the optical properties of ordered lattices of the dichroic carotenoid molecules and anisotropic ordered vesicles respectively [5]. Other nanoscale architectures create structural reflections and colouration which have also been adapted to create biological polarizers. Some biological optical structures can affect the ellipticity of reflected light, for instance the chiral structures of the chitin of species of scarab beetles preferentially reflect either left-or right-handed circularly polarized light [19,20], as does the telson of the stomatopod crustacean Gonodactylaceus falcatus [5,21].
However, this description of the polarization of light and our understanding of how the polarization is changed and manipulated in nature has created two challenges in the literature. The first is that no animals directly measure the AoP or DoLP and obviously know nothing of these human mathematical descriptions. In understanding the polarization ecology of different animals, the focus should always be on the way animals process this information [22]. This first challenge is not discussed further in this paper, and is left to other authors. The second challenge is that because we do not see polarization information as other animals do, it is hard to correctly measure the absolute values of polarization information or understand what is ecologically relevant. This point is the main focus of this paper, and in particular we aim to provide an understanding of why so many papers incorrectly report absolute values of DoLP, especially measurements of black or dark objects, by failing to account for noise in the images. We will demonstrate how these potential errors can be made when DoLP is measured using both imaging polarimeters and also when using a spectrometer. We will then discuss steps for mitigating the effects of noise, by considering illumination of scenes and camera settings, maximum likelihood estimation and finally a new denoising algorithm we have developed. In this paper, we have chosen to focus of measurement of DoLP rather than AoP, as the measurement of DoLP is a much greater source of error in the literature.

Representing polarization
A polarizer is an optical filter which maximally transmits light of a given linear polarization. The angle between the transmitted light and the horizontal is known as the polarizer orientation. Let I represent the total light intensity and I i represent the intensity of light which is transmitted through a polarizer orientated at i • to the horizontal. The standard way of representing light polarization is by using Stokes parameters (S 0 , S 1 , S 2 ), which are defined as follows [16]: Note that S 0 is simply the total light intensity. Also note that assuming that ellipticity of the light is 0, then I = I 0 + I 90 = I 45 + I 135 , so the above can be rewritten just using I 0 , I 45 and I 90 as: The degree of linear polarization (DoLP) and the angle of polarization (AoP) are defined as: Traditional techniques visual ecologists use to measure the polarization of light have centred on using spectrometers [23]. Placing a rotatable polarizer in the path of the input beam of the spectrometers and measuring different spectra at 0 • , 45 • and 90 • have enabled the characterisation of several polarized light environments [15,[24][25][26]. More recently imaging polarimeters have been able to measure real-time polarization video which provide key information about the dynamics of polarization signalling [7]. Imaging polarimeters (also known as 'polarization cameras') are devices which, in addition to measuring the intensity of light at each pixel in an array, also measure the light polarization at each pixel location. There are many designs of imaging polarimeter, summarised in [27], the most recent of which are able to gather spectral information simultaneously with polarization [28][29][30][31]. The common feature they share is measuring light intensity which passes through polarizers of multiple orientations, (I i1 , I i2 , . . . , I in ), possibly with additional measurements of circular polarization, at each pixel location in an array [32]. The measurements for multiple orientations are taken either simultaneously (in 'division of amplitude', 'division of focal plane' and 'division of aperture' polarimeters) or sequentially in the case of a completely static scene ('division of time' polarimeters) and Stokes parameters are then derived at each pixel, or in the case of 'division of focal plane' polarimeters, each superpixel (group of 3 of 4 pixels). The most common arrangement is to measure I 0 , I 45 and I 90 at each pixel location, and the rest of this paper considers imaging polarimeters of this type. It is straightforward to generalise this work to other arrangements and the methods presented here are applicable to every type of imaging polarimeter. As the interests of this paper lie in polarization measurements across a whole image array, for the rest of this paper the symbols I 0 , I 45 , I 90 , S 0 , S 1 , S 2 , DoLP and AoP will refer to the array of values, as well as single measurements. I 0 , I 45 and I 90 are known as the camera components, and S 0 , S 1 and S 2 as the Stokes components.

Darkness and noise
A difficulty that arises in the use of imaging polarimeters (which also applies to spectrometers) is the issue of noise. As this work will show, the measurement of DoLP and AoP is very sensitive to noise and even low levels can have dramatic effects on the measured values. This paper focuses on the measurement of DoLP because a current issue in the literature is the incorrect reporting of the absolute values of DoLP. This point is illustrated in figure 1, where the polarization image of an unpolarized scene was simulated by assuming each camera component was identical, and the DoLP was calculated. Noise was then added to the camera components and the DoLP was calculated again. The amount of noise added was small enough that the visual difference between the original images and those with noise added is small. The correct DoLP of the simulated scene is zero everywhere. In contrast, the DoLP when calculated from the camera components which have had noise added is non-zero across most of the image, and DoLP is large in areas of the image which have low intensity (i.e. dark in the I 0 , I 45 and I 90 images). In fact for 1.3% of the pixels in the image DoLP > 1, which is a physical impossibility. As the true DoLP = 0, this shows that even a small amount of noise can create a large amount of error in a DoLP image.
Because it is the dark areas of the original camera components that generate these incorrect absolute values of the DoLP, this leads to a very different explanation for results reported in the literature where those studies have concluded that black objects or coloration display very high DoLP values [33]. Although in some cases darker objects do exhibit greater polarization that lighter ones owing to the Umov's rule [34], care must be taken so as not to conflate this increase in DoLP with that which is due to noise. Unless it can be demonstrated that very high DoLP in dark areas of an image is not due to noise, the inference that darker objects are very highly polarized cannot be supported as a correct conclusion.
To quantify the effects of noise on DoLP a numerical simulation was performed. Gaussian noise of standard deviation σ = (S 0 t + R 2 ) 1 2 (as in [35]) was simulated with an image with DoLP = 0, as above, where S 0 , t and R represent intensity, exposure time and read-out noise respectively, the S 0 t term represents shot noise. Figure 2 shows how the expected value of the error of the DoLP, E( ), varies as S 0 , t and R vary. Note that if δ is the measured DoLP, δ is the true DoLP and = δ − δ is the error, then in this case, since δ 0 = 0, = δ. It can be seen that generally the smaller the intensity, S 0 , the greater the error, as was observed in the unpolarized scene. Additionally as the read-out noise, R, increases, the expected error increases, but this effect becomes negligible as the exposure time, t, increases. The exposure time, t, makes a large difference to the expected error. As it is increased the error due to both shot noise and read-out noise is greatly reduced, to almost nothing in the case of a long exposure except for very small values of S 0 . Note this simulation does not include pixel non-uniformity, which can lead to an increase in error for long exposures when intensity is low.
This sensitivity of DoLP measurements to noise leads to problems when taking polarization images in the field. Figure 3(a) shows an image containing several beetles which reflect partially polarized light under unpolarized illumination. The three camera components (I 0 , I 45 and I 90 ) are shown along with the DoLP and AoP. The top row shows the images taken with negligible amounts of noise, the bottom row shows the images taken with a moderate amount of noise (the change being effected by decreasing the exposure setting on the camera and normalising the intensity). Visual differences seen between the camera components are clear, but not extreme. However, the differences between the DoLP images are much greater. The white areas indicate where the measured DoLP is greater than 1, again a physical impossibility. No white pixels are present in the 'low noise' DoLP image, however in the 'noisy' DoLP image 7.3% of the image has DoLP > 1. In the 'noisy' image there are also large regions on each beetle that have high DoLP, which are not present on the 'low noise' image. Such results could be very misleading in an experimental context.
Under lab conditions, with stationary subjects, it is possible to take polarization images with arbitrarily small amounts of noise. A low noise level can be achieved by firstly having adequate lighting, and using a suitably long exposure. The noise level can then be made arbitrarily small by taking multiple pictures and then taking the mean of the camera components. The 'low noise' images in figure 3(a) were produced by taking 10 images, which were aligned and the mean was taken. Figure 3(b) shows the DoLP and AoP computed from the first 5 cumulative means of the camera components of the images of the beetles. It can be seen that as the number of images included in the mean is increased, the DoLP in parts of the image reduces, as the amount of noise reduces. There is very little visible difference between the DoLP and AoP images which are calculated from the camera components consisting of the mean of 4 and 5 images, and by the time there are 10 images, as in figure 3(a), the amount of noise can be seen to be very small. The noise standard deviation (and therefore the SNR) of N images averaged in this way is reduced by a factor of √ N , so for 10 images is reduced by a factor of √ 10. Under field conditions, the above method is usually not possible because it requires multiple identical images to be taken, and therefore requires a completely static scene. Additionally, only slight movement in the scene, as would be expected outside in a natural scene, requires a short exposure setting on the camera, which  increases the noise level. In figure 3(a) the 'noisy' image was captured by using a shorter exposure than the 'low noise' image, and used a single image (i.e. no averaging) (the intensity values were then shifted so that the 'noisy' and 'low noise' images have the same median pixel intensity). These conditions represent a realistic amount of noise, which, as figures 1-3 show, can have a dramatic affect on the measured DoLP, and could easily be misleading in a field situation.
So far we have only seen the effects of low intensity on imaging polarimetry, but spectrometer measurements are affected in a similar way. To demonstrate this phenomenon, the DoLP of a modified LCD panel [10] was measured across the 8bit pixel scale (values of 0-255) at six different levels of illumination using a spectrometer (QE6500, Ocean Optics, Dunedin, Florida) with an integration time of 1.0 s and 1.5 s. Since the intensity and the DoLP of the light source vary independently, there should be no change in the DoLP as intensity varies. However, as shown in figure 4(a), at low light levels (absolute irradiance <0.0012 Wm −2 nm −1 ), the measured DoLP exceeds 1, again a physical impossibility. The lower the light intensity, the greater the incorrectly calculated DoLP. This error is greatly reduced with a longer inte-gration time; the measured DoLP is largely unaffected by the illumination level when the integration time of the spectrometer was set to 1.5 s ( figure 4(b)). To conclude, increasing the signal-to-noise ratio by extending the integration time greatly reduces the overestimate of the degree of polarization in low-intensity highnoise regions when using spectrometric polarimetry.

How noise can be corrected and the correct absolute values of DoLP measured
There are several basic steps which should be undertaken when taking imaging polarimetry in order to decrease the amount of error introduced by noise. Firstly, the scene being captured must be well illuminated, with the ISO setting as low as possible. Figure 2 demonstrates that the expected error for an unpolarized scene is greatly reduced if S 0 is as large as possible. If the intensity of the scene has a large dynamic range then only the parts of the scene that are well illuminated should be regarded as having a reliable measurement of DoLP. Secondly the exposure should be as long as possible without overexposing. Although increasing the exposure duration, t, increases shot noise (by a factor of √ t), the signal is also increased (by a factor of t), so the SNR increases and therefore the expected error decreases, as shown in figure 2. Any parts of a scene that are overexposed, even if only in one camera component, cannot be used to provide a measure of the DoLP as they will result in a false measurement. If all three camera components are overexposed then clearly DoLP = 0 will be measured.
The calculation of a DoLP image using equation (7) leads to bias. Consider noisy measured Stokes parameters, (S 0 , S 1 , S 2 ), which are normally distributed around the true Stokes parameters, (S 0 , S 1 , S 2 ). Let the true DoLP be given by δ. If the measured DoLP, δ , is computed the naive way, using (7), with the measured Stokes parameters, then E(δ ) = δ (where E is expected value). This can be seen by the fact that if the true DoLP, δ, is zero, then any error in S 0 and S 1 results in δ > 0. To counteract this bias the 'maximum likelihood estimate' (MLE) [36] of the DoLP, δ ml , can be calculated from δ and the measured intensity S 0 . Figure 5(a) shows the surface of the δ ml values and examples of δ ml for given vales of S 0 . The derivation and calcul ation of δ ml is given in appendix A.1.
The MLE of the DoLP image for three polarization images: 'zebra', 'beetles' and 'plant' (note 'zebra' is not a true polarization image but the simulated image detailed in section 3.1) with added noise, were computed for several values of σ, the standard deviation of noise. The mean squared error (MSE) between the pixels of the original DoLP images and naively computed DoLP images, and between the pixels of the original DoLP images and the MLE of the DoLP images were then calculated, with the results shown in figure 5(e). It can be seen that for all three polarization images, if σ > 0.02, the MLE of the DoLP image has an MSE which is approximately half of the MSE of the naive DoLP image. Figure 6 shows the ground truth, naive, MLE and denoised (which will be discussed in section 3.3) DoLP images for all three polarization images with σ = 0.026. It can be seen that the MLE images are visually more similar to the ground truth than the naive images, however, the improvement is limited.
The derivation of δ ml , as given in appendix A.1 relies on the assumption that every value of AoP is equally likely, which will often not be true in practice and will lead to the bias due to the effects of noise not being eliminated, but only reduced. It also relies on the measured S 0 as an estimator for the true value. If the exact value of S 0 can be measured or inferred then the MLE method will provide a greater reduction in error. Even with these limitations, the modest reduction from using MLE is still likely to be useful in many applications, especially when computer processing power is limited or for real time applications as using MLE has low computational complexity.

Denoising
Maximum likelihood estimation can reduce the amount of error in a polarization image, but is of limited effectiveness in that it simply shifts DoLP values without taking into account spatial information from the image. To return images with significantly less error, denoising algorithms, which take spatial information into account, can be used. A disadvantage of denoising is that it is of much greater complexity than maximum likelihood estimation, so cannot be used for real time applications, or when computational resources are limited. Many image denoising algorithms exist in the literature, but few of these are specifically tailored to polarimetry. The most recently developed denoising algorithm specifically for polarimetry is PBM3D by Tibbs et al [37]. PBM3D is an adaptation of Dabov's BM3D algorithm [38] for polarization. BM3D was chosen as a primarily for its robustness and effectiveness [39], and secondarily because many extensions to BM3D have been published, demonstrating the versatility of the core algorithm. Extensions have been published for color images (CBM3D) [40], multispectral images (MSPCA-BM3D) [41], volumetric data (BM4D) [42] and video (VBM4D) [43].
Other methods for denoising polarimetry in general include [44,45] which have been evaluated and compared to PBM3D [37] and shown to give inferior For the shorter integration time, the DoLP is overestimated at low light levels, to the extent that the degree exceeds 1, which is unphysical. With a longer integration time, the signal to noise ratio of the data is sufficient to greatly reduce the low-light overestimation. denoising performance. [46] presents a method for the denoising of polarimetric 3D integral imaging and [47] deals with noise when demosaicing 'division of focal plane' polarimeter data. [48] proposes a method for demosaicing 'division of focal plane' data as well as a general denoising algorithm for polarimetry based on PCA, based on [49]. A similar algorithm from the same author [50] has been shown to be easily outperformed by BM3D for intensity imaging [39], so PCA-based methods were therefore not considered when developing PBM3D.
It should be noted that rather than acting on DoLP images as with MLE, the denoising acts on a three comp onent polarization image, resulting in a denoised three component polarization image. Figure 6 dem-onstrates the effectiveness of applying PBM3D to polarimetry. The DoLP images in the 'denoised' column are the result of applying PBM3D to the images whose DoLP is shown in the 'naive' column. They are far more visually similar to the ground truth images than the 'naive' and 'maximum likelihood' images. Figure 5(e) shows that for all three images tested, those denoised using PBM3D have a much smaller MSE than the MLE images. On average across all three images and all noise levels the MSE of the denoised images is smaller than the MLE images by a factor of 36.4.
To further test PBM3D with real rather than simulated noise (as used in figure 6), we used a DSLR camera with rotatable polarizer to capture the three camera  In 'naive' the DoLP is calculated in the standard way using (7), in 'maximum likelihood' the maximum likelihood estimate is used, in 'denoised' the image has been denoised using PBM3D (which will be discussed in section 3.3). In the DoLP images the color bar represents DoLP values from 0 to 1, DoLP > 1 is represented by white. Photo credit (zebra): Tim Caro.  Figure 7(a) shows the camera components of the captured image. Figure 7(b) shows the components of the captured image denoised using PBM3D. While the level of noise in the images is low, the effect of this noise is evident in the DoLP images of the original and denoised images (figures 7(c) and (d)). The perceptible noise in the noisy DoLP image (figure 7(c)) is clearly greater than for the DoLP image for the denoised camera components ( figure 7(d)).
In addition to imaging polarimetry (figure 7), we also measured the DoLP of several regions of the scene (figures 7(c) and (d)) using a spectrometer with an integration time sufficient for the illumination level (and therefore sufficient SNR) to greatly reduce the effects of noise discussed in section 3.1. The intensity count was averaged across the wavelength range corresponding to the camera sensitivity (400-700 nm) at three different orientations of a rotatable polarizer, 0 • , 45 • and 90 • . These mean intensities, I 0 , I 45 , I 90 , were used to calculate the DoLP using (7). The DoLP of the corresponding regions in the polarization images was also calculated using (7) with a weighting on each of the camera components (I 0 , I 45 , I 90 ) to account for the separate RGB channels, I i = 0.299R + 0.587G + 0.110B, which corresponds to the luminance, Y, of the YUV colorspace. Due to the slight differences in spectral sampling, some small disparity between the DoLP values calculated using image polarimetry and spectrometry is to be expected. The DoLP values from the spectrometry and from the imaging polarimetry with the noisy image and the same image denoised using PBM3D are shown in table 1. While there is some disparity between the imaging polarimetry DoLP and the spectrometry DoLP, the values of the DoLP from the noisy image are significantly different from those calculated from spectrometry (Wilcoxon test, V = 0, n = 7, p = 0.016). In contrast, the DoLP values from the denoised images are not significantly different from the spectrometry DoLP values (Wilcoxon test, V = 5, n = 7, p = 0.156). Denoising using PBM3D significantly reduces the effect of noise on the measurement of DoLP when using imaging polarimetry.

Conclusions
Imaging polarimetry is an important tool for understanding an animal's visual ecology in the context of the natural environment. Additionally, polarimetry is increasingly being incorporated into bio-inspired vision systems, which seek to optimize the utilization of visual information in a similar way to natural visual systems. In this work, we have highlighted the adverse effects of image noise in correctly calculating the absolute values of polarization. The degree of polarization in areas of low intensity is particularly susceptible to overestimation error due to noise. We have demonstrated that, in order to mitigate the effects of noise, attention must be paid to lighting conditions and camera settings, though care must be taken to avoid overexposure. In applications involving limited computational power or for real time applications, our method for maximum likelihood estimation may also be used to reduce the noise by a factor of 2. However, the biggest reduction in error in the DoLP calculation was obtained by using the PBM3D denoising algorithm, reducing the error by a factor of 36 on average. Using the PBM3D denoising algorithm on a single polarimetry image results in measurements of the DoLP that are not significantly different from those obtained using a photospectrometry method.
Let (S 0 , S 1 , S 2 ) represent the true Stokes parameters, and let S 1 = I 0 − I 90 and S 2 = −I 0 + 2I 45 − I 90 represent the measured the Stokes parameters. Then, for given S 1 , S 2 :  From the probability density function (PDF) of the normal distribution, the conditional density functions f S 1 |S1 and f S 2 |S2 are given by: We seek the joint probability density function f S 1 ,S 2 . To do so, let 1 2 A be the true AoP, and assume that all values are equally likely, thus A ∼ U(0, 2π), where U represents the uniform distribution, with PDF f A (α) = 1/2π. Then, by the law of total probability we have (A.8) Now, letting the true DoLP, Δ, take the value δ and assuming that the true intensity is given by S 0 , note that A = α ⇔ S 1 = S 0 δ cos α and S 2 = S 0 δ sin α, We now seek the probability density function,f ∆ , of ∆ , the measured DoLP, which can be calculated from the cumulative density function, F ∆ as follows: (A.12) F ∆ is given by F ∆ (δ ) = P(∆ < δ ) = P(S 2 1 + S 2 2 < δ 2 S 2 0 ), which is given by: (A.16) Note the similarity to [36], which concerns a similar problem.
For given values of S 0 , σ and δ, f ∆ is shown in figure A1. It can be seen that the lower the value of the true DoLP, δ, the greater the probable difference between δ and δ . For high values of δ, δ is distributed symmetrically around δ, meaning that on average the measured value will equal the true value.
The maximum likelihood estimate of δ, δ ml , is defined as the value of δ which maximises F ∆ (δ ; δ) [36]. This has been calculated numerically for a range of S 0 and δ values, which are shown in figure 5(a). These have been used to find the maximum likelihood esti- Figure A1. The probability density function, f ∆ , of the measured DoLP, ∆ , with varying true DoLP, δ.
mates of the pixel values given in figure 6. As the true S 0 value is unknown, the measured value is used instead, which is an unbiased estimator of the true value.