Display of polarization information by coherently moving dots

It is known that human eyes are effectively polarization-blind. Therefore, in order to display the polarization information in an image, one may require exhibiting such information using other visual cues that are compatible with the human visual system and can be easily detectable by a human observer. Here, we present a technique for displaying polarization information in an image using coherently moving dots that are superimposed on the image. Our examples show that this technique would allow the image segments with polarization signals to "pop out" easily, which will lead to better target feature detection and visibility enhancement. Comments Copyright 2003 Optical Society of America, Inc. Postprint version. Published in Optics Express, Volume 11, Issue 13, June 2003, pages 1577-1584. Publisher URL: http://www.opticsexpress.org/abstract.cfm?id=72903 This journal article is available at ScholarlyCommons: https://repository.upenn.edu/ese_papers/274


Introduction
Polarization is an important feature of electromagnetic waves, and it can be affected by surface shapes, materials, local curvature and features, and relative location of sources and objects, and thus it can provide useful information about the observed scene and objects. Of the several characteristics of visible light, only two intensity and wavelength are received and encoded by the human eye and mapped by the visual system into perceptual qualities of brightness and color. In contrast, without appropriate instruments human eyes cannot effectively utilize light's polarization. However, it is well known that eyes of some animal species are sensitive to the polarization of light (see e.g., [1]- [5]). Polarization sensing by some of these species has mostly been shown to provide information for navigation [5], as originally discovered by the Nobel laureate Karl von Frisch in his work with honeybees [1], and it has been suggested that some species may have evolved polarization sensitivity as a mechanism of enhancing the contrast of targets in scattering media [6]. In our previous work, we have shown that an optical imaging system utilizing such contrast enhancement through polarization differencing can increase the distance over which targets can be detected, and their critical features discriminated [7]- [9]. Thus, "polarization imagery" is a naturally occurring and demonstrably successful strategy for enhancing vision.
However, since the human eye cannot "see" the polarization information, when this information is captured by a polarimetric imaging system, it has to be displayed into some form of visual cues/information that can be detectable by a human observer. In other words, some form of "sensory substitution" should be exploited for representing polarization "signals". In our group's earlier work, we have shown one such bio-inspired mapping, in which polarization information was pseudo color-coded, based on the opponent-colors model of human vision [9]. The results established several promising imaging strategies for displaying in a natural way the contrast enhancement of polarization imagery of objects in scattering media. Such mapping of polarization information into pseudo-colors ignores the "true" colors of the scene, namely, the spectral information contained in the scene. So if one wants to preserve the spectral and luminance information in an image, one will need to map the polarization into visual cues other than the color and brightness.
We are interested to explore and investigate certain bio-inspired display methodologies for mapping polarization information into visual information that can be readily perceived by the human visual system. In this Letter, we describe one such mapping, namely representation of polarization information by a set of coherently moving dots superimposed on an image. It is known that human vision is capable of motion perception, which includes coherent motion detection, form from motion, and biological motion (e.g., [10]- [12]). We exploit the sensitivity of coherence detection in human eye in displaying polarization information in an image. One of the motivations behind using coherently moving dots in such mapping is to preserve, by and large, various features of the image, such as color and luminance, while "displaying" the polarization information.

Mapping of Polarization-Difference (PD) Signal into Coherently Moving Dots
In our earlier work, we introduced the concept of polarization-difference imaging [7]. In this technique, at every pixel of the image, the intensities of the two orthogonal polarization components of the light reaching the camera, ( , ) I x y and ( , ) I x y , are obtained, and then the sum and difference of these two quantities are evaluated where ( ) , x y identifies the pixel position in the image, and indicate two orthogonal linear polarizations, PD stands for the "polarization-difference" signal, and PS for the "polarization-sum" signal. If an ideal linear polarization analyzer is used to capture orthogonal polarization components, the PS image will be equivalent to a conventional intensity image.
In order to use moving dots to represent the polarization information in a scene, we first obtain the two orthogonal polarization components at every pixel, and then form the PS image of the scene following Eq. (1). We then superimpose dots on this PS image. The original locations of dots are chosen as follows: The entire PS image is divided into small identical square-shape cells each having m m × pixels, e.g., 10 10 × pixels. In each cell, a single square dot is placed at a random location within the cell. The size of the dot is n n × pixels, with n m << , and can be properly chosen to make the dots small enough so that the original image would be practically undistorted by the presence of these dots and large enough so the dots can be detected by an observer. In the results presented in this Letter, the dot size is taken to be 2 2 × pixels for an observer comfortably seated in front of a computer while viewing the image on the computer monitor at viewing distance of about 50 cm. So in each cell, the 2 2-pixel × dot replaces a randomly chosen 2 2 × -pixel set in the cell. The density of the dot depends on the cell density. The brightness of the dot is chosen according to one of the two algorithms described later. To produce dot motion, we make sequence of several frames of the same image, but in each frame the location of the single dot in each cell is changed with respect to its location in the previous frame as follows. (1) if the averaged PD signal, i.e., the average of ( ) , PD I x y , in that cell is greater than certain "threshold" value , the direction of change of location of the dot, which becomes the direction of perceived motion of dot in that cell will be chosen along a vector parallel with polarization component; (2) if the averaged PD signal in that cell is less than , the direction of dot motion will be selected along the polarization component; and finally (3) if the averaged PD value is between and within that cell, the sequence of location of the dot in that cell will be picked randomly, leading to a random motion of dots in this case (or the dot location can be fixed resulting in dots with no movement). So when these frames are played sequentially, the dots in all the cells with average PD signals greater than appear to be moving coherently in the direction parallel with the polarization component, while the dots in cells with average PD signals less than move coherently along the orthogonal direction. The dots in cells with average PD signal between and move randomly (or they do not move at all). Since the human visual system can detect and perceive coherent motion, the two regions with dots moving coherently in two orthogonal directions can be pre-attentively segregated and "popped out" against the rest of the image where the dots are moving randomly (or they are fixed). Thus, in this technique, the observer views the PS image with its details as a conventional image, while detecting three regions in the image representing areas with PD > , PD < , and PD < < signals.
The brightness/intensity of the dots must be chosen such that they are easily detectable against the background image. Here we suggest two different recipes; (1) the intensity of each pixel of a dot in each cell is selected to be in "contrast" with the PS signals of the pixels the dot is replacing. Specifically, if ( ) , PS I x y of each of the pixels which are being replaced by the -pixel m m × dot is greater than 128 pixel intensity in an 8-bit display system, the intensity of each pixel of the dot in that cell will be assigned as . In this scheme, which we call the "contrast scheme", the dot intensity is "complemented" against the background intensity in each cell; (2) in this method, the intensity of each pixel in a dot in each cell is 4 chosen to be M% less than the PS signal in the image pixel which it is replacing. So . This approach, we call "percentage scheme".

Results
To demonstrate this mapping strategy, we develop the above algorithm in the MATLAB environment. The frame sequences generated using this algorithm are presented as movies with 20 frames/second in the .avi format. These movies can be viewed by any player, e.g., Microsoft Windows Media ™ Player, Apple QuickTime ™ Player, or the likes.
First, in a dark background with no target, we show how a region with coherently moving dots can be easily segregated against the rest of the image with randomly moving dot. Figure  1 shows a collection of randomly located white dots on a dark background. When one clicks on the link to the movie file of this Figure, a movie appears, and one can see that dots in certain rectangular region move coherently while the dots in the rest of the image move randomly. The coherent motion of dots can be easily detected, and the region containing these dots is readily noticed by the observer.
We now apply this algorithm to the images of a target that was previously used by our group in the study of polarization-difference imaging [7]- [9]. We use the same PS and PD images used in (Fig. 1 in [9]) in order to allow the fair comparison between mapping of polarization using the new algorithm described in this Letter with that of the colorimetric PDI algorithm we employed in [9]. The target was an aluminum disk with 3.8-cm diameter. The surface of this disk, except for two 1-cm 2 square patch areas, was sandblasted to make it Lambertian. The patches were raised a few mills and were abraded with the emory paper in orthogonal directions. This target was specifically constructed as such in order to have a surface that may give rise to scattered light with partial polarization from the patch areas (each with partial polarization parallel with the direction of abrasion), and scattered light with essentially no polarization from the sandblasted surface. Figure 2 shows the PS and PD images of this target when in our previous study it was immersed in a 40-L plexiglas tank of water to which 5mL of milk was added, and it was front-illuminated [9]. The PD image was scaled using a "symmetric" affine transformation [9] to utilize the full dynamic range of the 8bit display. In other words, if for instance the range of PD values is between a and b + (with b a > ), the zero value of PD signal is mapped to pixel intensity 128, while b + and b are mapped to 255 and 0, respectively. These images were originally obtained and used in our study of colorimetric PDI by Tyo, Pugh, and Engheta [9]. Figure 3 presents implementation of the current mapping scheme on the image of our target. The threshold value is taken to be 32 = in the affine transformed PD image (which is effectively a threshold intensity value of 2 = in the raw PD image with PD value range between 8 ± .) The cell size is 10 10 × pixels and the dot size is 2 2 × pixels. Dot intensity is assigned using the "contrast scheme" in Observing these movies, one is able to distinguish between the regions with PD > and PD < signals (which is equivalently 128 PD > + and 128 PD < + in the affine transformed PD image), and the region with PD < < signal (or equivalently 128 128 PD + < < + in the affine transformed PD image), while the original information contents of the PS image is essentially intact. With regard to the two schemes for dot intensity, we notice that the "contrast scheme" may work better in images where the PS intensity distribution may be somewhat uniform over certain segment of the image (such as the disk face in our example). The "percentage scheme", on the other hand, may be better for situations in which unimportant segments of an image possess the PS intensity near zero (for example, the dark background in the images shown here). In this way, the dots in such regions have low intensity, and thus do not distract the observer from noticing the dots with higher intensity in regions of importance. Figure 4 shows the corresponding results when the threshold value is chosen to be 48 = in the affine transformed PD image (i.e., effectively threshold value of 3 in the raw PD image.) (Click on appropriate links to the movie file of each Figure to start the movies.) Here we notice that smaller regions of the image possess coherently moving dots, which represent patch areas where PD signals have higher absolute values. So the choice of threshold value is important is segregating regions of the image with specific values of PD signals.
Finally, Fig. 5, when clicked on appropriate links, illustrates a similar mapping of polarization into moving dots. The difference here, however, is that the dots form short lines with time-varying lengths as they move, providing additional cues for perception of polarization direction. This mapping, of course, results in higher number of pixels to be replaced with dots and lines, but it may provide stronger cues for polarization representation.  Fig. 2. Polarization information from the affine transformed PD image (Fig. 2B) is mapped as moving dots onto the PS image ( Fig. 2A). Here the threshold value is chosen to be 32 = for the affine transformed PD values. In (A), the dot intensity is prescribed using the "contrast scheme", while in (B) it is chosen using the "percentage scheme" with M = 30%. Viewing the moving dots, our visual system can distinguish among the regions with PD > , PD < , and PD < < PD signals.  The above examples clearly demonstrate the possibility of exploiting the sensitivity of human visual systems to coherent motion as one possible representational mechanism for visualization of polarization information in an image. This provides the observer with additional information about the scene with little or no distortion to the other nonpolarization-related information, such as color and luminance, in the image. This mapping represents one possible "sensory substitution" for displaying polarization information to human observers while retaining the conventional image information.

Conclusions
We have introduced a visualization technique for mapping polarization information into an image without essentially altering the information contents of the original image. Dots have been superimposed on the image, and the motion of such dots has been implemented. Regions with PD > and PD < signals are given dots that move coherently in two orthogonal directions representing main directions of partial polarization, and the regions with PD < < signals are assigned randomly moving dots. Since human vision can sense and perceive coherent motion, segments of the image with coherently moving dots can be readily detected and distinguished against the region with random motion. In this way, the segments with various ranges of PD signals "pop out". Such polarization representation can lead to visibility enhancement, better target detection and feature extraction.