Dynamic contrast enhancement in widefield microscopy using projector-generated illumination patterns

We present a simple and cost-effective optical protocol to realize contrast-enhancement imaging (such as dark-field, optical-staining and oblique illumination microscopy) of transparent samples on a conventional widefield microscope using commercial multimedia projectors. The projector functions as both light source and mask generator implemented by creating slideshows of the filters projected along the illumination planes of the microscope. The projected optical masks spatially modulate the distribution of the incident light to selectively enhance structures within the sample according to spatial frequency thereby increasing the image contrast of translucent biological specimens. Any amplitude filter can be customized and dynamically controlled so that switching from one imaging modality to another involves a simple slide transition and can be executed at a keystroke with no physical filters and no moving optical parts. The method yields an image contrast of 89–96% comparable with standard enhancement techniques. The polarization properties of the projector are then utilized to discriminate birefringent and non-birefringent sites on the sample using single-shot, simultaneous polarization and optical-staining microscopy. In addition to dynamic pattern generation and polarization, the projector also provides high illumination power and spectral excitation selectivity through its red-green-blue (RGB) channels. We exploit this last property to explore the feasibility of using video projectors to selectively excite stained samples and perform fluorescence imaging in tandem with reflectance and polarization reflectance microscopy.


Introduction
Modern widefield microscopes are equipped with imaging modalities that enable them to probe specimens specific to their endemic optical parameters including, but not limited to, index of refraction, birefringence, scattering, absorption, transmission and fluorescence. This is accomplished by optical accessories that transform normal bright-field illumination to darkfield, polarized-light, Rheinberg optical-staining, oblique illumination and other specialized imaging techniques. Despite the differences of the measured optical variables, all of these methods share a common imaging attribute: they introduce spatially-modulated illumination to the specimen [1] by placing physical masks and filters before the substage condenser lens of the microscope [2]. However, these masks are static and a change in illumination technique necessitates filter replacement and optical re-alignment. It is therefore difficult to image multiple optical parameters from live samples in real time because it necessitates rapid switching between different contrast-enhancement techniques. Magnification matching introduces another difficulty which strictly requires pairing of specific filters with particular objective lenses requiring the end user to accumulate filter sets. This elevates the cost and places them beyond the reach of medium-scale biological laboratories. Low-cost controllable filters whose physical dimension, illumination intensity and spectral content can dynamically be adjusted would therefore significantly remedy this need.
Spatial light modulators (SLMs) have already been utilized to project illumination and imaging patterns to correct aberrations and to sculpt the point-spread-functions of imaging systems [3]. Liquid crystal-based SLMs have been used to perform phase stepping in conjunction with phase contrast microscopy. This resulted in high contrast imaging and the possibility of performing high precision measurements [4]. They were also utilized to generate microscopic images by scanning dynamic apertures across the sample [5]. Digital micromirror devices (DMDs) were initially employed as programmable optical masks for precision imaging and spectroscopy [6] and have currently found utility in dynamic optical lithography [7]. DMDs were eventually included in the external illumination design of a widefield microscope in order to perform dark-field and confocal imaging [8,9]. These systems rely on external laser excitation whose long coherence length introduces speckles into the widefield image.
In this research, we address active filter control and device cost by utilizing simple, commercial multimedia projectors to introduce dynamic spatial patterns on the illumination 3 planes of a standard widefield microscope. Because video projectors can project any image generated by a computer, a large array of filters can easily be customized by simply creating patterned slide presentations. Transitioning from one slide to another rapidly switches the imaging modality, transforming bright-field illumination to dark-field microscopy, for example, with a single keystroke. Additionally, its rapid refresh rate of 60 Hz provides an ideal imaging environment for the growing field of digital video microscopy of live biological samples, requiring a variety of inexpensive contrast-enhancement protocols with no moving optical parts.
Apart from filter control, the projector also provides over 2000 ANSI lumens of excitation power which can easily be regulated by change of projected color or grayscale values. It further affords spectral selectivity through the RGB channels presenting over 16 million colors to design filters from. We exploit this flexibility by constructing and evaluating the imaging properties of spatial modulation-dependent techniques: bright-field, dark-field, Rheinberg and oblique illumination. This is extended to simultaneous optical-staining and polarization imaging by utilizing the polarization characteristics of liquid crystal display (LCD) projector technology. The optical protocol is performed to probe birefringence and increase image contrast of complementary structures in transparent biological samples. This is expensive and cumbersome when done by standard methods relying on static filters. The last experimental procedure takes advantage of the spectral selectivity within the RGB projector channels to demonstrate the feasibility of utilizing projectors in fluorescence imaging.

Methodology
A schematic of the optical set-up is shown in figure 1(a), where a multimedia projector was coupled to an upright diascopic microscope (BX51, Olympus) using intermediary lenses of focal lengths 300 mm (L1), 150 mm (L2), and 300 mm (L3). The video projector is utilized both for illumination and dynamic generation of the different amplitude masks specific to the chosen contrast-enhancement technique. The masks are created as simple slideshows projected on to the back focal aperture of the condenser lens and imaged through the illumination planes of the microscope to ensure Köhler illumination. Because the light source is not focused at the specimen level, illumination is essentially grainless and extended, and does not suffer deterioration from dust and imperfections on the glass surfaces of the condenser. The transmitted and scattered light from the sample collected by the objective lens (Olympus, NA = 0.30, 10X) is directed by a tube lens to a 12-bit digital color camera (DP71, Olympus), which captures widefield images of the sample.
Two projector technologies are independently utilized in the experiments: DMD (Acer XD1150) technology and LCD (Epson EMP-50) projectors. The DMD projectors and their commercial moniker Digital Light Processing (DLP) technology have higher light throughput and contrast than equivalently-priced LCD devices but do not possess significant polarization properties. Conversely, the nature of crystal orientation and color separation in LCD projectors endow each primary color a linear polarization state. In the LCD projector employed, the red and blue color constituents have the same linear polarization and are mutually perpendicular to the green component as shown in figure 1. By orienting a polarization analyzer orthogonal to green illumination, narrow-band polarization imaging is possible. The spectral content of both projectors cover the 400-700 nm range, with the DMD projector providing more uniform coverage of the visible spectral window. There are spectral bands where there is no considerable spectral overlap between the adjacent colors: λ B (400-450 nm), λ G (525-575 nm), λ R (625-725 nm) for the blue, geen and red channels, respectively. The minimal overlap and the absence of strong spectral tails may help relax the filter requirement for fluorescence excitation if the illumination wavelength falls within the band.
The constant movement of each projector micromirror and filter wheel whenever the pixel values are updated causes fluctuations on the illumination. By observing intensity variations for a single captured pixel over a period of 500 s, a short exposure time (1/100 s) yielded a 16% variation about the mean value. The fluctuations can be minimized by extending the exposure time to 1/20 s exposures, yielding 1.0% variance about the mean. This would give an effective acquisition rate of 20 frames-per-second (fps).
The spectral selectivity of the projector coupled to an episcopic (reflectance) microscope set-up, extends the application to fluorescence imaging along with reflectance and polarization reflectance imaging modes.

Contrast-enhancement using projector-generated amplitude filters
We first demonstrate the flexibility of the set-up to transform a standard microscope into various illumination modes to improve specimen contrast, among them: bright-field, dark-field, Rheinberg, and oblique illumination.
The montage in figure 2 clearly illustrates the various imaging modalities and the corresponding filter pattern generated by the projector. The sample is comprised of translucent siliceous starfish spicules which are difficult to visualize under conventional bright-field microscopy (figure 2(a)) because the minimal refractive gradient between the interface and air scatters too little light to produce any contrast. For this section, we utilize a DMD projector because of its higher contrast ratio over LCD technology. The inset illumination pattern used is a simple white background projected on to the condenser diaphragm which overfills the objective back aperture (dotted circle). The optical pathway is then aligned to adhere to Koehler illumination.

Dark-field microscopy.
Dark-field illumination requires blocking out the central light rays that ordinarily pass through the specimen using 'spider stops' and allowing only conical 6 oblique rays to illuminate the specimen [1]. In the optical set-up, this is implemented with a single keystroke that switches the slideshow to a different frame containing a white annulus on a black background. The inset in figure 2(b) presents the projected annulus at the objective back aperture (dotted circle) viewable by looking through the trinocular tube without the eyepieces. The annulus subtends a mean numerical aperture of N A ann = 0.70 designed such that the cone of light it produces does not enter the objective lens (N A obj = 0.30). The increase in contrast is evident in figure 2(a), where the edges or boundaries of the spicules are clearly imaged in white against a dark background, i.e. the structures having high spatial frequencies are emphasized and only light scattered by the sample contributes to image formation.
In standard dark-field microscopy, the central ray is physically blocked by an opaque stop which rejects the zeroth-order illumination and results in a dark field of view or background. With the projector set-up, there is no actual blockage of the zeroth-order beam. Rejection is implemented by projecting a central black aperture whose degree of darkness depends on the projector contrast. Better projector contrast results in increased light rejection (and darker background) because the central dark pattern contributes less to illumination and image formation on the rear focal plane of the objective. It is therefore important that high contrast projector technology is used, otherwise the resulting image is no longer formed exclusively from higher order diffraction intensities scattered by the specimen. We also point out that the annulus depends on the magnification of the objective lens and separate amplitude masks must be paired with different lenses. With pattern projection, the annulus can easily be re-sized and even nudged to accommodate magnification changes and alignment requirements without additional optics.

Rheinberg illumination.
Sample visibility can also be increased by introducing opticalstaining via Rheinberg illumination (figure 2(c)) which was achieved by using an amplitude filter similar to the dark-field filter but consisting of a green annulus on a red background. The filter color was chosen because a green-red pairing is said to be a visually-effective combination [2]. Zeroth-order light from the red central filter pervades the background. The edges of the spicules scatter the green component, which accentuate these higher-order structures.
The spectral output of the projector can also result in preferential color rendering centered on the wavelength where the projector intensity output is highest. This would swamp the other colors leading to false color portrayal or simply non-detection of the weaker colors. This consideration has influenced the choice of filter colors where green is assigned as the annulus color due to the very high intensity output of the video projector of this color, ideal for weaker scattered signals. Otherwise, the effects of a red annulus on the widefield image will be swamped out by the green signal. Color flooding can also be tempered by changing the color number of the projected patterns from [0, 255] to attenuate color intensity. Using projectors for filter control therefore not only balances or enhances visibility of specific structures but also matches image formation to the spectral sensitivity of the either camera or human vision.

Oblique illumination.
When incident light is restricted to a segment of the condenser light cone, the specimen is illuminated from one side only at a highly specific angle and the net effect is to reveal details in high-contrast, pseudo-relief from an otherwise transparent, unstained specimen. In figure 2(d), an off-axis white pattern was utilized to set-up oblique illumination. The obliquity of the incident light pushes the zeroth-order just within the periphery of the objective aperture and biases the collection of the higher diffraction orders (or sidebands) 7 on a single side and exclusion of the orders on the opposing side. The asymmetric sidebands contribute to image formation leading to shadowing and apparent three-dimensional structures because of the imbalanced interference that occurs between the detected sidebands and the zeroth-order light. The white sector in the illumination pattern can also be easily rotated to any azimuthal angle to highlight different structures in the sample or study the effect of rotation on image formation. From figure 2, it is evident that simple slide transition of custom masks improves the image contrast two-fold from brightfield to dark-field.

Comparative evaluation of imaging modalities and projector technologies
3.2.1. Contrast comparison. The plots in figure 3(a) represent grayscale histograms of the acquired images from the various illumination modes of the optical system. The image contrast provided by the different techniques can be easily deduced from these histograms. For our purposes, contrast is defined as the maximum range of pixel intensities obtained in an image, and for an 8-bit image, 100% contrast is achieved if the image exhibits the complete 256 intensity range. In this context, a broader histogram signifies higher image contrast.
From the plots, the bright-field image exhibits a narrow histogram as compared to the other three techniques; hence, higher image contrast is obtained with the three illumination methods as expected. The contrast of each image is indicated in figure 2. Referenced against the brightfield image, an increase up to 92.9% in image contrast is attainable.

LCD versus DLP projector technology contrast and imaging performance.
It should be noted though that the achieved contrast in the images of the video projector microscope is dependent on the video projector being used. We investigate this by replacing the Acer DLP projector with the Epson LCD projector that has a lower contrast ratio. The Epson and Acer projectors' advertised contrast ratios are 400 : 1 and 2200 : 1, respectively.
A comparison of the performance obtained from the two projectors is shown in figure 3(b) through histograms of images obtained using the LCD and DLP projectors in dark-field mode. The image obtained using the DLP projector has a broader histogram as compared to images acquired using the LCD projector which achieved 59.0% contrast. Projectors having higher contrast ratios provide better illumination since higher image contrast can be realized.
We next evaluate the performance of the projector microscope, against a true dark-field (true DF) microscope, i.e. a microscope that uses a physical annulus mask. The black plot in figure 3(b) represents the image histogram of a true DF system. It has a broader histogram than the DLP projector and the contrast achieved is approximately 98.9%. In the context of contrast ratios of video projectors, the illumination produced by this kind of annular stop has a high contrast ratio since low intensities are achieved by physical light blockage.

Modulation transfer function (MTF)
. The imaging quality of the optical system can further be evaluated through its MTF. The MTF is defined as the magnitude response of an imaging system to sinusoids of different spatial frequencies [10,11] which establishes performance similarity of the imaging system with a conventional microscope.
There are various ways of obtaining the MTF of a microscope system. In this study, an MTF target (NT56-077, Edmund Optics) is utilized, consisting of Ronchi rulings with spatial frequencies ranging from 240 lines mm −1 to 600 lines mm −1 . The Ronchi rulings were imaged where I max and I min are the maximum and minimum intensity in the resulting image. The MTFs obtained for the bright-field (red) and oblique illumination (green) projector modes are shown figure 3(c). The conventional bright-field microscope MTF (blue) is also shown as reference. Error bars were calculated for all the data points on figure 3(c) derived from the standard deviation σ of 300 resolution target images. The true bright-field yielded an average 9 percentage deviation of σ/MTF = 3.6% while the projector MTF yielded larger deviations of 14.6 and 10.1% for bright-field and oblique illumination, respectively. The error bars indicate that 83% of the projector MTF falls within the conventional MTF values and allows us to say at most, that the bright-field mode of a video projector microscope has a similar modulation response to a conventional microscope. Both exhibit decreasing modulation at higher spatial frequencies expected for bright-field illumination.
The error bars also reveal the variability of image formation whenever objects of high spatial frequencies (MTF targets) are imaged. For the biological samples in figure 2, an exposure time of 1/20 s, yielded 1.0% variance about the mean. The large error bars in figure 3(c), however, forces us to proceed with caution whenever high-frequency samples are involved and considerably lengthen the exposure time.
The dip in the MTF of the oblique illumination may be attributed to the relatively low contrast ratio achieved with projected filters compared to physical filters. In oblique illumination using physical filters (which totally reject light), the zeroth-order light is completely translated to the edge of the condenser aperture allowing higher-order light to also enter. This also happens in the projector case but the black background is not dark enough resulting to a persistent brightfield background (i.e. the zeroth-order due to the black background is not entirely shifted to the edge). This birght-field contribution and the difference in the location of their diffraction orders may result to selective reduction in the projector MTF. We note that it is possible to control the location of the modulation dip by adjusting the size of the white oblique pattern in relation to the size of the condenser aperture. This will tune the higher-order frequencies allowed to enter the aperture of the condenser.

Optical-staining, polarization and fluorescence imaging
In the previous sections, it was demonstrated that with amplitude modulation it is possible to derive three imaging modalities to improve image contrast. Although DMD projectors were proven to have better imaging contrast than LCD technology, the latter presents advantages dependent on the nature of its polarized illumination. This property shall be utilized to extract and highlight complementary structures within the biological sample that exhibit birefringence or fluorescence.

Polarization microscopy.
The projection microscope takes advantage of the nature of polarization of the light coming from the video projector itself. Each primary color produced has a different polarization. For the LCD projector, the green component has a linear polarization state that is orthogonal to its red and blue components.
A simple addition of an analyzer placed after the objective lens and oriented orthogonal to the green component of the LCD projector increases the polarization sensitivity of the set-up. In the absence of any birefringent material, the only colors that should be detected by the camera are blue and red, since their polarization states are parallel to that of the analyzer. Figure 4 exhibits the optical properties of the lily anther when illuminated by red, green, blue, and white light whose polarization states are depicted by arrows in the figure inset and referenced against the analyzer (A) orientation. As expected, the red-and blue-illuminated images (figures 4(a) and (b)) did not show any birefringence information and maintain their background color because their polarization states are parallel to the analyzer orientation. The structures that appear are only those that have varying opacities characteristic of bright-field images. In contrast, the green-illuminated images in figure 4(c) registered structures that have known birefringence properties, primarily the fibrous cell wall, and the pollen sacs where mitotic and meiotic cells that normally contain filamentous mitotic spindles are present [12]. The dark background signifies that the polarization of green light from the projector is indeed orthogonal with the analyzer. The resulting polarization contrast ratio is 1 : 100. Although this is 90% weaker than the ratio of a real polarization microscope (1 : 1000), the birefringent structures have still been mapped out.
White illumination in figure 4(d) generated a magenta field because only the red and blue components contribute to background formation. The structures generated are the superposition of the three components showing magenta non-birefringent sites and white birefringent structures since all component colors are transmitted through the analyzer.

Simultaneous polarization and Rheinberg imaging.
We next demonstrate simultaneous polarization imaging and optical staining of complementary structures in translucent biological samples. A Rheinberg filter is used in tandem with an analyzer orthogonal to the green component placed after the objective lens. The image in figure 5(a) clearly discriminates the birefringent (green) and non-birefringent (red) composites of the lily anther. The red annular filter provides vivid optical staining to the pollen sac and parts of the cell wall while the green components are birefringent. It should be noted here that the colors indicated are not pseudo-colors nor assigned via post-processing of the image. All the colors presented are directly and simultaneously captured by the system's digital camera. The pictures are not superpositions of several images and there were no spectral filters necessary. A standard polarization microscope only marks out the birefringent sites. Our method can differentially stain both birefringent and non-birefringent structures simultaneously. Due to the nature of the amplitude filter used and of the illumination source, the red-or blue-colored structures in figure 5(b) are characterized as having high spatial frequencies (since these are the colors of the annuli in the filters) and, being consistent with the optical-staining stated above, do not exhibit birefringence. The green-colored structures, on the other hand, have low spatial frequencies and are birefringent. The implementation of such a concept in standard microscopes will be very costly because the annular and central portions of the static filters need to have orthogonal polarization, different spectral transmission properties and to be designed specific to each objective lens.

Structure-dependent polarization imaging.
We extend optical discrimination to spatial frequency and birefringence properties. In particular, we extract localized birefingent information from low and high spatial frequency structures in the sample. This is achieved by designing filters that probe birefringence from zeroth-and higher-order scattering. Figure 5(c) demarcates birefringence from low spatial frequencies (green). The dark background and blue sites are non-birefringent containing the zeroth and higher scattering orders, respectively. From the same sample, birefringence at high spatial frequencies is shown in the green regions highlighted by arrow points in figure 5(d). The red background is non-birefringent while the low scattering filament is birefringent resulting in a darker outline because it has suffered rotation and partial rejection by the analyzer compared to the background. It is therefore possible to actively select specific spatial frequencies, probe the birefringence properties at those frequencies and then image and isolate the sites that exhibit such behavior, all dependent on the filters being projected into the microscope.

Fluorescence microscopy.
The spectral range of the projector significantly overlaps the spectral excitation window of most biological dyes which makes it applicable to fluorescence imaging. This is demonstrated in figures 6(c) and (d) which feature a commercially-stained cabbage flower bud cross-section taken with an Olympus U-MNG filter (ex: 530-550 nm, DM: >570 nm, em: >590 nm) and FITC filter set (ex: 450-490 nm, DM: >505 nm, em > 515 nm), respectively. The sample was illuminated with white light from the projector. The reflectance using non-polarized white light (figure 6(a)) and linear polarized reflectance images ( figure 6(b)) establish the flexibility of the optical system when used in episcopic mode.

Summary and conclusions
We have demonstrated the feasibility of transforming a standard widefield microscope into dark-field, Rheinberg optical-staining and oblique illumination microscopes by utilizing a commercial multimedia projector as an arbitrary pattern generator. The filters associated with each imaging mode could easily be created on slideshows and projected along the illumination planes of the microscope. Converting to another optical technique was casually executed by a slide transition to the appropriate mask at 60 Hz rates without re-alignment and with no moving optical parts. Image contrast ratios as high as 96% were achieved making it comparable to a true DF microscope. This is expected to improve with better projector technologies.
The linear polarization of the LCD projector was then utilized to map out the anisotropic (birefringent) regions situated in the pollen sacs and cell walls of a translucent lily anther. A customized filter enabled simultaneous polarization and Rheinberg imaging. This permitted optical discrimination of isotropic and anisotropic sites by optically-staining each with different color without the need for spectral filters. We extend this to optical screening of birefringence generated by structures containing low or high spatial frequencies. It is therefore possible to actively delineate spatial frequencies, probe the birefringence properties at those frequencies and then image and isolate the sites that exhibit such behavior. The production of conventional static filters for such applications would be very costly, tedious and beset with less flexibility.
The high luminance in specific spectral ranges enabled excitation of pre-stained samples and fluorescence imaging using appropriate dichroic filters. With the exception of UV-dyes, the projector illumination spectra intersect the excitation window of most biological dyes. Spectral selectivity and dynamic pupil generation holds great potential for fluorescence imaging applications using patterned illumination. By integrating real-time projection and acquisition it is possible to use changes in birefringence, spatial frequency, scattering property and even fluorescence emission as indicators of biochemical or biophysical cellular dynamics.
With the rapid improvements in projector design, image resolution, contrast and brightness will still improve. And with broad public access to affordable technology, any bioimaging laboratory may soon have the capability to perform multi-functional microscopy at low entry cost.