Image simulation for biological microscopy: microlith

Image simulation remains under-exploited for the most widely used biological phase microscopy methods, because of difficulties in simulating partially coherent illumination. We describe an open-source toolbox, microlith (https://code.google.com/p/microlith), which accurately predicts three-dimensional images of a thin specimen observed with any partially coherent imaging system, including coherently illuminated and incoherent, self-luminous specimens. Its accuracy is demonstrated by comparing simulated and experimental bright-field and dark-field images of well-characterized amplitude and phase targets, respectively. The comparison provides new insights about the sensitivity of the dark-field microscope to mass distributions in isolated or periodic specimens at the length-scale of 10nm. Based on predictions using microlith, we propose a novel approach for detecting nanoscale structural changes in a beating axoneme using a dark-field microscope.

Use of partially coherent (i.e., angularly diverse) illumination has been popular for biological imaging in the form of contrast mechanisms of dark-field, phase-contrast, differential interference contrast, differential phase contrast, polarized-light imaging, among others. As compared to coherent illumination, partially coherent illumination provides higher resolution, improved immunity against optical imperfections in the light path, and a degree of depth sectioning. Among emerging partially coherent methods in optical and X-ray regimes, emphasis is given to the reconstruction of the specimen's optical properties (absorption and phase or their polarization-resolved counterparts that report molecular order, dichroism and birefringence) and their underlying molecular and structural properties. Both types of quantitative analysis and the design of an optimal instrument require accurate modelling of image formation. However, partially coherent illumination leads to bilinear image formation, i.e., the image intensity at a given point in image space depends not only on the corresponding point of the specimen, but on pairs of points of the specimen. The intrinsic bilinearity has hampered the use of partially coherent imaging systems for quantitative biology despite the experimental benefits.
Simulation and analysis of coherent and incoherent systems have been well established [1] thanks to their linear image formation that allows the description of an imaging system as a point spread function in space or a transfer function in spatial frequency for a general 3D specimen. Most of the partially coherent phase reconstruction and segmentation algorithms employ approximate linear models -either of a weak phase variation [2], slowly varying gradient [3,4,5], slowly varying curvature [6], or dictionary of diffraction patterns [7] -to retrieve a quantitative representation of the specimen phase. For partially coherent microscopes, the appropriate model for the specimen and the appropriate transfer function that relates the optical property of interest with the intensity recorded (e,f) are images corresponding to (c,d), but assuming slight over etching. Scale bars: (a,d,g) 5µm, the central disk at the center in the rest of the images has diameter of 1.2µm. (g) Illustration (drawn to scale) shows the angles of diffraction orders generated by the periodic phase grating at the radius of 4µm due to the steepest illuminating ray. (h) shows the angles of the diffraction orders (as described in the text) generated by the periodic grating at radii ranging from 0.6 − 5µm. by the imaging system depend on the optical contrast method used and the object features to be retrieved. Therefore, flexible, problem-dependent simulations are necessary for accurate results. While the optical lithography community regularly employs partially coherent simulation tools to optimize and reduce the feature sizes that can be imprinted on silicone wafers, accurate simulation has remained under-exploited for biological partially coherent imaging. Apart from commercial image simulation tools such as PROLITH TM , lithography community has made available some free simulation tools (e.g., LAVA at http://cuervo.eecs.berkeley.edu/volcano/), but they are not suitable for simulating biological contrast methods and none are open source. With this Letter and the associated open-source MATLAB toolbox, microlith (https://code. google.com/p/microlith), we aim to bridge this gap. The design goals for microlith are to enable quantitative comparison of biological microscopy methods, quantitative analysis of partially coherent images, and design of retrieval approaches. microlith permits the flexibility of simulating 3D image of a thin specimen under any scalar partially coherent system. It allows simulation of spatially coherent imaging (when the illumination originates from a point) or incoherent imaging (when imaging fluorescent/self-luminous specimen) as special cases. The toolbox may serve as an open source alternative for optical lithography simulation -hence the name microlith. The project website provides a tutorial, documentation, and simulated images for simple imaging problems for which analytical expressions are available. In this letter, we demonstrate the utility of the toolbox by revealing the sensitivity of dark-field (DF) imaging to subresolution features and the effect of specimen birefringence on differential interference contrast (DIC) imaging by comparing simulated and experimental images of a Siemens star phase object from the MBL/NNF test target [8].
Partially coherent image simulation based on Hopkins's transmission cross coefficient (TCC) has been standard for optical lithography. Previously we have employed the TCC model for simulations of 1D specimens to clarify effects of coherence in differential interference contrast [9]. Hopkins's TCC is a 4D quantity (for 2D images) with the key benefit that the model separates the imaging system and the specimen. Such a separation is attractive for lithography simulations despite the need to store 4D quantities, because the goal is to evaluate images produced by a library of small features under the same imaging system. To evaluate biological microscopy methods, however, one needs to simulate relatively large patterns (such as Siemens star from MBL/NNF phase target used in this Letter) with variable microscope settings (such as defocus) or different contrast methods. To store the complex TCC coefficients as single-precision floating point numbers for simulating a DIC image of size 256x256, one needs (256 × 256 × 8) 2 = 2.75 × 10 11 bytes, which is a prohibitively large amount of data. Therefore, the current version of microlith implements the sum over source (SOS) algorithm, first described by Zernike [10]. Using the SOS algorithm, we could perform parallel simulation of the images of the MBL/NNF target over a 4001x4001 grid under two contrast methods (DF and DIC) with less than 1.5 GB of memory in total. We implemented numerical optimizations that ensure accuracy and radiometric consistency of the simulated image, i.e., the simulated intensity is proportional to the image recorded by an ideal photodetector. To ensure radiometric consistency (over the chosen simulation grid) when either the specimen or the imaging system is varied, we normalize the computation so that the image of a sub-resolution pixel (that approximates a point) is proportional to the area of the imaging pupil (NA 2 o ), area of the illumination pupil (NA 2 c ), and area of the pixel. We choose normalizations such that the image of a point has the peak value of unity for NA o = 1, NA c = 1 and pixel size of 100nm. To ensure artifact-free simulation, the simulation grid should extend beyond any feature that transmits light by at least the size of the image of a point. When above simulation conditions are met, the simu-lated integrated intensity is invariant to defocus, as expected in the absence of confocal gating in the imaging path. Further details about the SOS algorithm and usage of microlith are provided on the project website. Fig. 1 shows the experimental image of the MBL/NNF target with a high-resolution DF microscope and images simulated using microlith for the same optical setup. The test target is a 90nm thick fused silica layer (refractive index 1.46) deposited onto a microscope cover glass. A Siemens star pattern of 36 wedge pairs is etched into the silica layer using E-beam lithography [8]. The wedge pairs create an azimuthal square grating whose periodicity (d) increases with increasing radius (r) from the center, specifically, d = 2πr/36. If wedges are over-etched or under-etched, the grating is slightly asymmetric (non-square). The cover glass with silica layer is mounted on a microscope slide with a thin water layer between silica and slide. Fig. 1(c) and (e) show simulated binary patterns assuming perfect square etching or over-etching by 4nm, respectively. The simulation grid is sampled at 2nm to capture such fine change in the simulated structure. Beyond the radius of 6µm, the experimental image in fig. 1(b) matches very well with simulated images in fig. 1(d) and (f). Going towards the center in the experimental image, the azimuthal period of intensity changes from a period of edges (2 periods/ 10 • ) to a period of wedge pairs (1 period/10 • ) at the radius of 4.8µm. However, the simulation assuming an exact square grating (36 symmetric wedge pairs) shows that the contrast vanishes at that radius. The experimental image shows intensity modulation of 1 period/10 • up to the radius of 3µm, uniform intensity up to the radius of 1.5µm, and finally almost no intensity up to the bright ring at radius of 0.6µm. Simulation in fig. 1(d) on the other hand shows uniform intensity over radii of 4.8 − 1.5µm, dark-band over 1.5 − 0.6µm similar to experimental image, but much dimmer central bright ring.
Thinking of the azimuthal phase profile as a linear grating, above result can be understood as follows: Since the target is immersed in water, it is illuminated by a hollow cone of light with an angular span θ i ∈ [sin −1 (1.2/1.33), sin −1 (1.1/1.33)]. The wavelength of the light within the specimen is given by λ = 0.546/1.46 = 0.374µm and the undiffracted light (zeroth order) leaves the specimen with an angular span of θ 0 ∈ [sin −1 (1.2/1.46), sin −1 (1.1/1.46)]. From the grating equation, the angles θ N in which the diffraction orders generated by a periodic phase grating can propagate are given by sin θ N = sin θ 0 − N λ/d, where d is the grating period and N the diffraction order. The interference of 1st and 2nd order gives rise to intensity modulation with the period of wedge pairs. The interference of 1st and 3rd order gives rise to intensity modulation with the period of edges. Since the imaging NA is 1, the steepest angle of the ray that can be collected is θ c = ± sin −1 (1/1.46) shown by black lines in fig. 1(g and h). Fig. 1(g) illustrates (to the scale) that at the radius of 4µm, only the 1st and 2nd orders can be collected. From fig. 1(h), we see that the 1st, 2nd, and the 3rd orders enter the collection aperture at radii of 1.5µm, 3µm, and 4.5µm, respectively. Therefore, above analysis accurately explains the experimentally observed transitions in the intensity modulation as a function of the interference of the collected diffraction orders.
However, if the azimuthal pattern is an exact square wave, the diffraction spectrum consists of only odd numbered orders. The 2nd order propagating at θ 2 (shown by a dashed line) is not generated by an exact square grating, leaving only the 1st order at θ 1 up to the radius of 4.5µm, which by itself cannot give rise to a grating structure. Thus, comparison of fig. 1(b) and 1(d) led us to the finding that the pattern must have been over or under-etched such that a slight asymmetry in the azimuthal square wave allows the generation of the 2nd order. As shown by fig. 1(f), we found that over-etching by 4nm provides the best fit between experimental and simulated data; including the bright ring of radius 0.6µm, which is due to a fully-etched ring around the central intact silica disk. The finding that target is overetched by 4nm was subsequently confirmed from the electron micrograph of the target. Thus, comparison of the simulated and experimental DF images led us to understand the sub-resolution deviation of a few nanometers from the intended design in the E-beam lithography process, illustrating the sensitivity of the dark-field microscope to sub-resolution structure and accuracy of our simulation code. We are exploiting the sensitivity of dark-field imaging in conjunction with microlith to measure and model structural changes in a biological nanomachine called axoneme that powers cilia and flagella of eukaryotic cells, which will be published elsewhere. Figure. 2 shows the experimental and simulated images of the same target with a high-resolution DIC microscope. The direction of shear is from top-left to bottom-right. The shear was estimated to be 0.48λ/NA o by imaging the interference fringes created by the DIC prism in the back focal plane of the objective [11]. The contrast in DIC is due to a relative phase-delay between two sheared beams, to which a constant phase-delay or bias can be added. For the simulation, the bias was estimated by matching the contrast of the experimental image and the simulated image in a region close to the periphery of the target. The simulated image in fig. 2(c) is based on the model of DIC imaging that we proposed [9]), which matches well with the experimental image in fig. 2(a). A different model for DIC imaging had been proposed by Preza [12], which predicts much reduced contrast (comparison available on microlith website). The reduced contrast is due to the omission of the coherence effects of the DIC prism in the illumination path in Preza's model (see [9, sec. 2.4]).
From the central regions of the experimental [ fig. 2(b)] and simulated [ fig. 2(d)] images, we notice two experimental effects not accounted for by our model -a) azimuthal variation in the brightness near the center, and b) imaging of edges along the shear direction with small yet finite contrast. Both effects occur due to the fact that sheared beams are polarized orthogonally to each other and the specimen is birefringent. microlith does not simulate polarization effects in its current version. In DIC, the two beams with lateral shear in object space are created using Nomarski prisms near the aperture planes of the condenser and objective lens. The prisms generate two beams with orthogonal polarizations, enabling a change in bias between two beams by adding a birefringent plate between the two prisms. However, this feature makes the technique prone to artifacts caused by the birefringence or dichroism of the specimen, as has been alluded to before [13]. Near the center, the ordered arrangement of dielectric interfaces gives rise to form birefringence with a radially aligned slow axis [14, Fig. 37]. Away from the center, the phase-edge due to etched silica exhibits edge birefringence [15]. The presence of localized specimen birefringence introduces a phase-delay that adds to the bias generated by the DIC prism. We call this additional phasedelay retardance bias. The retardance bias depends on the specimen birefringence and on the angle between the slow axis of birefringence and the directions of polarization of the light (at 45 • to the direction of shear). Near the center, the target exhibits a radially oriented slow-axis and constant retardance, leading to a periodic change in relative angle between the slow axis and the constant direction of shear along the azimuth. Thus retardance bias changes periodically along the azimuth and gives rise to periodic change in brightness of DIC contrast. Further analysis of the notion of the retardance bias may lead to a scheme for separating the phase-gradient and the specimen birefringence.
Apart from gaining insights about the image formation as described in this Letter, a promising use of microlith is to compare the transfer properties of quantitative phase-retrieval methods in terms of the spatial frequencies of the specimen. Phase-retrieval [3,6,13,16] or retrieval of specimen anisotropy [17] requires acquisition of multiple images with systematically varied pupils and algorithmic recovery. The entire process can be characterized by simulating the chosen model specimen with appropriate pupils, and by evaluating the contrast transfer from the phase image produced by the retrieval algorithm. In the future, we intend to extend microlith to incorporate simulation of polarized light microscopy and aberrations besides defocus. We hope that the accuracy and the open source character of microlith toolbox will facilitate quantitative use of partially coherent systems for biological imaging. We welcome feedback and collaborative input from the microscopy and lithography communities.
This study was funded by National Institutes of Health grant RO1 EB002583. SM acknowledges fellowship from Human Frontier Science Program and useful discussions with Colin Sheppard.