Simulating digital micromirror devices for patterning coherent excitation light in structured illumination microscopy

Digital micromirror devices (DMDs) are spatial light modulators that employ the electro-mechanical movement of miniaturized mirrors to steer and thus modulate the light reflected off a mirror array. Their wide availability, low cost and high speed make them a popular choice both in consumer electronics such as video projectors, and scientific applications such as microscopy. High-end fluorescence microscopy systems typically employ laser light sources, which by their nature provide coherent excitation light. In super-resolution microscopy applications that use light modulation, most notably structured illumination microscopy (SIM), the coherent nature of the excitation light becomes a requirement to achieve optimal interference pattern contrast. The universal combination of DMDs and coherent light sources, especially when working with multiple different wavelengths, is unfortunately not straight forward. The substructure of the tilted micromirror array gives rise to a blazed grating, which has to be understood and which must be taken into account when designing a DMD-based illumination system. Here, we present a set of simulation frameworks that explore the use of DMDs in conjunction with coherent light sources, motivated by their application in SIM, but which are generalizable to other light patterning applications. This framework provides all the tools to explore and compute DMD-based diffraction effects and to simulate possible system alignment configurations computationally, which simplifies the system design process and provides guidance for setting up DMD-based microscopes. This article is part of the Theo Murphy meeting ‘Super-resolution structured illumination microscopy (part 1)’.

TH, 0000-0003-2348-7416; MM, 0000-0002-2264-3643 Digital micromirror devices (DMDs) are spatial light modulators that employ the electro-mechanical movement of miniaturized mirrors to steer and thus modulate the light reflected off a mirror array. Their wide availability, low cost and high speed make them a popular choice both in consumer electronics such as video projectors, and scientific applications such as microscopy. High-end fluorescence microscopy systems typically employ laser light sources, which by their nature provide coherent excitation light. In super-resolution microscopy applications that use light modulation, most notably structured illumination microscopy (SIM), the coherent nature of the excitation light becomes a requirement to achieve optimal interference pattern contrast. The universal combination of DMDs and coherent light sources, especially when working with multiple different wavelengths, is unfortunately not straight forward. The substructure of the tilted micromirror array gives rise to a blazed grating, which has to be understood and which must be taken into account when designing a DMD-based illumination system. Here, we present a set of simulation frameworks that explore the use of DMDs in conjunction with coherent 2021 The Authors. Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/ by/4.0/, which permits unrestricted use, provided the original author and source are credited.

Introduction
Spatial light modulators (SLMs) offer a robust and fast way to pattern the excitation light in a fluorescence microscope. This can be employed for various illumination schemes [1], for example to achieve selective (de)activation of photo-switchable dyes [2], and most notably, to achieve background suppression and resolution enhancement in structured illumination microscopy (SIM) [3][4][5][6][7][8], a widely used, fast super-resolution microscopy technique [9][10][11][12][13]. Typical SLMs are based on liquid crystal technology, and thus operate by electrically modulating the phase (or amplitude) of light through their active material. Digital micromirror devices (DMDs), on the other hand, work electro-mechanically, by individually flipping the orientation of each mirror between two pre-defined states. Because of their widespread use in consumer devices such as video projectors, DMDs are available at relatively low cost and in a variety of sizes. They also offer high switching speeds, they can handle high light intensities, and depending on coating, are not sensitive to light polarization. This makes them an interesting option for many SLM applications in microscopy [14][15][16][17][18][19][20][21][22]. However, the jagged nature of the micromirror array gives rise to the blazed grating effect that becomes rather annoying and detrimental when using DMDs in combination with a coherent light source [23,24]. Thus, if DMDs are selected as active light modulation systems in a fluorescence microscope based on laser light sources, this effect must be well understood and needs to be taken into consideration.
The work presented here was motivated by our wish to expand the range of applications of a DMD-based SIM system (figure 1), where a coherent laser source is the primary source for the SIM interference pattern, giving rise to optimal pattern contrast [14]. We have developed a set of simulation frameworks that allow us to simulate the propagation of coherent light reflected off a DMD at different angles of incidence, at different wavelengths and with the DMD displaying arbitrary patterns. This enables us to explore the feasibility of optical layouts, to determine which simplifications (e.g. keeping optical elements in a single plane on a table) are possible, and to choose the proper wavelength combinations that could be used in a multicolour DMD-based system. While our work is motivated by and centred around SIM, most of the findings should apply to any other, arbitrary DMD-based microscope systems that employ coherent illumination.

Methods
If DMDs are to be used with coherent illumination, it is essential to know and understand the resulting diffraction patterns generated by both the pattern displayed on the DMD and, much more importantly, the structure of the tilted micromirrors that is native to the device. Harvey & Pfisterer [25] provide a one-dimensional introduction to the blazed grating effect itself, but this model needs to be extended when used to describe a two-dimensional DMD with mirrors tilting along their diagonal. A white paper [23] by Texas Instruments (itself a large producer of DMD chips) offers an introduction to the blazed grating effect in conjunction with DMDs, and Brown et al. [22] recently provided a (semi-)analytic framework to model DMDs. Simultaneously, we started from a purely numerical approach to simulate light propagation in a DMD-based system, and developed simplifications to this model in the process. This results in three simulation (a) General modelling of coherent light and DMDs The general model was derived for the previously developed DMD-based SIM system [14], and, for completeness, is summarized here, as it forms the basis of our simulation approaches. For incident and diffracted directions we use the normalized vectors a = (a x , a y , a z ) and For every incident direction, all diffracted directions must be considered. Since the DMD is basically a two-dimensional array of mirrors, we start with modelling a single mirror with defined dimensions Here, w is the width and height of the mirror, and s and t are parameters for the x-and y-directions, respectively. To model the tilt state each single mirror in a DMD array can be rotated by the angle γ around the normalized diagonal axis n = (n x , n y , n z ) = 1 To shift a single mirror, we use the native grid m(m x , m y ) = m(m x , m y , 0) with the micromirror pitch m = w + g. The gap between the non-tilted mirrors is g. m x ∈ N 0 and m y ∈ N 0 are the indices of the grid points (the single mirrors). This leads us to the following expression to describe each single mirror on the DMD: We can model monochromatic coherent light using the Fraunhofer/far-field approximation with a time-independent electric field, which depends on the beam profile I 0 ( a, d) = |E 0 ( a, d)| 2 projected onto the DMD (2.5) E 0 represents the incidence amplitude and Φ the resulting phase, which is dependent on the path length l = l a + l b = d a − d b and the wavelength λ (figure 2a). For the electric field diffracted at a specific point on the DMD we obtain (2.6) To calculate the diffracted field distribution, we need to integrate over each single mirror and calculate the sum over the entire DMD with N x and N y as the number of single mirrors in x-and y-direction This expression depends on many parameters and generally cannot be simplified further analytically. In order to be able to calculate diffraction patterns numerically and in adequate time, it is thus necessary to make further assumptions to simplify the calculation. Depending on royalsocietypublishing.org/journal/rsta Phil. Trans. R. Soc. A 379: how these assumptions are chosen, different approaches emerge which are capable of modelling different system constraints.

(b) Ray tracing approach
We approximate integrating over every single mirror and summing over the DMD (equation (2.7)) by running a Monte Carlo simulation with rays which will be summed up ( figure 2b). This yields a computationally feasible (if somewhat slow) simulation that does not introduce further constraints or approximations not inherent to the Monte Carlo process. To simulate a collimated Gaussian incident laser beam, we use a Gaussian probability density for the generation of K randomized incident beams with this equation of a line and with h k as support vector, a as normalized incident direction vector and the parameter u.
Here, h k · a = 0 because h k and a are perpendicular to each other. h k is Gaussian randomized for each ray f k . To obtain the point of diffraction on the DMD, we must calculate the intersection A Gaussian beam profile is already considered by the probability distribution of the rays. The field composed of contributions from all rays needs to be summed up to result in the final ray tracing expression for the diffracted field distribution (figure 2e, left). This simplifies equation In this ray-tracing approach, both the assumption of a Gaussian beam profile and especially of fully monochromatic (i.e. coherent) light could easily be changed. Thus, by using a different ray distribution, and by allowing for the single rays (equation (2.9)) to follow a wavelength and phase distribution when being summed up (equation (2.10)), it would easily be possible to simulate for example an arbitrary profile of a partially coherent source.

(c) Analytic phase shifting approach
For this approach, we assume that the field amplitude over each single mirror is constant γ m x ,m y ) and each single mirror can only be in the tilt state γ − or γ + (as reasonable assumption for the steady state of a DMD). A Gaussian beam profile can still be approximated by varying E 0 m x ,m y over the different single mirrors, and as typically a reasonably sized array of mirrors is illuminated, the error introduced by this approximation is small. This provides us with the opportunity to solve the integral in equation (2.7) over a single mirror for γ − and γ + , with the analytically known solution for E single mirror γ ± ( a, b, w, λ) (see dmd.nb in the attached repository). Instead of calculating the field for each mirror individually, we can use the fields of a reference mirror for γ − and γ + and apply the phase shift for the desired grid point (m x , m y ) shown in figure 2c. This simplifies equation (2.7) for the diffracted field (figure 2e, middle) to  This approach is computationally much simpler, while only introducing a minor approximation. It was thus used for the design of our compact DMD-based SIM system [14].

(d) Grating approach
Additionally to the assumptions for the analytic phase shifting approach, we assume here that the field amplitude over the entire DMD is constant (E 0 m x ,m y ( a, γ m x ,m y ) = 1) and that all single mirrors are in the γ − or γ + state. The field E single mirror γ ± ( a, b, w, λ) can be considered as an envelope over the diffracted native DMD grid E grating ( a, b, m, N x , N y , λ). This gives the opportunity to write equation (2.7) as follows: (2.12) To get the intensity I DMD , we calculate and multiply both diffraction patterns I envelope and I grating (figure 2d), very similar to the case of the Young double slit experiment (2.13) Of course, this approach is constrained to the simulation of a DMD without a pattern being displayed. However, if only the diffractive nature of the DMD itself is of concern, this offers a computationally very effective solution.

(e) Blaze condition approach
We assume that the incident beam is perpendicular to the tilting axes n of the single mirrors. This leads to a = (a x , −a x , a z ) and −ϕ a = ϑ a . To imagine it simply we rotate the DMD by 45 • (figure 3a). To account for the rotation, we switch to new angle coordinates (figure 3b): (2.14) As new coordinate axes, we define k = (1, −1, 0). Along k we get a new lattice constant m = (1/ √ 2)m. For perfect constructive grating interference, the path difference l must be an integer multiple of the wavelength λ nλ = m (sin(α) + sin(β)). (2.16) If we have perfect constructive interference in the centre of the envelope, the blaze condition is fulfilled. The centre of the envelope can then be assumed as a reflection of the light incident on the surface of a single mirror and we define this as our diffraction angle β = −α + 2γ . For this reflected envelope, we can now calculate the corresponding diffraction order If n is an integer the blaze condition is fulfilled. We use v to visualize this relationship v = |sin(nπ)| = sin π m λ (sin(α) + sin(−α + 2γ )) . (2.18) Here, v 0 , whenever the blaze condition is fulfilled. The choice of |sin(nπ)| leads to a symmetric (negative and positive deviations contribute equally) and slightly nonlinear (small deviations contribute stronger) metric, which is well suited to visualize the blaze condition for different wavelengths λ and different incident angles α along the diagonal of the DMD (figure 3b).
While this approach is even more constrained, it has an important real-world application: it allows us to directly calculate incident and reflective angles when placing a DMD rotated by 45°o n an optical table, where collimated beams are expected to run parallel to the table's surface.
Here, it provides a quick solution for the instrument design. A similar approach was pursued by Li et al. [20].

Results and discussion
Each of the algorithms described above has certain advantages and disadvantages, which means that each algorithm has its own area of application. As shown in figure 1e, the ray-tracing approach, the analytic phase shifting approach and the lattice approach each deliver qualitatively the same results under identical boundary conditions. The positions of the individual diffraction orders are identical in the three algorithms. All diffraction images shown in this article were simulated with m = 7.56 µm micromirror pitch and γ ± = ±12 • as tilt angle, which corresponds to the dimensions of the DLP ® LightCrafter™ 6500 (Texas Instruments) [26] and some other DMDs. To obtain a clear visualization of the diffraction orders with large visible spots (Fourier broadening), the simulation is limited to an array of 50 × 50 mirrors being illuminated. Of course, when no or a different visualization is needed, the algorithms can be run for larger mirror arrays. Especially, when a small subset of possible angles is computed with a high sampling rate, a larger number of mirrors can and should be used, up to the full DMD size.
The ray tracing approach is most flexible because it requires the least simplifying approximations. It can be used to calculate the diffraction patterns for any DMD-pattern. In addition, the approach could be extended relatively easily for arbitrary exposure beam profiles and incoherent light by specifying a corresponding spatial distribution and wavelength distribution and/or phase distribution for the random rays. Also, mechanical deformation of the DMD, i.e. non-flat surface structured, could be integrated into the model, and aid in determining their influence on the diffraction and thus illumination properties of the device. The main disadvantage of the ray tracing approach is that it is by far the slowest of the presented algorithms. Therefore, this algorithm is especially suitable for the simulation of incoherent or partially coherent light. The analytical phase shifter has the only disadvantage that, when compared with the ray tracing approach, it cannot be readily modified for use with incoherent illumination, or if done so, it would lose its speed advantage. Also, the granularity with which illumination   intensity and mechanical deformations (which would be represented as phase shifts) could easily be simulated is limited to a single intensity and offset per mirror, introducing an approximation. For the illumination, this effect is not noticeable in practice. As its main advantage, the algorithm works much faster and is especially suitable for the calculation of diffraction images where the DMD is provided with a pattern ( figure 4). The grating approach is not able to simulate any patterns displayed on the DMD. It assumes that all mirrors are tilted in the same direction. Due to its high speed, however, the algorithm is very well suited for investigating the effects of different boundary conditions such as changes in wavelength and angle of incidence on the diffraction image generated natively by the structure of the micromirror array (figure 5). It can be used to determine the distance between the centre of the envelope and the brightest native grating order (figure 5a). If this distance is 0°, the blaze condition (Littrow configuration) is fulfilled. For the use of coherent light in e.g. a SIM microscope, this is exactly what is desired, because then an isotropic envelope field and intensity distribution is present in the Fourier plane. This has the consequence that the disadvantages of the blazed grating effect of DMDs can be negated by exploiting the blaze condition. This is true for a range of angles of incidence which are shown in black in figure 5b. These angles of incidence that fulfil the blaze condition then result in the angles of diffraction marked in figure 5c.
If a DMD-based SIM microscope aims to provide multiple excitation wavelengths that share a single DMD, then it is necessary that for each wavelength the blaze condition is fulfilled. A possible approach to solving this problem is to use the γ − position of the micromirrors for one wavelength and the γ + position for the other wavelength. As shown for example for 488 nm or 638 nm in combination with 561 nm in figure 7a, there are two configurations in which the blaze condition is fulfilled for both wavelengths. This choice of wavelengths is desirable, as arguably 488 nm, 561 nm and 638 nm (all of course allowing for some variation) are very compatible with typical dyes used in fluorescence microscopy. This configuration is certainly possible but somewhat complex to achieve experimentally (as multiple independent tip/tilt axis have to be precisely aligned) and it is also very susceptible to the slightest change in angle of the micromirrors (we found e.g. 0.3°in [14] Figure 5. Analysis of the displacement between the envelope and the brightest diffraction order of the DMD. The columns from left to right correspond to the wavelengths 488 nm, 532 nm and 561 nm. Row (a) illustrates the displacement between the centre of the envelope (grey, linear intensity representation) and the brightest diffraction order of the native DMD grating (cyan, green, yellow, logarithmic intensity representation) using red arrows. The field of view ranges from −1°to 7°for ϕ b and −7°bis 1°for ϑ b with a sampling rate of 0.1 • at an angle of incidence of ϕ a = −ϑ a = −21 • . Row (b) visualizes the offset which can be determined from (a) for the angles of incidence ϕ a and ϑ a ranging from −60°to 60°with a sampling rate of 0.2°. The dark areas with a displacement close to 0°indicate angles which fulfil a blaze condition and are well suited for illumination with coherent light. The cyan dotted lines between (a) and (b) indicate an angle of incidence of ϕ a = −ϑ a = −21 • for the diffraction images seen in (a). Row (c) shows the areas in the diffraction space of ϕ b and ϑ b from −80°to 80°, each, in which the displacement between the centre of the envelope and the brightest diffraction order of the DMD is not more than 0.1°. Therefore, angles of incidence in a range of −60°to 60°used in (b) were used for ϕ a and ϑ a . The red boxes mark the diffraction space shown in (a). The white dotted lines in (b) and (c) correspond to the diagonal angles of incidence that are considered in the blazed condition approach. (Online version in colour.) 635 nm). They use a (semi-)analytic approach to model the blazed grating effect, developed independently of the approaches presented here, while providing compatible results, which we view as an important cross-check. Their experimental realization, including a voice-coil mirror for the dynamic switching between incidence angles, is very functional but also highlights the complexity it introduces. For a multi-colour implementation of a cost-effective DMD SIM [14], we aim to reduce opto-mechanical complexity, ideally avoiding dynamic opto-mechanical components or twodimensional alignment (non-horizontal beam paths). Therefore, we propose to limit the angle of incidence and thus also the outgoing angle for the optical axis of the DMD-SIM microscope to the diagonal perpendicular to the tilting axis of the micromirrors. This corresponds to a rotation  This reduces the number of degrees of freedom of the incident laser beam (ϕ a , ϑ a ) → α and the optical axis (ϕ b , ϑ b ) → β from two to one each, as described in equation (2.14). The experimental implementation thus becomes much easier, because the exposure path of the microscope can be set up parallel to the optical table, as usual. Now, however, the wavelengths must be chosen skilfully. For a more precise analysis of the blaze condition along the diagonal, both the grating approach and the blazed condition approach are suitable ( figure 6). Both approaches show qualitatively the same results, therefore, it is sufficient to refer to the blaze condition approach in the following. It is possible to use two wavelengths λ 1 and λ 2 with the same angle of incidence α 1,2 and diffraction β with the same tilt state γ − = −12 • . Starting from equation (2.16), it follows that n 1 λ 1 = n 2 λ 2 . (3.1) As integer diffraction orders n 1 = −4 and n 2 = −3 are suitable for the visible range. If we pretend λ 1 = 488 nm, we obtain  If a third excitation colour is supposed to use the other tilt state γ + = 12 • , we get α 3 = −β + 2γ + = 7, 4 • as incidence angle and must assume n 3 = 4 to be in the visible range, which then limits the wavelength to λ 3 = m n 3 (sin(α 3 ) + sin(β)) = 553.93 nm. (3.3) This combination of these three wavelengths, arranged at (−40.6 • , 488 nm), (−40.6 • , 651 nm) and (7.4 • , 554 nm) is indicated in figure 6 as a possible configuration for a three-colour DMD-SIM microscope. Given that these wavelengths are close to the typical combinations of blue (472 nm to 488 nm), yellow (561 nm to 568 nm) and red (630 nm to 650 nm) laser excitation sources used in fluorescence microscopy, and that suitable diode laser sources (even offering temperature-based fine-tuning of their emission wavelength) are readily available, this combination should make for a surprisingly capable three-colour DMD-based microscope.
Further possible configurations that closely match typical excitation wavelength are triples of e.g. 473 nm, 631 nm and 551 nm or 491 nm, 655 nm and 555 nm. It is important to note here that λ 3 depends very much on the tilt angles of the micromirrors, and thus might have to be adjusted based on the parameters of the specific DMD device. For λ 1 and λ 2, the tilt angle is of less importance. Looking at the diffraction space in which the blaze condition is fulfilled for typical wavelengths (figure 7a), it is noticeable that, using the same tilt state, no matches are found. The ring-like structures in figure 7 represent the different diffraction orders. Their size depends on wavelength and diffraction order and for clarity they are shown in different colours. As mentioned earlier, using the other tilt state results in matches, which are not on the diagonal and therefore not suitable for experiments. But if we choose the wavelengths carefully for multicolour applications (figure 7b), the circular structure of the -4th and -3rd diffraction order of 488 nm and 651 nm using the same tilt state match exactly. If we use the other tilt state for 554 nm, there is, besides the matches outside the diagonal, another one on the diagonal, which is formed by a  collapsing/emerging circular structure of the 4th diffraction order. This configuration, already described above, is therefore ideal for a three-colour DMD-SIM microscope.

Conclusion
The framework and the software tools provided here provide a rather universal set of simulation tools for experimentalists planning to use DMDs as SLMs with coherent light sources in SIM microscopes or similar applications. Compared to ferro-electric light modulators (FLCoS-SLM), DMDs are faster and more cost efficient, and they allow for a more compact design due to the smaller micromirror/pixel pitch. Also, the use of FLCoS-SLMs requires the implementation of a much more complicated timing scheme because of the limited duration during which a specific pattern can be projected by these devices. DMDs have no such constraints and can switch at speeds up to an order of magnitude faster than FLCoS-SLMs. We have developed four different approaches to simulate coherent light diffracted and reflected by a DMD (table 1). The results provide practical design suggestions for circumventing the undesired blazed-grating effect of DMDs. Our simulation results agree with the independent findings of Brown et al., who were able to successfully construct a multi-colour DMD SIM offering three excitation wavelengths [22]. Following our analysis of suitable wavelength, and building on top of an existing opto-mechanical platform [14], it should now also be possible to design a multicolour DMD-based SIM microscope with simple, passive opto-mechanics which can fully use all of the speed and cost advantages of a DMD mentioned before. Additionally, our simulation platform offers the flexibility to be extended, so that the use of partially coherent light sources and the effects of mechanically deformed DMDs can be explored. Especially for the latter, we hope that with a combination of inteferometric measurements and extensions of the ray tracer simulation, we can in the future explore how the optical quality (flatness, etc.) of the DMD influences illumination properties. We are currently building a two-colour DMD SIM microscope and plan to add a third colour to it in its final implementation.
The results shown in this article can easily be reproduced with the ImageJ/Fiji plugin which we provide as an open access download. The plugin contains all approaches and algorithms presented in this article, which can be adapted to different system conditions. For a more detailed insight into the plugin and some mathematical additions, we recommend a look at the electronic supplementary material.
Following the spirit of open science, we provide all source code and raw data for the results presented in this manuscript to the scientific community. All code is openly accessible under GPLv2 (or later) license and can be found in online repositories under github.com/fairSIM and github.com/biophotonics-bielefeld.
Data accessibility. A software package to simulate with the introduced algorithms the diffraction patterns of DMDs and the introduced further analysis depending on the angle of incidence, light wavelength and DMD pattern has been created. The software is written in Java to work as a plugin to scientific image processing package ImageJ/FIJI [27,28] and provides GPU acceleration through the jCuda framework [29]. The source code is available under an open-source (GPLv3) licence in an online repository: github.com/biophotonics-bielefeld/coherent-dmd-sim-simulator.