Tunable SIM: observation at varying spatiotemporal resolutions across the FOV

This document provides supplementary information to “Tunable SIM: observation at varying spatiotemporal resolutions across the FOV”. Section 1 describes the motion artifacts induced by fast moving objects in conventional SIM images. Sections 2 and 3 describe the generation of sinusoidal illumination pattern and details for pixel-by-pixel alignment between camera and DMD coordinates. Sections 4 and 5 describe the image acquisition sequence and the image processing workflow to generate adaptive tunable SIM patterns. Sections 6 and 7 describe the reconstruction of tunable SIM images and resolution verification using SIM reference targets. Sections 8 and 9 describe details for fluid flow analysis and tunable SIM videos visualizing the effect of shear stress on actin cytoskeleton dynamics in U87MG-EGFP-CD9 cells. Section 10 describes optical sectioning SIM as an extended application of tunable SIM.


Motion artifacts in SIM
Motion artifacts can be induced by the displacement or deformation of objects of interest during a single SIM measurement sequence. Slow movements induce shear while faster movements appear as distortion artifacts in the reconstructed images [1]. Fig.  S1 shows motion artifacts in a reconstructed SIM image of live U87MG-EGFP-CD9 (U87) cells immersed in fluid flow visualized with 1 µm fluorescent beads. Conventional 3D-SIM patterns were illuminated across the entire field of view. As U87 cell movement was smaller than the final resolution (~100 nm) during the acquisition, artifacts were not significant [2]. On the other hand, fast flowing beads resulted in severe artifacts compromising quantitative analysis based on the images. As the beads moved to different positions during the illumination pattern sequence, the orientation of the structured illumination patterns were visible in the reconstructed images [ Fig. S1 (b)]. When fast dynamics occur at regions in the proximity of U87 cell, motion artifacts induced by moving beads can cause misinterpretation of slowly moving objects (U87 cells) in the reconstructed SIM images as shown in Figs. S1 (c). For areas of slower fluid flow, the beads appear as honeycomb artifacts which disrupt observation of the fine actin structure and also preventing accurate fluid stream line retrieval as shown in Figs. S1 (d), and (e) [3].

Generation of sinusoidal illumination
A DMD is a binary amplitude modulator. To directly display an analog sinusoidal illumination intensity pattern, an option is to use the DMD to directly display the analog intensity while operating in grayscale mode. This is realized by repeatedly displaying fast onand-off patterns that the camera detector will perceive as grayscale intensities. Although our current implementation does not require the maximum refresh rate of our DMD, this approach sacrifices the effective DMD refresh rate. More importantly, this approach requires many pixels of the DMD to be used to correctly display a single analog sinusoid period in order to minimize pixelization errors.
To avoid such limitations, we utilized the DMD to display binary patterns combined with spatial filtering to obtain the analog grayscale sinusoidal illumination intensities. By displaying a binary amplitude grating on the DMD, multiple diffraction orders are obtained at the Fourier plane. A spatial filter fabricated with anodized aluminum foil was placed at the center of the Fourier plane along the optical axis to low pass filter the illumination into a SIM relies on interference of highly inclined beams and therefore is sensitive to the polarization of each beam. Ideally, s-polarized light gives perfect modulation contrast regardless of the pattern orientation angle. For 2-beam SIM, an azimuthal polarizer is a simple solution to generate s-pol for all orientation angles. However, for 3-beam SIM, the 0 th order beam has to be rotated with the pattern orientation to achieve s-pol for all orientations which requires additional hardware and synchronization. To mitigate such complexity, we utilized circular polarization for the 3 beams which does not necessitate such difficulties. Another simple option is to use a segmented azimuthal polarizer that enables spolarization for the ± 1st order beams while maintaining circular polarization for the 0 th order beam. To compare the different options, we performed simulations of 3-beam interference to validate the modulation contrast as shown in Fig. S3 [4]. The modulation contrast was defined as   pattern orientation angle. If the 3 beams are perfectly s-pol modulated, we obtain perfect contrast irrespective of the SIM pattern orientation angle. If all 3 beams are circularly polarized as in our work, we obtain a modulation contrast of approximately 0.7 irrespective of the orientation angle. If we use a segmented azimuthal polarizer so that the ± 1st order beams can maintain spolarization while the 0 th order beam is circular polarized, we can obtain an increase in modulation contrast to about 0.9. If the 0 th order beam is circularly polarized while the ± 1st order beams are at a fixed linear polarization (s-pol only for the 0 o and 180 o orientations), we can see that the modulation contrast diminishes drastically for different orientation angles.

DMD and camera alignment
In tunable SIM experiments, the camera pixel coordinates should directly correspond to DMD pixels for adaptive illumination pattern generation based on the observed object dynamics. We carried out calibration experiments for pixel-by-pixel alignment between the DMD and camera using fluorescence signals measured using a thin layer of fluorescein solvent solution (46960, SIGMA Life Science). Since the DMD and camera had different aspect ratios (16:9, and 1:1, respectively), we first had to find the appropriate pixels of the DMD that would fit into the camera while satisfying the Nyquist limit. We turned on a pair of pixels on the DMD along the upper and lower boundaries and checked the resulting fluorescence images on the camera. By simply shifting these ON pixels, the DMD pixels corresponding to the camera FOV edges were verified. Since the fluorescence images measured on the camera are diffraction-limited, this process was performed iteratively several times with Gaussian fitting for higher accuracy. DMD patterns with a pair of ON-state pixels at four vertexes, and the acquired image are shown in Figs. S4 (a), and (c), respectively. Using the acquired DMD pixel coordinates and their respective imaged coordinates on the camera, we extracted the projection matrix connecting the two planes. Therefore, we could illuminate the excitation light to suitable regions in our FOV, via a projection matrix compensating both slightly tilted angle and magnification between DMD and camera.
Next, we evaluated the accuracy of the acquired projection matrix. We generated an illumination pattern containing 10 randomly selected ON-state macropixels (3 × 3 pixels). Using the calibrated projection matrix, the appropriate DMD pattern was obtained as shown in Fig. S4 (b). The fluorescence image obtained using this illumination pattern showed perfect correspondence as we aimed [ Fig. S4(d)]. We compared the coordinates of the ON-state macropixels on the DMD and the acquired image on the camera.
The extracted coordinates are shown in Table S1. The difference in coordinates between the illumination pattern and acquired image was less than one pixel which is under the diffraction-limited resolution of our system.

Image acquisition
To realize real-time imaging with accurate synchronization among different electrical devices (DMD, sCMOS camera, and laser), the TTL output signal from the DMD was employed as the master Fig. S5. Dynamic modulation of the illumination across the FOV. (a) Image acquisition sequence diagram for tunable SIM. The DMD and camera were synchronized using a TTL trigger signal from the DMD. The first row of pixels in the camera starts the exposure in synchrony with the DMD trigger signal. Using the output trigger signal from the camera indicating when all rows in the camera are exposed (Row 1 to n), the laser pulse sequence was also synchronized. For stroboscopic illumination, the DMD can be further modulated in the widefield ROIs (b) Observation of moving fluorescent beads under stroboscopic illumination (switched ON/OFF 12 times during a single acquisition) and continuous widefield illumination. Scalebar: 10 µm. clock. The DMD sends an output TTL signal simultaneously as a new pattern is displayed. The camera acquisition was initiated using this trigger signal. Although our sCMOS camera (Zyla 4.2, Andor) operated in rolling shutter mode, we realized semi-global shutter measurements by generating laser illumination pulses that illuminated the sample only when all pixels in the camera were exposed [ Fig. S5 (a)].
The DMD maximum framerate is 12 kHz when binary patterns are kept on DMD storage memory. However, because our aim was to use DMD patterns self-adapting to time-varying objects, we could not simply upload fixed illumination patterns onto storage. We instead utilized the real-time video update mode of the DMD which enabled 120 Hz refresh rate for 24-bit pattern updates via display cable connection.
As the DMD can be controlled to display arbitrary ON and OFF states on a mirror by mirror basis, the illumination for the widefield regions in tunable SIM can also be additionally controlled. For example, to identify fast moving objects where the exact shape and instantaneous speeds are important, we can employ stroboscopic illumination to the target regions of interest (ROIs). An example application is shown as red dashed line in Fig. S5 (b) where fast moving fluorescent beads were tracked in two ROIs simultaneously. In the region above, the DMD was flipped ON and OFF 12 times during a single acquisition in a stroboscopic fashion while the region below was illuminated continuously. We can clearly observe the benefits of strobe illumination and identify the shape of the fluorescent bead as well as the instantaneous displacements that otherwise resulted in a single blur for continuous illumination [ Fig.  S5 (b)].

Adaptive tunable SIM pattern generation
We carried out image segmentation to generate tunable SIM illumination patterns, based on the acquired images. After verifying that a continuously acquired sequential image pair (i-th and i+1-th image) have no spatial correlation for fast dynamic processes (flowing beads), we adopted two continuously acquired images for segmentation [ Fig. S6 (a), and (b)]. The two images were processed with Gaussian and Top-hat filters to reduce noise and background signals, then binarized as shown in Figs. S6 (c-h). We then performed an AND operation on the two binarized masks to remove temporally varying information between the two images We neglected the small segments as they did not correspond to a continuous cell boundary [ Fig. S6 (m)]. Using the calibrated projection matrix, the segmented image was projected onto DMD pixel coordinates [ Fig. S6 (n)]. The tunable SIM Fig. S6. Tunable SIM mask generation workflow. To identify regions that feature slow dynamics that SIM can follow, two sequential (a) i-th and (b) i+1-th images were used. In order to suppress noise and background signal, (c-d) Gaussian and (e-f) Tophat filters were applied to both images. (g-h) The two images were thresholded and binarized and an (i) AND operation was performed to remove fast moving objects. Morphological image processing operations, such as (j) dilation, (k) filling holes, and (l) labeling, were applied to continuously define the boundary of the cell and remove redundant segments. (m) The extracted tunable SIM regions in camera coordinates were projected onto the (n) DMD coordinates by a 2-D projective geometric transform. (o) Finally, the mask was combined with structured and homogenous illumination to generate tunable SIM patterns. Scalebar; 10 µm patterns were generated by multiplying binary structured patterns with the segmented mask, and turning on all other regions for continuous illumination [ Fig. S6 (o)].

Tunable SIM image reconstruction
Based on linearity of the Fourier transform, different ROIs can be separately reconstructed in tunable SIM using the original reconstruction algorithms developed for SIM. To demonstrate the feasibility of tunable SIM, we show simulations of independent image recovery for arbitrary shaped parts of the FOV [Fig. S7]. Using a reference image [ Fig. S7 (a)], a low pass filtered image was obtained using a numerically generated OTF [ Fig. S7 (b)]. Using conventional 3-D SIM illumination, five copies of shifted information content in the Fourier domain can be clearly seen [ Fig.  S7 (c)]. SIM reconstruction extends the resolution by a factor of 2 [ Fig. S7 (d)]. By selectively choosing an arbitrary shaped portion of the FOV (tunable SIM), we can directly apply the same principles of SIM [ Fig. S7 (e,g)]. This enables orthogonal image recovery of different parts of the FOV as required [ Fig. S7 (f,h)]. This approach also has the additional benefit that the orientation angle and phase of the illumination patterns can be chosen to be different for each part of the FOV. Fig. S8 shows a magnified view of the reconstructed regions demonstrating the same reconstructed image quality for conventional SIM and tunable SIM.
To test the tunable SIM reconstruction fidelity as the selected ROI is reduced, we imaged 100 nm fluorescent beads. Conventional 3-D SIM illumination was applied to the entire FOV, however, selected ROIs were used to reconstruct the images in the respective ROIs. Fig. S9(a) shows the SIM image reconstructed using the raw data from the entire FOV. A white square marks the part of the image that was compared for image reconstruction quality. Conventional widefield imaging results in diffraction limited resolution [ Fig. S9 (b)]. Figs. S9 (c-g) shows reconstruction results using different reconstruction ROI sizes. The reconstruction is found to be robust even for 16 x 16 pixel ROIs [ Fig. S9 (f)] which only corresponds to a 1.04 µm x 1.04 µm ROI. The reconstruction starts to show artifacts when the ROI is reduced to 8 x 8 pixels [ Fig. S9 (g)].
In this case, a single ROI corresponds to a region only 500 nm x 500 nm across. As the ROI is smaller than the image shown in Fig. S9 (g), 9 overlapping regions were used to reconstruct the image shown. We therefore anticipate that tunable SIM reconstruction using smaller ROIs will not pose a limitation for practical applications.

Resolution tests using SIM reference target
We evaluated the resolution enhancement using a nanoruler target (SIM 140B, GATTA-quant), which have 140 nm distant pairs of fluorescent nanobeads. Since this intra-pair distance of the GATTA-SIM nanoruler was below the diffraction limit (229 nm) of our system, the bead pairs were not resolved in conventional widefield or Wiener filtered images [ Fig. S10 (a) and (b)]. Using the (c-g) Tunable SIM reconstruction using ROIs of 128 x 128 pixels, 64 x 64 pixels, 32 x 32 pixels, 16 x 16 pixels, and 8 x 8 pixels respectively. The visualized area corresponds to 16 x 16 pixels of the raw data. Therefore, the reconstruction in (g) was carried out using 9 overlapping areas as the ROI is smaller than shown. Scalebar; 2 µm, and 0.5 µm in (a), and (b-g), respectively.   Fig. S5 (a-c). Scalebar; 1 µm.

Fluid flow analysis
Flowing fluorescent beads surrounding U87 cells were imaged to visualize the fluid flow in the microfluidic channel chamber. Structured illumination patterns were illuminated only to the cell specific regions, whereas the rest of the FOV was illuminated with a homogeneous plane wave. In order to quantitatively analyze the fast flow dynamics, moving beads should be correctly extracted from each frame. Different segmentation processes were applied to the dynamic and static regions ( d r , and s r , respectively

Tunable SIM videos
Tunable SIM experiments were performed under two different conditions, with or without flow induced shear stress on the U87 cells [Visualization 2]. Without fluid flow in the surrounding medium, inherent spontaneous actin movement was rather limited. In contrast, rapid oscillation and shear related bending of actin fine structure was clearly observed upon the application of shear stress especially near the peripheral regions of the cell directly in contact with the fluid.

Optical sectioning SIM
As tunable SIM can modulate the illumination arbitrarily for different ROIs, we performed the following experiment using a brain tissue slice to demonstrate the feasibility of obtaining optical sectioning using tunable SIM. The idea is demonstrated in Fig. S12. Fig. S12 (a-c) shows the three images acquired with sinusoidal illumination with varying phases. In contrast to super-resolution  r s r d SIM, optical sectioning SIM using three images obtains efficient depth sectioning with modest illumination pattern periods (overlap in and information collection is required to extend ). Therefore, we displayed binary patterns with longer periods on the DMD and also filtered the diffracted beams at smaller spatial frequencies as shown in Fig. S12