Effect of depth dependent spherical aberrations in 3 D structured illumination microscopy

We model the effect of depth dependent spherical aberration caused by a refractive index mismatch between the mounting and immersion mediums in a 3D structured illumination microscope (SIM). We first derive a forward model that takes into account the effect of the depth varying aberrations on both the illumination and the detection processes. From the model, we demonstrate that depth dependent spherical aberration leads to loss of signal only due to its effect on the detection response of the system, while its effect on illumination leads to phase shifts between orders that can be handled computationally in the reconstruction process. Further, by using the model, we provide guidelines for optical corrections of aberrations with different complexities, and explain how the proposed corrections simplify the forward model. Finally, we show that it is possible to correct both illumination and detection aberrations using a deformable mirror only on the detection path of the microscope. © 2012 Optical Society of America OCIS codes: (110.0110) Imaging systems; (110.0180) Microscopy; (110.6880) Threedimensional image acquisition; (180.2520) Fluorescence microscopy; (110.2990) Image formation theory; (110.1080) Active or adaptive optics. References and links 1. M. Gustafsson, A. Agard, and J. Sedat, “Doubling of lateral resolution of wide-field fluorescence microscopy using structured illumination,” Proc. SPIE 3919, 14–150 (2001). 2. P. Kner, B. Chhun, E. Griffis, L. Winoto, and M. Gustafsson, “Super-resolution video microscopy of live cells by structured illumination,” Nat. Meth. 6, 339–342 (2009). 3. M. Gustafsson, L. Shao, P. Carlton, C. Wang, I. Golubovskaya, W. Cande, D. Agard, and J. Sedat, “Threedimensional resolution doubling in wide-field fluorescence microscopy by structured illumination,” Biophys. J. 94, 4957–4970 (2008). 4. M. Gustafsson, “Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution,” Proc. Natl. Acad. Sci. U.S.A. 102, 13081–13086 (2005). 5. L. Schermelleh, P. Carlton, S. Haase, L. Shao, L. Winoto, P. Kner, B. Burke, M. Cardoso, D. Agard, M. Gustafsson, H. Leonhardt, and J. Sedat, “Subdiffraction multicolor imaging of the nuclear periphery with 3D structured illumination microscopy,” Science 320, 1332–1336 (2008). 6. M. Trammell, N. Mahoney, D. Agard, and R. Vale, “Mob4 plays a role in spindle focusing in Drosophila S2 cells,” J. Cell Sci. 121, 1284–1292 (2008). 7. C. Wang, P. Carlton, I. Golubovskaya, and W. Cande, “Interlock formation and coiling of meiotic chromosome axes during synapsis,” Genetics 183, 905–915 (2009). 8. J. Fitzgibbon, K. Bell, E. King, and K. Oparka, “Super-resolution imaging of Plasmodesmata using threedimensional structured illumination microscopy,” Plant Phys. 153, 1453 –1463 (2010). #153649 $15.00 USD Received 1 Sep 2011; revised 30 Nov 2011; accepted 5 Dec 2011; published 6 Mar 2012 (C) 2012 OSA 12 March 2012 / Vol. 20, No. 6 / OPTICS EXPRESS 6527 9. S. Gibson and F. Lanni, “Experimental test of an analytical model of aberration in an oil-immersion objective lens used in three-dimensional light microscopy,” J. Opt. Soc. Am. A 8, 1601–1613 (1991). 10. B. Hanser, M. Gustafsson, D. Agard, and J. Sedat, “Phase-retrieved pupil functions in wide-field fluorescent microscopy,” J. Microsc. 216, 32–48 (2004). 11. Z. Kam, P. Kner, D. Agard, and J. Sedat, ”Modelling the application of adaptive optics to wide-field microscope live imaging,” J. Microsc. 226, 33–42 (2007). 12. P. Kner, J. Sedat, and D. Agard, “Applying adaptive optics to three-dimensional wide-field microscopy,” Proc. SPIE 6888, 688–809 (2008). 13. C. Preza and J. Conchello, “Image estimation account for point-spread function depth variation in threedimensional fluorescence microscopy,” Proc. SPIE 4964, 1–8 (2003). 14. C. Preza and J. Conchello, “Depth-variant maximum likelihood restoration for three-dimensional fluorescence microscopy,” J. Opt. Soc. Am. A 21, 1593–1601 (2004). 15. M. Arigovindan, J. Shaevitz, J. McGowan, J. Sedat, D. Agard, “A parallel product-convolution approach for representing the depth varying point spread functions in 3D widefield microscopy based on principal component analysis,” Opt Express 18, 6461–6476 (2010). 16. D. Débarre, E. Botcherby, T. Watanabe, S. Srinivas, M. Booth, and T. Wilson, “Image-based adaptive optics for two-photon microscopy,” Opt. Lett. 34, 2495–2497 (2009). 17. J. W. Goodman, Introduction to Fourier Optics, 3rd ed. (Roberts and Company Publishers, 2004). 18. S. Wiersma, P. Torok, T. Visser, and P. Varga, “Comparison of different theories for focusing through a plane interface,” J. Opt. Soc. Am. B 14, 1482–1490 (1997).


Introduction
Structured illumination microscopy (SIM) is the widefield super-resolution technique that results in a resolution that is twice that of classical optical resolution by combining only a small number of raw images [1][2][3].SIM involves illuminating the sample with a diffraction-limited sinusoidal pattern and acquiring images with different phase shifts and orientations of the illumination pattern.The acquired raw images are transformed in such a way that they represent the output of several optical transfer functions (OTFs) having frequency support beyond the classical diffraction limit; these extended OTFs are simply the shifted versions of the original diffraction limited OTF covering an isotropically extended volume in the Fourier space.Typically the transformed data are put together via Wiener filtering to obtain a properly-weighted super-resolved image.The nonlinear extension of SIM known as saturated SIM achieves a resolution that is more than twice the classical resolution limit by using nonlinear response of the fluorophores to laser irradiation [4].Example applications that led to new biological findings by the use of 3D SIM include study of nuclear envelope [5], spindle structure [6], chromosome structure during cell division [7], and plasmodesmata [8], and so on.
One of the major factors leading to the loss of resolution in any widefield-based technique is the depth dependent spherical aberration caused by mismatch between the mean refractive index of imaging specimen and the refractive index of the immersion medium.As with any other widefield system, 3D SIM suffers loss of resolution due to spherical aberration while imaging thick samples.In conventional widefield microscopy, the effect of depth dependent spherical aberration has been well studied [9,10] and optical [11,12] and computational [13][14][15] methods to correct these aberrations have been proposed.In SIM, the effects of depth dependent spherical aberrations have so far only been considered for systems that use 2D illumination patterns for z-sectioning [16], but never for the systems that achieve three-dimensional superresolution.
To treat the effect of depth dependent spherical aberration on three-dimensional SIM systems, we derive a compact model representing the object to image formation that accurately accounts for the effect of spherical aberration on both illumination and detection.From the model, we demonstrate that the aberration has different effects on the illumination and detection due to the fact that the detection works on the incoherent fluorescence signal, whereas the illumination pattern is generated using the mutual coherence of the illuminating wavefronts.Specifically, we show that the aberration effect on detection causes to a depth dependent attenuation leading to deviation of the actual transfer function from the standard convolution model.On the other hand, aberration of the illumination pattern causes only a deviation from the standard imaging model with no increased attenuation at high frequencies.Using the new 3D model, we provide guidelines for optical corrections of aberrations with different complexities, and explain how the proposed corrections simplify the forward model.In particular, we show that it is possible to correct the aberration for both illumination and detection using a deformable mirror only on the detection path of the microscope.

Derivation of the depth-variant model
In a widefield imaging system, there are two quantities representing axial distances: (i) the distance of the focal plane of objective lens from the coverslip, which is the user-controlled sectioning variable, z, and (ii) the distance of a given plane in the imaging sample from the coverslip, which is the object depth variable z .Assuming a depth independent imaging response is equivalent to supposing that the transfer function depends only on the difference between these quantities; this leads to a 3D representation of the imaging transfer function.In reality, the transfer function depends on both the axial variables, and the term "depth variant response" signifies the dependence of the imaging response on the depth variable z .
In the following, we will describe a schematic of a SIM system emphasizing the dependence of the transfer function on the above-mentioned axial variables.We will then describe the 4D amplitude transfer function that includes z and z as its independent variables which is responsible for making the SIM depth-variant.Next, from the amplitude transfer function, we formulate the depth dependent imaging transfer function for detection.Next we derive the expression for the illumination intensity pattern in terms of z and z and finally derive the overall imaging model of a SIM system.In this paper, only the forward model that represents the raw images in terms of the imaging specimen is developed.We do not consider the reconstruction problem.

Description of the imaging set-up
Figure 1(a) depicts the imaging set-up of a 3D SIM.The light exiting the fiber is collimated by the lens CL and directed to a linear transmission phase grating located at the plane P IM , which diffracts the beam into several orders.A beam block in an intermediate pupil plane (not shown) discards all diffraction orders except orders 0 and ±1.The three beams are refocused by the tube lens T L so that each forms an image of the fiber end-face in the back focal plane (P BF ) of the objective lens.The beams produced as diffraction orders +1 and −1 were focused near opposing edges of the P BF aperture, and order 0 at its center.The line connecting the fiber images at the P BF will be orthogonal to the grating lines.The objective lens recollimates the beams and makes them intersect with each other in the objective lens's focal plane, where they interfere to form an illumination intensity pattern with both axial and lateral structure.The overall intensity variation is essentially two-dimensional with the principal directions of variation are the z-axis and the direction orthogonal to the pattern direction of the diffraction grating.In particular, the 2D intensity pattern at the focal plane of the objective lens (P F ) is a demagnified and lowpass-filtered image of the grating.
The fluorescent object to be studied is mounted under a glass coverslip and placed below the objective lens.The space between the objective lens and the coverslip is filled with the immersion medium (oil) whose refractive index matches with that of glass.Hence the coverslip, even though its thickness is non-negligible, can be represented by a single plane, P I .The refractive index of the object differs from that of the coverslip and oil, and is typically assumed to be uniform.Hence, the entire medium below the objective lens is composed of two regions of differing refractive indices separated at plane P I .Fluorescence light emitted by the sample as a response to the above-mentioned illumination pattern is gathered by the same objective lens and deflected by a dichroic mirror to the tube lens, T L, which focusses the light onto the image sensor located at the image plane P IM .gives an alternative schematic of the SIM system that will facilitate the derivation of the depth dependent model in terms of the depth variable z and z defined at the beginning of this section.A 3D image acquisition involves a sequence of 2D acquisitions with different values of z such that P F samples the entire object depth with an appropriate step-size.For each value of z, the acquired 2D image includes the sum of contributions from all the planes in the object corresponding to different values of z .It is important to note that the image plane P IM and the grating plane P IM are optically identical with the respect to the sample space, i.e., the transfer functions from these planes to the sample space are identical.It should also be emphasized that this function is mainly determined by transfer function from P BF to the sample space, since the light propagation from the planes P IM and P IM to P BF is a simple Fourier transformation, and aberrations therein are negligible.This transfer function is a 4D function, which we will denote by T , and is composed of a set of 2D functions representing the transformation from a given sample plane P S to P BF for all relevant values of z and z .In the section 2.2, we specify the above mentioned amplitude transfer function, T .From this T , in the section 2.3, we express the 4D intensity transfer function, h, that relates the measured image in terms of the emitted fluorescence intensity.The transfer function h has been well characterized [9,10,[13][14][15].We however re-derive in the section 2.3 in terms of present notations to facilitate the readability.Then in the section 2.4, we determine the effect of T on the illumination intensity.Finally, we derive the complete SIM transfer function.

The transfer function from a sample plane to the back-focal plane
In order express this transfer function from P S to P BF in Fourier space (see Fig. 1), we first consider the transfer function for z = 0 and z = 0.For an ideal objective lens (objective lens with perfect immersion correction), this is an aperture function in 2D Fourier space whose value is identity inside a circular region and zero outside.In practice however, this function will contain phase variations inside the circular region to represent the phase aberrations in the objective lens.Denoting the lateral Fourier Frequencies by X and Y (note the capitalization), let P(X,Y ) = A(X,Y ) exp( j2πQ(X,Y )) be this transfer function, where Q is the function representing the aberrations, and A(X,Y ) is the aperture function.The function A(X,Y ) is given by where NA is the numerical aperture of the objective lens.To express the transfer function from P S to P BF for nonzero values z and z , first note that, a positive value of z causes a reduction in the optical path length through the immersion medium, whereas, a positive value of z causes an increase in the optical path length through the mounting medium.Hence, the transfer function in the Fourier domain can be written as (1) where N(., ., n) represents the phase dispersion function representing the transmission of light for unit distance through a medium of refractive index n.Here, n ob j and n imm are the refractive indices of the object and the immersion medium respectively.The function N is given by [17] where λ is the wavelength.T (X,Y, z, z ) represents the wavefront at P BF when a point source is placed at P S .By reciprocity, it is also the Fourier transform of the wavefront at P S if the image plane P IM is illuminated with a point source.Equivalently, it is the Fourier transform of the wavefront at P S if the P BF is illuminated by a plane wave.T (X,Y, z, z ) forms the basis for the derivation of the imaging model with or without structured illumination.Note that T is in a mixed representation, where lateral variables are in the Fourier space, and the axial variables are in real space.

The detection transfer function
Let S a (x, y, z ) be the emitted fluorescence amplitude distribution and let S a (X,Y, z ) be its section-wise Fourier transform.For a given value of z, the wavefront amplitude distribution at P BF is given by Let R(x, y, z) be the corresponding intensity image at P IM ; this is the squared modulus of the inverse Fourier transform of W BF (X,Y, z).By representing the section-wise inverse Fourier transform in x − y plane with F −1 xy , R(x, y, z) can be represented in terms of the object intensity as with ⊕ xy representing section-wise 2D convolutions.Equation ( 2) is the depth-variant convolution model, and g(x, y, z, z ) is the depth variant PSF (DV-PSF).Under this notion, when the system is assumed to be depth independent, for example when n imm = n ob j , the emission response is a function of only the difference z − z .In other words, the depth invariance corresponds to following relation: and Eq. ( 2) becomes a simple 3D convolution given by R(x, y, z) = g 0 (x, y, z) ⊕ S(x, y, z), where g 0 (x, y, z) is the standard depth independent PSF given by g 0 (x, y, z) = g(x, y, z, 0).
In order to derive the overall depth dependent response in a form that is comparable with the conventional depth invariant model, we first transform Eq. ( 2) such that it has highest resemblance to a standard convolution.To this end, we apply a z dependent shift on z, such that the value of latter satisfies the following condition: z = 0 represents the position of the focal plane P F such that the 2D image of the object plane at z reaches the image plane P I with minimum distortion.This shift is known as the focus shift [18].It is known that this shift is proportional to z by a factor that depends on refractive indices involved.Let the transformed DV-PSF be given by h(x, y, z, z where a is the focus shift (designated as the "detection focus shift").The forward model with new representation becomes The above equation is in a form suitable for incorporating the illumination pattern in order to derive the overall imaging model.In the following sections, we will use the above expression to derive the final imaging model that will combine the effect of aberration of illumination and detection.We denote the operation in the above equation by R(x, y, z) = DVC(S(x, y, z), h(x, y, z, z )), where DVC stands for depth variant convolution.

The effect of depth dependent spherical aberration on the illumination pattern
With reference to Fig. 1(b), let d k be the vector in lateral plane representing the direction orthogonal to grating lines.The complex amplitudes of the selected plane waves exiting the grating (P IM ) can be represented by , where x = [x, y, z] represents 3D spatial position.Here, {v 0 , v k + , v k − } are the wave vectors.The vector v 0 is parallel to the z-axis and is of the form v 0 = [0 0 1/λ ] T , where λ is the wavelength of illumination.The remaining vectors will be of the form The angle of the sub-vector [X k Y k ] T coincides with that of the vector d k .The phases {φ k0 , φ k + , φ k − } are the instrumentation-dependent (unknown) phase shifts that have to be estimated from the data, and φ s is the phase shift introduced for each 3D stack acquisition by controlling the position of the grating.Finally, the vectors {u 0k , u k + , u k − } represent the polarization of the plane waves.The tube lens T L performs a Fourier transform of the image of the grating at P IM and projects onto the back-focal plane P BF of the objective lens.Assuming that the fiber end-face can be approximated by delta functions, and ignoring the aperture effect on the plane waves, the illumination wavefront at the back-focal plane P BF can be expressed as The objective lens performs an inverse transform of the wavefront at P BF and projects into the sample space with a 3D intensity pattern that is dependent on both z and z.As mentioned before, the principal directions of intensity variation are the z-axis and the direction orthogonal to the pattern direction of the diffraction grating.For each direction k, five 3D stacks are acquired with different values of the phase φ s , and this process is repeated for directions k = 0, 1, 2 such that the directional vectors {(X k ,Y k ), k = 0, 1, 2} are 120 o apart in the x − y plane.Let E k (X,Y, z, z ) represent the set of 2D Fourier transform of the illumination of the object plane located at z with focal plane positioned at z.This function is determined by the transfer function T (X,Y, z, z ) and is given by Finding the effect of depth dependent spherical aberration is equivalent to finding the illumination intensity as function of four variables, x, y, z, and z ; in contrast, as we will show, ignoring the depth dependent aberration is equivalent to assuming the illumination intensity to be a function of three variables only, i.e., a function of x, y, and z − z.Let L (x, y, z, z ) be the illumination intensity function, and let L(X,Y, z, z ) be its partial Fourier transform taken along x and y directions.It is given by where Ēk is the complex conjugate of E k , and ⊕ X,Y represents the convolution only in the variables (X,Y ).Here E k (X,Y, z, z ) = T (X,Y, z, z )E BF,k (X,Y ) with T (X,Y, z, z ) and E BF,k (X,Y ) being specified by Eqs. ( 1) and ( 5).
In the appendix A, we show that for a sinusoidal grating (when only the 0th and ±th orders are projected on the back focal plane), L (x, y, z, z ) can be expressed as where 8) The factor a f can be considered as the illumination focus shift.In the absence of depth dependent aberration, for example, when n imm = n ob j , a f becomes unity, and the axial component of the illumination pattern will depend only on the difference z − z as assumed in the standard methods of processing SIM data.Now we consider the effect of the finite size of the fiber end-face.First we note that light from the laser source typically goes through a phase scrambler before passing through diffraction grating, and hence each pair of points at the fiber end-face are mutually incoherent.However, each photon exiting the fiber is split by the grating into the three orders, satisfying the need for mutual coherence required to create the interference pattern as given by Eq. (7).Further,for all such triplets, the relative distances and the phase differences among three points will be the same.Consequently, the illumination patterns generated by each point of the fiber end-phase will all be identical, and hence the present analysis equivalently accommodates the finite size of the fiber.

The complete imaging model
Under structured illumination, the emitted fluorescence intensity is the product of the object dye structure and the illumination intensity.As discussed above, due to the depth dependent spherical aberration, the illumination intensity is z-dependent.The fluorescence intensity is given by where S(x, y, z ) is the actual fluorescence dye structure.To obtain the 3D imaging model, F I (x, y, z, z ) has to be substituted in place of S(x, y, z ) in Eq. ( 4).Before expressing the model, we need to define the following: With this definition, we show in the appendix B that the imaging model can be expressed as R(x, y, z) = DVC S(x, y, z), h(x, y, z, z ) , +DVC S k (x, y, z) cos(P z (z)), h(x, y, z, z ) cos(2π f z z − ϕ kz ) , −DVC S k (x, y, z) sin(P z (z)), h(x, y, z, z ) sin(2π f z z − ϕ kz ) , +DVC S k2 (x, y, z), h(x, y, z, z ) , (10) where and DVC (•, •) represent the depth dependent detection model defined in Eq. ( 4). Figure 2 gives the flow chart for the complete depth variant SIM model given in Eq. ( 10).The first term in Eq. ( 10) is represented by the branch A in the flow-chart and it expresses the widefield imaging operation with depth dependent aberration as developed in [15].The fourth term is represented by the branch B and it is the widefield detection on the laterally modulated signal S k2 (x, y, z) = S(x, y, z) cos(2P(x, y)) with P(x, y) = 2π(X k x + Y k y) + ϕ k − φ s .Note that the modulation frequency here is twice that of the wavefront at the back-focal plane P BF .This term is responsible for the doubling of the lateral resolution.The second and third terms correspond to the axial resolution extension.They are represented by the branches C.1 and C.2 and express the depth variant convolutions by functions h(x, y, z, z ) cos(2π f z z − ϕ kz ) and h(x, y, z, z ) sin(2π f z z − ϕ kz ), where multiplications by cos(2π f z z − ϕ kz ) and sin(2π f z z − ϕ kz ) provide the axial resolution extension.Inputs for these two terms are derived from S k (x, y, z) = S(x, y, z) cos(P(x, y)) by multiplying with axial functions cos(P z (z)) and sin(P z (z)) that represent the effect of aberration on the illumination, where P z (z) = 2π f z (a − a f )z.Note that the function S k (x, y, z) is obtained by laterally modulating the original signal with the frequency of the illumination amplitude at the back-focal plane.Further, recall that a is the detection focus shift, and a f is the illumination focus shift given by a f = f z / f z with the frequencies f z and f z being given by Eqs. ( 8) and (9).
When the refractive indices match, i.e., when n ob j = n imm , the DV-PSF h(x, y, z, z ) in Eq. ( 10) becomes independent of z and hence the depth variant convolution DVC becomes a standard convolution.Further, the detection focus shift becomes a = 1, and, with reference to Eqs. ( 8) and ( 9), the illumination focus shift also becomes a f = 1.As a result, P z (z) becomes zero and hence the branch C.2 disappears; further, the multiplicative factor in the branch C.1 becomes equal to 1.The equivalent imaging equation is given by where h(x, y, z) = h(x, y, z, 0) and h c (x, y, z) = h(x, y, z) cos(2π f z z − ϕ kz ).This is indeed the model assumed in original SIM paper [3].
We will now qualitatively compare the effect of depth dependent aberration on the detection and illumination.To this end, we compare Eq. ( 10) with the depth invariant imaging model given in Eq. (11).The main difference is that the detection response is taken care by a simple convolution by h(x, y, z) in the depth-invariant model, and by the depth-variant convolution by h(x, y, z, z ) in the complete model.It is well known that this difference is more than a difference in the complexity.For z > 10 μm, the Fourier magnitude of the 3D transfer function falls-off steeper meaning that signal strength of the high frequency components is much lower than the case without spherical aberration [9].Hence, spherical aberration will lead to the loss of signal even if the exact model is used in the reconstruction.Next, to analyze the effect of aberration on illumination alone, we first replace DVC by the convolution by h(x, y, z).We get where h s (x, y, z) = h(x, y, z) sin(2π f z z − ϕ kz ).Comparing Eq. ( 12) with Eq. ( 11), we observe that the differing terms are given by where M 1 (•) is the term that acts on S k (x, y, z) in the model of Eq. ( 11), and M 2 (•) is the term that acts on the same signal in the model of Eq. ( 12).It is straightforward to verify that impulse responses of M 1 (•) and M 2 (•) at any depth have identical frequency magnitude (shown in Appendix C).The only difference is that M 2 has an higher computational complexity.This shows that spherical aberration on illumination causes only an increase in the complexity of the imaging model without any increase in the high frequency attenuation.Of course, illumination aberration effects will lead to a distorted reconstruction if standard depth-invariant model is incorrectly assumed.

Imaging with adaptive optics
In this section, we analyze the effect of adaptive optics schemes proposed in the papers [11,12] on the imaging model, and show how the models become simplified under these schemes.An electronically controlled deformable mirror is placed in a complementary plane that is optically identical to P BF , and mirror shape is controlled in a z-dependent way such that the spherical aberration is compensated.There are two type of schemes of compensating for depth dependent spherical aberration as explained below.
In the first scheme, the physical position of P F is kept fixed at P I (see Fig. 1), and z-sectioning is performed only by controlling the shape of the mirror.For each value of z, mirror shape is controlled to match the function DM 1 (X,Y, z) = −zn ob j N(X,Y, z, n ob j ) [11].The resultant transfer function is the product of T (X,Y, 0, z ) and exp( j2πDM 1 (X,Y, z)), which is given by In the second scheme, P F is physically positioned at distance z from the P I , and the z-dependent function for the deformable mirror is set to [11] We generalize the above expression with an addition of a new parameter as follows: where ã is a design parameter.The resultant transfer function will be now the product of the original transfer function T (X,Y, z, z ) and exp( j2πDM 2 (X,Y, z)), which is given by Here, the factor ã is a free parameter whose significance will be explained in the next paragraph.Note that the transfer function in Eq. ( 15) can be considered as a special case of the one in Eq. ( 16), and hence it is sufficient to analyze the effect of the latter alone on the imaging model.Our goal is now to find the resulting modifications in the model of Eq. ( 10) when the above adaptive optics scheme is used.To this end, we need to find the effect of replacing the amplitude transfer function T (X,Y, z, z ) with the new transfer function T ao (X,Y, z, z ) in the derivation of Eq. (10).
To derive the effect of adaptive optics on the detection, we first write the detection equation based on the new coherent transfer function T ao (X,Y, z, z ): xy T ao (X,Y, z, z ) 2 S(x, y, z )dz By using Eq. ( 16), the above equation can be written as where To find the implication of this modified detection equation on the imaging model, we compare this detection equation with the one that was used in the derivation of original imaging model, which is Eq. ( 4).This comparison implies that incorporating the effect of adaptive optics amounts to replacing DVC with convolution by h ao , and replacing the detection focus shift a by the parameter ã.The parameter hence allows the user to choose the detection focus shift.The imaging model in Eq. ( 10) now becomes R(x, y, z) = 3h ao (x, y, z) ⊕ S(x, y, z) + 4m 0k h ao,c (x, y, z) ⊕ [cos(P z (z))S k (x, y, z)] −4m 0k h ao,s (x, y, z) ⊕ [sin(P z (z))S k (x, y, z)] + 2m 1k h ao (x, y, z) ⊕ S k2 (x, y, z) (17) where P z (z) = 2π f z ( ã − a f )z, h ao,c (x, y, z) = cos(2π f z z − ϕ kz )h ao (x, y, z), and h ao,s (x, y, z) = sin(2π f z z − ϕ kz )h ao (x, y, z).Contrary to the general belief, it is possible to make the imaging model entirely free of depth dependent aberration by applying adaptive optics only on the detection path.To show this, we consider the model resulting from applying adaptive optics on the detection, which is given in Eq. (17).If we set the user-defined focus shift ã to be equal to the focus shift of illumination a f , then P z (z) becomes zero, and hence Eq. ( 17) becomes which is clearly depth invariant.This shows that it is possible to compensate the effect of depth dependent aberration both on illumination and detection by applying the adaptive optics only on the detection path of the microscope.Interestingly, it is possible to correct the aberration for illumination alone by using a single adaptive element.To this end, we construct a complementary back-focal plane for illumination and apply the following form of z-dependent function as an adaptive mirror: Note that this is equivalent to applying a single adaptive element that is approximately a point at the center of the back-focal plane.Applying this function on the back-focal plane is equivalent to applying a z-dependent phase factor exp( j2π fz z) to the zeroth order illuminating plane wave, where fz is the design parameter.The resultant coherent transfer function is given by where the subscript ill indicates that this transfer function is applied only on the illumination.
By following the same step as in the derivation of Eq. ( 7), it can be shown that the resultant illumination pattern can by obtained by replacing f z by f z + fz in Eq. ( 7).As a result, P z (z) in Eq. (10), which represents aberration in the illumination pattern becomes This means that illumination aberration can be eliminated by setting fz = f z /a − f z .Consequently, the imaging model becomes R(x, y, z) = DVC S(x, y, z), h(x, y, z, z ) , +DVC S k (x, y, z), h(x, y, z, z ) cos(2π( f z /a)z − ϕ kz ) , +DVC S k2 (x, y, z), h(x, y, z, z ) , which reveals that the aberration in illumination has been eliminated.If the images obtained by using this type of aberration correction are processed assuming standard depth-invariant model, resolution loss at higher depths will be incurred only by the aberration effect on the detection; there will be no other forms of distortion that are normally encountered in the case of systems without any aberration correction.In the context of handling the finite size of the optical fiber, it is only required to add a phase equal to fz z for all points in the center-image of the fiber endface at the back-focal plane (see the last paragraph of section 2.4).Hence the adaptive element should be flat with its size equal to or greater than the size of the fiber image at the back focal plane.

Conclusions
We have developed an imaging model for structured illumination microscopes that takes into account the full effect of depth dependent spherical aberration caused by refractive index mismatch between the immersion and mounting mediums.The model explicitly reveals the effect of aberration on the detection response and the illumination pattern distinctly, and will allow their effect in terms of signal loss to be compared by implementing computational correction independently for each effect.We demonstrated that depth dependent spherical aberration leads to loss of signal only due to its effect on the detection response of the system, and its effect on the illumination only amounts to an increased computational complexity in the forward model.Thus signal loss due to illumination aberration only occurs if an incorrect model is used in the reconstruction process.This contrasts with detection aberrations, which lead to loss of signal even if the exact depth variant model is used during data processing. where The inverse Fourier transform of Eq. (20) becomes L (x, y, z, z ) = 3 + 2m 0k cos(2π(X k x +Y k y + f z z − f z z ) − ψ a + ψ b ) +2m 0k cos(2π(X k x +Y k y − f z z + f z z ) + ψ c − ψ b ) +2m 1k cos(2π(2X k x + 2Y k y) − ψ a + ψ c ), The above equation can further be reformatted as follows: L (x, y, z, z ) = 3 + 4m 0k cos(2π(X k x +Y k y) + ϕ k − φ s ) cos(2π( f z z − f z z) + ϕ kz ) +2m 1k cos(2π where

Figure 1 (
Figure1(b) gives an alternative schematic of the SIM system that will facilitate the derivation of the depth dependent model in terms of the depth variable z and z defined at the beginning of this section.A 3D image acquisition involves a sequence of 2D acquisitions with different values of z such that P F samples the entire object depth with an appropriate step-size.For each value of z, the acquired 2D image includes the sum of contributions from all the planes in the object corresponding to different values of z .It is important to note that the image plane P IM and the grating plane P IM are optically identical with the respect to the sample space, i.e., the transfer functions from these planes to the sample space are identical.It should also be emphasized that this function is mainly determined by transfer function from P BF to the sample space, since the light propagation from the planes P IM and P IM to P BF is a simple Fourier transformation, and aberrations therein are negligible.This transfer function is a 4D function, which we will denote by T , and is composed of a set of 2D functions representing the transformation from a given sample plane P S to P BF for all relevant values of z and z .In the section 2.2, we specify the above mentioned amplitude transfer function, T .From this T , in the section 2.3, we express the 4D intensity transfer function, h, that relates the measured image in terms of the emitted fluorescence intensity.The transfer function h has been well characterized[9,10,[13][14][15].We however re-derive in the section 2.3 in terms of present notations to facilitate the readability.Then in the section 2.4, we determine the effect of T on the illumination intensity.Finally, we derive the complete SIM transfer function.

Fig. 2 .
Fig. 2. Flow diagram of SIM imaging model with depth dependent spherical aberration.