Correcting for physical distortions in visual stimuli improves reproducibility in zebrafish neuroscience

Optical refraction causes light to bend at interfaces between optical media. This phenomenon can significantly distort visual stimuli presented to aquatic animals in water, yet refraction has often been ignored in the design and interpretation of visual neuroscience experiments. Here we provide a computational tool that transforms between projected and received stimuli in order to detect and control these distortions. The tool considers the most commonly encountered interface geometry, and we show that this and other common configurations produce stereotyped distortions. By correcting these distortions, we reduced discrepancies in the literature concerning stimuli that evoke escape behavior, and we expect this tool will help reconcile other confusing aspects of the literature. This tool also aids experimental design, and we illustrate the dangers that uncorrected stimuli pose to receptive field mapping experiments.

Breakthrough technologies for monitoring and manipulating single-neuron activity provide unprecedented opportunities for whole-brain neuroscience in larval zebrafish (Ahrens et al., 2012;Ahrens et al., 2013;Portugues et al., 2014;Prevedel et al., 2014;Vladimirov et al., 2014;Dunn et al., 2016b;Naumann et al., 2016;Kim et al., 2017;Vladimirov et al., 2018). Understanding the neural mechanisms of visually guided behavior also requires precise stimulus control, but little prior research has accounted for physical distortions that result from refraction and reflection at an air-water interface that usually separates the projected stimulus from the fish (Sajovic and Levinthal, 1983;Stowers et al., 2017;Zhang and Arrenberg, 2019). In a typical zebrafish visual neuroscience experiment, an animal in water gazes at stimuli on a screen separated from the water by a small (~500 mm) region of air (Figure 1a, top). When light traveling from the screen reaches the air-water interface, it is refracted according to Snell's law (Hecht, 2016;Figure 1b, bottom). At flat interfaces, a common configuration used in the literature (Ahrens et al., 2012;Vladimirov et al., 2014;Dunn et al., 2016a), this refraction reduces incident light angles, thereby translating and distorting the images that reach the fish (black vs. brown arrows in Figure 1a, bottom). By solving Snell's equations for this arena configuration (Appendix 1), we determined the apparent position of a point on the screen, , as a function of its true position, 0 ( Figure 1b). Snell's law implies that distant stimuli appear to the fish at the asymptotic value of 0 ð Þ (~48.6˚). This implies that the entire horizon is compressed into a 97.2˚"Snell window" whose size does not depend on the distances between the fish and the interface (d w ) or the screen and the interface (d a ), but the distance ratio d a /d w determines the abruptness of the 0 ð Þ transformation. We also calculated the total light transmittance according to the Fresnel equations (Figure 1b, right). These two effects have a profound impact on visual stimuli (Figure 1c). The plastic dish that contains the water has little impact (Appendix 1). Physical distortions thus have the potential to affect fundamental conclusions drawn from studies of visual processing and visuomotor transformations.
The quantitative merits of correcting for refraction are apparent when comparing two recent studies of visually evoked escape behavior in larval zebrafish. Although Temizer et al. (2015) and Dunn et al., 2016a both found that a critical size of looming stimuli triggered escape behavior, they reported surprisingly different values for the critical angular size (21.7˚±4.9˚and 72.0˚±2.5˚, respectively, mean ±95% CI). This naively implies that the critical stimulus of Dunn et al. occupied 9 times the solid angle of Temizer et al. (1.02 [+0.14,] steradians and 0.11 [+0.06,-0.04] steradians, respectively, mean [95% CI]) (Materials and methods). This large size discrepancy initially raises doubt to the notion that a stimulus size threshold triggers the escape (Hatsopoulos et al., 1995;Gabbiani et al., 1999;Fotowat and Gabbiani, 2011). However, a major difference in experimental design is that Temizer et al. showed stimuli from the front through a curved air-water interface, and Dunn et al. showed stimuli from below through a flat air-water interface (Figure 2a Figure 1. Snell's law describes visual stimulus distortions that occur via air-water interfaces encountered in a typical experiment. (a) Top, In a typical zebrafish neuroscience experiment, an image is presented via projection onto a screen underneath an animal in a water-filled plastic dish. Middle, A small layer of air separates the screen from the dish and water. Bottom box, This configuration causes the image received at the eye (brown arrow) to be distorted and translated relative to the projected image (black arrow). We can describe this transformation as a relationship between the true position of a projected point ( 0 ) and its apparent position (), depending on the ratio between the distance from the air-water interface to the screen (d a ) and the distance from the eye to the air-water interface (d w ). To solve the transformation, we use Snell's law (illustrated in inset and panel b), which relates the angle at which a light ray leaves the air-water interface ( w ) to the angle at which it hits the interface ( a ), depending on the refractive indices of the media (air, n a ¼ 1; water, n w ¼ 1:333). Note that the effects of the plastic dish are typically minor (Appendix 1). (b) Top left, the apparent position of a point () as a function of its true position ( 0 ), and its inverse (inset), for d a =d w ¼ 10 (pink) and d a =d w ¼ 0:1 (blue). Top right, fraction of light transmitted into the water as a function of 0 for the same two values of d a =d w . Bottom box, Using Snell's law, we derived 0 ð Þ (top left inset), whose inverse we take numerically to arrive at 0 ð Þ (top left). (c) Simulated distortion of a standard sinusoidal grating. Yellow circle denotes the extent of the Snell window (~97.2˚visual angle). The virtual screen is modeled as a 4 Â 4 cm square with 250 pixels/cm resolution, and we fixed the total distance between the fish and the virtual screen, d a þ d w , to be 1 cm. Note that only a fraction of the screen is apparent when d a =d w is small (bottom left), but a distorted view of the full screen appears within the Snell window when d a =d w becomes large (bottom right). Contrast axes are matched across panels and saturate to de-emphasize the ring of light at the Snell window, whose magnitude would be attenuated by unmodeled optics in the fish eye (Materials and methods). configurations were used to probe the neural circuitry processing looming stimuli that expand over time. In one, fish were embedded off-center in a curved plastic dish and a screen presented stimuli in front of the animal through the curved interface of the dish (Temizer et al., 2015). In the other, fish were embedded (or swam freely) in a similar dish, but stimuli were presented on a screen below the dish (as in Figure 1a The brown circle denotes the extent of the Snell window. As RFs are mapped directly to screen pixels, the axes are nonlinear in terms of angle relative to the fish (top left corner). Each blue "x" denotes the peak position of the RF displayed in each plot. The dashed blue border denotes the half-maximum value of each RF, and the size of the RF is the solid angle within one of these borders.
The online version of this article includes the following video for figure 2: Figure 2-video 1. Optical refraction distorts the appearance of looming visual stimuli. https://elifesciences.org/articles/53684#fig2video1 markedly reduced this discrepancy in the literature, shrinking a 9-fold size difference down to 3-fold (Figure 2c, blue). The small remaining difference could indicate an ethologically interesting dependence of behavior on the spatial location of the looming stimulus (Dunn et al., 2016a;Temizer et al., 2015). Accounting for optical distortions will be critical for understanding other fundamental properties of the zebrafish visual system. For example, a basic property of many visual neurons is that they respond strongest to stimuli presented in one specific region of the visual field, termed their receptive field (RF) (Hartline, 1938;Ringach, 2004;Zhang and Arrenberg, 2019). When we simulated the effect of Snell's law on RF mapping under typical experimental conditions (Figure 2d), we predicted substantial errors in both the position and size of naively measured receptive fields (Figure 2e, Materials and methods). Depending on the properties of the true RF, its position and size could be either over-or under-estimated (Figure 2e-f), with the most drastic errors occurring for small RFs appearing near the edge of the Snell window.
Future experiments could avoid distortions altogether by adjusting experimental hardware. For instance, fish could be immobilized in the center of water-filled spheres (Zhang and Arrenberg, 2019;Dehmelt et al., 2019), or air interfaces could be removed altogether, such as by placing a projection screen inside the water-filled arena. But in practice the former would restrict naturalistic behavior, and the latter would reduce light diffusion by shrinking the refractive index mismatch between the diffuser and transparent medium (water vs. air) that typical light diffusers use to transmit stimuli over a large range of angles. An engineering solution might build diffusive elements into the body of the fish tank (Stowers et al., 2017;Franke et al., 2019). Alternatively, we propose a simple computational solution to account for expected distortions when designing stimuli or analyzing data. Our tool (https://www.github.com/spoonsso/snell_tool/) converts between normal and distorted image representations for the most common zebrafish experiment configuration (Figure 1a), and other geometries could be analyzed similarly. This tool will therefore improve the interpretability and reproducibility of innovative experiments that capitalize on the unique experimental capabilities available in zebrafish neuroscience.

Materials and methods
See Appendix 1 and Appendix 2 for the geometric consequences of Snell's law at flat and curved interfaces, respectively.

Implications of the Fresnel equations
Only a portion of the incident light is transmitted into the water to reach the eye. We calculated the fraction of transmitted light according to the Fresnel equations. Assuming the light is unpolarized, where T is the fraction of light transmitted across an air-water interface at incident angle a ¼ a ð Þ (See Appendices 1, 2), w ¼ is the angle of the refracted light ray in water, and are the reflectances for s-polarized (i.e. perpendicular) and p-polarized (i.e. parallel) light, respectively. When including the plastic dish in our simulations, we modified these equations to separately calculate the transmission fractions across the air-plastic and the plastic-water interfaces. We assumed that the full transmission fraction is the product of these two factors, thereby ignoring the possibility of multiple reflections within the plastic.

Illustrating distorted sinusoidal gratings
For all image simulations in Figure 1c, we neglected the plastic and fixed the total distance between the fish and the virtual screen, d a þ d w , to be 1 cm, a typical distance in real-world experiments. The virtual screen was considered to be a 4 Â 4 cm square with 250 pixels/cm resolution. Here we assumed that the virtual screen emits light uniformly at all angles, but this assumption is violated by certain displays, and our computational tool allows the user to specify alternate angular emission profiles. To transform images on the virtual screen, we shifted each light ray (i.e. image pixel) according to Snell's law, scaled its intensity according to the Fresnel equations, and added the intensity value to a bin at the resulting apparent position. This simple model treats the fish eye as a pinhole detector, whereas real photoreceptors blur visual signals on a spatial scale determined by their receptive field. Consequently, our simulation compresses a large amount of light onto the overly thin border of the Snell window, and we saturated the grayscale color axes in Figure 1c to avoid this visually distracting artifact.
To make the image as realistic as possible, we mimicked real projector conditions using gammaencoded gratings with spatial frequency 1 cycle / cm, such that x t ð Þ ½ 1=2:2 ¼ sin t with x t ð Þ ranging from 1.0 to 500.0 lux, a standard range of physical illuminance for a lab projector. The exponent on the left represents a typical display gamma encoding with gamma = 2.2. To reduce moiré artifacts arising from ray tracing, we used a combination of ray supersampling (averaging the rays emanating from 16 sub-pixels for each virtual screen pixel) and stochastic sampling (the position of each ray was randomly jittered between -1 and 1 sub-pixels from its native position) (Dippé and Wold, 1985). In Figure 1c, we display the result of these operations followed by a gamma compression to mimic the perceptual encoding of the presented stimulus.

Corrections to looming visual stimuli
We approximated the geometric parameters from Dunn et al. (2016a) (flat air-water interface, d a = 0.5 mm, d w = 3 mm, d p = 1 mm, stimulus offset from the fish by 10 mm along the screen) and Temizer et al., 2015 (curved air-water interface, d a = 8 mm, d w = 2 mm, d p = 1 mm, r = 17.5 mm, stimulus centered) to create Snell-transformed images of circular stimuli with sizes growing over time (Figure 2a-c). We used a refractive index of n p = 1.55 for the polystyrene plastic. While Dunn et al. collected data from freely swimming fish, the height of the water was kept at approximately 5 mm, and 3 mm reflects a typical swim depth. Since freely swimming zebrafish can adjust their depth in water, it's an approximation to treat d w as constant.
We quantified the size of each transformed stimulus with its solid angle, the surface area of the stimulus shape projected onto the unit sphere. To calculate the solid angle for Temizer et al., we used the formula for a spherical cap, A ¼ 2p 1 À cos ð Þ, where A is the solid angle and 2 is the apex angle. To calculate the solid angle for Dunn et al., in which stimuli were not spherical caps, we first represented stimulus border pixels in a spherical coordinate system locating the fish at the origin. The radial coordinate does not affect the solid angle, so we described each border pixel by two angles: the latitude, a, and longitude, b. To calculate the area, we used an equal-area sinusoidal (Mercator) projection given by which projects an arbitrary shape on the surface of a sphere onto the Cartesian plane. While distances and shapes are not preserved in this projection, area as a fraction of the sphere's surface area is maintained. Thus, we could calculate the solid area of the stimulus in this projection by finding the area of the projected 2D polygon. To calculate the absolute and relative discrepancy 95% confidence intervals in Figure 2c, we used error propagation formulae for the difference and division of two distributions, respectively.

Receptive field mapping
We simulated receptive field (RF) mapping experiments by tracing light paths from single pixels on a virtual screen to the fish (Figure 2d-f). We modeled a neuron's RF as a Gaussian function on the sphere, defined the "true RF" to be the pixel-wise response pattern that would occur in the absence of the air-water interface, and defined the "apparent RF" as the pixel-wise response pattern that would be induced with light that bends according to Snell's law at an air-water interface. More precisely, we modeled the neural response to pixel activation at position x as where T a x ð Þ ð Þ is the fraction of light transmitted (Fresnel equations), RF and s RF are the mean and standard deviation of the Gaussian RF, x; RF ð Þ is the distance along a great circle from the center of the RF to the pixel's projected retinal location, and P ; s 2 RF À Á ¼ e À 2 = 2s 2 RF ð Þ is the Gaussian RF shape. We calculated the great circle distance between points on the sphere as where a RF ; b RF ð Þ are the latitude and longitude coordinate of the RF center, and a x ; b x ð Þ are the latitude and longitude coordinates of the projected pixel location. We quantified the position of the RF as the maximum of F x ð Þ, converted to an angular coordinate along the screen. We quantified RF area as the solid angle of the shape formed by thresholding F x ð Þ at half its maximal value.

Computational tool for simulating and correcting optical distortions
With this paper, we provide a computation tool for visualizing and correcting distortions (https:// github.com/spoonsso/snell_tool/). The tool is written in Python and uses standard image processing libraries. The tool can be launched virtually over the web, without any need to install new software, using the MyBinder link in the README file hosted on the github repository. The source code can also be downloaded and run on the user's local machine.
The uses and parameters of the tool are described in detail in an example notebook in the repository (snell_example.ipynb). In brief, the tool is implemented only for flat interfaces with the assumptions described in Appendix 1, and it can model distortions through three media (i.e. with a plastic interface between air and water). It can also model displays that emit light with non-uniform angular profiles. Key customizable parameters include the screen size, screen resolution, screen distance, media thicknesses, media refractive indices, and gamma encoding. As described in Illustrating distorted sinusoidal gratings, the tool uses a combination of ray super-sampling and stochastic sampling to reduce moiré artifacts arising from ray tracing.
The Python notebook illustrates two primary use cases of the tool, though the tool's library is flexible enough to be adopted for other tasks. First, it allows the user to input an image to see its distorted form under the assumptions of the model. Thus, it recreates Figure 1c, but for any arbitrary grayscale stimulus, and for a range of user-specified experimental configurations. Second, it allows the user to input an undistorted target image, and the tool inverts the distortion process to suggest an image that could be displayed during an experiment to approximately produce the target from the point of view of the fish. In the tool's example notebook, we demonstrate this inversion process using a checkered ball stimulus. Importantly, note that some stimuli will be physically impossible to correct (e.g. undistorted image content cannot be delivered outside the Snell window). The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication. Additional files

Supplementary files
. Transparent reporting form

Data availability
No data were collected for this theoretical manuscript.
analytically inverting 0 ð Þ is non-trivial, we noted from the graph of 0 ð Þ that the inverse function exists and calculated 0 ð Þ with a numerical look-up table (e.g. Figure 1b).
Appendix 1-figure 1. Illustration of mathematical variables used to analyze optical distortions in arena geometries where flat air-plastic and plastic-water interfaces separate the fish from the projection screen. The brown line denotes the trajectory of a light ray traveling from the screen to the fish. We quantify the image transformation by relating the true angular position of each projected point ( 0 ) to its apparent position (). The derivation involves several distances (e.g. d w ) that summarize the ray's trajectory through air (white region), plastic (gray region), and water (blue region). Refraction angles ( a ; p ; w ) describe the bending of light at each interface.