Optical-flow analysis toolbox for characterization of spatiotemporal dynamics in mesoscale optical imaging of brain activity

Highlights

• Graphical-user-interface toolbox based in Matlab® for investigating the spatiotemporal dynamics of mesoscale brain activity using optical-flow analyses.

• Graphical-user-interface toolbox for preprocessing mesoscale optical imaging data i.e. finding ΔF/F₀ and applying temporal or spatial filtering.

• Comparison of the efficacy of three optical-flow methods namely Horn-Schunck, Combined Local-Global, and Temporospatial algorithms.

• Combined local-global method yields the best results for estimating wave dynamics.

• The automated approach permits rapid and effective quantification of mesoscale brain dynamics.

Abstract

Wide-field optical imaging techniques constitute powerful tools to investigate mesoscale neuronal activity. The sampled data constitutes a sequence of image frames in which one can investigate the flow of brain activity starting and terminating at source and sink locations respectively. Approaches to the analyses of information flow include qualitative assessment to identify sources and sinks of activity as well as their trajectories, and quantitative measurements based on computing the temporal variation of the intensity of pixels. Furthermore, in a few studies estimates of wave motion have been reported using optical-flow techniques from computer vision. However, a comprehensive toolbox for the quantitative analyses of mesoscale brain activity data is still lacking. We present a graphical-user-interface toolbox based in Matlab® for investigating the spatiotemporal dynamics of mesoscale brain activity using optical-flow analyses. The toolbox includes the implementation of three optical-flow methods namely Horn-Schunck, Combined Local-Global, and Temporospatial algorithms for estimating velocity vector fields of flow of mesoscale brain activity. From the velocity vector fields we determined the locations of sources and sinks as well as the trajectories and temporal velocities of flow of activity. Using simulated data as well as experimentally derived sensory-evoked voltage and calcium imaging data from mice, we compared the efficacy of the three optical-flow methods for determining spatiotemporal dynamics. Our results indicate that the combined local-global method we employed, yields the best results for estimating wave motion. The automated approach permits rapid and effective quantification of mesoscale brain dynamics and may facilitate the study of brain function in response to new experiences or pathology.

Introduction

Mesoscale optical imaging of brain activity allows the study of physiological models of brain functions at the network and system levels. The activity of thousands of neurons can be recorded simultaneously to study how networks of neurons communicate with each other. Important regional-scale information has been revealed with mesoscale optical imaging and it is increasingly being used to further our understanding of cortical functional connectivity and flow of information over the cortical surface (Frostig, 2009). In rodent brain imaging, mesoscale assessments correspond to spatial dimensions ranging from hundreds of micrometers to a few millimeters. Various methods have been developed to monitor mesoscale brain activity including optical imaging approaches and multielectrode electrophysiology (Nunez and Srinivasan, 2006, Hillman, 2007, Townsend et al., 2015, Grinvald et al., 2016, Obien et al., 2015). In optical imaging, the brain is illuminated with light and imaged to detect neuronal activity at the population level. This neuronal activity is transduced into an optical signal by either intrinsic or extrinsic reporters and interpreted as changes in the intensity of captured light (Frostig, 2009, Grinvald et al., 2016, Kalatsky and Stryker, 2003, Scanziani and Häusser, 2009). In intrinsic imaging, neuronal activity is inferred from either changes in reflectance of light from tissue due to oxygenation states of hemoglobin (Nemoto et al., 1997, Biswal et al., 2011) or from changes in auto-fluorescence of endogenous molecules such as flavoproteins (Reinert et al., 2004, Husson and Issa, 2009, Shibuki et al., 2009). In extrinsic imaging on the other hand, exogenous fluorescent molecules are introduced into the brain and neuronal activity is inferred from fluorescence changes modulated by either neuronal membrane voltage (voltage imaging), or concentration of an intracellular ion (e.g. calcium imaging) or an extracellular molecule (e.g. glutamate) (Xie et al., 2016, Storace et al., 2015, Abdelfattah et al., 2016, Akemann et al., 2012, Chen et al., 2013). Whichever method is selected, the data obtained with optical imaging is a set of images whereby the intensity of each pixel is indicative of the summed activity of nearby neurons. Mesoscale brain activity is also recorded with multielectrode electrophysiology in which electrodes are spread over surface of the brain to enable spatially discrete sampling of neuronal activity. The electrophysiology data can also be visualized as a set of 2D images where the intensity of each pixel would represent the voltage measured by an electrode (Buzsáki et al., 2012). Thus, mesoscale sampling of brain activity can capture the spatiotemporal dynamics of networks of neurons resulting in data with one temporal and two spatial dimensions (Fig. 1).

There are a variety of features that can be observed from optical imaging of brain activity. For example, when a group of neurons become active in response to a stimulus or from ongoing spontaneous activity, a source-like structure can be observed whereby pixel intensities corresponding to the site of activation gradually increase. The observed shape of the source-like activity in 2D can vary and may take the form of a point, line, circle, ellipse, or a combination of these shapes depending upon the spatial organization of the active neurons (Huang et al., 2010, Takagaki et al., 2011, Mohajerani et al., 2013). As time passes from the moment of activation, the source activity might weaken and disappear or sink in the same spatial region it originated from. In contrast, it might travel like a wave (Lilly, 1954) to another region following a continuous path/trajectory before weakening and disappearing into a sink where pixel intensities gradually decrease. The sink might also have shapes in 2D similar to the source activity. For propagating activity, the paths might be a mixture of translational, rotational, expansion or compression trajectories. If multiple brain areas are simultaneously active, multiple sources, travelling waves, and sinks would be observable. The observation of brain activity at the mesoscale hence mimics the motion or flow of waves.

The most common analysis used for the extraction of physiological features from optical imaging of brain activity is the quantitative assessment of temporal variation of intensity by plotting the average intensity of pixels versus time for defined regions of interest. More recently, the temporal correlations of optical signals were used to determine the functional connectivity between different regions (Fig. 1) (Mohajerani et al., 2013, Vanni and Murphy, 2014, Chan et al., 2015, McVea et al., 2012, White et al., 2011, Minderer et al., 2012, Kuhn et al., 2008). To identify sources, sinks, and activity trajectories, visual inspection was often used (Mohajerani et al., 2013, Han et al., 2008). However, few studies have reported estimates of neuronal activity spread using “optical-flow” methods which were first developed in the field of computer vision. In “optical-flow” approaches, velocity vector flow fields are calculated to determine speeds and directions of motion (http://link.springer.com/10.1007/0-387-28831-7, Sun et al., 2010). From the velocity vector fields, locations of sources and sinks are estimated using vector calculus methods. Inouye et al., 1994, Inouye et al., 1994, Inouye et al., 1995 reported the use of Horn-Schunk (HS) method (Horn and Schunck, 1981) to determine the flow of brain oscillations over the human scalp in flattened (3D surface to 2D) electroencephalography data. Takagaki et al. (2011) developed and reported a new method (Temporospatial – TS) in which temporal correlation of a given pixel with neighboring pixels was used to estimate local motion in voltage sensitive dye (VSD) imaging data. Recently, Mohajerani et al. (2013) used the combined local-global (CLG) algorithm (Bruhn et al., 2002, Jara-Wilde et al., 2015) to determine velocity vector fields of flow in wide-field VSD imaging data. They also manually determined the location of sources and sinks in space and time. Following Mohajerani et al., Townsend et al. (2015) also used the CLG method to estimate wave patterns in cortical activity recorded with multielectrode arrays.

Only HS, CLG, and TS optical flow methods have been used for the analysis of brain activity but there are numerous other optical flow algorithms that have been developed (http://link.springer.com/10.1007/0-387-28831-7, Sun et al., 2010, Otte and Nagel, 1994, Tavakoli et al., 2008, Liu et al., 2015, Kroeger et al., 2016). However, a quantitative and comparative evaluation of the performance and analytical efficacy of optical-flow methods for determining velocity vector fields in brain activity remains lacking. There also has been a scarcity of quantitative analysis for extracting spatiotemporal dynamics in brain activity from velocity vector fields. Off the shelf tools are also missing for streamlining data analysis i.e. estimating velocity vector fields followed by automatically identifying sources and sinks, and calculating trajectories and velocities of brain activity with respect to time. In addition to identifying sources and sinks in space-time, it would also be useful to quantitatively characterize their properties such as outlines (what is the shape of a source or a sink), sizes (how big a source or sink is in space) and strengths (how much activity outflow or inflow is there in time).

In this paper, we present for the neuroscience community, a graphical user interface based Optical-Flow Analysis Toolbox in Matlab® (Mathworks Inc.) for investigating the spatiotemporal dynamics of mesoscale brain activity (OFAMM). We compared the performance and analytical efficacy of three optical-flow methods namely, Horn-Schunck (HS), combined local-global (CLG), and temporospatial (TS) for determining velocity vector fields of perceived flow in brain activity monitored using voltage and calcium imaging. Also, we compared the performance of HS and CLG method-based analyses in determining sources and sinks in space-time as well as trajectories of brain activity waves and their temporal speeds. These three optical flow methods were selected from numerous others (Paragios et al., 2006), primarily, due to their previous use with brain activity. Since ground truth values are unknown in real experimentally derived data sets of brain activity, simulated data was first used to investigate the accuracy of our analysis and its sensitivity to the addition of noise. Later, we tested and validated the application of our analysis on real data acquired with wide-field voltage and calcium imaging from mouse cortical activity. Similar to previous findings (Berger et al., 2007), higher instantaneous and temporal speeds were estimated with voltage imaging as compared to calcium imaging. The results are consistent with voltage signals reporting predominantly subthreshold activity and calcium signals reporting suprathreshold spiking activity (Berger et al., 2007).