The EthoVision video tracking system—A tool for behavioral phenotyping of transgenic mice
Introduction
The behavior of animals, including transgenic mice, is commonly recorded in either a manual or semiautomated way. Traditionally, a researcher observes the animal, and if he or she considers that a certain behavior pattern is displayed, the behavior is noted — either by writing it down, or by entering the data into an event-recording program such as The Observer [1], [2]. Although manual recording of behavior can be implemented with a relatively low investment, and for some behaviors it may be the only way to detect and record their occurrence, automated observation (such as video tracking) can provide significant advantages. The behaviors are recorded more reliably because the computer algorithm always works in the same way, and the system does not suffer from observer fatigue or drift. In contrast to manual observation, video tracking carries out pattern analysis on a video image of the observed animals to extract quantitative measurements of the animals' behavior (for more details about how this works, see below).
Technology for automated detection and recording of behaviors has evolved dramatically in the past decade. Early systems, using hard-wired electronics, were only able to track a single animal in highly artificial environments. For example, an open field can be sampled with a grid of infrared beams either as the sole detectors [3], [4], [5] or in combination with other methods such as placing a series of strain gauge transducers under the arena to estimate the animal's position [6]. The magnitude of an animal's motion can also be estimated using various types of touch-sensitive sensors ranging from a crude estimate of movement by placing the animal on a bass loudspeaker and monitoring the loudspeaker's electrical output when its cone was moved by the rat [7], or measuring the movement of a mouse's wheel by attaching the wheel's axle in place of the ball of a computer mouse [8], to sensors that can also measure the position of the animal (and hence locomotion), for instance by changes in the capacitance of a plate when the animal is in proximity to it [9], or changes in body resistance [10]. Other comparable detection methods have included use of ultrasound [11] and microwave Doppler radar [12]. The position of a rat in an open field has been recorded by attaching the rat to a computer joystick via a series of rods attached by a collar to the rat's neck [13]. Animals' behavior can also be measured using a computerized version of a Skinner box [14], in which the subject has to tap a touch-sensitive monitor [15], [16]. The motion of individual limbs can be monitored automatically using actigraph sensors, which detect movement by means of a piezoelectric accelerometer [17].
Early video tracking systems offered clear advantages of flexibility, precision, and accuracy over the various hardware devices listed above. However, the actual path of the animal still had to be entered manually by the experimenter, either by following the track of the animal with the computer mouse [18] or joystick [19], or a digitizing tablet or similar device [20].
Another early method was to feed the analog video signal to a dedicated video tracking unit, which detected peaks in the voltage of the video signal (indicating a region of high contrast between the tracked animal and background), and used this to produce the x, y coordinates of the tracked animals as output which was then fed to the serial port of a computer [21], [22]. These analog systems have the disadvantage of being relatively inflexible (dedicated to particular experimental setups) and can normally only track one animal, in rather restricted lighting and background conditions.
The first video digitizers were of the column scan type. These had no internal memory of their own, and were only able to sample the video signal at rather low rates [23]. In contrast, a modern frame grabber uses a high-speed analog–digital converter to enable real time conversion of the entire video image to a high-resolution grid of pixels [23].
It is also possible to acquire digitized video images by first converting the video input to a digital video format such as AVI, and then using the AVI file as the input for object detection [24]. However, this method has disadvantages that the AVI file quickly gets very large (and so only trials of a limited duration can be carried out), and the method does not allow for real-time live data acquisition. It is also possible to use a digital overlay board to obtain positional data of tracked animals without the need for a frame grabber [25], [26].
Modern systems, which are based on full-color video frame grabbers and have flexible software, can track multiple animals simultaneously against a variety of complex backgrounds. Whereas an infrared detector system typically operates in an arena of 0.5×0.5 m with 12–32 beams (though systems do exist with a grid of 24×24 beams, in an arena of 2 m diameter [4]), the frame grabber used by EthoVision has a resolution of 768×576 pixels and can track a rat in an arena with a variable shape and up to 20 m diameter.
Video tracking is particularly suitable for measuring three types of behaviors: behaviors that occur briefly and are then interspersed with long periods of inaction (rare behaviors [27]), behaviors that occur over many hours [24], [28] (such as diurnal variation in behavior), and spatial measurements (e.g., [29], [30]) (distance, speed, turning, etc.) that the human observer is unable to accurately estimate. If a behavior can be automatically detected using a video tracking system, that can greatly reduce the human effort required [28]. This not only reduces costs, but enables a larger and therefore statistically more responsible number of replicates to be used, whilst at the same time reducing the total number of animals used in an experiment (because each mouse can be used much more efficiently). This is particularly important for transgenic mice because in that case the experiment examines a Heredity×Environment interaction and a larger sample size is required to test if interaction effects are significant [31].
It is usually obvious if an animal is experiencing an extreme behavior such as an epileptic seizure. However, certain other behaviors, such as behaving in a stressed manner, may be more open to interpretation in borderline cases. A key advantage of automated video tracking is that it forces researchers to define exactly what they mean by a given behavior, and so enables standardization of methodologies [32] — which is one of the most pressing needs in mouse behavioral phenotyping [31], [33], [34].
In this paper we present EthoVision—a general-purpose video tracking, movement analysis, and behavior-recognition system, and illustrate its use in behavioral phenotyping with a number of case studies. Noldus Information Technology introduced the first version of EthoVision in 1993. The system has undergone numerous updates over the years, based on feedback from users around the world, which has resulted in a very comprehensive package for studies of movement and behavior [35]. The software has recently been comprehensively redesigned for 32-bit Windows platforms, with a highly interactive graphical user interface for display of experimental design, experimental arena, and tracks of movement.
Section snippets
How video tracking works
In a video tracking system such as EthoVision, the basis of the system is that a CCD video camera records the area in which the subject animals are (the scene). The camera's signal can either be fed directly to a computer, or via a video cassette recorder (see Fig. 1).
In the case of an analog camera (the most practical option with current technology) the signal is then digitized (by a hardware device called a frame grabber) and passed on to the computer's memory. The video signal consists of a
Practical aspects of video tracking
When using a video tracking system such as EthoVision, the data obtained will only be as good as the quality of the experimental design. Of course, basic considerations such as sufficient replication and avoiding pseudoreplication [37] must still be adhered to. Likewise, the better the practical setup, the better the quality of the data obtained. If the video signal is of poor quality (for example, containing reflections), the tracked animals may not be detected, or other objects (such as the
The EthoVision system
The EthoVision for Windows software has a number of features that make it particularly suitable as a tool in studies such as those described in the Case Studies section (below).
Case studies
The following case studies are presented in order to illustrate the use of EthoVision for detection and quantification of various behaviors in mice. Therefore, full details of neither the experimental setup nor full conclusions drawn from the data are given; the reader is referred to the researchers or the original publications for that information [55], [70]. However, the raw data from some of these case studies have been placed on the web; follow the link from //www.noldus.com/products/ethovision/
Discussion and conclusions
The four case studies illustrate different aspects of using EthoVision to track mice. The first case study (knock-out of the Munc 18-1 gene) looked at mice in an open field which were hypothesized to be hyperactive as a result of the knocked-out gene. Plots of two simple variables quantifying the mice's behavior demonstrated the experiment's effects. Simple, straightforward parameters such as distance moved or time in center can be powerful measures to quantify differences in behavior between
Acknowledgements
We are grateful of those who provided us with their data for use in the case studies section of this paper. All members of the EthoVision team at Noldus Information Technology have contributed towards the product since it was initially launched in 1993; their hard work and dedication is acknowledged here.
References (81)
- et al.
A novel apparatus for measuring rat locomotor behavior
J Neurosci Methods
(1995) - et al.
TouchWindows and operant behaviour in rats
J Neurosci Methods
(1994) - et al.
A method for measuring locomotor behavior in rodents: contrast-sensitive computer-controlled video tracking activity assessment in rats
Neurotoxicol Teratol
(1992) - et al.
Applications of video mixing and digital overlay to neuroethology. Methods
Companion Methods Enzymol
(2000) - et al.
Applications of multimedia computers and video mixing to neuroethology
J Neurosci Methods
(1997) - et al.
Quantification of rodent catalepsy by a computer-imaging technique
Pharmacol, Biochem Behav
(1992) - et al.
Prolonged animal observations by use of digitized videodisplays
Pharmacol, Biochem Behav
(1983) - et al.
Org2766 improves performance of rats with unilateral lesions in the fimbria fornix in a spatial learning task
Brain Res
(1990) - et al.
A video image analyzing system for open-field behavior in the rat focusing on behavioral asymmetries
J Neurosci Methods
(1993) - et al.
Spatial analysis of animals' movements using a correlated random walk model
J Theor Biol
(1988)
Optimal sinuosity in central place foraging movements
Anim Behav
Approach, avoidance and contact behavior of individually recognized animals automatically quantified with an imaging technique
Physiol Behav
Automation of the social interaction test by a video tracking system: behavioural effects of repeated phencyclidine treatment
J Neurosci Methods
The MSH/ACTH(4–9) analog Org2766 counteracts isolation-induced enhanced social behavior via the amygdala
Peptides
Y chromosomal and sex effects on the behavioral stress response in the defensive burying test in wild house mice
Physiol Behav
Y chromosome, urinary chemosignals, and an agonistic behavior (offense) of mice
Behav Physiol
Development of a water-maze procedure for studying spatial learning in the rat
J Neurosci Methods
Behavioral approaches to study function of corticosteriods in brain
Methods Neurosci
Differential modulation of lateral septal vasopressin receptor blockade in spatial learning, social recognition, and anxiety-related behaviors in rats
Behav Brain Res
CREB: A mediator of long-term memory from mollusks to mammals
Cell
Gene-targeting studies of mammalian behavior: is it the mutation or the background genotype?
Trends Neurosci
Assessment of learning by the Morris water task and fear conditioning in inbred mice strains and F1 hybrids: implications of genetic background for single gene mutations and quantitative trait loci analyses
Neuroscience
A new computer program for detailed off-line analysis of swimming navigation in the Morris water maze
J Neurosci Methods
Deficient long-term memory in mice with a targeted mutation of the cAMP responsive element binding (CREB) protein
Cell
Essential role for TrkB receptors in hippocampus-mediated learning
Neuron
The Observer: a software system for collection and analysis of observational data
Behav Res Methods, Instrum Comput
The Observer Video-Pro: new software for the collection, management, and presentation of time-structured data from videotapes and digital media files
Behav Res Methods, Instrum Comput
An infrared device for detecting locomotor activity
Behav Res Methods, Instrum Comput
A method to analyze the spatial distribution of behavior
Behav Res Methods, Instrum Comput
A computerized infrared monitor for following movement in aquatic animals
Behav Res Methods, Instrum Comput
A simple method for recording the path of a rat in an open field
Behav Res Methods, Instrum Comput
A microcomputer-controlled system for measuring reactivity in small animals
Behav Res Methods, Instrum Comput
Monitoring running-wheel movement using a serial mouse and an IBM-compatible system
Behav Res Methods, Instrum Comput
A programmable proximity-contact sensor to detect location or locomotion of animals
Behav Res Methods, Instrum Comput
An electronic device for detecting activity in caged rodents
Behav Res Methods, Instrum Comput
The use of an ultrasonic monitor for recording locomotor activity
Behav Res Methods, Instrum Comput
A microwave Doppler radar activity monitor
Behav Res Methods, Instrum Comput
The behavior of organisms
The touch-screen system in the pigeon laboratory: an initial evaluation of its utility
Behav Res Methods, Instrum Comput
A new method of monitoring motor activity in baboons
Behav Res Methods, Instrum Comput
Cited by (228)
Comparison of automated video tracking systems in the open field test: ANY-Maze versus EthoVision XT
2023, Journal of Neuroscience MethodsMulti-object Mice Tracking Based on YOLO v8n - seg and Improved Strongsort
2024, Nongye Jixie Xuebao/Transactions of the Chinese Society for Agricultural MachineryA comparison of machine learning methods for quantifying self-grooming behavior in mice
2024, Frontiers in Behavioral NeuroscienceDetecting and Tracking of Multiple Mice Using Part Proposal Networks
2023, IEEE Transactions on Neural Networks and Learning Systems