Semi-automatic behavior analysis using robot/insect mixed society and video tracking

https://doi.org/10.1016/j.jneumeth.2010.06.013Get rights and content

Abstract

This paper proposes a novel robot/insect mixed society setup which enhances the possibilities for insect behavioral research and can be used as a powerful tool for interdisciplinary studies on insect behavior. Micro-robots are equipped with decoys so as to allow a controlled dynamic interaction with crickets, Gryllus bimaculatus. A camera records the interaction and the video is later processed for the automatic tracking of each encounter between cricket and robot. A novelty of our method lies in using the robots as tools for the controlled evoking of specific insect behaviors rather than trying to build an insect-like robot. The possibility for performing controlled repeatable movements allows the stimulation of certain insect behaviors that are usually difficult to trigger using insects alone, allowing consistent behavioral research. A set of experiments were performed in order to validate the proposed setup. We also demonstrate the use of our setup for stimulating agonistic behavior during an electromyography recording session.

Introduction

This paper presents a method of using micro-robots for the behavioral study of crickets. Robotics has been increasingly used for the validation of behavior models of animals ranging all the way from insects (Webb, 2006) to human infants (Asada et al., 2009). In particular, with respect to insect/robot interactions, Gaustrais et al. (2004) used autonomous insect-like robots as a complementary tool for the study of the robustness of cockroach aggregation behavior against high interindividual variability. They explored the dynamics of mixed societies where one or more individuals were modified to test whether there is only a gradual change on the collective level or whether non-linear changes would occur. Later Halloy et al. (2007) showed how autonomous miniature robots modeled to mimic cockroach behavior can affect the aggregation behavior of real cockroaches. A key difference between our approach and that of Gaustrais et al. (2004) and Halloy et al. (2007) is that we take a more analytic rather than synthetic approach, that is, we do not attempt to build a robot that faithfully mimics an insect's behavior. This allows us to be less restrictive on the constraints on the robot and setup, shifting the focus toward the studied subject itself. We believe the use of small miniature robots in such a multidisciplinary setup is a powerful tool for general and systematic investigation of insect behavior. We also describe an algorithm capable of automatically parsing and classifying agents encounters through video tracking using an overhead camera.

In order to validate our proposed setup we performed a series of experiments focusing on how subordinate and dominant crickets behave after an agonistic dispute is settled.

The current state of behavior research of cricket (and other insects in general) can be classified into three main categories of experimental setup: (1) one shot, (2) treadmill and (3) free moving.

In the one shot setup, the insect, arena and other apparatus are repeatedly reset into a given fixed initial condition and some stimulus is presented in a controlled fashion. Typically, the analysis is focused on the behavior that immediately follows, thus allowing carefully controlled investigations once the experimenter knows how to trigger the desired behavior. For instance, Tauber and Camhi (1995) and Baba and Shimozawa (1997) use this kind of setup with the aid of video analysis for studying wind-evoked escape behavior in crickets. Another example is the use of a system of individual choice chambers (August, 1971). In the one shot setup it is also common to use stationary artifacts as cues (Adamo and Hoy, 1995, Tachon et al., 1999, Nagamoto et al., 2005).

The treadmill setup (typically a free-rotating styrofoam ball) is popular for long course tracking. Some examples are studies on phonotaxis Doherty (1985), research on audio and visual stimuli influences on course control (Bohm et al., 1991), and the study of insects’ reactions to mechanical stimulation of hindwings (Hiraguchi and Yamaguchi, 2000). More recently, optical mouse sensors have been employed for more accurate measurements (Lott et al., 2007). This setup is a great solution for tracking the trajectory of a single individual in a very controlled environment. Here again the experimenter must typically have a fairly accurate idea of how to trigger the behavior to be investigated. This system can monitor only one individual at a time and insects are usually constrained so they cannot jump, flip-over or accelerate abruptly.

Relying on rigid controlled conditions prevents the unfolding complexity typical of longer interactions. The free moving setup is one where insects are left inside an arena while their behavior is observed, often in a more qualitative way, but also with the aid of cameras as tracking devices. This type of setup is ideal for reducing the amount of prior assumptions and for investigating social behaviors. It allows the experimenter to observe the behaviors as they arise from the stochastic complexity of the interactions among the individuals. Unfortunately, so far this could only be done exclusively with real insects, in which case the biologist can observe an animal's behavior, but cannot control it.

The novelty of our proposed method, in comparison with more classical behavioral studies, is that our setup allows the experimenter to actively produce movements which are controlled and repeatable. Along with other cues, the mechanical stimuli may have a significant influence on the likelihood of triggering crickets’ behavior as if they were interacting with conspecifics. Small attachments can be added to this robot in order to convey behavioral cues such as chemical attachments, textures or other tactile cues like antennae, visual lures, etc., in a non-stationary way. Additionally our setup brings the possibility of programmatically modulating the robot's movement in real time according to some hypothetical model as a function of cricket's movements. This allows the researcher to test different hypotheses regarding the influence of opponents’ behavior on insects’ choices.

Section snippets

Materials and methods

The experiment setup is illustrated in Fig. 1. A laptop equipped with an infrared transmitter was used for controlling a robot's movement while an overhead camera recorded each trial. The resulting footage was later processed for the tracking of both robots and crickets. The arena was a rectangular area of dimensions 300 mm × 225 mm delimited by walls of 150 mm of height, separating it from the external environment of the lab.

Results

The results described here demonstrate the use of our setup in practice. In our experiments we used our setup to automatically detect following and escaping behaviors of subordinate and dominant crickets after they encountered another agent (robot or cricket).

In each trial two new male crickets were left to freely interact with each other in the arena for 5 min or longer until a dominant/subordinate relationship could be clearly observed, and then depending on the group being studied either

Discussion

A behavioral research framework based on the multidisciplinary mixing of micro-robots and insects has been presented. Our setup allows us to trigger specific insect behaviors, enticed by controlled stimulus cueing assisted by the use of micro-robots, triggering, for instance, courtship or agonistic behavior.

Okada et al. (1999) have showed how electrophysiological recording using copper wire can aid the study of neuronal activities in a free moving insect. However, if we focus on the agonistic

Acknowledgments

This work was partially supported by grants-in-aid for Scientific Research from the MEXT, Scientific Research on Priority Areas (Area No. 454) to H.A. (17075001). The authors would also like to thank Citizen Watch Co. for helping in the design and construction of such tiny robots.

References (24)

  • H. Bohm et al.

    Visual and acoustic course control in the cricket Gryllus bimaculatus

    J Exp Biol

    (1991)
  • J.A. Doherty

    Phonotaxis in the cricket, Gryllus bimaculatus DeGeer: comparisons of choice and no-choice paradigms

    J Comp Physiol

    (1985)
  • Cited by (11)

    View all citing articles on Scopus
    1

    Tel.: +81 11 706 3832; fax: +81 11 706 3832.

    2

    JST ERATO Asada Synergistic Intelligence Project, Dept. of Adaptive Machine Systems, Osaka Univ., M4-202, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan. Tel.: +81 6 6876 8884; fax: +81 6 6876 8994.

    3

    JST ERATO Asada Synergistic Intelligence Project, Dept. of Adaptive Machine Systems, Osaka Univ., FRC1-4F, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan. Tel.: +81 6 6876 8884; fax: +81 6 6876 8994.

    View full text