Controlling underwater robots with electronic nervous systems

Department of Biology and Marine Science Center, Northeastern University, East Point, Nahant, MA 01908, USA; Information Systems Laboratories, Inc., 10070 Barnes Canyon Road, San Diego CA 92121, USA; Marine Science Center, Northeastern University, East Point, Nahant, MA 01908, USA; Department of Electrical and Computer Engineering, Northeastern University, 360 Huntington Ave. Boston, MA 02115, USA; Institute for Nonlinear Science-0402, UCSD, La Jolla, CA 92093-0402, USA


Introduction
The ability of even simple invertebrates to outperform the mobility of the most sophisticated robots has suggested a biomimetic approach to the problem of how to achieve truly autonomous robotic devices (Taubes 2000;Ayers et al. 2002). Recent advances in biomimetics have made it feasible to construct robots ( Figure 1) that to some degree approximate their animal models Kato et al. 2004). The existence of such robots provides an embodied technique to model underlying mechanisms of the control of behaviour (Webb 2000;Webb 2001). Biorobotic studies can provide unique insights into the critical variables in the control of behaviour by neuronal networks (Horchler et al. 2003). We have previously demonstrated biomimetic robots built with finite-state machines that mimic the operation of endogenous central pattern generators (Ayers et al. 2000;Ayers and Witting 2007). These controllers capture the dynamics and logic of the motor pattern generators of the model animals (Ayers 2002). More recently, we have begun to construct electronic nervous systems based on two nonlinear dynamical models of neurons. The first are analogue computers developed at UCSD (UCSD ENs) that solve the Hindmarsh-Rose Equations (Pinto et al. 2000). The second are based on one or two degree maps that * Corresponding author. Email: lobster@neu.edu model the dynamics of neurons and synapses using difference equations (Rulkov 2002). Here we compare these architectures and report a feasibility test of the potential to develop a hybrid central pattern generator-based controller for a biomimetic robot based on a combination of UCSD electronic neurons (Pinto et al. 2000) and discretetime map-based computed neurons (Rulkov 2002).

Background
Decapod crustacea have long been models for the study of sensory-motor integration and have provided important insights into the organisation of locomotory systems (Kennedy and Davis 1977). Decapods exhibit tactile navigation capabilities using antennae and bump detectors to literally feel their way through complex rock-delineated fields (Ayers 2004). By sweeping antenna over different subsets of their workspace, they can determine their proximity to objects as well as gauge their height (Sandeman 1985). Lobsters can walk with equal facility in any direction, rotate in place and change their walking direction on a step-by-step basis (Ayers 2000a).

Artificial and natural control architectures
The quest to couple sensing devices with motor control led to behaviour-based control architectures in the 1980  (Brooks 1991). The use of motors to control these early systems, however, necessitated the use of electronic interfaces that differ profoundly from the control principles used by animals. For example, animals grade muscular force by recruiting increasingly larger numbers of motor units based on axon size (Stuart and Enoka 1985). Increasing the force produced by a motor however requires feedback and a mechanical gear arrangement. The absence of actuators that have a realistic resemblance to biological muscle has been a challenge to the control of biomimetic robots. However, recent advances in sensor and artificial muscle technologies have made it feasible to consider the development of robots organised along more physiological principles (Taubes 2000).
Studies over the past 40 years have demonstrated that the innate rhythmic behaviour of animals is generated by central pattern generators (CPGs) distributed throughout the nervous system (Delcomyn 1980;Marder et al. 2005) and modulated by sensory feedback. CPGs are networks of neurons that can generate an excellent replica of the motor neuron discharge patterns underlying innate behavioural acts in the total absence of sensory feedback or patterned input from higher centres (Selverston and Moulins 1987). Unit CPGs are organised by intersegmental interneurons that modulate and coordinate their activity (Kennedy and Davis 1977;Stein 1978;Namba and Mulloney 1999). Coordinating neurons pass information from a governing CPG to a governed CPG that, depending on the nature of the synapse, can cause a phase advance or delay that maintains intersegmental phase or gait. Command neurons exhibit parametric modulation of CPGs (Pinsker and Ayers 1983) to initiate operation and control the average period and amplitude. In some cases, different commands can select different motor programs from the same CPG (Bowerman and Larimer 1974;Ayers and Davis 1977).

CPGs for locomotion
There are numerous examples of central pattern generator networks (Selverston 1999). In many cases, the underlying circuitry has been established by paired neuronal recordings and the cellular properties of the different component neurons defined in terms of their underlying ionic conductances (Harris-Warrick et al. 1992). Central pattern generators may produce more than one behaviour. An example of a hypothetical central pattern generator network for the control of walking in different directions is illustrated in Figure 2. In this system, a neuronal oscillator generates a three-phase pattern and command inputs select different behaviours by gating synapses within the network (Ayers 2004). Figure 1 illustrates an existing biomimetic robot based on the lobster (Ayers and Witting 2007). The robot consists of an 8 in. × 5 in. hull actuated by eight three-degree of freedom legs and stabilised by anterior and posterior hydrodynamic control surfaces. It is be powered by NiMH batteries and, at present, can be controlled by an on board neuronal-circuit based controller implemented as a finite state machine on a microprocessor or via a serial interface from the same code running on a laptop for interactive debugging. The watertight hull contains the motherboard, leg current driver boards, motor controller board, sonar board and current drivers for the trim appendages. The motherboard houses power management circuitry, the compass and pitch and roll inclinometers. Eight modular walking leg assemblies are attached to a flange on the hull. Each leg assembly is composed of vertical posts that contain muscle modules that protract and retract the leg around a capstan that supports the more distal joints. Two other segments house paired antagonistic actuators that cause elevation/depression and extension/flexion.

Myomorphic actuators
The leg state machines gate current drivers that actuate antagonistic shape memory alloy (SMA) artificial muscles to move the different leg joints. The SMA actuators are formed from nitinol wire (Witting and Safak 2002). When cooled by the surrounding seawater, the wires can be deformed and stretched to a martensite structure. When heated to the transition temperature by electrical current, the martensite converts to a more compact structure (austenite) and the wire contracts by about 5% from its deformed martensite length. A 250 µ m wire can lift a kilogram in about 150 ms (Witting and Safak 2002). Pairs of SMA actuators can produce alternating contractions or can be co-activated to maintain the stiffness of the joint. Pulse width duty cycle modulation of trains of current pulses allows graded contractions to regulate the attitude and speed of movements. Figure 2. Hypothetical Neuronal Circuitry underlying the EN CPG. Triangles represent excitatory synapses. Coloured circles projecting from the neuronal oscillator neurons represent inhibitory synapses. The inhibition from Elev (elevator) to Stance is stronger than that from Elev to DEP resulting in a delay between the recovery of swing from Elev relative to the recovery of Dep. This delay represents the late swing phase of the step period. Coloured circles projecting from the command neurons indicate presynaptic inhibitory synapses. These synapses gate connections from the Swing and Stance phase inter-neurons appropriate to the direction of walking. The excitatory synapses from the commands to the bifunctional muscles (protractor, retractor, extensor and flexor) represent the recruiting pathway by which the speed of walking is controlled.
Each actuator can be activated with three different duty cycles to produce low, medium and high amplitude contractions corresponding to the recruitment states of the controller.

Neuromorphic sensors
We have developed several biomimetic sensors necessary to mediate reactive tactile navigation on the ocean bottom (Ayers and Witting 2007). All sensors code information with a labelled line code. Each sensor is represented by a byte, each bit of which corresponds to a labelled line. The labelled line represents three characteristics of the stimulus: (1) the sensory modality; (2) the receptive field or orientation relative to the body; and (3) the amplitude of the stimulus. All sensors are polled by the state machine and return a byte representing their status ( Figure 3). The lobster robot is equipped with an exteroceptive sensor suite that includes: (1) Compass -mediates sense of direction; (2) Pitch and Roll inclinometers -mediate orientation in the pitch and roll plane; (3) Antennae -multidimensional sensors that respond to collision, active sweeps and water current; (4) Bump Detectors -respond to collisions by particular appendages such as the claws.

A state machine-based CPG
At present this vehicle is controlled by a finite state machine controller (Ayers and Witting 2007). The outputs of the finite state machine are control signals that specify the timing and amplitude of actuator action. These signals are used to gate power transistors at different duty cycles to activate contractions of the artificial muscle just as motor neuron action potentials activate muscle. Antagonist muscles of joints that serve a postural function in a particular walking direction are co-activated at low amplitude (Ayers and Clarac 1978).

Behavioural choice and sequencing
The higher order control of behaviour of the robot is based on command neurons (Bowerman and Larimer 1974). There are nine internal state variables or commands; each of one had two to five states. Modulation of these commands occurs at two levels (Ayers and Witting 2007). For example, in exteroceptive reflexes, sensor feedback modulates on one command such as walking speed on the two sides on an ongoing basis to modulate yaw relative to flow. Similarly, changes in pitch will evoke a reflex that levels the thorax. More complex, linked sequences involve lists of both fixed and goal achieving command transition subsequences. These sequences are stored as tables in a behavioural library and each is triggered by a specific sensory releaser. Goal achieving sub-sequences maintain ongoing states until a goal is achieved (e.g. turning to a particular compass heading). Achievement of the goal triggers the next sub-sequence in the list (Ayers 2000b).

Why chaotic electronic neuron networks?
These state machine models are, at best, approximations of the operation of the biological systems and rely on entirely different mechanisms. As a result, their dynamical responses to perturbation must differ from that of natural architectures. When biological neurons from lobster CPGs are isolated from other neurons and phase portraits made of their burst patterns, they show clear regimes of chaotic activity. This chaotic activity is an important element that has been built into the behaviour of the Hindmarsh-Rose electronic neurons. It enables both the biological and the electronic version of individual and small networks of neurons to solve the problem of robustness and flexibility, two mutually antagonistic properties. Networks made up of chaotic neurons display extremely robust local minima but the trajectories can be displaced by transient perturbations such as inputs from sensory receptors that allow the pattern to be momentarily altered. This can smoothly adjust the gait of a robot to irregularities on the ocean floor while maintaining the stability of the overall locomotory pattern.
We propose an alternative way of controlling robots: construct the biological networks in electronic neurons and synapses as an electronic nervous system (ENS). Existing conductance models of neurons are too computationally intensive to permit real-time robot control by even simple neural circuits (Hammarlund and Ekeberg 1998). Application of non-linear dynamical analysis to isolated lobster neurons has indicated that they express only four degrees of freedom (Abarbanel et al. 1996). This allows models of low enough complexity to capture the dynamics of neurons and instantiate them in simple analog circuits. Similarly, map-based neurons can be modelled with difference equations and allow the real-time operation of more complex circuits. We advocate the use of a hybrid architecture of electronic neurons (Pinto et al. 2000) and prototyping with map-based neurons (Rulkov 2002) to allow adaptive development of electronic nervous systems that embody the CPG components and their interrelations with sensors in real time.
We chose these architectures for two reasons. First an electronic neuron-based system accurately reproduces the rhythmic spatiotemporal motor patterns used by animals and inherently captures the dynamical processes that lead to stability and response to perturbation. Secondly, due to their low dimensionality, these phenomenological models can operate in real time and respond to perturbations as rapidly as the real nervous system.

Electronic neurons
Our electronic neurons are analogue computational units that solve modified Hindmarsh-Rose (HR) equations that define the four degrees of freedom observed experimentally in lobster neurons (Pinto et al. 2000).
a, b, c, d, I, e, f, g, µ , S, h, ν , k, r and l are constants embodying neural dynamics (modified from Hindmarsh and Rose 1984); x(t) corresponds to membrane voltage; y(t) represents a 'fast' current; we choose µ 1, so z(t) is a 'slow' current. The first three equations (3-D model) can reproduce several modes of spiking-bursting activity seen in STG cells. Adding the fourth equation (for w(t); 4-D model) introduces an even slower process (ν < µ 1), intended to represent intracellular Ca ++ dynamics (Falcke et al. 2000). The w(t) dynamics is an additional degree of freedom with a time constant three times slower than the characteristic bursting times. Both 3-D and 4-D models have regions of chaotic behaviour, but the 4-D neuron has much larger regions in parameter space where chaos occurs. Thus modulation of w(t) can vary the level of chaos in a network constructed of electronic neurons. ENs can be configured for different levels of complexity. When the integrators for z and w are disabled, the resulting 2-D ENs behave like FitzHugh-Nagumo neurons (Hindmarsh and Rose 1982). When the w integrator is disabled, the resulting 3-D ENs behave like regular bursters (Abarbanel et al. 1996). When the w integrator is enabled, the resulting 4-D ENs become capable of regular as well as chaotic behaviour over a broad range of parameter values. When isolated, lobster neurons have different dynamical personalities ranging from silence to chaotic bursting and their behaviour can be altered by injected current and neuromodulation (Harris-Warrick and Johnson 1989). The ENs have trim pots that allow variation of seven of the parameters in the HR equations.

Electronic synapses
Electronic chemical synapses (CSs) instantiate both preand post-synaptic EN potentials, presynaptic release threshold and slope, postsynaptic strength and reversal potential as previously modelled (Sharp et al. 1993;Destexhe et al. 1994). In our realisation of chemical synapses, we used the mathematical model of a synapse that can be presented in the form where: The circuit first calculates the voltage using the five segments piecewise linear function generator (block 'tanh'). This voltage is used as input signal to calculate the value of S(t). As one can see the equation for S(t) is a linear differential equation in which the time constant τ depends on the parameter S ∞ as τ = τ 0 (1 − S ∞ ). Our circuit approximates this dependence in the following way: The output voltage of this block multiplied by g(V rev − V post ) is proportional to the synaptic current.

Analogue VLSI electronic neurons and
synapses UCSD electronic neurons are relatively large circuits based on discrete components, require large (±15 V) power supplies and are not suitable for a robotic implementation. We have already simulated the implementation of HR ENs in low voltage sub-threshold analogue VLSI (Lee et al. 2004;). As the UCSD ENs are based on integrated circuits, it was first necessary to instantiate Op Amps and multipliers using sub-threshold VLSI ( Figure  4). The modelled controller is based on a standard 0:25 mm CMOS process with 2V supply voltage. In order to achieve low power consumption, CMOS sub-threshold circuit techniques are used. The simulated power consumption is 4.8 mW and die size including I/O pads is 2.2 mm × 2.2 mm. We have also been able to implement and simulate HR 2 and 3 degree electronic neurons, chemical synapses and presynaptic inhibition ).

Discrete-time map-based neurons and synapses
For modelling the sensory inputs to CPGs, huge numbers of ENs would be required. Testing and optimisation of networks of large numbers of ENs is best done with a computationally efficient simulation models and not ENs. The critical issue is to simulate nested exteroceptive reflexes in real time on low power-embedded processors while being able to reconfigure and anneal the networks readily to test different hypotheses. A 2-D discrete-time map ( Figure 5) that describes the spiking and spiking-bursting behaviour of a neuron model (Rulkov 2002) can be written as follows: x n+1 = f α (x n , x n−1 , y n + β n ) , y n+1 = y n − µ(x n + 1) + µσ + µσ n , where x n is the fast and y n is the slow (due to 0<µ 1) dynamical variable. The non-linear function is written in the following form where the third argument u = y n or a combination of input variables that depend on the model type. Input variables β n and σ n incorporate the action of synaptic inputs I syn and can be written as β n = β E I syn , σ n = σ E (I syn ), where β E is a constant that controls how quickly neurons respond to the input and supports dynamical mechanisms for spike frequency deceleration for DC pulses of current. A simplified 1-D analog of the model can be written as: x n+1 = f α (x n , x n−1 , b rs + β e I n ) , where b rs is a constant defining the resting state of the DMT model. This type of model does not allow the onset of a self-sustained regime of bursting. The simplest map-based model for a synaptic current can be written as: where g syn is the strength of synaptic coupling, and indexes pre and post stand for the presynaptic and postsynaptic potentials, respectively. Here γ controls the relaxation rate of the synapse (0 < γ < 1) and x rp defines the reversal potential and, therefore, the type of synapse: excitatory or inhibitory.

Simulation of behavioural control with the hybrid architecture
We use LabView TM (National Instruments, Austin, TX) to examine interactions between hybrid networks of discretetime map-based (DTM) and electronic neurons (Knudsen et al. 2006). The DTM neurons and synapses are instantiated as virtual instruments (subVIs) in the Labview TM G graphical programming language. 1 Their input, outputs and internal parameters α and σ are connected with each other and the LabView TM instrument controls with the LabView TM 'Wire' tool. In order to create a functional synapse between the DTMs and the ENs, it was necessary to create an analogue output from the waveform of the presynaptic DTM. To achieve this, the DTM voltage is used as the output of a M-series board D/A converter. The analogue output was interfaced to the presynaptic input of an electronic chemical synapse that was in turn connected to the desired postsynaptic EN. In order to create a functional synapse between an EN and a DTM neuron, the EN voltage is input to the LabView TM synapse subVI through the A/D converter and becomes the presynaptic voltage parameter of the of the synapse subVI. Figure 6 demonstrates the successful integration of hybrid DTM EN central pattern generators, coordinating neurons and commands. In the first experiment (Figure 6a), two ENs and one DTM neuron are linked by recurrent cyclic inhibition (DTM inhibits the first EN, the first EN inhibits the second EN and the second EN inhibits DTM). This connectivity causes a sequence of bursting where the bursts proceed between the neurons in the reverse direction of the inhibition. Thus hybrid CPG networks can be constructed. Figure 6b, the free run operation of an electronic neuron is demonstrated configured in bursting mode. In Figure 6c, the EN is entrained by a DTM neuron through an excitatory synapse by a DTM neuron configured to burst at a slightly higher frequency. In Figure 6d, the EN is entrained by a DTN through an inhibitory synapse configured to burst at a lower frequency than the unperturbed EN. Thus, hybrid connections between DTM Neurons and ENs can both perturb and entrain EN CPG neurons to increase or decrease their inherent frequencies (Selverston and Ayers 2006). In Figure 6e, we initiate the operation of an EN recurrent cyclic inhibition CPG with synaptic input provided by a DTM trigger command neuron (Stein 1978). Similar results can be obtained with a gate command where the command neuron fires tonically throughout the motor program.
We have constructed and simulated central pattern generators for walking based on Figure 2 from analog VLSI-based neurons and synapses (Lee YK et al. 2007). A simulation of the operation of a neuronal circuit formed of electronic neurons based on Hindmarsh-Rose neuron dynamics and first order chemical synapses is modelled in Figure 7. The controller generates an excellent replica of the walking motor program (Figure 7) and allows switching between walking in different directions in response to different command inputs. The results of a simulation where the commands for forward and lateral leading are turned on to generate a diagonal walking pattern are shown in Figure 7.
This result indicates that it is feasible to build walking CPG chips and to adaptively modulate their behaviour. Using such chips, it will be feasible to integrate an electronic nervous system into the robotic vehicle. We have been successful at controlling a nitinol-based leg with electronic neurons by thresholding a power transistor to directly drive the actuators with the EN action potentials. 2 The interface necessary to do this consists of a comparator and a threshold circuit to activate a power transistor. The DTM components of the ENS can operate on a DSP chip. Using a serial bus, serial to analogue converters and analogue to serial converters can provide bidirectional input-output connectivity to the aVLSI CPGs. Figure 8 illustrates an experiment activating walking commands using simulated input from the robotic antennae. The command and inter-neuronal network is indicated in the upper panel. In the lower left panel, lateral surge activates rotational walking where in the lower right panel, off centre axial surge activates a yawing turn into the surge. Further layers of such exteroceptive reflexes incorporating optical flow, bump, gravitational and perhaps chemosensory sensors can realise an elementary brain with capabilities for behavioural choice and sequencing.

Conclusion
The common bridge between the analog VLSI CPGs and the DTM neurons is the command neuron voltages (Figure 6d). If these are generated through D/A converters interfaced to a serial to analog chip they can be used to connect the 'brain' (Figure 8) to the segmental CPGs ( Figure 6). This capability demonstrates that the instantiation of a hybrid electronic nervous system for adaptive behaviour is both feasible and capable of control of a self-contained autonomous robot.