Effects of temporal clumping and payoff accumulation on impulsiveness and cooperation

Abstract

Animals show impulsiveness when they prefer a smaller more immediate option, even though a larger more delayed option produces a higher intake rate. This impulsive behavior has implications for several behavioral problems including social cooperation. This paper presents two experiments using captive blue jays (Cyanocitta cristata) that consider the effects of payoff accumulation and temporal clumping on impulsiveness and cooperation. Payoff accumulation refers to a situation where the benefits gained from each choice trial accumulate from one trial to the next, and only become available to the animal after it has completed a fixed number of trials. We hypothesized that this would reduce impulsiveness because it removes the advantage of quickly realizing food gains. Clumping refers to situation in which the animal experiences several choice trials in quick succession followed by a long pause before the next clump. We hypothesized that if payoffs accumulated over a clump of trials this would enhance the effect of accumulation. We tested the effects of accumulation and clumping on impulsiveness in a self-control situation. We found a significant interaction between clumping and accumulation. Payoff accumulation reduced impulsiveness, but only when trials were clumped. Post hoc analyses suggest that clumping alone increases impulsiveness. A second experiment applied these results to cooperation. This experiment reveals an interaction between payoff accumulation and trial's position within the clump. Jays were more likely to cooperate on the first trial of a clump, but the likelihood of cooperation dropped after the first trial. However, this drop was larger when payoffs did not accumulate. This observation suggests that the difference between accumulated and unaccumulated treatments that we reported previously (Science 198: 2216–2218) may be largely due to differences in how animals behave in the first trial of a clump.

Introduction

Animals show impulsiveness when they prefer an immediate food reinforcer even though a more delayed alternative yields a higher rate of food intake. This preference for immediacy could have important implications for behavioral models of cooperation, like the famous Iterated Prisoner's Dilemma, which focus on situations in which defecting (that is, not cooperating) leads to an immediate gain. Indeed some workers have argued that exhibiting self-control rather than impulsiveness is almost identical to cooperating rather than defecting (Platt, 1973, Rachlin, 2000, Rachlin, 2002). To cooperate in situations like the Prisoner's Dilemma, animals must forego the immediate temptation to defect in anticipation of a larger stream of gains in the long run. Since, by definition, impulsive animals prefer immediate rewards, strong impulsiveness may present a significant barrier to cooperation.

Beyond this theoretical link, there is an intriguing empirical parallel between impulsiveness and cooperation. In both topics, human and non-human results disagree. Experiments suggest that non-human animals are much more impulsive than humans (Mazur, 1987, Rachlin et al., 1991, Richards et al., 1997), and much less likely to cooperate in experimental games (Clements and Stephens, 1995, Sally, 1995). This correlation agrees with our predictions: subjects that are more impulsive should be less cooperative (see Harris and Madden, 2002 for evidence of this correlation in human subjects). However, we must view this correlation with some caution because human and non-human studies use quite different procedures (Jackson and Hackenberg, 1996).

Our goal was to investigate the idea that strong impulsiveness in animals prevents cooperation in experimental Prisoner's Dilemmas. To test this, one would like some manipulation or treatment that switches impulsiveness off. With such a treatment, we should be able to show that animals cooperate in a Prisoner's Dilemma when we experimentally “switch off” their impulsiveness, but defect otherwise. The difficulty here is that we know relatively little about the causes of impulsiveness, although the literature gives several examples of impulsiveness varying across contexts in both humans and non-humans (Mazur, 1994, Mischel et al., 1989, Stephens and Anderson, 2001, Stephens and McLinn, 2003, Wilson and Daly, 2004). A manipulation that should, in theory, have a large effect on impulsiveness is payoff accumulation (Stephens, 2000).

In a typical choice experiment, an animal makes a choice and receives the benefits of its decision, and this pattern of choice and reward continues over many trials. Imagine instead that the benefits of each choice accumulate over a sequence of trials, becoming available for collection only after the subject completes four trials (for example). If impulsiveness occurs because more immediate reinforcers are more valuable, then payoff accumulation should reduce or eliminate impulsiveness because it eliminates the advantage of immediacy. There should be no advantage to choosing the more immediate option because the subject must wait for all its accumulated gains. We present two experimental studies that explore the potential of accumulation to influence impulsiveness. The first experiment directly asks whether accumulation reduces impulsiveness, and the second asks whether accumulation increases cooperativeness.

When we say that the bushes in a garden are clumped, we mean that several bushes are clustered together and clearly separated from other clumps. By analogy, this paper refers to events that are clumped in time. When events are clumped in time, groups of events occur in quick succession, but relatively long intervals separate one group from another. We reasoned that if some process created temporal clumps of trials, this would make accumulation more evident to the subjects, and possibly enhance accumulation's effect on impulsivity. The rationale of this claim is that when trials are clumped and accumulated the subject experiences a quick sequence of trials separated by a long interval, collecting reinforcement after every clump of trials. The long gap between clumps should make it easier to recognize clumps of trials, and we reason that it may also make it easier to recognize that payoff accumulation combines the benefits derived from these clumps. In contrast, if accumulated trials were equally spaced in time, this may blur the distinction between one set of accumulated trials and the next, and so possibly reduce the effect of accumulation. Our first experiment factorially combined accumulated and unaccumulated treatments with clumped and unclumped treatments using a conventional impulsiveness test. This experiment offered subjects a choice between a small immediate and a large delayed option using an experimental situation that psychologists call the self-control paradigm.

Experiment 2 applied the results of experiment 1 to cooperation by testing blue jays in an experimental Prisoner's Dilemma with and without accumulation. We reported the main effects of accumulation on cooperation elsewhere (Stephens et al., 2002). Here, we focus on patterns related to clumping that we have not reported previously.