On the joint control of preference by time and reinforcer-ratio variation

doi:10.1016/j.beproc.2013.02.001

Behavioural Processes

Volume 95, May 2013, Pages 100-112

https://doi.org/10.1016/j.beproc.2013.02.001 Get rights and content

Abstract

Five pigeons were trained in a procedure in which, with a specified probability, food was either available on a fixed-interval schedule on the left key, or on a variable-interval schedule on the right key. In Phase 1, we arranged, with a probability of 0.5, either a left-key fixed-interval schedule or a right-key variable-interval 30 s, and varied the value of the fixed-interval schedule from 5 s to 50 s across 5 conditions. In Phase 2, we arranged either a left-key fixed-interval 20-s schedule or a right-key variable-interval 30-s schedule, and varied the probability of the fixed-interval schedule from 0.05 to 1.0 across 8 conditions. Phase 3 always arranged a fixed-interval schedule on the left key, and its value was varied over the same range as in Phase 1. In Phase 1, overall preference was generally toward the variable-interval schedule, preference following reinforcers was initially toward the variable-interval schedule, and maximum preference for the fixed-interval schedule generally occurred close to the arranged fixed-interval time, becoming relatively constant thereafter. In Phase 2, overall left-key preference followed the probability of the fixed-interval schedule, and maximum fixed-interval choice again occurred close to the fixed-interval time, except when the fixed-interval probability was 0.1 or less. The pattern of choice following reinforcers was similar to that in Phase 1, but the peak fixed-interval choice became more peaked with higher probabilities of the fixed interval. Phase 3 produced typical fixed-interval schedule responding. The results are discussed in terms of reinforcement effects, timing in the context of alternative reinforcers, and generalized matching. These results can be described by a quantitative model in which reinforcer rates obtained at times since the last reinforcer are distributed across time according to a Gaussian distribution with constant coefficient of variation before the fixed-interval schedule time, changing to extended choice controlled by extended reinforcer ratios beyond the fixed-interval time. The same model provides a good description of response rates on single fixed-interval schedules.

This article is part of a Special Issue entitled: SQAB 2012.

Section snippets

Subjects

Five pigeons, numbered 161–165, were maintained at 85% ± 15 g of their ad-lib body weights by supplementary feeding of mixed grain at 9.30 am daily. These pigeons had served in a previous experiment (Davison and Elliffe, 2010), but Pigeon 165's data were not reported there due to some problems maintaining its behavior. Grit and water were available in their home cages, in which the experiment was conducted, at all times.

Apparatus and procedure

The apparatus was the same as used by Davison and Elliffe (2010). The

Results

The data used in analyses were from the last 20 sessions of each condition. Generally, response and reinforcer numbers were collected in 2.5-s bins for all conditions except those arranging FI 5 s, for which 1.25-s bins were used. From these data, we calculated local response and reinforcer rates in each bin, and analyzed only data obtained on trials in which VI food deliveries were arranged in Phases 1 and 2. The reason for not using FI-trial data was that, of course, these data were truncated

Some general considerations

There are two ways in which the schedules arranged here can be viewed. First, they can be seen in an extended sense, in which we might expect, according to the generalized matching law, that the extended allocation of behavior between the keys would be a simple linear function of the extended log reinforcer ratio between the keys. In Phase 1, the sessional obtained log reinforcer ratio was close to zero throughout, so this approach would suggest that choice should be close to indifferent and

References (27)

R.M. Church et al.
Temporal memory of interfood interval distributions with the same mean and variance
Learn. Motiv.
(2001)
M. Davison
Interresponse times and the structure of choice
Behav. Process.
(2004)
W.M. Baum
On two types of deviation from the matching law: bias and undermatching
J. Exp. Anal. Behav.
(1974)
W.M. Baum et al.
Choice in a variable environment: visit patterns in the dynamics of choice
J. Exp. Anal. Behav.
(2004)
L.A. Bizo et al.
Biasing the pacemaker in the behavioural theory of timing
J. Exp. Anal. Behav.
(1995)
N. Boutros et al.
Examining the discriminative and strengthening effects of reinforcers in concurrent schedules
J. Exp. Anal. Behav.
(2011)
A.C. Catania et al.
A quantitative analysis of responding maintained by interval schedules of reinforcement
J. Exp. Anal. Behav.
(1968)
R.M. Church et al.
Temporal search as a function of the variability of interfood intervals
J. Exp. Psychol. Anim. Behav. Process.
(1998)
S. Cowie et al.
Reinforcement: food signals the time and location of future food
J. Exp. Anal. Behav.
(2011)
M. Davison et al.
Choice in a variable environment: effects of blackout duration and extinction between components
J. Exp. Anal. Behav.
(2002)

M. Davison et al.

Do conditional reinforcers count?

J. Exp. Anal. Behav.

(2006)

M. Davison et al.

Divided stimulus control: a replication and a quantitative model

J. Exp. Anal. Behav.

(2010)

M. Davison et al.

Stimulus discriminability, contingency discriminability, and schedule performance

Anim. Learn. Behav.

(1985)

Cited by (15)

Choosing a future from a murky past: A generalization-based model of behavior
2022, Behavioural Processes
Citation Excerpt :
Here, we outline how a quantitative approach might be used to explore how generalization affects the way we navigate our environment. The development of such an approach has provided the foundation for a range of research in the past (e.g., Cowie et al., 2013, 2014; Davison et al., 2013; Davison and Cowie, 2019; Cowie and Davison, 2020a, 2020b); this paper might provide direction for future research that seeks to understand how past experience guides behavior in the present. The generalization modeling approach (see also Cowie et al., 2016) uses reinforcers obtained in the past to predict behavior in similar stimulus contexts.
Remembering the past appears critical in allowing organisms to detect order in an environment, and hence to behave in accordance with likely future events. Yet the shortcomings of remembering and perceiving typically mean that the remembered past differs from the actual past, and hence that behavior does not perfectly track the structure of the environment. Here, we outline how the process of generalization might be used to understand differences between what an organism does, and the structure of the past and potential structure of the environment. We explore how different sources of generalization – both from within the same stimulus situation, and from different stimulus situations – might be modeled quantitatively, and how predictions made by this modeling approach are supported by research. Finally, we discuss how generalization from multiple stimulus situations, longer-term experience, and from stimulus situations in the past that are not identical to the stimulus situation in the present, might contribute to our understanding of how an organism’s experience translates into behavior.
The effects of changeover delays on local choice
2018, Behavioural Processes
Citation Excerpt :
These local-level effects of the COD on responding appear to be partly responsible for the difference between overall choice in concurrent schedules with and without a COD (e.g., Dreyfus et al., 1982; Pliskoff et al., 1978; Shahan and Lattal, 1998; Silberberg and Fantino, 1970). In addition to creating a change in reinforcer availability after switches, the COD alters local reinforcer contingencies after reinforcer deliveries (Cowie et al., 2011; see also Boutros et al., 2009, 2011; Cowie and Davison, 2016; Davison et al., 2013). After a reinforcer delivery, switching to the other alternative (the not-just-productive alternative) initiates the COD, and hence reinforcers are not available until the COD ends.
In concurrent schedules with a changeover delay (COD), choice often strongly favours the just-reinforced alternative immediately after a reinforcer delivery. These ‘preference pulses’ may be caused by a change in reinforcer availability created by the COD, and/or because the COD decreases the overall probability of switching. We investigated which explanation better accounts for preference pulses by arranging concurrent schedules that allowed us to separate the COD’s effects on reinforcer availability from its effects on the probability of switching. When the reinforcer ratio was 1:1, pulses were inconsistently accompanied by changes in reinforcer availability, but consistently accompanied by longer visits. These pulses appeared to be related only to the decreased probability of switching caused by the COD, providing the first evidence of pulses after reinforcers caused by the probability of switching alone. When the reinforcer ratio was 1:5 or 5:1; preference pulses were accompanied by changes in reinforcer availability and by longer visits. These pulses appeared to be related to the COD’s effects on reinforcer availability, although a small portion appeared to be related to low probability of switching. These findings suggest that the COD affects preference pulses by both decreasing the probability of switching and creating a change in reinforcer availability.
Reinforcer distributions affect timing in the free-operant psychophysical choice procedure
2016, Learning and Motivation
Citation Excerpt :
While LeT may be modified so that the speed of the pacemaker is not affected by overall density (see Machado & Guilhardi, 2000), consistent with the present findings, such a modification does not aid the model in predicting the effect of the distribution of reinforcers across time on discrimination. An alternative approach is to consider temporal-discrimination performance as choice under stimulus control by elapsed time (e.g., Cowie et al., 2013; Cowie, Davison, & Elliffe, 2014; Davison et al., 2013). Where LeT would predict that a behavioral state occasions a timing response, a stimulus-control approach would assume that elapsed time functions as a discriminative stimulus signaling the response more likely to produce a reinforcer.
In procedures used to study timing behavior, the availability of reinforcement changes according to time since an event. Manipulation of this reinforcer differential often produces violations of scalar timing, but it is unclear whether such effects arise because of a response bias or a change in temporal discrimination. The present experiment investigated the effects of the overall and relative probability of obtaining a reinforcer on performance in the free-operant psychophysical procedure. We arranged short and long trials with unequal reinforcer ratios, at high or low overall reinforcer rates. Changes in the overall reinforcer rate produced only small changes in timing behavior. Changes in relative reinforcer probability, which caused differences in the likely availability of reinforcers across time within a trial, produced a change in both bias and discrimination. We suggest reinforcers affect timing, and that discrimination in timing tasks depends on the distribution of reinforcers in time, as well as on the interval to be timed.
Being there on time: Reinforcer effects on timing and locating
2020, Journal of the Experimental Analysis of Behavior
Some weaknesses of a response-strength account of reinforcer effects
2020, European Journal of Behavior Analysis
Timing or counting? Control by contingency reversals at fixed times or numbers of responses
2019, Journal of Experimental Psychology: Animal Learning and Cognition

View all citing articles on Scopus

^☆: We thank the students enrolled in Psych 309 Learning in 2010 and 2011, who conducted this experiment as their laboratory requirement, supervised by Sarah Cowie. We also thank the members of the Experimental Analysis of Behaviour Research Unit who helped run this experiment out of term time, and Mick Sibley who looked after the pigeons. The research was approved by the University of Auckland Animal Ethical Committee (Approval TR-909). Reprints may be obtained from Michael Davison (email: [email protected]).

View full text

On the joint control of preference by time and reinforcer-ratio variation☆

Abstract

Section snippets

Subjects

Apparatus and procedure

Results

Some general considerations

Learn. Motiv.

Behav. Process.

On two types of deviation from the matching law: bias and undermatching

J. Exp. Anal. Behav.

Choice in a variable environment: visit patterns in the dynamics of choice

J. Exp. Anal. Behav.

Biasing the pacemaker in the behavioural theory of timing

J. Exp. Anal. Behav.

Examining the discriminative and strengthening effects of reinforcers in concurrent schedules

J. Exp. Anal. Behav.

A quantitative analysis of responding maintained by interval schedules of reinforcement

J. Exp. Anal. Behav.

Temporal search as a function of the variability of interfood intervals

J. Exp. Psychol. Anim. Behav. Process.

Reinforcement: food signals the time and location of future food

J. Exp. Anal. Behav.

Choice in a variable environment: effects of blackout duration and extinction between components

J. Exp. Anal. Behav.

Do conditional reinforcers count?

J. Exp. Anal. Behav.

Divided stimulus control: a replication and a quantitative model

J. Exp. Anal. Behav.

Stimulus discriminability, contingency discriminability, and schedule performance

Anim. Learn. Behav.