Abstract
Mathematical principles of reinforcement were developed in order to (1) account for the interaction of target responding and other behavior; (2) provide a simple graphical representation; (3) deal with measurement artifacts; and (4) permit a coherent transition from a statics to a dynamics of behavior. Rats and pigeons were trained to make a target response while general activity was measured with a stabilimeter. The course of behavioral change was represented as a trajectory through a two-dimensional behavior space. The trajectories rotated toward or away from the target dimension as the coupling between the target response and the incentive was varied. Higher rates of reinforcement expanded the trajectories; satiation and extinction contracted them. Concavity in some trajectories provided data for a dynamic generalization of the model.
Article PDF
Similar content being viewed by others
References
Allison, J. (1981). Economics and operant conditioning. In P. Harzem & M. D. Zeiler (Eds.),Advances in the analysis of behavior: Vol. 2, Predictability, correlation, and contiguity (pp. 321–353). Chichester, U.K.: Wiley.
Allison, J. (1983).Behavioral economics. New York: Praeger.
Allison, J. (1993). Response deprivation, reinforcement, and economics.Journal of the Experimental Analysis of Behavior,60, 129–140.
Amsel, A., &Rashotte, M. E. (1984).Mechanisms of adaptive behavior: Clark L. Hull’s theoretical papers, with commentary. New York: Columbia University Press.
Anderson, M. C., &Shettleworth, S. J. (1977). Behavioral adaptation to fixed-interval and fixed-time food delivery in golden hamsters.Journal of the Experimental Analysis of Behavior,25, 33–49.
Arabie, P. (1991). Was Euclid an unnecessarily sophisticated psychologist?Psychometrika,56, 567–587.
Baum, W. M. (1974). On two types of deviation from the matching law: Bias and undermatching.Journal of the Experimental Analysis of Behavior,22, 231–242.
Bizo, L. A.,Bogdanov, S. V., &Killeen, P. R. (in press). Satiation causes within-session decreases in instrumental responding.Journal of Experimental Psychology: Animal Behavior Processes.
Bizo, L. A., &White, K. G. (1994). Pacemaker rate in the behavioral theory of timing.Journal of Experimental Psychology: Animal Behavior Processes,20, 308–321.
Bouton, M. E. (1993). Context, time, and memory retrieval in extinction and the interference paradigms of Pavlovian learning.Psychological Bulletin,114, 80–99.
Bouton, M. E. (1994). Conditioning, remembering, forgetting.Journal of Experimental Psychology: Animal Behavior Processes,20, 219–231.
Brodbeck, D. R. (1997). Picture fragment completion: Priming in the pigeon.Journal of Experimental Psychology: Animal Behavior Processes,23, 461–468.
Buckner, R. L., Green, L., &Myerson, J. (1993). Short-term and long-term effects of reinforcers on choice.Journal of the Experimental Analysis of Behavior,59, 293–307.
Catania, A. C., &Reynolds, G. S. (1968). A quantitative analysis of the responding maintained by interval schedules of reinforcement.Journal of the Experimental Analysis of Behavior,11, 327–383.
Davison, M., &Jenkins, P. E. (1985). Stimulus discriminability, contingency discriminability, and schedule performance.Animal Learning & Behavior,13, 77–84.
Davison, M., &Jones, B. M. (1995). A quantitative-analysis of extreme choice.Journal of the Experimental Analysis of Behavior,64, 147–162.
Forster, F. C. (1995). Prior and concurrent learning increases object exploration in the rat (Rattus-norvegicus).Australian Journal of Psychology,47, 147–151.
French, A. P., &Kennedy, P. J. (Eds.) (1985).Niels Bohr: A centenary volume. Cambridge, MA: Harvard University Press.
Gallo, A., Duchatelle, E., Elkhessaimi, A., Lepape, G., &Desportes, J. P. (1995). Topographic analysis of the rat’s bar behaviour in the Skinner box.Behavioural Processes,33, 319–327.
Gibbon, J. (1995). Dynamics of time matching: Arousal makes better seem worse.Psychonomic Bulletin & Review,2, 208–215.
Harper, D. N., &McLean, A. P. (1992). Resistance to change and the law of effect.Journal of the Experimental Analysis of Behavior,57, 317–337.
Herrnstein, R. J. (1979). Derivatives of matching.Psychological Review,86, 486–495.
Hineline, P. N. (1978). Warmup in avoidance as a function of time since prior training.Journal of the Experimental Analysis of Behavior,29, 87–103.
Hogan, J. A. (1997). Energy models of motivation: A reconsideration.Applied Animal Behavior Science,53, 89–105.
Hull, C. L. (1943).Principles of behavior. New York: Appleton-Century-Crofts.
Killeen, P. R. (1975). On the temporal control of behavior.Psychological Review,82, 89–115.
Killeen, P. R. (1978). Superstition: A matter of bias, not detectability.Science,199, 88–90.
Killeen, P. R. (1992). Mechanics of the animate.Journal of the Experimental Analysis of Behavior,57, 429–463.
Killeen, P. R. (1994a). Mathematical principles of reinforcement.Behavioral & Brain Sciences,17, 105–135.
Killeen, P. R. (1994b). Rats, responses and reinforcers: Using a little psychology on our subjects.Behavioral & Brain Sciences,17, 157–172.
Killeen, P. R. (1995). Economics, ecologics, and mechanics: The dynamics of responding under conditions of varying motivation.Journal of the Experimental Analysis of Behavior,64, 405–431.
Killeen, P. R. (1998). The first principle of reinforcement. In C. Wynne & J. E. R. Staddon (Eds.),Models of action (pp. 127–156). Mahwah, NJ: Erlbaum.
Killeen, P. R., Fetterman, J. G., &Bizo, L. A. (1997). Time’s causes. In C. M. Bradshaw & E. Szabadi (Eds.),Time and behaviour: Psychological and neurobiological analyses (pp. 79–131). Amsterdam: Elsevier.
Killeen, P. R., Hanson, S. J., &Osborne, S. R. (1978). Arousal: Its genesis and manifestation as response rate.Psychological Review,85, 571–581.
Locurto, C. M., Terrace, H. S., &Gibbon, J. (Eds.) (1981).Autoshaping and conditioning theory. New York: Academic Press.
Mazur, J. E. (1996). Past experience, recency, and spontaneous recovery in choice behavior.Animal Learning & Behavior,24, 1–10.
Myerson, J., &Hale, S. (1988). Choice in transition: A comparison of melioration and the kinetic model.Journal of the Experimental Analysis of Behavior,49, 291–302.
Myerson, J., &Miezin, F. M. (1980). The kinetics of choice: An operant systems analysis.Psychological Review,87, 160–174.
Nevin, J. A. (1988). Behavioral momentum and the partial reinforcement effect.Psychological Bulletin,103, 44–56.
Nevin, J. A. (1992). An integrative model for the study of behavioral momentum.Journal of the Experimental Analysis of Behavior,57, 301–316.
Nevin, J. A., Tota, M., Torquato, R. D., &Shull, R. L. (1990). Alternative reinforcement increases resistance to change: Pavlovian or operant contingencies?Journal of the Experimental Analysis of Behavior,53, 359–379.
Reid, A. R., &Dale, R. H. I. (1985). Dynamic effects of food magnitude on interim-terminal interaction.Journal of the Experimental Analysis of Behavior,39, 135–148.
Schwendiman, J. W. (1997, May).Extreme choice: Contingency-discriminability or generalized matching? Poster presented at the Society for Quantitative Analysis of Behavior, Chicago.
Seligman, M. E. P. (1970). On the generality of the laws of learning.Psychological Review,77, 406–418.
Silva, F. J., &Pear, J. J. (1995). Stereotypy of spatial movements during noncontingent and contingent reinforcement.Animal Learning & Behavior,23, 245–255.
Skinner, B. F. (1938).The behavior of organisms. New York: Appleton-Century-Crofts.
Skinner, B. F. (1948). Superstition in the pigeon.Journal of Experimental Psychology,38, 168–172.
Staddon, J. E. R. (1977). On Herrnstein’s equation and related forms.Journal of the Experimental Analysis of Behavior,28, 163–170.
Staddon, J. E. R. (1979). Operant behavior as adaptation to constraint.Journal of Experimental Psychology: General,108, 48–67.
Staddon, J. E. R. (1988). Quasi-dynamic choice models: Melioration and ratio invariance.Journal of the Experimental Analysis of Behavior,49, 383–405.
Staddon, J. E. R., &Simmelhag, V. (1971). The “superstition” experiment: A re-examination of its implications for the principles of adaptive behavior.Psychological Review,78, 3–43.
Ten Cate, C., &Ballintijn, M. (1996). Dove coos and flashed lights: Interruptibility of “song” in a nonsongbird.Journal of Comparative Psychology,110, 267–275.
Timberlake, W. (1993). Behavior systems and reinforcement: An integrative approach.Journal of the Experimental Analysis of Behavior,60, 105–128.
Timberlake, W. (1994). Behavior systems, associationism, and Pavlovian conditioning.Psychonomic Bulletin & Review,1, 405–420.
Timberlake, W., &Silva, F. J. (1994). Observation of behavior, inference of function, and the study of learning.Psychonomic Bulletin & Review,1, 73–88.
Wasserman, E. A., &Miller, R. R. (1997).What’s elementary about associative learning? Annual Review of Psychology (Vol. 48, pp. 573–607). Palo Alto, CA: Annual Reviews.
Wearden, J. H. (1994). Fifty years on: The new “principles of behavior”?Behavioral & Brain Sciences,17, 155.
Weingarten, H. P., Duong, A., &Elston, D. (1996). Interpretation of sham feeding data: Curve-shift studies.American Journal of Physiology: Regulatory Integrative & Comparative Physiology,40, R1009-R1016.
White, N. F., &Milner, P. M. (1992). The psychobiology of reinforcers.Annual Review of Psychology,43, 443–471.
Zeiler, M. D. (1996). What behavers do.Behavioral & Brain Sciences,19, 549–550.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by NSF Grants IBN-9408022 and BNS 9021562 and by NIMH Award MH01293 K05. We thank Geof White, John Wixted, and an anonymous reviewer for their generous help in configuring the manuscript.
Rights and permissions
About this article
Cite this article
Killeen, P.R., Bizo, L.A. The mechanics of reinforcement. Psychonomic Bulletin & Review 5, 221–238 (1998). https://doi.org/10.3758/BF03212945
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03212945