Elsevier

European Economic Review

Volume 61, July 2013, Pages 186-204
European Economic Review

Learning, forecasting and optimizing: An experimental study

https://doi.org/10.1016/j.euroecorev.2013.04.003Get rights and content

Highlights

  • We experimentally study whether and how agents learn a rational expectations equilibrium.

  • The learning problem is decomposed into forecasting or optimizing tasks or both.

  • We also consider the performance of individuals versus teams of individuals.

  • Learning is fastest in the forecasting task and slowest in the forecasting and optimizing task.

  • Teams outperform individuals in the combined forecasting and optimizing task.

Abstract

Rational Expectations (RE) models have two crucial dimensions: (i) agents on average correctly forecast future prices given all available information, and (ii) given expectations, agents solve optimization problems and these solutions in turn determine actual price realizations. Experimental tests of such models typically focus on only one of these two dimensions. In this paper we consider both forecasting and optimization decisions in an experimental cobweb economy. We report results from four main experimental treatments: (1) subjects form forecasts only, (2) subjects determine quantity only (solve an optimization problem), (3) they do both and (4) they are paired in teams and one member is assigned the forecasting role while the other is assigned the optimization task. All treatments converge to Rational Expectation Equilibrium (REE), but at different speeds. We observe that performance is the best in treatment 1 and the worst in Treatment 3. We further find that most subjects use adaptive rules to forecast prices. Given a price forecast, subjects are less likely to make conditionally optimal production decisions in Treatment 3 where the forecast is made by themselves, than in Treatment 4 where the forecast is made by the other member of their team, which suggests that “two heads are better than one” in term of the speed of finding the REE.

Introduction

Rational Expectations (RE) macroeconomic models have two crucial dimensions: (i) Rational agents on average correctly forecast future prices given all available information, that is, they do not make systematic forecast mistakes; (ii) Given agents' rational expectations, these same agents solve optimization problems that determine their consumption and/or production decisions, which then, via market clearing, determine the realizations of prices and wages the agents were seeking to forecast; these data are then used to update forecasts. Thus, RE systems are self-referential; beliefs affect outcomes and outcomes affect beliefs.

Testing rational expectation models with field data is problematic as agents' expectations are not generally observable and economists may disagree as to what constitutes the “true” model in which agents' expectations are formed. An alternative approach is to test rational expectations models in the laboratory where it is possible to control the model that determines economic data and to elicit and use agents' expectations of future variables in the determination of that same data. However, the self-referential nature of RE models makes it difficult to test these models in the laboratory. As Sargent (2008) observes:

“Laboratory experiments using macroeconomics are rarer than those using microeconomics...I suspect that the main reason for fewer experiments in macro than in micro is that the choices confronting artificial agents within even one of the simpler recursive competitive equilibria used in macroeconomics are very complicated relative to the settings with which experimentalists usually confront subjects.”

Experimentalists seeking to test RE macroeconomic models have dealt with the complicated nature of these models by reducing the dimensionality of the problem that subjects face. Two approaches have been taken.

In a “learning to forecast experiment,” (LtFE) – a design first proposed by Marimon and Sunder (1993) – subjects are asked to submit a forecast for a future economic variable (e.g., a price, inflation rate, foreign exchange rate, etc.), and they are rewarded solely on the basis of the ex-post accuracy of their forecast. Their forecast is used as an input by a computer program to determine each individual's optimal quantities as if the subjects themselves were capable of solving the optimization problem conditional on their forecast. The computer-determined quantities together with market clearing conditions then determine the actual price realizations (the object of the subjects' forecasts), and these realizations are then used to assess the accuracy of the subjects' forecasts. Subjects, however, are not necessarily made aware of how their forecasts affect outcomes; the mechanism by which subjects' forecasts determine the actual realizations of forecasted variables often amounts to a “black-box” process.

In a second, older experimental approach, known as a “learning to optimize experiment” (LtOE), subjects are asked to make economic decisions (to consume, invest, trade, produce, etc.) directly, without any elicitation of their forecasts of the relevant endogenous variables such as the market price, interest rate or wages. (e.g., Arifovic, 1996, Smith et al., 1988) Of course, such forecasts can be determined implicitly based on subjects' decisions or are sometimes determined separately via some market mechanism (e.g., a double auction or a call market) that is often external to the theory being tested.

Studies using the LtFE approach find mixed evidence as to whether subjects are able to learn a rational expectations equilibrium (REE) (see, e.g., Hommes, 2011 for a survey). In some instances, subjects learn a REE via some adaptive learning process while in other instances subjects behave as trend extrapolators resulting in persistent deviations or cycles around the rational expectations equilibrium (e.g., Anufriev and Hommes, 2012). Similarly, findings from LtOE studies have sometimes confirmed competitive equilibrium predictions and associated comparative statics predictions, but in other instances have generated outcomes that are at odds with RE model predictions, for instance, non-rational bubbles, excess volatility, etc.

In this paper we compare the LtFE and LtOE approaches in a common, economic decision-making task. Importantly, we also consider how behavior improves or deteriorates if we combine these two approaches. Our combined LtFE and LtOE design gets at the heart of the belief–outcome interaction that is the signature property of rational expectations models. We ask if convergence to the REE and efficiency are affected when subjects are asked to play both roles as forecaster and optimizer or if specialization of tasks by individuals alone (as in LtFE and LtOE designs) or within two-agent teams leads to a significant improvement in performance. One aim of this research is to assess whether the results obtained in the LtFE literature are robust when the optimization task is performed by an individual rather than by a computer program. Moreover, our novel team specialization treatment has a very natural, real-world interpretation: Organizational investors such as investment banks and pension funds usually employ both professional forecasters (researchers and economists) and production managers or traders. This type of team specialization set-up has not been previously explored in the laboratory.

The experimental environment we study is a simple, N-firm cobweb model economy—a negative expectation feedback system. This kind of feedback system arises naturally in commodity markets that were the inspiration for Ezekiel's (1938) development of the cobweb model. Furthermore, Muth (1961) proposed rational expectations in the context of this very same negative feedback cobweb model. Prior research indicates that under a LtFE design, market prices will converge very quickly to the RE equilibrium in this environment. In addition to LtFE, we consider three additional treatments where subjects must submit their production decision directly without a forecast (LtOE), or together with a forecast, or subjects are paired in teams and one team member submits a forecast which the other team member can use to determine a production decision.

We find a tendency for the market price to converge to the REE price in all four treatments. Thus, the stabilizing effect of a negative feedback market is a robust feature of our experiment. However, when the speed of convergence is compared across treatments, we find that the market price converges most quickly and reliably when subjects only make price forecasts as in the computer-aided LtFE design. There is not much difference in performance between the treatment where subjects only make production decisions (LtOE) and the treatment where they form teams that specialize in one of the two tasks. However, the market price and quantity fluctuate the most and are the slowest to converge when subjects are required to perform both forecasting and production decision-making (optimizing) tasks. Our findings have important implications not only for the design of experiments, but more importantly for how we might think about the representative agent firm: should it be viewed as an individual actor (e.g., the C.E.O.) or is it better to think of the representative firm as consisting of teams of individuals specialized in various tasks, such as forecasting and production? Further, our decomposition of the forecasting and optimization tasks suggests that bounded rationality with respect to optimization decisions appears to be as important a consideration in the learning of rational expectations equilibria as is bounded rationality in expectation formation. Nevertheless, most research on learning in macroeconomics (e.g., Evans and Honkapohja, 2001) has focused only on bounded rationality with respect to expectation formation and not with respect to optimizing behavior.

The rest of the paper is organized as follows: Section 2 discusses the related literature. Section 3 describes our experimental design. Section 4 presents the experimental results. Finally, Section 5 concludes.

Section snippets

Related Literature

Our work is related to prior studies using either the LtFE or LtOE designs. Smith et al. (1988), Lim et al. (1994), Arifovic (1996), Lei et al. (2001), Noussair et al. (2007) and Crockett and Duffy (2012) are some examples of LtOE studies. Adam (2007), Marimon et al. (1993), Marimon and Sunder, 1993, Marimon and Sunder, 1994, Hommes et al., 2005, Hommes et al., 2007, Heemeijer et al. (2009) and Bao et al. (2012) are some representative studies using the LtFE design.

As we also have a treatment

Theoretical model

The model behind our experiment is one of demand and supply for a single, non-storable good. Denote by D the non-negative and monotonically decreasing aggregate demand function for this good and by Sh,t, the non-negative supply function for the good by firm h, derived from expected profit maximization at period t. Let ph,te be the price forecast made by firm h in period t. The supply function may be rewritten as S(ph,te). We assume that all firms have the same supply function. Subjects play

Aggregate market price

Fig. 2 plots the average market prices in each treatment against the REE price, p(⁎)=30.73. This figure reveals that the average price in all five treatments gets very close to the REE price, especially in the later periods of the experiment. Thus, the general tendency for a negative feedback system to converge to REE is not greatly affected by the type of task (forecasting or optimizing or both) that is assigned to market participants. However, the adjustment toward the REE at the beginning of

Conclusion

Rational Expectations (RE) macro models have two crucial dimensions: (i) Agents correctly forecast future prices using all available information (i.e., they do not make systematic mistakes) and (ii) Given these expectations, agents solve optimization problems and their solutions (together with market clearing conditions) then determine actual price realizations, that is, there is belief–outcome interaction. These two important dimensions are difficult to simultaneously observe in field data,

Acknowledgments

The authors thank the editor Jörg Oechssler and two anonymous referees for helpful comments. We are grateful to Jasmina Arifovic, Dirk Engelmann, George Evans, Ernst Fehr, Theo Offerman, Andrew Schotter, Lise Vesterlund, Stephanie Wang, seminar participants at the University of Pittsburgh, CESS-CREED Workshop at New York University, the Amsterdam Symposium on Behavioral and Experimental Economics, RES 2012 Annual Conference at the University of Cambridge, 3rd LICTEM conference at Universitat

References (39)

  • M. Anufriev et al.

    Evolutionary selection of individual expectations and aggregate outcomes

    American Economic JournalMicro

    (2012)
  • J. Arifovic

    The behavior of the exchange rate in the genetic algorithm and experimental economies

    Journal of Political Economy

    (1996)
  • M. Blanco et al.

    Belief elicitation in experimentsis there a hedging problem?

    Experimental Economics

    (2010)
  • A.S. Blinder et al.

    Are two heads better than one? Monetary policy by committee

    Journal of Money, Credit and Banking

    (2005)
  • G. Bornstein et al.

    Individual and group behavior in the ultimatum gameare groups more ‘rational’ players?

    Experimental Economics

    (1998)
  • T. Cason et al.

    A laboratory study of group polarization in the team dictator game

    Economic Journal

    (1997)
  • D.J. Cooper et al.

    Are two heads better than one? Team versus individual play in signaling games

    American Economic Review

    (2005)
  • J.C. Cox

    Trust, reciprocity, and other-regarding preferencesgroups vs. individuals and males vs. females

  • Crockett, S., Duffy, J., 2012. A dynamic general equilibrium approach to asset pricing experiments. Working...
  • Cited by (0)

    View full text