Abstract
After the aims and processes, prediction in economics requires the analysis of the issues on evaluation and limits. Chapter 12 begins with the use of prediction as a test, both in economic theory and in applied economics. The evaluation of predictions in the context of economic models is considered taking into account the problem of uncertainty and, consequently, forecast uncertainty. The appraisal of economic predictions is focused on the criteria of prediction as a test, which follows several steps: (a) main criteria in the appraisal of predictions; (b) methodological processes to the assessment of predictions (i.e., different kinds of testing); (c) the case of econometrics (as a tertium quid between laboratory experimentation and thought experiments); and (d) the existence of predictive errors and their economic costs. The limits and obstacles of prediction in economics are also studied: the limits of predictability—both epistemological and ontological—and the obstacles to predictors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
It was not always the case that prediction was a main tool for evaluation, at least in the field of econometrics: it was in the mid-1930s that “economists had began to use economic forecasting for testing their models: Tinbergen (1939) was one of those responsible for developing forecasting tests of econometric models, partly in response to the criticism of Keynes (1939) and Frisch (1938)” (Clements and Hendry 1998, p. 7).
- 3.
“The view that the worth of a theory is to be judged solely by the extent and accuracy of its predictions seems to me wrong. (…) We are not interested simply in the accuracy of its predictions. A theory also serves as a base for thinking. It helps us to understand what is going on by enabling us to organise our thoughts. Faced with a choice between a theory which predicts well but gives us little insight into how the system works and one which gives us this insight but predicts badly, I would choose the latter, and I am inclined to think that most economists would do the same” (Coase 1994b, pp. 16–17).
- 4.
This content is the 1991 Alfred Nobel Memorial Prize lecture in economic sciences.
- 5.
On the special difficulties that arise in testing and appraising mainstream economic theory, cf. Hausman (1998b).
- 6.
According to Michael Evans, who is thinking of business forecasting, there are many ways in which the economic forecast can be presented in applied economics: point estimate or interval; absolute or conditional; alternative scenarios weighed by probabilities; asymmetric gains and losses; single-period or multi-period; short run or long range; forecasting single or multiple variables (Evans 2013, esp., Sect. 1.3, pp. 18–22).
- 7.
“Taking energy forecasting as an example, all of the following are possible components: (a) energy use for industrial, commercial, and residential purposes; (b) energy use per unit of each component of the GNP; (c) energy use per mode or form of energy (electricity, fossil fuels, etc.); and (d) energy use per capita, accompanied by a population projection” (Ascher 1978, p. 9).
- 8.
Fernández-Jardón, C., Personal communication, 27 January 2014.
- 9.
- 10.
For Makridakis, Wheelwright, and Hyndman, “setting uncertainty at realistic levels, separating objective predictions from wishful thinking or the attainment of desired objectives, and realizing that unusual, threatening events have occurred and will continue to do so in the future are critical aspects that must be dealt with while forecasting” (1998, pp. 552–553).
- 11.
Fernández-Jardón, C., Personal communication, 23 February 2014.
- 12.
On the origin of predictable behavior, see Heiner (1983).
- 13.
Rescher has pointed out that, in some cases, too much information on something specific—such as a concrete event—makes thing harder for prediction: “the access of further information can sometimes make the future less predictable” (1998, p. 58).
- 14.
Central Banks are among those institutions concerned with this issue, cf. Hatch (2001). The Bank of England uses “fan charts” in this regard.
- 15.
The debate has followed upon quite different lines, according to the diverse views on probability.
- 16.
It is noticeable that Michael Dummett, who has developed an anti-realist semantics, where the notion of “proof”—a justified assertion—has more credentials than the concept of “truth,” has written that “the defenders of truth-conditional theories of meaning are thus right to argue that the concept of truth is indispensable. The concept of truth is the pivot about which a theory of meaning is brought to bear on metaphysics. Metaphysics is concerned with the general nature of reality, and, as the opening remarks of the Tractatus state, reality is constituted not by the totality of objects that exists but by the totality of facts that obtain. Facts are true propositions: so metaphysics concerns itself with what truths hold good in general” (Dummett 2004, p. 35).
- 17.
Precision is important insofar as prediction cannot remain in the sphere of vague statements (i.e., in the realm of large confidence intervals). But it might be useless if there is a very precise statement whose actual value is outside of the confidence interval studied. Then, the criterion of accuracy is more relevant because combines precision (random errors) and bias (systematic errors) in a single measure. José Ramón Cancelo, Personal communication, January 2007.
- 18.
- 19.
“Negligibility assumptions state that some factor has a negligible effect upon the phenomenon under investigation. Domain assumptions specify the domain of applicability of the theory. Heuristic assumptions are a means of simplifying the logical development of the theory” (Musgrave 1981, p. 386).
- 20.
Even though the origin of this distinction is in the analysis of ceteris paribus conditions, it seems to me that the main differences that are drawn in this differentiation have a rather general consideration for the methodological process.
- 21.
“In the laboratory an artificial economic reality is constructed, for example a market or an auction” (Selten 2003, p. 63).
- 22.
That is the idea of T. Haavelmo, cf. Morgan (1990, p. 245).
- 23.
A methodological question can be considered here: two independent analysts, using the same model, will they introduce identical corrections when the model shows that is not working properly?
- 24.
Popper uses this terminology (1957, p. 13). He gives the name “Oedipus effect” to the influence of an item of information (historical or economical) upon the situation to which the information refers to.
- 25.
- 26.
According to Granger and Machina, “forecasts are invariably subject to error” (2006, p. 89).
- 27.
In the case of econometric models, among the potential sources of error are specification error, conditioning error, sampling error, and random error (Kennedy 1998, p. 289).
- 28.
It is odd that they have included “inflation” and “output” among the stochastic terms, because—in a strict sense—inflation and output are statistical series rather than “stochastic terms.”
- 29.
According to Makridakis et al., “judgmental predictions must supplement the statistical ones when and where they can contribute the most: in identifying forthcoming changes and predicting the direction and extent that they will influence the future so that statistical predictions, which can more objectively and correctly identify and extrapolate established patterns and/or existing relationships, can be appropriately modified” (1998, p. 551).
- 30.
For Rosenberg, “the debate would threaten to be inconclusive, mainly because we have neither a natural unit of predictive power nor a good measuring device to calculate the changes in predictive power that we might expect over a period of a decade or even a century” (1993, p. 164).
- 31.
The methodological limits includes that the methodological universalism is untenable (Gonzalez 2012b). In this regard, it is quite interesting to see that Popper did not assume this idea of “the” scientific method (understood as a systematic way to achieve well founded scientific results), cf. Worrall (2001a, p. 114).
- 32.
An analysis of these issues can be found in Nieto de Alba (1998).
- 33.
Herbert Simon developed an initial attempt, when I explicitly asked him for a paper in this regard. The article was published in a monographic issue that I coordinated on philosophy and methodology of economics: Simon (1998).
- 34.
- 35.
Simon, H. A., Personal communication, 31 July 1996.
- 36.
“Chaos in this sense [‘sensitive dependence on initial conditions’] has been observed in cardiac disorders, turbulence in fluids, electronic circuits, dripping faucets, and many other seemingly unrelated phenomena. These days, the existence of chaotic systems is an accepted fact of science” (Mitchell 2009, p. 20). For a number of years the topic of chaos has received enormous attention, and exceeds the limits of this chapter: Brock (1991), Ruelle (1991), Winnie (1992), Kellert (1993), Batterman (1993), Stewart (1993), etc. Regarding its relation to complexity, besides the book by Smith (1998), see Bertuglia and Vaio (2005), Strevens (2003), and Rosser (2011).
- 37.
References
Abraham, B., and J. Ledolter. 1983/[2005]. Statistical methods for forecasting. New York: Wiley.
Ascher, W. 1978. Forecasting: An appraisal for policy-makers and planners. Baltimore: J. Hopkins University Press.
Batterman, R. W. 1993. Defining chaos. Philosophy of Science 60 (1): 43–66.
Bertuglia, C. S., and F. Vaio. 2005. Nonlinearity, chaos and complexity. The dynamics of natural and social systems. Oxford: Oxford University Press.
Blaug, M. 2002. Ugly currents in modern economics. In Fact and fiction. Models, realism, and social construction, ed. U. Mäki, 35–56. Cambridge: Cambridge University Press.
Boumans, M., and M. S. Morgan. 2001. Ceteris paribus conditions: Materiality and the application of economic theories. Journal of Economic Methodology 8 (1): 11–26.
Brock, W. A. 1991. Causality, chaos, explanation and prediction in economics and finance. In Beyond belief. Randomness, prediction and explanation in science, ed. J. L. Casti and A. Karlqvist, 230–279. Boca Raton: CRC Press.
Buck, R. C. 1963. Reflexive predictions. Philosophy of Science 30:359–369.
Burns, T. 1986. The interpretation and use of economic predictions. In Predictability in science and society, ed. J. Mason, P. Mathias, and J. H. Westcott, 103–125. London: The Royal Society and The British Academy.
Burns, T. 2001. The costs of forecast errors. In Understanding economic forecasts, ed. D. F. Hendry and N. R. Ericsson, 170–184. Cambridge: The MIT Press.
Clements, M. P., and D. F. Hendry. 1998. Forecasting economic time series. Cambridge: Cambridge University Press.
Clements, M. P., and D. F. Hendry. 1999. Forecasting non-stationary economic time series. Cambridge: The MIT Press.
Clements, M. P., and D. F. Hendry. 2002c. Explaining forecast failure in macroeconomics. In A companion to economic forecasting, ed. M. Clements and D. F. Hendry, 539–571. Oxford: Blackwell.
Coase, R. H. 1994a. The institutional structure of production. In Essays on economics and economists, ed. R. H. Coase, 3–14. Chicago: The University of Chicago Press.
Coase, R. H. 1994b. How should economists choose. In Essays on economics and economists, ed. R. H. Coase, 15–33. Chicago: The University of Chicago Press.
Dummett, M. 2004. Truth and the past. New York: Columbia University Press.
Ericsson, N. R. 2001. Forecast uncertainty in economic modeling. In Understanding economic forecasts, ed. D. F. Hendry and N. R. Ericsson, 68–92. Cambridge: The MIT Press.
Ericsson, N. R. 2002. Predictable uncertainty in economic forecasting. In A companion to economic forecasting, ed. M. Clements and D. F. Hendry, 19–44. Oxford: Blackwell.
Evans, M. K. 2013. Practical business forecasting. Oxford: Blackwell.
Fildes, R., and K. Ord. 2002. Forecasting competitions: Their role in improving forecasting practice and research. In A companion to economic forecasting, ed. M. Clements and D. F. Hendry, 322–353. Oxford: Blackwell.
Franses, Ph. H. 2006. Forecasting in marketing. In Handbook of economic forecasting, vol. 1, ed. G. Elliot, C. W. J. Granger, and A. Timmerman, 983–1012. Amsterdam: Elsevier.
Friedman, M. 1953. The methodology of positive economics. In Essays in positive economics, M. Friedman, 3–43. Chicago: University of Chicago Press (6th repr., 1969).
Gonzalez, W. J. 1996b. Prediction and mathematics: The Wittgensteinian approach. In Spanish studies in the philosophy of science, ed. G. Munevar, 299–332. Dordrecht: Kluwer.
Gonzalez, W. J. 2003d. Racionalidad y Economía: De la racionalidad de la Economía como Ciencia a la racionalidad de los agentes económicos. In Racionalidad, historicidad y predicción en Herbert A. Simon, ed. W. J. Gonzalez, 65–96. A Coruña: Netbiblo.
Gonzalez, W. J. 2007a. The role of experiments in the social sciences: The case of economics. In General philosophy of science: Focal issues, ed. T. Kuipers, 275–301. Amsterdam: Elsevier.
Gonzalez, W. J. 2010b. Recent approaches on observation and experimentation: A philosophical-methodological viewpoint. In New methodological perspectives on observation and experimentation in science, ed. W. J. Gonzalez, 9–48. A Coruña: Netbiblo.
Gonzalez, W. J. 2011a. Complexity in economics and prediction: The role of parsimonious factors. In Explanation, prediction, and confirmation, ed. D. Dieks, W. J. Gonzalez, S. Hartman, Th. Uebel, and M. Weber, 319–330. Dordrecht: Springer.
Gonzalez, W. J. 2012b. Methodological universalism in science and its limits: Imperialism versus complexity. In Thinking about provincialism in thinking, Poznan Studies in the Philosophy of the Sciences and the Humanities, vol. 100, ed. K. Brzechczyn and K. Paprzycka, 155–175. Amsterdam: Rodopi.
Granger, C. W. J. 2001. Evaluation of forecasts. In Understanding economic forecasts, ed. D. F. Hendry and N. R. Ericsson, 93–103. Cambridge: The MIT Press.
Granger, C. W. J. 2012. The philosophy of economic forecasting. In Philosophy of economics, ed. U. Mäki, 311–327. Amsterdam: Elsevier.
Granger, C. W. J., and M. J. Machina. 2006. Forecasting and decision theory. In Handbook of economic forecasting, vol. 1, ed. G. Elliot, C. W. J. Granger, and A. Timmerman, 81–98. Amsterdam: Elsevier.
Grünbaum, A. 1956. Historical determinism, social activism and predictions in the social sciences. The British Journal for the Philosophy of Science 7:236–240.
Grünbaum, A. 1963. Comments on Professor Roger Buck’s paper “Reflexive predictions.” Philosophy of Science 30:370–372.
Grunberg, E., and F. Modigliani. 1954. The predictability of social events. Journal of Political Economy 62:465–478.
Hahn, F. 1993. Predicting the economy. In Predicting the future, ed. L. Howe and A. Wain, 77–95. Cambridge: Cambridge University Press.
Hatch, N. 2001. Modeling and forecasting at the bank of England. In Understanding economic forecasts, ed. D. F. Hendry and N. R. Ericsson, 124–148. Cambridge: The MIT Press.
Hausman, D. M. 1997. Theory appraisal in neoclassical economics. Journal of Economic Methodology 4 (2): 289–296.
Hausman, D. M. 1998b. Confirming mainstream economic theory. Theoria 13 (2): 261–278.
Heiner, R. 1983. The origin of predictable behavior. American Economic Review 73:560–595.
Hendry, D. F. 1986. The role of prediction in evaluating economic models. In Predictability in science and society, ed. J. Mason, P. Mathias, and J. H. Westcott, 25–34. London: The Royal Society and The British Academy.
Hendry, D. F. 2000a. Econometrics: Alchemy or science? Essays in econometric methodology (new ed.). Oxford: Oxford University Press (1st ed., 1993).
Hendry, D. F., and N. R. Ericsson. 2001c. Epilogue. In Understanding economic forecasts, ed. D. F. Hendry and N. R. Ericsson, 185–191. Cambridge: The MIT Press.
Hoover, K. D. 2002. Econometrics and reality. In Fact and fiction: Foundational issues on economics and the economy, ed. U. Mäki, 152–177. Cambridge: Cambridge University Press.
Keating, G. 1985. The production and use of economic forecasts. London: Methuen.
Kellert, S. E. 1993. In the wake of chaos. Unpredictable order in dynamical systems. Chicago: University of Chicago Press.
Kennedy, P. 1998. A guide to econometrics. 4th ed. Cambridge: The MIT Press.
Llewellyn, J., S. Potter, and L. Samuelson. 1985. Economic forecasting and policy-The international dimension. London: Routledge and K. Paul.
Makridakis, S., S. C. Wheelwright, and R. J. Hyndman. 1998. Forecasting: Methods and applications. 3rd ed. Hoboken: Wiley.
Mariano, R. S. 2002. Testing forecast accuracy. In A companion to economic forecasting, ed. M. Clements and D. F. Hendry, 284–298. Oxford: Blackwell.
Mathies, B. P., and A. Diamantipoulos. 1994. Towards a taxonomy of forecast error measures. A factor-comparative investigation of forecast error dimensions. Journal of Forecasting 13:409–416.
McNees, S. K. 1989. Why do forecasts differ? New England Economic Review Jan/Feb:42–54.
McNees, S. K. 1992. The uses and abuses of “consensus” forecasts. Journal of Forecasting 11:703–710.
Mitchell, M. 2009. Complexity: A guided tour. Oxford: Oxford University Press.
Morgan, M. S. 1990. The history of econometric ideas. Cambridge: Cambridge University Press.
Musgrave, A. 1981. “Unreal assumptions” in economic theory: The F-Twist untwisted. Kyklos 34 (3): 377–387.
Nieto de Alba, U. 1998. Historia del tiempo en Economía: Predicción, caos y complejidad. Madrid: McGraw Hill.
Popper, K. R. 1957. The poverty of historicism. London: Routledge and K. Paul. (Reprinted by Routledge in 1991).
Radnitzky, G. 1978. The boundaries of science and technology. In The search for absolute values in a changing world. Proceedings of the VIth International Conference on the Unity of Sciences, vol. II, 1007–1036. New York: International Cultural Foundation Press.
Rescher, N. 1998. Predicting the future: An introduction to the theory of forecasting. Albany: State University of New York Press.
Romanos, G. 1973. Reflexive predictions. Philosophy of Science 40:97–109.
Rosenberg, A. 1992. Economics-Mathematical politics or science of diminishing returns? Chicago: The University of Chicago Press.
Rosenberg, A. 1993. Scientific innovation and the limits of social scientific prediction. Synthese 97:161–182.
Rosser, J. Barkley Jr. 1999. On the complexities of complex economic dynamics. Journal of Economic Perspectives 13 (4): 169–192.
Rosser, J. Barkley Jr. 2011. Complex evolutionary dynamics in urban-regional and ecologic-economic systems: From catastrophe to chaos and beyond. Dordrecht: Springer.
Ruelle, D. 1991. Chance and chaos. Princeton: Princeton University Press.
Salmon, W. C. 1981. Rational prediction. The British Journal for the Philosophy of Science 32:115–125. (Reprinted in Grünbaum, A. and W. C. Salmon, eds. 1988. The limitations of deductivism, 47–60. Berkeley: University of California Press).
Selten, R. 2003. Emergence and future of experimental economics. In Observation and experiment in the natural and the social sciences, ed. M. C. Galavotti, 63–70. Dordrecht: Kluwer.
Simon, H. A. 1989. The state of economic science. In The state of economic science. Views of six Nobel laureates, ed. W. Sichel, 97–110. Kalamazoo: W. E. Upjohn Institute for Employment Research.
Simon, H. A. 1998. Economics as a historical science. Theoria 13 (32): 241–260.
Simon, H. A. 2001b. Science Seeks Parsimony, not Simplicity: Searching for Pattern in Phenomena. In Simplicity, inference and modelling. Keeping it sophisticatedly simple, ed. Zellner, A., Keuzenkamp, H. A., and McAleer, M., 32–72. Cambridge: Cambridge University Press.
Smith, P. 1998. Explaining chaos. Cambridge: Cambridge University Press.
Sober, E. 1996. Parsimony and predictive equivalence. Erkenntnis 44:167–197.
Stewart, I. 1993. Chaos. In Predicting the future, ed. L. Howe and A. Wain, 24–51. Cambridge: Cambridge University Press.
Strevens, M. 2003. Bigger than chaos: Understanding complexity through probability. Cambridge: Harvard University Press.
Vetterling, M. K. 1976. More on reflexive predictions. Philosophy of Science 43:278–282.
West, K. D. 2006. Forecast evaluation. In Handbook of economic forecasting, Vol. 1, ed. G. Elliot, C. W. J. Granger, and A. Timmerman, 99–134. Amsterdam: Elsevier.
Winnie, J. A. 1992. Computable chaos. Philosophy of Science 59 (2): 263–275.
Wold, H. O. A. 1969. Econometrics as pioneering in non-experimental model building. Econometrica 37:369–381.
Worrall, J. 2001a. De la Matemática a la Ciencia: Continuidad y discontinuidad en el Pensamiento de Imre Lakatos. In La Filosofía de Imre Lakatos: Evaluación de sus propuestas, ed. W. J. Gonzalez, 107–128. Madrid: UNED.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Gonzalez, W. (2015). Evaluation and Limits of Prediction in Economics. In: Philosophico-Methodological Analysis of Prediction and its Role in Economics. Theory and Decision Library A:, vol 50. Springer, Cham. https://doi.org/10.1007/978-3-319-08885-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-08885-3_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-08884-6
Online ISBN: 978-3-319-08885-3
eBook Packages: Humanities, Social Sciences and LawPhilosophy and Religion (R0)