Abstract
Forecasting, in general, has been described as an unavoidable yet impossible task. This irony, which comprises the ‘rock’ and the ‘hard place’ in the title, creates a high level of cognitive dissonance, which, in turn, generates stress for those both making and using forecasts that have non-trivial impacts. Why? Because the forecasted numbers that are invariably accorded a high degree of precision inexorably reveal their inevitable imprecision when the numbers forming the actuality finally take place and the numbers comprising the forecast's errors are precisely measured. The current state of the art in demography for dealing with the ‘rock’ and the ‘hard place’ is a less-than-successful strategy because it is based on an acceptance of accuracy as the primary evaluation criterion, which is the source of cognitive dissonance. One way to reduce cognitive dissonance is to change the relationship of the very cognitive elements creating it. We argue that forecast evaluations currently focused on accuracy and based on measures like RMSE and MAPE be refocused to include utility and propose for this purpose the ‘Proportionate Reduction in Error’ (PRE) measure. We illustrate our proposal with examples and discuss its advantages. We conclude that including PRE as an evaluation criterion can reduce stress by reducing cognitive dissonance without, at the same time, either trivializing the evaluation process or substantively altering how forecasts are done and presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agresti, A. (1990).Categorical data analysis. New York: John Wiley.
Armstrong, J. (1982–83). The 15 most common pitfalls and how to avoid them,Journal of Business Forecasting 1(6): 12–15.
Ascher, W. (1978).Forecasting: An appraisal for policy-makers and planners. Baltimore, MD: The Johns Hopkins Press.
Brettschneider, S. & Gorr, W. (1992). Alternatives to forecast error based evaluation: Communicability, manipulability, credibility, and policy relevance, pp. 114–123, in:Proceedings of the Federal Forecasters Conference, 1992.
Burch, T., Swanson, D.A. & Tedrow, L. (1996). What is applied demography? What can it be? How can we get there from here?,Population Research And Policy Review, forthcoming in a special issue (Spring, Vol. 15).
Costner, H. (1965). Criteria for measures of association,American Sociological Review 30: 341–353.
D'Allesandro, F. (1987). Should applied demographers take out liability insurance?,Applied Demography 3(Fall): 1–3.
Dorn, H. (1950). Pitfalls in population forecasts and projections,Journal of the American Statistical Association 45: 311–334.
Espenshade, T. & Grummer-Strawn, L. (1991). Evaluating the accuracy of US population projection models. Paper presented at the Population Association of America Conference, Washington, DC.
Festinger, L. (1957).A theory of cognitive dissonance. Evanston, IL: Row, Peterson.
Festinger, L., Riecken, H., & Schachter, S. (1956).When prophesy fails. Minneapolis, MN: University of Minnesota Press.
Hajnal, J. (1995). The prospects for population forecasts,Journal of The American Statistical Association 50: 309–322.
Keyfitz, N. (1972). On future population,Journal of the American Statistical Association 67: 347–363.
Keyfitz, N. (1981). The limits of population forecasting,Population and Development Review 7: 579–594.
Keyfitz, N. (1982). Can knowledge improve forecasts?,Population and Development Review 8: 729–751.
Keyfitz, N. (1987). The social and political context of population forecasting, pp. 235–258, in: W. Alonso & P. Starr (eds.),The Politics of Numbers. New York: Russell Sage Foundation.
Kintner, H. & Swanson, D. (1994). Forecasting health benefit populations. Paper presented at the 14th International Symposium on Forecasting, Stockholm, Sweden.
Makridakis, S. & Hibon, M. (1979). Accuracy of forecasting: An empirical investigation,Journal of the Royal Statistical Society, Series A: 142: 97–145.
Moen, E. (1984). Voodoo forecasting: Technical, political, and ethical issues regarding the projection of local population growth,Population Research and Policy Review 3: 1–25.
Murdock, S. & Leistritz, L. (1980). Selecting socio-economic assessment models: A discussion of criteria and selected models,Journal of Environmental Management 10: 1–12.
Murdock, S., Hamm, R., Fannin, D., Pecotte, B. & Voss, P. (1989).Evaluating small-area population estimates and projections. Applied Community Research Monograph E3. Alexandria, VA: American Chamber of Commerce Researchers Association.
Murdock, S., Leistritz, F.L. Hamm, R., Hwang, S. & Parpia, B. (1984) An assessment of the accuracy of a regional economic-demographic projection model,Demography 21: 383–404.
Musham, H.V. (1965). The use of cost functions in making assumptions for population forecasts,Proceedings of the World Population Conference. New York: United Nations.
Ohio Data Users Center (1985).Population projections, Ohio and counties by age and sex: 1980 to 2000. Columbus, OH: Ohio Department of Development.
Pittenger, D. (1978). The role of judgement, assumptions, techniques, and confidence limits in forecasting population,Socio-Economic Planning Sciences 12: 271–276.
Rainford, P. & Masser, I (1987). Population forecasting and urban planning practice,Environment and Planning A 19: 1463–1475.
Reynolds, H.T. (1977).Analysis of nominal data. Beverly Hills, CA: Sage.
Robinson, J.G., Ahmed, B., Das Gupta, P. & Woodrow, K. (1991). Estimating coverage of the 1990 United States census: Demographic analysis. Paper presented at the Annual Meeting of The American Statistical Association, Atlanta, Georgia.
Romaniuc, A. (1994). Reflection on population forecasting: From prediction to prospective analysis,Canadian Studies in Population 21(2): 165–180.
Smith, S. (1987). Tests of forecast accuracy and bias for county population projections,Journal of the American Statistical Association 82: 991–1003.
Smith, S. & Bayya, R. (1992). An evaluation of population forecasts for Florida and its counties,Applied Demography 7 (Spring): 1–5.
Smith, S. & Shahidullah, M. (1995). An evaluation of population projection errors for census tracts,Journal of The American Statistical Association 90(1): 69–71.
Smith, S. & Sincich, T. (1990). The relationship between the length of the base period and population forecast errors,Journal of the American Statistical Association 85: 367–375.
Smith, S. & Sincich, T. (1992). Evaluating the forecast accuracy and bias of alternative projections for States,International Journal of Forecasting 8: 495–508.
Stoto, M. (1983). The accuracy of population projections,Journal of the American Statistical Association 78: 13–20.
Starr, P. (1987). The sociology of official statistics, pp. 7–58, in: W. Alonso & P. Starr (eds.),The politics of numbers. New York: Russell Sage Foundation.
Swanson, D.A. (1986). Evaluating population estimates and short-term projections,Applied Demography 2 (November): 5–6.
Swanson, D.A. & Beck, D. (1994). New short-term county population projection method.Journal of Economic and Social Measurement 20: 1–26.
Swanson, D.A. & Tayman, J. (1994). Measuring the utility of population projections. Paper presented at the Annual Meeting of the Ohio Academy of Science, Toledo, Ohio.
Swanson, D.A., Tayman, J. & Beck, D. (1995). On the utility of lagged ratio-correlation as a short-term county population projection method: A case study of Washington State,Journal of Economic and Social Measurement 21: 1–16.
Sykes, Z. (1969). Some stochastic versions of the matrix model for population dynamics,Journal of the American Statistical Association 44: 111–130.
Tayman, J. (1993). How accurately can we forecast small area population? Presented at the Annual Meeting of the American Statistical Association, San Francisco, California.
Tayman, J. (1996). The accuracy of small area population forecasts based on a spatial interaction land use modelling system,Journal of The American Planning Association (forthcoming).
Tayman, J., Schafer, E. & Carter, L. (1994). Confidence intervals for small area population forecast error: A repeated sampling approach. Paper presented at the Annual Meeting of the Population Association of America, Miami, Florida.
Tayman, J. & Swanson, D. (1995). Alternative measures for evaluating population forecasts: A comparison of state, county, and subcounty areas. Paper presented at The Annual Meeting of the Population Association of America, San Francisco, California.
US Bureau of the Census (1975).Coverage of population in the 1970 census and some implications for public programs. Current Population Reports, P-23, No. 56. Washington, DC: US Government Printing Office.
US Bureau of the Census (1982).Coverage of the national population in the 1980 census by age, sex, and race. Current Population Reports, P-23, No. 115. Washington, DC: US Government Printing Office.
US Bureau of the Census (1992).Population projections of the United States. by age, sex, race, and Hispanic origin: 1992–2050. Washington, DC: US Government Printing Office.
Washington State, Office of Financial Management (1986).Forecasts of the State and County populations by year for selected age groups: 1980–2000. F86–11. Olympia, WA: Population Estimation and Forecasting Unit.
Wheelwright, S. & Makridakis, S. (1980).Forecasting methods for management, 3rd ed. New York: John Wiley.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Swanson, D.A., Tayman, J. Between a rock and a hard place: The evaluation of demographic forecasts. Popul Res Policy Rev 14, 233–249 (1995). https://doi.org/10.1007/BF01074460
Issue Date:
DOI: https://doi.org/10.1007/BF01074460