Assessing Accuracy in Postcensal Estimates: Statistical Properties of Different Measures

Abstract

When developing new estimation or projection methods or assessing current techniques, demographers must deal with evaluating several options for constructing estimates for national and sub-national areas, such as states, counties, tracts, etc. Assessing an option for the estimation involves evaluating the estimates as a set rather than one at a time. A technique often used is to compare the estimates against a standard, such as recent census counts, believed to be more accurate than the estimates under evaluation. To do this, it is necessary to summarize the errors into performance measures that can be applied to a sub-national level. These measures need to reflect the different uses of the estimates. Usually several measures need to be examined because a single one is not able to capture all the different accuracy requirements for the varied uses of the estimates, such as fixed pie allocations or absolute numbers of the population. Different measures have evolved to reflect the accuracy requirements, some focus on the accuracy of the distribution of the population across areas while others focus on numerical accuracy in the sub-national areas. Some measures give all component error estimates equal weight while others weight the error estimate by population size. This paper presents a systematic overview of these methods. It explores the relationship between the population estimates and their important uses, such as fund allocations. Included is a demonstration of the properties of the measures and an illustration of the necessity of understanding the role of error in the standard of comparison when forming the measures. Finally, the paper presents a method to facilitate comparisons across measures when assessing a set of population estimates or studying a proposed estimation methodology.

This report is released to inform interested parties and encourage discussion of work in progress. The views expressed on statistical, methodological, and operational issues are those of the authors and not necessarily those of the U.S. Census Bureau.