Skip to main content
Log in

Estimation of the reliability of ratings

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

A procedure for estimating the reliability of sets of ratings, test scores, or other measures is described and illustrated. This procedure, based upon analysis of variance, may be applied both in the special case where a complete set of ratings from each ofk sources is available for each ofn subjects, and in the general case wherek 1,k 2, ...,k n ratings are available for each of then subjects. It may be used to obtain either a unique estimate or a confidence interval for the reliability of either the component ratings or their averages. The relations of this procedure to others intended to serve the same purpose are considered algebraically and illustrated numerically.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Clark, Edward L. Reliability of college grades. Unpublished paper presented to American Psychological Association in September, 1950.

  2. Cureton, Edward E. Reliability of a set of averages of unequal numbers of scores. Unpublished paper read at the March, 1951, meeting of the Southern Society for Philosophy and Psychology.

  3. Fisher, R. A. Statistical methods for research workers. (10th Ed.) Edinburgh: Oliver and Boyd, 1946.

    Google Scholar 

  4. Gulliksen, Harold. Theory of mental tests. New York: John Wiley and Sons, 1950.

    Google Scholar 

  5. Horst, Paul. A generalized expression for the reliability of measures.Psychometrika, 1949,14, 21–31.

    Google Scholar 

  6. Jackson, R. W. B., and Ferguson, George A. Studies on the reliability of tests. Toronto: Univ. of Toronto Press, 1941.

    Google Scholar 

  7. Lindquist, E. F. Statistical analysis in educational research. Boston: Houghton Mifflin Company, 1940.

    Google Scholar 

  8. Peters, Charles C., and Van Voorhis, Walter R. Statistical procedures and their mathematical bases. New York: McGraw Hill Book Company, 1940.

    Google Scholar 

  9. Snedecor, George W. Statistical methods. (4th Ed.) Ames, Iowa: Iowa State Coll. Press, 1946.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Additional information

The writer wishes to acknowledge the helpful comments and suggestions of Professors E. E. Cureton, Harold Gulliksen, and E. F. Lindquist.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ebel, R.L. Estimation of the reliability of ratings. Psychometrika 16, 407–424 (1951). https://doi.org/10.1007/BF02288803

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02288803

Keywords

Navigation