Abstract
Scientometric predictors of research performance need to be validated by showing that they have a high correlation with the external criterion they are trying to predict. The UK Research Assessment Exercise (RAE) — together with the growing movement toward making the full-texts of research articles freely available on the web — offer a unique opportunity to test and validate a wealth of old and new scientometric predictors, through multiple regression analysis: Publications, journal impact factors, citations, co-citations, citation chronometrics (age, growth, latency to peak, decay rate), hub/authority scores, h-index, prior funding, student counts, co-authorship scores, endogamy/exogamy, textual proximity, download/co-downloads and their chronometrics, etc. can all be tested and validated jointly, discipline by discipline, against their RAE panel rankings in the forthcoming parallel panel-based and metric RAE in 2008. The weights of each predictor can be calibrated to maximize the joint correlation with the rankings. Open Access Scientometrics will provide powerful new means of navigating, evaluating, predicting and analyzing the growing Open Access database, as well as powerful incentives for making it grow faster.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Brody, T. (2003), Citebase search: Autonomous citation database for e-print archives. ECS Technical Report. http://eprints.ecs.soton.ac.uk/10677/
Brody, T., Harnad, S., Carr, L. (2006), Earlier web usage statistics as predictors of later citation impact. Journal of the American Association for Information Science and Technology (JASIST), 57(8): 1060–1072. http://eprints.ecs.soton.ac.uk/10713/
Garfield, E., (1955) Citation indexes for science: A new dimension in documentation through association of ideas. Science, 122: 108–111, http://www.garfield.library.upenn.edu/papers/science_v122(3159)p108y1955.html
Giles, C. L., Bollacker, K., Lawrence, S. (1998), CiteSeer: An automatic citation indexing system. Digital Libraries 98: Third ACM Conf. on Digital Libraries, ACM Press, New York, 1998, pp. 89–98. http://www.neci.nj.nec.com/homepages/lawrence/citeseer.html
Guédon, J.-C. (2002), In Oldenburg’s Long Shadow: Librarians, Research Scientists, Publishers, and the Control of Scientific Publishing. Washington, DC: The Association of Research Libraries. http://www.arl.org/resources/pubs/mmproceedings/138guedon.shtml
Hajjem, C., Harnad, S., Gingras, Y. (2005), Ten-year cross-disciplinary comparison of the growth of open access and how it increases research citation impact. IEEE Data Engineering Bulletin, 28(4) 39-47. http://eprints.ecs.soton.ac.uk/12906/
Harnad, S. (2006), Online, continuous, metrics-based research assessment. Technical Report, ECS, University of Southampton. http://eprints.ecs.soton.ac.uk/12130/
Harnad, S., Brody, T. (2004), Comparing the impact of Open Access (OA) vs. non-OA articles in the same journals, D-Lib Magazine, 10(6) June. http://www.dlib.org/dlib/june04/harnad/06harnad.html
Harnad, S., Carr, L., Brody, T., Oppenheim, C. (2003), Mandated online RAE CVs Linked to University Eprint Archives: Improving the UK Research Assessment Exercise whilst making it cheaper and easier. Ariadne, 35 (April 2003). http://www.ariadne.ac.uk/issue35/harnad/
Hirsch, J. E. (2005), An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46): 16569–16572, http://www.pnas.org/cgi/content/abstract/102/46/16569
Kleinberg, J. M. (1999), Hubs, authorities, and communities. ACM Computing Surveys, 31(4), http://www.cs.brown.edu/memex/ACM_HypertextTestbed/papers/10.html
Kline, P. (2000), The New Psychometrics: Science, Psychology and Measurement. Routledge.
Lawrence, S. (2001), Online or invisible? Nature, 411(6837): 521, http://www.neci.nec.com/:_lawrence/papers/online-nature01/
Landauer, T. K., Foltz, P. W., Laham, D. (1998), Introduction to latent semantic analysis. Discourse Processes, 25: 259–284.
Moed, H. F. (2005), Citation Analysis in Research Evaluation. NY Springer.
Oppenheim, Charles (1996), Do citations count? Citation indexing and the research assessment exercise, Serials, 9: 155–61. http://uksg.metapress.com/index/5YCDB0M2K3XGAYA6.pdf
Page, L., Brin, S., Motwani, R., Winograd, T. (1999), The PageRank Citation Ranking: Bringing Order to the Web. http://dbpubs.stanford.edu:8090/pub/1999-66
Shadbolt, N., Brody, T., Carr, L., Harnad, S. (2006), The open research web: a preview of the optimal and the inevitable. In: Jacobs, N. (Ed.), Open Access: Key Strategic, Technical and Economic Aspects, chapter 21. Chandos. http://eprints.ecs.soton.ac.uk/12453/
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Harnad, S. Open access scientometrics and the UK Research Assessment Exercise. Scientometrics 79, 147–156 (2009). https://doi.org/10.1007/s11192-009-0409-z
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-009-0409-z