Pragmatic issues in calculating and comparing the quantity and quality of research through rating and ranking of researchers based on peer reviews and bibliometric indicators from Web of Science, Scopus and Google Scholar

The purpose of this paper is to analyse the findings of two recently published papers (Norris and Oppenheim, 2003; and Li et al., 2010).

The findings were analysed from the practitioner's perspective about the procedures involved in calculating the indicator values and the ranks and ratings. This was done with the purpose of playing the devil's advocate, contemplating the reservations and arguments of those who do not want to use metrics based on database searches.

One advantage of this project is that its results can be compared at least partially with the findings of the three earlier RAEs (although its grade classes have changed), as well as with some of the other ranking lists in library and information management areas.

Very importantly, the authors concluded that “it would be premature in the extreme to suggest that citation‐based indicators could be used as a cost‐effective alternative to expert judgments”. This is a strong, very realistic and fair statement. Even this recent project's results are very valuable in spite of the problems mentioned.