Skip to main content
SpringerLink
Log in

Ranking journals using the dominance hierarchy procedure: an illustration with IS journals

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Ranking journals is an important exercise in academia. While several approaches to rank journals exist, an inherent assumption of these approaches is that there is indeed a hierarchy of journals, which is captured by the methods used for ranking them. We address a more fundamental question: Is there a linear hierarchy within journals? In this article, we introduce the dominance ranking approach that investigates the extent of hierarchy in a given set of objects by examining the extent of intransitivity in the system of interactions. We test the efficacy of the approach to ranking information systems journals based on citation data spanning a 3 year period from 2009 to 2011. Results indicate that the approach is very effective in identifying the extent of hierarchy within journals, and subsequently in ranking the journals. With its statistical underpinnings, the approach brings greater objectivity to the ranking of journals than prior approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. We would like to thank the anonymous reviewer for highlighting this limitation.

References

  • Adams, D., & Johnson, N. (2008). The journal list and its use: Motivation, perceptions, and reality. European Journal of Information Systems, 17(2), 158–162.

    Article  Google Scholar 

  • Amin, M., & Mabe, M. (2000). Impact factors: Use and abuse. Perspectives in publishing, 1(2), 1–6.

  • Appleby, M. C. (1983). The probability of linearity in hierarchies. Animal Behaviour, 31(2), 600–608.

    Article  Google Scholar 

  • Baskerville, R. (2008). For better or worse: How we apply journal ranking lists. European Journal of Information Systems, 17(2), 156–157.

    Article  Google Scholar 

  • Cohen, Jacob. (1990). Things I have learned (so far). American Psychologist, 45, 997–1003.

    Google Scholar 

  • Davis, P. M. (2008). Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts? Journal of the American Society for Information Science and Technology, 59(13), 2186–2188.

    Article  Google Scholar 

  • de Vries, H. (1995). An improved test of linearity in dominance hierarchies containing unknown or tied relationships. Animal Behaviour, 50(5), 1375–1389.

    Article  Google Scholar 

  • de Vries, H. A. N. (1998). Finding a dominance order most consistent with a linear hierarchy: A new procedure and review. Animal Behaviour, 55(4), 827–843.

    Article  Google Scholar 

  • Galliers, R. D. (2008). A discipline for a stage? A Shakespearean reflection on the research plot and performance of the Information Systems field. European Journal of Information Systems, 17(4), 330–335.

    Article  Google Scholar 

  • Harshman, R. A., Green, P. E., Wind, Y., & Lundy, M. E. (1982). A model for the analysis of asymmetric data in marketing research. Marketing Science, 1(2), 205–242.

    Article  Google Scholar 

  • Holsapple, C. W. (2008). A publication power approach for identifying premier information systems journals. Journal of the American Society for Information Science and Technology, 59(2), 166–185.

    Article  Google Scholar 

  • Holsapple, C. W. (2009). A new map for knowledge dissemination channels. Communications of the ACM, 52(3), 117–125.

    Article  Google Scholar 

  • Iivari, J. (2008). Expert evaluation vs bibliometric evaluation: Experiences from Finland. European Journal of Information Systems, 17(2),169–173.

  • Iverson, G. J., & Sade, D. S. (1990). Statistical issues in the analysis of dominance hierarchies in animal societies. Journal of Quantitative Anthropology, 2(1), 61–83.

  • Katerattanakul, P., & Han, B. (2003). Are European IS journals under-rated? An answer based on citation analysis. European Journal of Information Systems, 12(1), 60.

    Article  Google Scholar 

  • Kendall, M. G. (1962). Rank correlation methods (3rd ed.). London: Charles Griffin.

    Google Scholar 

  • Kendall, M. G., & Smith, B. B. (1940). On the method of paired comparisons. Biometrika, 31(3/4), 324–345.

    Article  MathSciNet  MATH  Google Scholar 

  • Landau, H. (1951). On dominance relations and the structure of animal societies: I. Effect of inherent characteristics. Bulletin of Mathematical Biology, 13(1), 1–19.

    Google Scholar 

  • Lowry, P. B., Romans, D., & Curtis, A. (2004). Global journal prestige and supporting disciplines: A scientometric study of information systems journals. Journal of the Association for Information Systems, 5(2), 29–77.

    Google Scholar 

  • Mingers, J., & Harzing, A.-W. (2007). Ranking journals in business and management: A statistical analysis of the Harzing data set. European Journal of Information Systems, 16(4), 303–316.

    Article  Google Scholar 

  • Mylonopoulos, N. A., & Theoharakis, V. (2001). Global perceptions of IS journals. Communications of the ACM, 44(9), 29–33.

    Article  Google Scholar 

  • Nelissen, M. H. J. (1986). The effect of tied rank numbers on the linearity of dominance hierarchies. Behavioural Processes, 12(2), 159–168.

    Article  Google Scholar 

  • Nerur, S., Sikora, R., Mangalaraj, G., & Balijepally, V. (2005). Assessing the relative influence of journals in a citation network. Communications of the ACM, 48(11), 71–74.

    Article  Google Scholar 

  • Peffers, K., & Tang, Y. (2003). Identifying and evaluating the universe of outlets for information systems research: Ranking the journals. Journal of Information Technology Theory and Application, 5(1), 63–84.

    Google Scholar 

  • Phillips, D. M., Baumgartner, H., & Pieters, R. (1999). Influence in the evolving citation network of the journal of consumer research. Advances in Consumer Research, 26(1), 203–210.

  • Polites, G. L., & Watson, R. T. (2008). The centrality and prestige of CACM. Communications of the ACM, 51(1), 95–100.

    Article  Google Scholar 

  • Rainer, R. K., Jr, & Miller, M. D. (2005). Examining differences across journal rankings. Communications of the ACM, 48(2), 91–94.

    Article  Google Scholar 

  • Salancik, G. R. (1986). An index of subgroup influence in dependency networks. Administrative Science Quarterly, 31, 194–211.

  • Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. British Medical Journal (BMJ), 314(7079), 498.

    Article  Google Scholar 

  • van Eck, N. J., & Waltman, L. (2009). How to normalize cooccurrence data? An analysis of some well-known similarity measures. Journal of the American Society for Information Science and Technology, 60(8), 1635–1651.

    Article  Google Scholar 

  • Whitman, M. E., Hendrickson, A. R., & Townsend, A. M. (1999). Research commentary. Academic rewards for teaching, research, and service: Data and discourse. Information Systems Research, 10(2), 99–109.

    Article  Google Scholar 

  • Willcocks, L., Whitley, E. A., & Avgerou, C. (2008). The ranking of top IS journals: A perspective from the London School of Economics. European Journal of Information Systems, 17(2), 163–168.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Management and Entrepreneurship Division, University of Colorado at Boulder, 419 UCB, Boulder, CO, 80309, USA

    Kishen Iyengar

  2. Decision and Information Sciences Department, School of Business Administration, Oakland University, Rochester, MI, 48309-4401, USA

    Venugopal Balijepally

Authors
  1. Kishen Iyengar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Venugopal Balijepally
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kishen Iyengar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Iyengar, K., Balijepally, V. Ranking journals using the dominance hierarchy procedure: an illustration with IS journals. Scientometrics 102, 5–23 (2015). https://doi.org/10.1007/s11192-014-1444-y

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-014-1444-y

Keywords

Search

Navigation