Skip to main content

Advertisement

Log in

Some bibliometric procedures for analyzing and evaluating research fields

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Nowadays, measuring the quality and quantity of the scientific production is an important necessity since almost every research assessment decision depends, to a great extent, upon the scientific merits of the involved researchers. To do that, many different indicators have been proposed in the literature. Two main bibliometric procedures to explore a research field have been defined: performance analysis and science mapping. On the one hand, performance analysis aims at evaluating groups of scientific actors (countries, universities, departments, researchers) and the impact of their activity on the basis of bibliographic data. On the other hand, the extraction of knowledge from the intellectual, social or conceptual structure of a research field could be done by means of science mapping analysis based on bibliographic networks. In this paper, we introduce some of the most important techniques and software tools to analyze the impact of a research field and its scientific structures. Particularly, four bibliometric indices (h, g, hg and q2), the h-classics approach to identify the classic papers of a research field and three free science mapping software tools (CitNetExplorer, SciMAT and VOSViewer) are shown.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://www.citnetexplorer.nl/

  2. http://sci2s.ugr.es/scimat

  3. 3 http://www.vosviewver.com

References

  1. Alonso S, Cabrerizo FJ, Herrera-Viedma E, Herrera F (2009) h-Index: a review focused in its variants, computation and standardization for different scientific fields. J Informet 3(4):273–289. https://doi.org/10.1016/j.joi.2009.04.001

    Article  Google Scholar 

  2. Alonso S, Cabrerizo FJ, Herrera-Viedma E, Herrera F (2010) Hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices. Scientometrics 82(2):391–400. https://doi.org/10.1007/s11192-009-0047-5

    Article  Google Scholar 

  3. Batagelj V, Cerinšek M (2013) On bibliographic networks. Scientometrics 96 (3):845–864. https://doi.org/10.1007/s11192-012-0940-1

    Article  Google Scholar 

  4. Batagelj V, Mrvar A (1998) Pajek - program for large network analysis. Connections 21(2):47–58

    MATH  Google Scholar 

  5. Boccaletti S, Latora V, Moreno Y, Chavez M, Hwang D (2006) Complex networks: structure and dynamics. Phys Rep 424(4–5):175–308. https://doi.org/10.1016/j.physrep.2005.10.009

    Article  MathSciNet  MATH  Google Scholar 

  6. Börner K, Chen C, Boyack KW (2005) Visualizing knowledge domains. Annual Review of Information Science and Technology 37(1):179–255. https://doi.org/10.1002/aris.1440370106

    Article  Google Scholar 

  7. Bornmann L, Mutz R, Daniel HD (2008) Are there better indices for evaluation purposes than theh index? a comparison of nine different variants of theh index using data from biomedicine. J Am Soc Inf Sci Technol 59 (5):830–837. https://doi.org/10.1002/asi.20806

    Article  Google Scholar 

  8. Burrell QL (2007) On the h-index, the size of the Hirsch core and Jin’s A-index. J Informet 1(2):170–177. https://doi.org/10.1016/j.joi.2007.01.003

    Article  MathSciNet  Google Scholar 

  9. Cabrerizo F, Alonso S, Herrera-Viedma E, Herrera F (2010) Q2-index: quantitative and qualitative evaluation based on the number and impact of papers in the hirsch core. J Informet 4(1):23–28. https://doi.org/10.1016/j.joi.2009.06.005

    Article  Google Scholar 

  10. Callon M, Courtial JP, Turner WA, Bauin S (1983) From translations to problematic networks: an introduction to co-word analysis. https://doi.org/10.1177/053901883022002003

  11. Cartes-Velásquez R, Manterola Delgado C (2014) Bibliometric analysis of articles published in ISI dental journals, 2007–2011. Scientometrics 98(3):2223–2233. https://doi.org/10.1007/s11192-013-1173-7

    Article  Google Scholar 

  12. Cobo MJ, Chiclana F, Collop A, De Oña j, Herrera-Viedma E (2014) A bibliometric analysis of the intelligent transportation systems research based on science mapping. IEEE Trans Intell Transp Syst 15(2):901–908. https://doi.org/10.1109/TITS.2013.2284756

    Article  Google Scholar 

  13. Cobo MJ, López-Herrera AG, Herrera F, Herrera-Viedma E (2012) A note on the ITS topic evolution in the period 2000-2009 at T-ITS. IEEE Trans Intell Transp Syst 13(1):413–420. https://doi.org/10.1109/TITS.2011.2167968

    Article  MATH  Google Scholar 

  14. Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F (2011) An approach for detecting, quantifying, and visualizing the evolution of a research field: a practical application to the fuzzy sets theory field. J Informet 5(1):146–166. https://doi.org/10.1016/j.joi.2010.10.002

    Article  MATH  Google Scholar 

  15. Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F (2011) Science mapping software tools: review, analysis, and cooperative study among tools. J Am Soc Inf Sci Technol 62(7):1382–1402. https://doi.org/10.1002/asi.21525

    Article  MATH  Google Scholar 

  16. Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F (2012) SciMAT: a new science mapping analysis software tool. J Am Soc Inf Sci Technol 63(8):1609–1630. https://doi.org/10.1002/asi.22688

    Article  MATH  Google Scholar 

  17. Cobo MJ, Martínez MA, Gutiérrez-Salcedo M, Fujita H, Herrera-Viedma E (2015) 25 Years at knowledge-based systems: a bibliometric analysis. Knowl-Based Syst 80:3–13. https://doi.org/10.1016/j.knosys.2014.12.035

    Article  Google Scholar 

  18. Cook DJ, Holder LB (2006) Mining graph data. Wiley-Interscience

  19. Costas R, Bordons M (2007) The h-index: advantages, limitations and its relation with other bibliometric indicators at the micro level. J Informet 1(3):193–203. https://doi.org/10.1016/j.joi.2007.02.001

    Article  Google Scholar 

  20. De la Flor-Martínez M, Galindo-Moreno P, Sánchez-Fernández E, Piattelli A, Cobo MJ, Herrera-Viedma E (2016) H-classic : a new method to identify classic articles in implant dentistry, periodontics, and oral surgery. Clin Oral Implants Res 27(10):1317–1330. https://doi.org/10.1111/clr.12749

    Article  Google Scholar 

  21. De Maio C, Fenza G, Loia V, Parente M (2015) Biomedical data integration and ontology-driven multi-facets visualization. In: 2015 international joint conference on neural networks (IJCNN). https://doi.org/10.1109/IJCNN.2015.7280395. IEEE, pp 1–8

  22. De Maio C, Parente M, Fenza G, Greco D (2016) Time aware knowledge extraction to analyze nanosafety cluster scientific activities. In: 2016 IEEE congress on evolutionary computation (CEC). IEEE, pp 1233–1240. https://doi.org/10.1109/CEC.2016.7743928

  23. Egghe L (2006) Theory and practise of the g-index. Scientometrics 69(1):131–152. https://doi.org/10.1007/s11192-006-0144-7

    Article  MathSciNet  Google Scholar 

  24. Egghe L, Rousseau R (2008) An h-index weighted by citation impact. Inf Process Manag 44(2):770–780. https://doi.org/10.1016/j.ipm.2007.05.003

    Article  Google Scholar 

  25. Feijoo JF, Limeres J, Fernández-Varela M, Ramos I, Diz P (2014) The 100 most cited articles in dentistry. Clinical Oral Investigations 18(3):699–706. https://doi.org/10.1007/s00784-013-1017-0

    Article  Google Scholar 

  26. Garfield E (1972) Citation analysis as a tool in journal evaluation. Science 178(4060):471–479. https://doi.org/10.1126/science.178.4060.471

    Article  Google Scholar 

  27. Garfield E (1977) Introducing citation classics. The human side of scientific reports. Current Comments 1 (1):5–7

    Google Scholar 

  28. Garfield E (1979) Citation indexing: its theory and application in science, technology, and humanities. Wiley, New York

    Google Scholar 

  29. Garfield E (1987) 100 citation classics from the journal of the american medical association. J Am Med Assoc 257(1):52. https://doi.org/10.1001/jama.1987.03390010056028

    Article  Google Scholar 

  30. Garfield E (1994) Scientography: mapping the tracks of science. Current Contents: Social & Behavioural Sciences 7(45):5–10

    Google Scholar 

  31. Garfield E, Pudovkin AI, Istomin VS (2003) Why do we need algorithmic historiography? J Am Soc Inf Sci Technol 54(5):400–412. https://doi.org/10.1002/asi.10226

    Article  Google Scholar 

  32. Glanzel W (2001) National characteristics in international scientific co-authorship relations. Scientometrics 51(1):69–115. https://doi.org/10.1023/A:1010512628145

    Article  MathSciNet  Google Scholar 

  33. Hirsch JE (2005) An index to quantify an individual’s scientific research output. Proc Natl Acad Sci 102 (46):16,569–16,572. https://doi.org/10.1073/pnas.0507655102

    Article  MATH  Google Scholar 

  34. Houari NS, Taghezout N (2016) Integrating agents into a collaborative knowledge-based system for business rules consistency management. International Journal of Interactive Multimedia and Artificial Intelligence 4(2):61–72. https://doi.org/10.9781/ijimai.2016.4210

    Article  Google Scholar 

  35. Huang MH, Chang CP (2014) Detecting research fronts in OLED field using bibliographic coupling with sliding window. Scientometrics 98(3):1721–1744. https://doi.org/10.1007/s11192-013-1126-1

    Article  Google Scholar 

  36. Hutchins BI, Yuan X, Anderson JM, Santangelo GM (2016) Relative citation ratio (RCR): a new metric that uses citation rates to measure influence at the article level. PLOS Biology 14(9):e1002,541. https://doi.org/10.1371/journal.pbio.1002541

    Article  Google Scholar 

  37. Ibrahim GM, Carter Snead O, Rutka JT, Lozano AM (2012) The most cited works in epilepsy: trends in the “Citation Classics”. Epilepsia 53(5):765–770. https://doi.org/10.1111/j.1528-1167.2012.03455.x

    Article  Google Scholar 

  38. Jin B (2006) h-Index: an evaluation indicator proposed by scientist. Science Focus 1(1):8–9

    MathSciNet  Google Scholar 

  39. Jin B, Liang L, Rousseau R, Egghe L (2007) The R- and AR-indices: complementing the h-index. Chin Sci Bull 52(6):855–863. https://doi.org/10.1007/s11434-007-0145-9

    Article  Google Scholar 

  40. Kessler MM (1963) Bibliographic coupling between scientific papers. Am Doc 14(1):10–25. https://doi.org/10.1002/asi.5090140103

    Article  Google Scholar 

  41. Li X, Zhou Y, Xue L, Huang L (2015) Integrating bibliometrics and roadmapping methods: a case of dye-sensitized solar cell technology-based industry in China. Technol Forecast Soc Chang 97:205–222. https://doi.org/10.1016/j.techfore.2014.05.007

    Article  Google Scholar 

  42. Martínez MA, Cobo MJ, Herrera M, Herrera-Viedma E (2015) Analyzing the scientific evolution of social work using science mapping. Res Soc Work Pract 5(2):257–277. https://doi.org/10.1177/1049731514522101

    Article  Google Scholar 

  43. Martínez MA, Herrera M, López-Gijón J, Herrera-Viedma E (2014) H-classics: characterizing the concept of citation classics through H-index. Scientometrics 98(3):1971–1983. https://doi.org/10.1007/s11192-013-1155-9

    Article  Google Scholar 

  44. Moed HF, Bruin RE, Leeuwen TN (1995) New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications. Scientometrics 33(3):381–422. https://doi.org/10.1007/BF02017338

    Article  Google Scholar 

  45. Moral-Muñoz JA, Cobo MJ, Chiclana F, Collop A, Herrera-Viedma E (2016) Analyzing highly cited papers in intelligent transportation systems. IEEE Trans Intell Transp Syst 17(4):993–1001. https://doi.org/10.1109/TITS.2015.2494533

    Article  Google Scholar 

  46. Moral-Muñoz JA, Cobo MJ, Peis E, Arroyo-Morales M, Herrera-Viedma E (2014) Analyzing the research in integrative complementary medicine by means of science mapping. Complement Ther Med 22(2):409–418. https://doi.org/10.1016/j.ctim.2014.02.003

    Article  Google Scholar 

  47. Murgado-Armenteros EM, Gutiérrez-Salcedo M, Torres-Ruiz FJ, Cobo MJ (2015) Analysing the conceptual evolution of qualitative marketing research through science mapping analysis. Scientometrics 102 (1):519–557. https://doi.org/10.1007/s11192-014-1443-z

    Article  Google Scholar 

  48. Noyons ECM, Moed HF, Luwel M (1999) Combining mapping and citation analysis for evaluative bibliometric purposes: A bibliometric study. J Am Soc Inf Sci 50(2):115–131. https://doi.org/10.1002/(SICI)1097-4571(1999)50:2⟨115::AID-ASI3⟩3.0.CO;2-J

    Article  Google Scholar 

  49. Perianes-Rodriguez A, Waltman L, van Eck NJ (2016) Constructing bibliometric networks: a comparison between full and fractional counting. J Informet 10(4):1178–1195. https://doi.org/10.1016/j.joi.2016.10.006

    Article  Google Scholar 

  50. Ponce FA, Lozano AM (2011) The most cited works in Parkinson’s disease. Mov Disord 26(3):380–390. https://doi.org/10.1002/mds.23445

    Article  Google Scholar 

  51. Rodriguez-Ledesma A, Cobo MJ, Lopez-Pujalte C, Herrera-Viedma E (2015) An overview of animal science research 1945-2011 through science mapping analysis. J Anim Breeding Genet 132(6):475–497. https://doi.org/10.1111/jbg.12124

    Article  Google Scholar 

  52. Rousseau R (2006) New developments related to the Hirsch index. Science Focus 1(4):23–25

    Google Scholar 

  53. SCImago (2007) SJR - SCIMago Journal & Country Rank

  54. Settouti N, Bechar MEA, Chikh MA (2016) Statistical comparisons of the top 10 algorithms in data mining for classi cation task. International Journal of Interactive Multimedia and Artificial Intelligence 4(1):46–51. https://doi.org/10.9781/ijimai.2016.419

    Article  Google Scholar 

  55. Sidiropoulos A, Katsaros D, Manolopoulos Y (2007) Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics 72(2):253–280. https://doi.org/10.1007/s11192-007-1722-z

    Article  Google Scholar 

  56. Small H (1973) Co-citation in the scientific literature: a new measure of the relationship between two documents. J Am Soc Inf Sci 24(4):265–269. https://doi.org/10.1002/asi.4630240406

    Article  Google Scholar 

  57. Small H (1999) Visualizing science by citation mapping. J Am Soc Inf Sci 50 (9):799–813. https://doi.org/10.1002/(SICI)1097-4571(1999)50:9⟨799::AID-ASI9⟩3.0.CO;2-G

    Article  Google Scholar 

  58. Smith D (2007) Ten citation classics from the New Zealand medical journal. N Z Med J 120(1267):2871–2875

    Google Scholar 

  59. Stack S (2013) Citation classics in deviant behavior: a research note. Deviant Behavior 34(2):85–96. https://doi.org/10.1080/01639625.2012.707539

    Article  Google Scholar 

  60. Tam WW, Wong EL, Wong FC, Cheung AW (2012) Citation classics in the integrative and complementary medicine literature: 50 frequently cited articles. Eur J Intern Med 4(1):e77–e83. https://doi.org/10.1016/j.eujim.2011.12.004

    Article  Google Scholar 

  61. Tang L, Shapira P (2011) China-US scientific collaboration in nanotechnology: patterns and dynamics. Scientometrics 88(1):1–16. https://doi.org/10.1007/s11192-011-0376-z

    Article  Google Scholar 

  62. Van Eck NJ, Waltman L (2009) How to normalize cooccurrence data? An analysis of some well-known similarity measures. J Am Soc Inf Sci Technol 60(8):1635–1651. https://doi.org/10.1002/asi.21075

    Article  Google Scholar 

  63. van Eck NJ, Waltman L (2010) Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 84(2):523–538. https://doi.org/10.1007/s11192-009-0146-3

    Article  Google Scholar 

  64. van Eck NJ, Waltman L (2014) Citnetexplorer: a new software tool for analyzing and visualizing citation networks. J Informet 8(4):802–823. https://doi.org/10.1016/j.joi.2014.07.006

    Article  Google Scholar 

  65. van Raan AFJ (2005) Measuring science. In: Moed HF, Glänzel W, Schmoch U (eds) Handbook of quantitative science and technology research, chap. measuring. Springer, Netherlands, pp 19–50

  66. Vanclay JK (2007) On the robustness of theh-index. J Am Soc Inf Sci Technol 58(10):1547–1550. https://doi.org/10.1002/asi.20616

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge FEDER funds under grants TIN2013-40658-P and TIN2016-75850-R, and also the financial support from the University of Cádiz Project PR2016-067.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to E. Herrera-Viedma.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gutiérrez-Salcedo, M., Martínez, M.Á., Moral-Munoz, J.A. et al. Some bibliometric procedures for analyzing and evaluating research fields. Appl Intell 48, 1275–1287 (2018). https://doi.org/10.1007/s10489-017-1105-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-017-1105-y

Keywords

Navigation