Skip to main content
SpringerLink
Log in

The control model for the selection of reference collections providing the impartial assessment of the quality of scientific and technological publications by using bibliometric and scientometric indicators

  • Systems Analysis and Operations Research
  • Published:
Journal of Computer and Systems Sciences International Aims and scope

“…What Is Good and What Is Bad?…” V. Mayakovsky

Abstract

This paper presents a method for the impartial assessment of the quality of scientific and technological natural language documents and their collections by using bibliometric and scientometric indicators. The method of computational document search and design of document collections that are adequate for calculating reference values to assess the quality of the documents and document collections is formulated. This method is based on the mathematical modeling of natural language texts. Examples of the obtained assessments are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. I. V. Marshakova-Shaikevich, “The role of bibliometrics in the evaluation of research activity of the science,” Upravl. Bol’sh. Sist. 44, 210–247 (2013).

    Google Scholar 

  2. L. Bornmann, A. Thor, W. Marx, and H. Schier, “The application of bibliometrics to research evaluation in the humanities and social sciences: an exploratory study using normalized google scholar data for the publications of a research institute,” J. Assoc. Inform. Sci. Technol. (in press).

  3. L. Bornmann and R. Haunschild, “The interest of the scientific community in expert opinions from journal peer review procedures,” Scientometrics 102, 2187–2188 (2015).

    Article  Google Scholar 

  4. J. Ruiz-Castillo and L. Waltman, “Field-normalized citation impact indicators using algorithmically constructed classification systems of science,” J. Informetrics 9, 102–117 (2015).

    Article  Google Scholar 

  5. N. J. van Eck and L. Waltman, “CitNetExplorer: a new software tool for analyzing and visualizing citation networks,” J. Informetrics 8, 802–823 (2014); arXiv:1404/5322.

    Article  Google Scholar 

  6. L. Waltman and R. Costas, “F1000 recommendations as a new data source for research evaluation: a comparison with citations,” J. Assoc. Inform. Sci. Technol. 65, 433–445 (2014); arXiv:1303.3875.

    Google Scholar 

  7. L. Waltman and N. J. van Eck, “A systematic empirical comparison of different approaches for normalizing citation impact indicators,” J. Informetrics 7, 833–849 (2013); arXiv:1301.4941.

    Article  Google Scholar 

  8. L. Waltman and N. J. van Eck, “Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison,” Scientometrics 96, 699–716 (2013); arXiv:1208.6122.

    Article  Google Scholar 

  9. L. Waltman and M. Schreiber, “On the calculation of percentile-based bibliometric indicators,” J. Am. Soc. Inform. Sci. Technol. 64, 372–379 (2013).

    Article  Google Scholar 

  10. L. Waltman, N. J. van Eck, and P. Wouters, “Counting publications and citations: is more always better?,” J. Informetrics 7, 635–641 (2013); arXiv:1301.4597.

    Article  Google Scholar 

  11. L. Bornmann and L. Leydesdorff, “Does quality and content matter for citedness? A comparison with para-textual factors and over time,” J. Informetrics 9, 419–429 (2015).

    Article  Google Scholar 

  12. L. Bornmann, “Redundancies in h index variants and the proposal of the number of top-cited papers as an attractive indicator,” Measurement 10, 149–153 (2012).

    Google Scholar 

  13. L. Waltman, A. F. J. van Raan, N. J. van Eck, and W. C. Peul, “Citation analysis may severely underestimate the impact of clinical research as compared to basic research,” PLoS ONE 8, e62395 (2013); arXiv:1210/0442.

    Article  Google Scholar 

  14. B. Z. Iliev, “Measuring the evaluation and impact of scientific works and their authors,” (2013); arXiv:1311/6948.

    Google Scholar 

  15. S. V. Bredikhin and A. Yu. Kuznetsov, Bibliometric Methods and the Electronic Scientific Periodicals Market (IVMiMG SO RAN, NEIKON, Novosibirsk, 2012) [in Russian].

    Google Scholar 

  16. O. P. Morozova, “Hirsch index as a scientometric indicator: comparative analysis and its modifications,” Nauch.-Tekh. Inform., Ser. 1: Organiz. Metodika Inform. Raboty, No. 2, 30–33 (2011).

    Google Scholar 

  17. V. V. Nalimov and Z. M. Mul’chenko, Scientometry (Nauka, Moscow, 1969) [in Russian].

    Google Scholar 

  18. N. S. Red’kina, “Bibliometrics: history and the present,” Molodye Bibliotech. Dele 2, 76–86 (2003).

    Google Scholar 

  19. O. V. Fedorets, “Using the weighted impact factor to create a list of the most-cited multidisciplinary journals,” Nauch.-Tekh. Inform., Ser. 1: Organiz. Metodika Inform. Raboty, No. 7, 22–27 (2011).

    Google Scholar 

  20. O. Kücüktunç, E. Saule, K. Kaya, and Ü. V. Catalyürek, “Diversifying citation recommendations,” ACM Trans. Intell. Syst. Technol. 5 (4), 55 (2015); arXiv:1209/5809.

    Google Scholar 

  21. J. K. Vanclay, “Impact factor: outdated artefact or stepping-stone to journal certification?,” Scientometrics 92, 211–238 (2012); arXiv:1201.3076.

    Article  Google Scholar 

  22. J. K. Vanclay and L. Bornmann, “Metrics to evaluate research performance in academic institutions: a critique of ERA 2010 as applied in forestry and the indirect H2 index as a possible alternative,” Scientometrics 91, 751–771 (2012).

    Article  Google Scholar 

  23. Numbers Game, or as We Now Appreciate the Work of the Scientist, Collection of Articles in Bibliometrics (MTsNMO, Moscow, 2011) [in Russian].

  24. T. Saracevic, “Effects of inconsistent relevance judgments on information retrieval test results: a historical perspective,” LIBRARY TRENDS.he Legacy of F. W. Lancaster 56, 763–783 (2008).

    Article  Google Scholar 

  25. M. G. Kreines, “Models and technologies for the extraction of aggregated Knowledge to control processes of the retrieval of non-structured information,” J. Comput. Syst. Sci. Int. 48, 272 (2009).

    Article  MathSciNet  MATH  Google Scholar 

  26. M. G. Kreines, “Methods of computational analysis of semantic models for quality assessment of scientific texts,” J. Comput. Syst. Sci. Int. 52, 226 (2013).

    Article  MATH  Google Scholar 

  27. A. N. Petrov, M. G. Kreines, and A. A. Afonin, “Semantic search of the unstuctured natural language texts in the management of the research and development programs’ expertise,” Informatiz. Obrazov. Nauki 18 (2), 54–67 (2013).

    Google Scholar 

  28. M. G. Kreines and A. A. Afonin, “Clusterization of text collections: help with a content search and analytical tool,” in Internet-portals: Content and Technologies (Prosveshchenie, Moscow, 2007), No. 4, pp. 510–537 [in Russian].

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. BaseTech Llc, Moscow, Russia

    M. G. Kreines & E. M. Kreines

Authors
  1. M. G. Kreines
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. E. M. Kreines
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. G. Kreines.

Additional information

Original Russian Text © M.G. Kreines, E.M. Kreines, 2016, published in Izvestiya Akademii Nauk, Teoriya i Sistemy Upravleniya, 2016, No. 5, pp. 73–89.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kreines, M.G., Kreines, E.M. The control model for the selection of reference collections providing the impartial assessment of the quality of scientific and technological publications by using bibliometric and scientometric indicators. J. Comput. Syst. Sci. Int. 55, 750–766 (2016). https://doi.org/10.1134/S1064230716040092

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1064230716040092

Search

Navigation