Design and Validation of the Conceptual Model of Scientometric System in Research Centers

Evaluation and ranking of universities and research centers had always been very important because they are among the most important centers of science production. However, in order to perform scientometrics-based evaluation properly and effectively, the system is needed to meet their needs at various levels. The purpose of this study is to design and validate the conceptual model of scientometrics system in order to provide the possibility of measuring and evaluating the scientific outputs of research centers and researchers according to various dimensions and indicators of scientometrics. This study has been conducted in three phases. In the first phase, scientometric indicators and system capabilities were extracted. In the second phase, the conceptual model was designed using UML diagrams and in the third phase, it was evaluated and validated using Delphi method. In this study, the conceptual model of scientometrics system of research centers is presented and its main components and applications are introduced. The implementation of such a system at the national and international levels can provide effective tools for scientific policy makers to advance a variety of scientometric tasks such as computational evaluations, more accurate rankings, as well as identifying the most suitable research centers and researchers.


INTRODUCTION
The scientific societies, universities, and institutes and their affiliated centers have always been looking for more practical methods in order to measure the progress of science and compare the scientific outputs of researchers and scientific organizations. In this regard, scientometrics has been able to provide practical and appropriate criteria for evaluating and comparison of the scientific products of individuals and scientific institutions. [1,2] Using quantitative analysis methods, [3] scientometrics can provides evaluation and policies of the scientific products of researchers, universities, research centers, specific topics and countries and also compares, evaluates and rank them. [4,5] Evaluation and ranking of universities and research centers have always been very important because the production of science and comparison of scientific outputs of organizations and countries are based on their educational activities and research results. [6] Due to the competitive and challenging climate at the national and international levels, universities and research centers are always looking to achieve a higher position and to be able to benefit from its advantages and obtain higher budget. Hence, determining and evaluating their performance is one of the main issues considered by officials. [7] However, the evaluation and ranking of universities and research centers is based on various objectives and criteria, an important part of which is to investigate their scientific products indexed in valid international databases and evaluate them based on common and existing indicators, such as h and g and the like.
Meanwhile, medical science research centers are considered as the most important centers of science production due to their serious objectives in the field of health development, and a thorough and comprehensive investigation of their scientific productivity and ranking requires serious determination. Because conducting such an evaluation can identify the most efficient and effective research centers and be used in research policies. In today's world where governments are always looking for more standard indicators and methods to improve the quantitative and qualitative evaluations of scientific products [1] , the need to use systems that take into account the PubMed, Scopus, Embase and Lista databases without any time limit until July 15, 2019. A total of 18,326 records were obtained that after excluding repeats, screening the title, abstract and full text of the papers, 173 papers were used to extract index information.
At the second stage of the first phase, in order to complete other capabilities required for the system, the observed resources, texts and related sites were used and the desired features and capabilities were identified and summarized and categorized in the form of a Table in a Word file.
Phase 2: providing a conceptual model of scientometric system In order to provide a database, after determining the database management system and the type of model used, the first task of a database designer will be to design a conceptual model of data that represents the structure of information in a database. The semantic data modeling is the presentation of a model of an operational space using concepts that are independent of the issues related to the logical and physical presentation of data. The semantic modeling is related to the perceptual or conceptual level of database design and DBMS. [9] In fact, a database or DBMS has a specific three-level architecture for managing and providing responses to users, which are: [10] -External level: The database users and applications dealing with it are at this level, which is also called "user vision".
-Perceptual or conceptual level: At this level, the structure designed for data and relation between them based on the needs of all system users and at the perceptual level, is provided and specified. At this level, the database designer defines and designs his desired structures for the stored data, and the perceptual schema of the system is designed.
-Internal or physical level: At this level, the method of data storage in the hardware system and the details of physical storage are specified that this information is hidden from users. [11][12][13] In this regard, there are various models and methods for semantic modeling of data, the most famous of which are the entity-relation model (ER), Unified Modeling Language model (UML) and object modeling technique (OMT). [14,15] In the system design phase, based on the data and information obtained from the first phase, based on object oriented programming and using the UML the Use Case diagrams, scenarios, sequence diagrams, activity diagrams and class diagram were drawn and presented. [16] Each chart was designed using PowerDesigner 16.5 software and its files were saved in the desired format of the software. Then, in the Word file, the Figures related to each diagram were placed and used. The designed diagrams were described and explained descriptively various quantitative and qualitative indicators of scientometrics, evaluate scientific performance and use the effectiveness of their scientific products is necessary.

Objectives of the study
Regarding the importance of comprehensive evaluations of scientific products and the need to facilitate matters, reduce time and cost and increase the accuracy and precision of scientometric-based evaluation processes, the existence of a comprehensive electronic system is necessary. The system that has the necessary comprehensiveness and ability to implement important and practical indicators of scientometrics and can be used nationally and internationally to evaluate scientific productivity as well as various aspects of scientific production of universities and research centers, doubled its necessity. [8] In addition, the availability of such scientometric systems in medical research centers, which, due to their special missions in the field of health, attribute great importance to timely and effective access to information, can be a priority.
The question that needs to be answered is what dimensions, details, features and conceptual model should such systems have to be able to provide such scientometric evaluations in research centers? Studying, designing and validating the conceptual model of such scientometric systems can provide the necessary grounds for its implementation in medical research centers as well as in other similar centers and draw the attention of policy makers and research planners to the effective capabilities and performance of these systems for wider and more accurate evaluation of the type of centers.

MATERIALS AND METHODS
In terms of the design of a conceptual model for a system, this study is a developmental and applied type that has been conducted by descriptive scientometric research method with a combination of quantitative and qualitative methods in three phases. In the following, for example, the evaluation of medical science research centers is considered: Phase 1: Determining the features required to provide a scientometric system In the first phase of the study, sources and texts were investigated in order to collect the elements and data required for the design of the scientometric system, and the main indicators along with additional capabilities were extracted. All evaluation indicators of researchers or organizations were extracted from papers, entered EndNote and then in Excel thematically categorized based on the objectives and application of each index. Also, some other useful and appropriate capabilities were extracted and used from scientific texts and papers.
In order to extract the required indicators in the system, the papers were searched and retrieved in Web of Science, and textually, and the process stages were explained in each of the diagrams and scenarios. In addition to the conceptual model that is the main topic of this study, in order to better display the model of this system, a prototype of this system was also designed based on the conceptual model next to this study.

Phase 3: Validation of the conceptual model of scientometrics system
The validity of the designed model was investigated using Delphi technique and a survey of experts in the field of informatics and scientometrics. At this stage, the researcher-made questionnaire was prepared which contained 11 groups of questions and 82 items and distributed by email among experts in this field and anonymously and separately.
We selected Delphi phase members through non-probabilistic sampling and a combination of purposeful and judgmental methods, and those with Ph.D. in medical informatics, health information management, library and information science with work and research experience and expertise in the field of information systems design and information technology that 12 experts participate in this phase. This number of members was according to the scientific literature that mentioned this number varies and has not yet been precisely determined or have determined standards [17] for instance, the review of the Delphi research by Rowe and Wright [18] showed that in most studies, the number of members participated in Delphi phase, ranged from 4 to 21 experts. Other studies mentioned that this number varies and is between 10-50, [19] 5-20, [20] 10-15, [21] or 16 is suitable. [22] So, we sent a questionnaire by email to them and after collecting all 12 completed questionnaires, the results of the first round were also analyzed using SPSS 24 software, which due to scores above 3.75, this stage was done in one round and did not lead to the second or third round, which is explained in detail below. For validity, the face validity of this questionnaire was confirmed by 5 experts in the field of scientometrics, informatics and information management, and its reliability was investigated in two ways, which are described below.

Split-half method
In this method, the questionnaire questions are divided into two categories. The reliability value for the first 42 questions is about 0.98 and (next 40 questions) equal to 0.94 for the second category. The numerical value of the relationship and correlation between these two categories is equal to about 0.84.

Cronbach's alpha test
In this method, for the prepared questionnaire, the numerical value of Cronbach's alpha coefficient for 82 questions of the questionnaire was equal to 0.98, which indicates that the reliability of the questionnaire is excellent.
For data analysis, the scores obtained from the items in the questionnaire (1= very poor, 2= poor, 3= moderate, 4= high and 5= very high) were entered SPSS software. Since the mean score indicates the distribution of scores, so it is very important and therefore, for comparison, the mean score is used. In this study, the mid-point of the measured interval was considered 3 and accordingly, the components that have obtained a mean of less than 2.5 were excluded from the questionnaire for the second phase of Delphi, the cases that scored between 2.5 and 3.75 will be corrected for the second phase, and the cases that scored more than 3.75 will remain for the second phase of Delphi. The question groups and their items consisted of 11 groups and 82 items: Group 1: Use Case diagrams, Group 2: scenarios, Group 3: sequence diagrams, Group 4: activity diagrams, Group 5: class diagrams, Group 6: prototype designed, Group 7: search requirements and capabilities of research centers in the system, Group 8: ability to apply filters, Group 9: page display capabilities, Group 10: reporting capabilities in the system, and Group 11: capabilities of storage in the system

RESULTS
What are the features required to provide a scientometric system?
In order to answer this question, at the first stage, data features and elements required for the system are provided and at the second stage, the functional requirements of the system were extracted and expressed. These indicators, which are the most important indicators extracted from scientific papers, include various aspects of evaluating the performance of researchers and organizations, which include 135 indicators retrieved in this study and are classified in all eight general groups including: Each of these groups contains a number of indicators that are similar in nature, function, objectives, or features in a group that evaluate a particular aspect.
What is the conceptual model of a scientometric system?
At this stage and after determining the required features for the scientometric system with the expected and specified capabilities, a conceptual model and a prototype of this system were presented, the conceptual model of which is described here. In order to draw the conceptual model of this system, UML diagrams were drawn and designed. In the following, we present the results of this section and the concept diagram including Use Case diagrams, the user search scenario, and then the sequence, activity, and class diagrams.

Use Case diagram
In order to draw these diagrams for the desired system, three actors have been identified, which are: -The main users, which include scientometrics and evaluation officials of research centers as well as policy makers in this field, -System manager in which a person is responsible for controlling, monitoring and managing the use of users of the system.
-System designer who includes a programmer and provider of the system. These diagrams show the different uses of the system and the pattern of behavior by its actors, which show an overview of how a system works. This type of diagram is one of the most important diagrams that are used during the design of the system as a model of the activity and expectations from the system and its performance. [23] In this section, in order to better specify the Use Case of the actors, their diagrams are provided separately.
Here all system actors can login if they have already registered by entering their ID and password. Then, the users and the system administrator can perform search operations, investigate the results, apply the desired indicators, report and save the results.
The system administrator can also control and confirm the use of users as well as control the system dashboard ( Figure 2).
In the section related to the system designer, in addition to controlling users, the designer can update and develop the system and prepare the required types of reports (Figure 3).

Scenarios
This section investigates and presents an example of scenarios and activities that the actors of a system perform sequentially and in fact show the process of operation in the system. [24]

Sequence diagrams
These types of diagrams show the stage-by-stage process of operations in a system, which are successive stages of activities over time. [25,26] The following are two examples of sequence diagrams related to the search, reporting and storage sequence of the system users. The following Figure 4 shows the user search sequence for selecting basic or advanced search and its stages.
The following Figure 5 shows the sequence of stages for reporting and storing results by the system and selecting the type of reports and format for storing results.

Activity diagram
These diagrams show the work and activity in the system in the form of flowcharts. [27] The flow of behaviors and decisions (conditions) is displayed at successive stages, which the following diagram, which is related to the user search, shows the flow of the search process to exit and the stages and decisions it takes to reach the end of the activity process ( Figure 6).

Class Diagram
The class diagram, which is the most important diagram for the design of a system, presents the classes and the relationships between them. [23,28] The following Figure 7 shows each class of the system along with its attributes and related operations. We showed four indicators in the classes for instance.

Select a sample of performance effect indicators of researchers (DS index) for calculation in the system Scenario
The user extracts DS-index for the target research center. Brief description

Users Actors
The user first selects or searches his or her research center from the search page. Precondition 6. The user selects "Analysis" option.
7. The user investigates the results of this index separately by the authors of that research center.
8. The user reports or stores the results in the desired formats.
1. If you need to edit the selected index to calculate, first go back to the previous stage and then display the results page.

Exception conditions
What is the validity of the conceptual model of the scientometric system?
In this section, after preparing the conceptual model and the prototype of the scientometric system, at the next stage, this model was evaluated by Delphi method.
According to the results of the analysis of item scores, the final scores were obtained and the results showed that the mean of each component, except in one case, was more than 3.75, and therefore, this stage of the research was performed in one round.
The Table 2 below also presents the groups studied and the sum of the means of the items along with the total mean, which in total all the means were higher than 3.75.    or based on the decisions of scientific policy makers at the national and international levels.
For the second objective of the study, the conceptual model of the scientometric system was designed based on the objectoriented unified modeling language. UML is one of the most important languages of object-oriented systems, in which information are in the form of objects, and the most important feature of UML is the ability to provide and display objects in the real world as illustrated. In general, UML method is diagram-based and uses diagrams to show the modeling and design of software and systems. Also, the main objective is to define a standard method to illustrate the system design path as an illustration language used to show the behavior and structure of a system. [15,29] In this regard, the most important diagrams for drawing a conceptual model with UML were the five main diagrams: Use Case, scenarios, sequence, activity and class. According to Reggio et al. study these five diagrams have had the highest percentage of use. [30,31] Also, related studies and dissertations in the field of design and implementation of information and software systems by UML method were reviewed, which have used these common diagrams. [32][33][34][35] For this reason, these diagrams were used to model the scientometric system and it was attempted to fully express the most important performance and activities related to the system.
In addition, in order to better understand these activities and details related to the capabilities and scheme of the system, the system prototype has been designed, which was not presented in this study to summarize the content, and therefore its interactivity and simulation of the reality of using the system were provided in practice. In this case, we attempted to put the details and the most important elements needed in the system so that the audience can communicate with the way of using the system and its parts in a visual and interactive way.
In this system, the login process, selecting menu or advanced search methods, the required filters and the important stage of selecting the desired index among the various categories of indicators were shown. Also, an index was determined for the sample, and the stages of selecting and applying the index are shown to analyze the data obtained from a sample research center that the selection of other indicators will be based on the same process and by selecting the desired option among the indicators, the system can perform calculations and provide the necessary text and image reports to users.
After preparing the reports, there is this feature in the system to store the results in different formats. Here, each user can create their own profile, and save the results and update when needed.
Therefore, in the first phase, the desired results were obtained and the questionnaire did not lead to the second round of Delphi phase. This shows that, based on the analysis performed, the various components and capabilities provided for the system have been appropriate and should be applied in the system.

DISCUSSION
In the first section of the study, the capabilities and indicators of the system were addressed, and several papers were studied and reviewed to extract the indicators used for evaluating the research performance of researchers and their affiliated organizations, comparing and ranking the related countries. At this stage, many indicators were retrieved that were provided and presented over the years by the efforts of various researchers with different fields in science. Many of these indicators are less considered while they could be used well for research evaluations. Hence, in this phase, this study aimed to identify the various indicators used for evaluating the performance of researchers and affiliated organizations and use them for the classifications used for the scientometric system as comprehensive as possible, and provide a possibility for research centers and researchers around the world to evaluate various aspects of scientific performance. Because there are many useful and practical indicators that have not been used yet, while the evaluation process and its tools are broader and deeper, so far less had been addressed.
The indicators, which were 135 indicators retrieved from the studied references, were extracted and these indicators were classified into eight categories based on their most important and most uses including various indicators that can be used based on the needs of each organization or research center, * This is the sum of the means of all items presented in the each group** This is the mean of all items presented in each group the system capabilities and requirements of this system were determined and therefore the features of the scientometric system were prepared and presented, which were used at the design and evaluation stage of the model.
The scientometric indicators are studied and evaluated by various researchers, and various criticisms and confirmations are published in scientific references by experts in this field, as well as other activists in the field of scientometrics, in which the strengths and weaknesses of the indicators are investigated and presented. Therefore, since no index can investigate all the scientific aspects of a study and / or the scientific performance of a researcher or a group of them alone, the use of different indicators and approaches can complement each other and make a more equitable evaluation with better and more accurate rankings of different views. Also, conducting various studies in this regard leads to the development of effective and practical concepts and indicators in the field of scientometrics.
In this regard, designing and providing a national and comprehensive system in the field of scientometrics to evaluate, rank and monitor performance of researchers and affiliated organizations can be useful and important and have the necessary accuracy, efficiency and comprehensiveness. Also, this system can be expanded and used to evaluate other organizations, universities and research centers at the international level, because its principles and infrastructure are to evaluate the scientific products and research performance of individuals and affiliated organizations that can also cover other centers.
Consistent with the second phase, which was the main objective of this study; the model of scientometric system of research centers was prepared and drawn. This model was designed to implement and illustrate the desired idea of a scientometric system with the required capabilities, its main and important components were displayed and the stages and activities in the use of users and interaction with the system were presented. Based on this model, a model can be provided for its implementation in the future, and by planning and providing the required budget, a useful and practical system can be prepared that can be implemented at a large international level and based on its basic infrastructure and can be used internationally by different countries to evaluate the research centers and researchers in different fields based on different scientometric indicators.
In addition, the design of such a system in order to evaluate the scientific performance of researchers, organizations and research centers in more detail and accuracy, use the results by science policy makers for future research studies and identify information gaps as well as strengths and weaknesses of existing national and international studies, as well as allocate PowerDesigner software was used to draw the relevant diagrams, which is much easier and more convenient to work with compared to other software, and has the ability to draw properly and with quality, and there is useful text and image help for it in the Internet.
In this section of the study, a conceptual model was designed and its details were displayed in the relevant diagrams and Figures, so that the general and conceptual structure could be shown as much as possible, and an introduction for more accurate implementation and various details can be provided.
For the third objective of the study, based on the results obtained from the analysis of the conceptual model evaluation questionnaire and its capabilities required, all the cases raised were accepted and can be applied in the system. In this regard, one case had a mean of less than 3.75 (the possibility of using the operators SAME, NEAR, and Wildcard in the system with a mean of 3.67), which was due to the lack of details of these features in the search and for this reason, less attention was paid to respondents, while it has been one of the most important features for the system leading to a more accurate retrieval of organizational affiliation of research centers through advanced search.
In general, most items had high scores, and the lowest scores were related to the prototype designed for the conceptual model, which seems to be due to a slight lack of focus on its various parts, and the reason is that the objective of the preparation of this prototype was to provide a schematic representation of the conceptual model to better understand the generalities of the model, and in fact it is a complementary part for illustrating this model, which certainly needs to provide more and more complete details to the audience and user during implementation.
In this questionnaire, it was attempted to provide the details of the conceptual model and the required capabilities from the perspective of experts in this field for review and analysis, and in the next phase of implementation in the future, the points and opinions of experts will be considered to increase research quality.
Also, given that Delphi phase of this questionnaire was done at a stage and did not lead to the second phase due to the higher scores than the specified range and the agreement on the capabilities and facilities provided in this model, this can indicate the appropriateness of the questionnaire parts and components in terms of quality and necessity.

CONCLUSION
In this study, it was attempted to extract the most important and practical indicators and data elements in the scientometric system including the most important part of the system. Also, appropriate funding to research, has an effect and better application in the development of science, technology and health as other advantages. Therefore, the design of such systems seems necessary and will play an important role in achieving more diverse evaluations with different objectives, rankings, budgeting and research investment on researchers, scientific products and related organizations.
Finally, the conceptual model prepared for the scientometric system in Delphi phase was evaluated, and the opinions and suggestions of experts in this field were collected and analyzed. The results can be used for the implementation phase of the main system, and in the required details and performance, make the relevant parts more complete and appropriate.