Postgraduate Training Performance Evaluation Applying Weighting and Comprehensive Fuzzy Evaluation Methods

Postgraduate training performance evaluation is an important part of higher education evaluation. In order to quantitatively evaluate the performance of postgraduate training combined with the needs of talent training in colleges and universities, a comprehensive evaluation system for postgraduate training performance is designed and proposed. The analytic hierarchy process and the entropy method are combined to determine the weights of seven input and output indicators, such as funding input, teacher source input, scientific research ability, and graduate output quality. The comprehensive fuzzy evaluation model is also applied. This paper tests the above models to evaluate the graduate training performance of 20 colleges at Tianjin University. It is found that the weight of student input and graduate output quality combined is 87.69%, which makes them the key indicators to measure the performance of graduate training. Furthermore, it is concluded that the performance of the science college was significantly higher than that of the humanities college. Keywords—Postgraduate training, performance evaluation, Combination weighting, fuzzy evaluation model


Introduction
The evaluation of the quality of higher education is an important part of higher education activities and is often based on a combination of judgment and quantitative criteria [1]. In China, graduate students are the top level of national education and the core element of the national innovation system. As a combination of education and science and technology, postgraduate education is responsible for the mission of supplying highly educated graduate students to the country's labor market. In 2018, the number of graduate students enrolled in China was about 858,000. A large number of graduate students makes an important contribution to the country's economic development. The contribution rate to China's economic growth is 3.1% [2], which is an important force to lead China's economic development. Based on the importance of education evaluation for higher education and the key role of graduate education, it is necessary to conduct an evaluation of graduate education efficiently. http://www.i-jet.org University performance evaluation has been included in the scope of policy leadership [3]. The evaluation of the training performance of colleges and universities, as the training units for graduate students and the main implementers of postgraduate education, is the focus of many academic studies. A large number of literature sources show that teaching conditions, infrastructure, and quality of postgraduate training are the main factors taken into account when assessing the performance of postgraduate training [4]. Among these，student satisfaction with the process of teaching can be appropriate for determining whether teaching quality is excellent or not, which are also helpful for university managers to judge performance [5]. In addition, accelerated genetic algorithm, projection pursuit classification model, principal component analysis, analytic hierarchy process, data envelopment method, fuzzy neural network, and input-output method [6,7,8] are all important means of evaluating postgraduate training performance. However, due to different evaluation motivations and indicators, different evaluation methods often yield different results, thus, it cannot be used for reference directly.
Based on these considerations, this paper constructs a comprehensive performance evaluation system for graduate training according to the needs of talent cultivation in Chinese universities. To achieve it, this study combines the analytic hierarchy process (AHP) with the entropy method to determine the main factors affecting the performance of postgraduate training in colleges and universities. In order to test the proposed method, this research collects data from 20 colleges of Tianjin University, and the comprehensive fuzzy evaluation model is used to further quantitatively evaluate the performance of postgraduate training in Colleges and universities. The results show that the input of students and the quality of postgraduate training are important factors affecting the performance evaluation of the postgraduates, and the performance of the Science college is significantly higher than that of the Humanities College.

Performance evaluation of university postgraduates' training
The term performance has been around for a long time. Bernadin [9] believes that performance is the result of work and a record and consideration of organizational work. However, Campbell insists that performance differs from results and points out that performance should be perceived as a process behavior [10]. In this paper, the concept of performance is considered to cover both behavior and results. Therefore, performance evaluation should consider both input (behavior) and output (result). Meanwhile, evaluation is generally defined as the implementation and acceptance of the basic criteria to determine the value of things according to the selected criteria [11]. Therefore, the performance evaluation of postgraduate training refers to judging the value of postgraduate training results based on the two levels of behavior input and behavior output of postgraduate training.

Evaluation system
Postgraduate training is a complex dynamic process, and its evaluation needs to take into account multiple factors. A large number of academic sources shows that some key input factors, such as human, financial, and facility resources, as well as some important outputs, including graduates, publications, and knowledge transfer (for example, patents and derivatives), are regarded as indicators of higher education activities [12,13,14,15,16,17]. Thus, based on the definition of performance, evaluation of postgraduate training and those classical literature, the paper proposes the performance evaluation index system for postgraduate training, as shown in Table 1.
The input in the evaluation system consists of three parts: student source input (the number and achievements of different types of postgraduate students), fund input (research funds and assistantship), and teacher input. The output consists of four parts: Scientific research ability (research project, high-level papers and the number of Key Research Centres or Laboratory Bases), course building ability (student satisfaction and number of classic courses), degree construction ability (number of academic degree authorization points, evaluation results), and graduate output quality (for instance, academic ability, employment competitiveness, and innovation ability).

Methodology
When evaluating a system with multiple rating indicators, a weighting factor is usually used to indicate the importance of different indicators for the evaluation results. The methods of weight determination include subjective weighting, objective weighting, and combined weighting. Subjective weighting (or empowerment) is conducted by experts, and it has certain reference significance. Among the common approaches, there are the Delphi and factor pair comparison methods. The traditional subjective empowerment often has the disadvantage of subjective preference. Nevertheless, the objective weighting method, which relies on actual data, can also have this shortcoming. The common objective weighting methods include principal component analysis, mean square error weighting, and entropy method. There are also shortcomings of the objective weighting method, which make it hard to reflect the importance of different indicators.
To address these shortcomings, this study applies the analytic hierarchy process and the entropy method to combine empowerment combined with the indicators sys-tem of graduate training performance evaluation and the applicability of various empowerment methods. On this basis, a comprehensive fuzzy evaluation model is constructed, and with its help, the objects are evaluated by the comprehensive scores obtained from calculations.

Combination weighting with analytic hierarchy process and entropy methods
Analytic Hierarchy Process (AHP) is a decision analysis method proposed by American Operations Research in the 1970s. The key to using AHP to empower is to construct a judgment matrix. From the first-level indicators of the system, a pairwise comparison matrix is constructed for the corresponding higher-level indicators. In this way, the n-dimensional judgment matrix is formed until the last layer. The importance of the correlation factors of the two pairs is the value in the matrix. This importance is assessed while referring to the nine-scale table proposed by Saaty [18]. This procedure is conducted by 12 experts, engaged in higher education administration, postgraduate teaching, scientific research, and employment. The experts construct the judgment matrix through three rounds of cross-scoring.
Entropy method19] is originally derived from thermodynamics, and it is a mathematical method to judge the discrete degree of the index by using the entropy value. If the value of entropy is large, the degree of dispersion is high. Thus, this particular index has a high contribution to the comprehensive evaluation and represents a large weight. When calculating the entropy value and corresponding weight of the evaluated object, the following steps are followed: Where, i is the i-th evaluated object, and j represents the j-th evaluation indicator. ′ is the value of the j-th indicator of the i-th object to be evaluated.

Comprehensive fuzzy evaluation
The comprehensive fuzzy evaluation method [20] is a mathematical method derived from the fuzzy set theory, and it is especially suitable for evaluation problems with multiple levels of influence factors that are difficult to quantify.
The performance of postgraduate training in colleges and universities is a complex system composed of various indicators and changes with the evaluation factors such as scientific research environment and teaching staff. The comprehensive fuzzy evaluation method is based on the fuzzy membership degree theory and is suitable for solving the above problems. When using this method to calculate the score of the evaluated object, it is necessary to construct an evaluation matrix, and based on this, the membership degree is obtained by the following formula: where, rab is a membership function, and is appropriately chosen so that 0 < rab < 1 rab represents the membership grade score of the index a for the fuzzy subset of the grade relative to the evaluated unit. The final weight vector is B=W1×30*R30× n=B1× n .

Preparation for survey
This paper considers 20 colleges of Tianjin University as the research object to conduct an empirical study on the performance evaluation of postgraduates. The data used are from the official public data of Tianjin University's colleges from 2014 to 2018, the Employment Quality Report of Tianjin University and the statistical data of various centers. Data processing is completed by EXCEL, and the model analysis is carried out by MATLAB. The data are real and reliable, the analysis tools are scientific, and the research results are representative.

Analysis of research results
The subjective weights are determined by the analytic hierarchy process. From the first level of indicators, the corresponding indicators of the subordinate higher level are constructed into a pair comparison matrix until the last level, forming a 7dimensional judgment matrix. In order to ensure rationality, it is necessary to verify the random consistency ratio (CR) of the judgment matrix: CI =(λmax-n)/(n-1). Then CI=0.0446，and CR=CI/RI=0.0267<0.1，therefore, the consistency test is passed, so the weights are reasonable. Then the objective weights of the seven indicators are determined according to the entropy method formula, and the weight vector is obtained using EXCEL for data processing. Finally, the followings are performed: the test of the consistency of subjective and objective weight results and Pearson con-sistency coefficient test. Since the P value is 0.02 < 0.05, the results show that there is significance. Then through the multiplication synthesis, the final calculation of the weight results is conducted. The results are shown in Table 2. It can be seen from Table 2 that the results that most affect the performance evaluation of postgraduate training are the student input, graduate output quality, scientific research ability, funding input, curriculum construction ability, teacher investment, and degree building ability successively. Among them, the weight of student input and graduate output quality combined is 87.69%, which is the key index to measure the performance of graduate training at Tianjin University.
Student input reflects the quality of student resources in colleges. Astin [21] has shown that students' educational background will affect students' educational activities during school. This study also proves that there is a close relationship between students' educational background and training performance. In addition, in a realm of postgraduate education, the quality of postgraduates has always been a key factor in evaluating the performance of postgraduate training in colleges and universities. It is also a necessary consideration to measure whether the training work of Tianjin University is qualified.
It is generally believed that the increased investments in human capital can improve the productivity of human capital itself; so, teachers are often seen as a key factor for the performance of graduate students [22]. However, the weight results show that the contribution of teacher investment to the evaluation of postgraduate training performance of Tianjin University is only 0.61%. The metrics of teacher input in this paper are the ratio of students to teachers, the proportion of associate professors among the teachers, and the rationality of the teachers [23]. In the process of data collection, it is found that there is not much difference in the investment of teachers across 20 colleges. This may be the expert score, and the entropy calculation results finally show the main reason for the low weight score.
With regard to human capital investment, in academia, it is generally believed that inadequate funding is a key factor affecting the quality of postgraduate training. However, the weight results show that the contribution rate of funding to the evaluation of postgraduate training performance is only 3.84%. This may indicate that at the present stage of the development of higher education, the dependence of postgraduate training on funding has weakened, and the role of funding in evaluating the performance of postgraduate training in colleges and universities has declined to a certain extent [24].
To further discuss the results of postgraduate training at Tianjin University, the paper constructed the evaluation matrix by using the comprehensive fuzzy evaluation model and the actual data on the object of study, which stipulates that the performance standards of graduate students in each college are five-level system, that is, V  = {v1, v2, v3, v4, v5} = {A  According to the calculated deviation, the comprehensive score of training perfor-mance of Software College is 2.6% higher than the average level of Tianjin University, while that of Microelectronics College is 4.2% lower than the average level.
The results of postgraduate training in the colleges of science, mathematics, and life sciences are at a general level. The schools that have qualified the results of performance evaluation of graduate training are the Colleges of Education, Law, Marxist and Foreign Language and Literature; the average score of these colleges is only 0.38. This result shows that the performance of postgraduate training in science or engineering Colleges is generally higher than that of Social Sciences Colleges, and the difference between the two reached 0.539. Thus, there is a significant gap in the effectiveness of postgraduate training in various institutes, and the postgraduate training work of Social Sciences colleges at Tianjin University needs to be improved.

Conclusion
This paper evaluates the postgraduate training work in 20 colleges of Tianjin University from 2014 to 2018 by using the combination of the weighting method and comprehensive fuzzy evaluation model. This study is conducted to inform legislation development and reveal existing problems of postgraduate training work in colleges and universities. The following four main conclusions have been drawn: Students' background is an important factor affecting postgraduate training performance. From 2014 to 2018, the proportion of students' background in the performance evaluation of postgraduate training in universities was as high as 52%. This result shows that there is a close relationship between students' background and training performance. Students' background is becoming more and more important for evaluating the training of Postgraduates in Colleges and universities.
The output quality of postgraduates is an important consideration to measure the quality of postgraduate training. According to the evaluation index, the output quality of graduate students mainly inspects five aspects: research ability, innovation and entrepreneurship ability, practical ability, social work ability, and employment competitiveness. Its weight in the performance evaluation of graduate students' training in universities is nearly 35.4%, which shows the importance of the output quality of graduate students.
The contribution rate of financial input to evaluating the performance of postgraduate training in Colleges and universities is low. According to the result of empowerment, the proportion of funds invested in the evaluation of graduate education in universities is only about 6%. This result shows that the correlation between funds investment and training performance is small. Continuous improvement of funds investment may not make the performance of graduate education excellent. In the future, the work of graduate education in universities should reasonably evaluate the funding gap and increase funds use efficiency.
There are obvious differences in the performance of postgraduate training among different colleges. The quality of postgraduate training in science and engineering colleges is generally higher than that in social science colleges. According to the analysis results and the characteristics of the research object, Tianjin University be-longs to the Polytechnic School, which pays more attention to the science and engineering, and its source quality, graduate output quality, scientific research environment, and development level are also relatively dominant. In contrast, the development of humanities and social sciences is still insufficient, and the overall level of graduate training performance of the Academy of Social Sciences is low, so there is a phenomenon of unbalanced development of disciplines.

8 Authors
Mei Wang has graduated from Tianjin University with a doctorate in management. She is an associate professor of higher education management and talent development in Tianjin University. She has published dozens of articles in core journals and hosted and participated in five national and provincial research projects.

18062111221@163.com
Zeng Zhang is a second-year master's degree in education economics and management student at the School of Education, Tianjin University. Her main research direction is the education quality assessment.
Mi Xu is a second-year master's degree in education economics and management student at the School of Education, Tianjin University. Her main research direction is the education quality assessment. 2019212099@tju.edu.cn