Comparative Study of Comprehensive Evaluation Model for Academic Quality of Food Journals

This study puts forward an academic level evaluation model of journal based on a rough-set-equivalent thinking and neural networks and tests the model’s efficiency and practicality by comparing it to the traditional evaluation methods. First of all, the forming of this evaluation model includes the simplification of journal evaluation with theories based on rough-set-equivalent thinking and the abandoning of miscellaneous evaluation indicators. Secondly, the remaining essential evaluation indicators would be used to form plenty of training samples for the neural networks’ building up. Lastly, the neural networks would use the BP algorithm to rank those samples in general and therefore forms the journal academic level evaluation model. In order to testify the effectiveness of this model, other methods of TOPSIS is used to evaluate these journals and gray-relation-based thinking is used to set the essential indicators’ weights, which provide another outcome for comparison. The instance analysis of food journals indicates that the process of building this evaluation model is secured and logical and the model could well fit into the actual food journals academic level evaluation.


INTRODUCTION
Academic evaluations' importance in the academic journals is growing.Even though it remains controversial on whether it's good for the academic journals, it becomes a secured fact that academic evaluations make academic journals more important (Hu and Fu, 2008).Currently, China's academic evaluation system is not quite satisfactory, which means it is not robust, incomplete and not standardized.In this case, the craving for interests and benefits and the phenomenon of exaggeration is inevitable (Qiu and Li, 2008).Therefore, it becomes essential to construct a logical journals evaluation system and methods in order to process the data mine fully and avoid man-made factors.Only if academic journals scientific evaluations' logicality is proved multiple times, can the academic's self-esteem be re-established and finally accepted by the society.
Academic evaluation of journals is a complicated systematic project, how to reasonably and objectively evaluate journals is one of hot issues in present research (Lou et al., 2009).Currently there are some methods and models, such as, principal component analysis (Chen, 2004), Attribute interval recognition model (Yan, 2006), normalization method (Liu et al., 2006), grey correlation degree method (Liu and Zeng, 2013) and some other methods (Wang and Yu, 2012;Lv et al., 2012).These methods are commonly affected by man-made factors in some extent and inevitably look poor on their results' objectivity.
The rough set theory is a mathematical tool proposed to quantify and analyze the imprecise, inconsistent and incomplete information and knowledge, which was raised by professor Pawlak (1982).By using the rough-set-equivalent thinking to fully datamine the statistics, the troubling man-made factors can be well avoid and therefore making the results more objective.Li and Yun (2010) made a specific description on the rough-set-based indicator system constructing and general evaluation methods.He et al. (2014) made an evaluation on a science journal by using the rough set theory and TOPSIS methods.Huang et al. (2015) made a comparison research on the journals using the rough set theory and the gray relation's evaluation for the journals.
On the other hand, as an artificial intelligence technology, the neural network possesses the ability of self-learning and self-organizing and is especially suitable for the non-linear modeling (Statsoft, 1999).Cai et al. (2014) did a research on establishment of food safety evaluation model based on neural network.Silva et al. (2015) used artificial neural network to make an evaluation of extra virgin olive oil stability.
This essay puts forward an academic level evaluation model of food journal based on the roughset-equivalent thinking and neural networks in order to make evaluations more objectively.Meanwhile this essay makes comparison between the model and the gray relation evaluation methods' evaluation results, in order to tests the model's efficiency and practicality.By referring to the reference documentation's (He et al., 2013) journal indicator statistics, the instance analysis presents that the model has precision and suitability.

METHOD
Algorithm of indicator reduction based on equivalence relation: Rough set theory extends classical set theory, the knowledge, which is used for classification, is embedded in set as a part (Pawlak, 1982).There are some key definitions as follows: Definition 1: Any information system can be abbreviated as a two-tuple , S U A   , U stands for the objects' non-empty set and A stands for attributes' set.

Definition 2: Assuming that ( )( ) ind D D A
 is an equivalence relation, the equivalence class formed by it is , then a is redundant in the knowledge system, otherwise a is necessary.
As for the evaluation indicator system for food journals' academic level, it contains differently based evaluation indicator sets (because of miscellaneous indicators) that form some same evaluation classes of the evaluation objects.Based on the rough-setequivalent thinking, finding the necessary indicator from the indicators set that form the same evaluation class could make this evaluation class identical to the original evaluation results which was based on all of the indicators.This method could realize effective reductions of evaluation indicators without losing any beacon information.
Thereby, heuristic algorithm of indicator reduction based on equivalence relation can be established as follows (Li and Yun, 2010): Denote IND(D) = B is the indicator system after reduction.
Step 1: For indicators system D { }( 1, ..., ) , then a i is a deletable miscellaneous indicator in indicators system D, otherwise a i is an undeletable necessary indicator.
Obviously, this method can effectively make to reduce the complexity of solution and the method is simple.

Comprehensive evaluation model with BP network:
Normalization: Due to the different dimensions of evaluation indicators, it needs to eliminate the dimension of the data to compare it.In order to make the data in [0, 1], we adopt the range transformation method to obtain normalization matrix X = (x ij ) m×n , that is: where, r ij is the value of the corresponding journal indicators, x ij is the dimensionless value and also .
Generation of sample data: According to reference (Lou et al., 2009), it shows that journal evaluation grade is decided by the upper and lower limit of evaluation indicator, that is, a certain indicator i a of the evaluation system should meet: where, a i1 = The lower limit of evaluation indicator a i2 = The upper limit of evaluation indicator Since we obtain the key indicator system, the importance degree of each indicator is quite high.To a certain extent, in order to simplify the model, we can consider the weight of all the indicators in the key indicator system is almost the same.Therefore, in order to reasonably expand the training sample data, we believe that, if the two indicators of journal evaluation system up and down reversely 10%, the grade of the journals evaluation is the same and the generated data can be relatively well training for neural network.Thus, as defined herein, when a two indicator a1, a2 of the key indicator system B of journals meet: That is, when indicator data c1 increases (or decreases) 10% to generate a new indicators data a3, at

Model structure of BP neural network:
In this study, we use a three-layer feed forward BP neural network structure as shown in Fig. 1 to map the nonlinear relationship between academic journal indicators and its grade.
In the neural network, x 1 , x 2 , …, x n is journal indicators, y is journal evaluation grade.Fun is Logarithmic sigmoid transfer function and the input layer and output layer neurons use pure linear transformation function.
The selected network training evaluation function is a function of the mean squared error:


where, y = The network predicted output t = The desired network output SSE = Normalized total error of neural network learning n = The number of the output unit p = The number of samples After that, we use the sample data to have trainings, after training and we obtain the evaluation model.And then we input the indicator data to get an output value, with the output value to judge journals' rankings.

Evaluation with grey correlation-Topsis:
In this study, we do a gray correlation weights analysis, by studying the correlation of evaluation indicator and journals ranks to obtain the importance of each factor and the weight according to their importance (Huang et al., 2015).
Base on the key indicators, we use grey correlation weights methods to calculate the correlation degree of journals and calculate the weight of key indicators and then we use TOPSIS method (He et al., 2014) to rank.

Reduction of evaluation indicators based on based on equivalence relation in rough set:
This study selects 20 food journals from CHINESE S&T JOURNAL REPORTS (Expanded Edition) as research objects.Let denote J1, J2, J3, J5, J6, J7, J8, J9, J10, J11, J12, J13, J14, J15, J16, J17, J18, J19 and J20 as these research journals in order: Food Science, Food Science and Biotechnology, Chinese Institute of Food Science, Food Industry, Chinese Cereals and Oils Association, Food Science, Food Science and Technology, China's dairy industry, China oil, Chinese mushroom, Food and machinery, Chinese spices, Food research and development, Dairy Science and Technology, Chinese food additives, Chinese food and nutrition, Food and medicines, Preservation and processing, Salt and Chemical Industry, Food industry.
We use 11 evaluation indicators from A Report on Chinese Academic Journals Evaluation and all of them are the maximum indicator.Let D1, D2, D3, D4, D5, D6, D7, D8, D9, D10, D11 respectively represent: extended total cited frequency, extended number of citing journals, extension immediacy index, extended cited rate, extended impact factor, expand discipline impact indicators, expand discipline diffusion index, extended H index, literature sources amount, average number of citations and fund citations number.
The evaluation indicator set D = {D1, D2, D3, D4, D5, D6, D7, D8, D9, D10, D11}, thereby we establish journal evaluation system.We divided all the selected journals into four levels based on these journals in academic journal ranking.Denote R as a rating score, with the number of 0.6, 0.7, 0.8 and 0.9 and the greater the score, the higher ranking.The raw data obtained are shown in Table 1.
Due to the different dimensions of evaluation indicators, In order to make the data in [0, 1].We adopt the range transformation method (Specific methods as above) to do with data in Table 1 and the results (portion) shown in Table 2.
Then we discretize the data of evaluation indicators in Table 2 by equidistance division method.As the journals selected are divided into 4 grades, so in our paper, the range between the minimum and maximum is also divided into 4 intervals.The integers from 1 to 4 are assigned to 4 intervals from small to large.The discrete results (portion) are shown in Table 3.
According to algorithm of indicator reduction based on equivalence relation, we write programs to solve the problem and obtain five key evaluation indicators after reduction: Extended cited rate, extended impact factor, extended H index, average number of citations and fund citations number.That is, the grade of evaluation according to the key evaluation indicators can fully consistent with the results according to the all evaluation indicators.According to the reference (Li et al., 2007), it shows that the impact factor, H index, citations are all common key indicators, so the five key indicators herein after reduction obtained are reasonable.
Thus, after reduction, we can extract key evaluation indicators of journals, so we denote B1, B2, B3, B4, B5, respectively as extended cited rate, extended impact factor, extended H index, average number of citations and fund citations number.The indicator set B = {B1, B2, B3, B4, B5}.Thus, we established a key evaluation system of food journals.

Comprehensive evaluation of food journals based on BP neural network: Generation of training sample:
The establishment of reliable and efficient BP neural network model requires a lot of training samples and necessary test samples (Lou et al., 2009).We use the method in the above section to deal with indicator in the key indicators system, any two indicators of five key indicators up and down reversely 10% and the remaining indicators unchanged.In this way, we believe that the evaluation grades of journals are the same.With the help of the approach to deal with all the indicators, we can get 400 training samples.

Evaluation results of neural network:
With the help of MATLAB 7.0 neural network toolbox, we use training function of momentum back propagation algorithm of gradient descent.In the training figure, the vertical coordinate represents all the error level of parameters and the abscissa represents the n-th iteration steps in the iterative training process forward.The curve describes the error performance trend and straight line is the desired error target.In the training process, we use 400 samples data, 10 hidden layer nodes, the minimum error of training objective is set to 0.001, the training times are set to 500 times and the learning rate is set to 0.05.The sample training results are shown in Fig. 2.
Figure 2, in accordance with the expected error level of 0.005, after about 90 times of iterations to achieve the desired target error.After the training is completed, we can input evaluation indicators of 20 food journals selected and we can get the output value.Taking into account all the training indicators are the maximum indicator.So the greater output value, the higher ranking.According to the output values, we can rank the food journals.
We compare the results of the model, the Grey correlation-TOPSIS method and the rankings in a A Report on Chinese Academic Journals Evaluation.In this way, we can verify the reasonableness of this model.Comparative results are shown in Table 4.

Evaluation results of Grey correlation-TOPSIS:
We use Grey correlation-TOPSIS method to evaluate the journals, firstly we get the values of each grey correlation degree in the key evaluation system of food journals: Lastly we use TOPSIS method with the obtained weights to rank food journals in Table 3 and we obtain the evaluation results shown in Table 4.

RESULTS
This study selects 20 food journals from A Report on Chinese Academic Journals Evaluation as research objects, as well as searches the data of seven common evaluation indicators of these journals from CHINESE S and T JOURNAL REPORTS (Expanded Edition).Combine the comprehensive evaluation model of food journals based on rough sets and neural network and Grey correlation-TOPSIS method, we obtain the evaluation results of neural comprehensive evaluation model (BP for short), those of Grey correlation-TOPSIS method (G-T for short) and also the ranking of these journals in A Report on Chinese Academic Journals Evaluation(R for short).Three rankings above shown in Table 4.
As can be seen from Table 4, overall, the comprehensive evaluation result of food journals based on rough sets and neural network obtained and the rankings in A Report on Chinese Academic Journals Evaluation, is relatively close, especially the rankings of the top seven are almost the same, indicating that the model used herein is reasonable.Because different evaluation systems use different evaluation principles and evaluation indicators, the result is a little bit different.But the overall ranking trend is consistent.

DISCUSSION
In this study, we use the absolute value of the deviation between the obtained results and the rankings in A Report on Chinese Academic Journals Evaluation to describe the level of the evaluation results.The deviation value of Grey correlation -TOPSIS method is 64, but the deviation value of the comprehensive evaluation model is 30.Therefore, the established method in this study is relatively better than grey correlation-TOPSIS method and also it can show the rationality and accuracy of the model.
But few individual rankings of the comprehensive evaluation model of food journals based on rough sets and neural network in Table 4 have a relatively larger deviation, for example, the ranking of J9 and J17 have a relatively big difference.Since the weights of the key indicator system are the same in this study, maybe one or two of key indicators could have a larger impact on the evaluation results of J9 and J17.So the result has a larger deviation to some extent.
In summary, in the process of evaluation of food journals, the key indicator system after reduction is reasonable, the principle of generating the sample is feasible and effective and the modeling process of BP neural network is reliable and reasonable.Hence, the model has a certain practicality and applicability.

CONCLUSION
Compared with grey correlation, we know that based on rough set theory and neural network for journals evaluation can have a fully data mining, providing a model for a more efficient and reasonable academic evaluation of journals.
We take the advantage of rough set to deal with imprecision and uncertainty of the data sample and use it to make a pretreatment to obtain key indicator system, reducing properties of sample, reducing dimension of sample.On this account to map the key indicator to the training samples of neural network, we can build the number of hidden layer and hidden layer neurons and in this way, making the network more logical and reducing the training time of neural network, enhancing accuracy and generalization of training.We compare the results of the model, the Grey correlation-TOPSIS method and the rankings in A Report on Chinese Academic Journals Evaluation and we find that the results of the model are reasonable and accurate.
The modeling process is credible, but because we do not take into accounts the specific weight of each indicator of the key indicator system, just supposing they are approximately the same, the further research in the future we can optimize this point.
Rough set theory and neural network are widely used in the field of academic evaluation, but they can also be applied to other aspects such as mine ventilation system evaluation (Wang et al., 2011), Optimization of Parameters (Yang et al., 2015).And in the future, we can have an in-depth study.A hybrid of these two methods can be applied not only evaluation studies, also be to other fields, such as dynamic variation analysis (Guo et al., 2015).

Fig. 1 :
Fig. 1: Structure of a three-layer feed forward neural network the same time and indicator data a2 decreases (or increases) 10% to a new indicators data a4 and other indicators of the key indicator system do not change.In this way, we believe that the grade of the journal is in the same level.

Fig. 2 :
Fig. 2: The sample training results of the best hidden layer nodes

Table 4 :
Comparative comprehensive ranking result Journal-NO.BP