Evaluation scheme design of college information construction based on a combined algorithm

Evaluation index

At present, many colleges adopt four stages of EI strategic planning goals. These include the primary construction goals, IC skills training of teachers and infrastructure, and promoting the construction of essential IC tools. The college innovation goals for IC programs, the strengthening goal of cultivating students’ collaborative learning and autonomous learning ability through IC, and information literacy goals for teacher training and cooperative learning (Sabiri, 2020). To provide a more in-depth evaluation of teachers’ education level from the three dimensions of informatization and sustainable education.

After the formulation of the 12th Five-Year Plan, China’s overall education information infrastructure construction and development level have been greatly improved, but there is still a serious imbalance between regions. After a detailed analysis of different aspects of the framework (Ministry of Education, 2013), the National Education Statistics in 2013 added information statistical indicators such as “Access to the Internet,” “Multimedia utilization rate,” and “Network multimedia classroom” for teachers’ teaching and students’ learning. In 2019, the EI and network security worked to refine the requirements for college network and information technology applications (Ministry of Education, 2019).

The existing evaluation index system is mainly aimed at the overall informatization development of colleges. Still, it is only a dimension in the evaluation of college informatization. However, the relevant evaluation indicators of management informatization are relatively simple, and there is no rich and comprehensive evaluation index system specifically for the EI. Most of the current indicators of EI are formulated for the result data, ignoring the potential value of process data. Since the big data era has arrived, we can fully utilize its potential and consider both the process and data when evaluating EI.

EI evaluation

In the field of EI evaluation, researchers have been applying new statistical methods and introducing computer technology. Different evaluation models are used to realize the application of varying evaluation scenarios. Lucas, Promentilla & Ubando (2017) introduced AHP to quantify the effect of teacher EI training according to the preference value of participants. Tao, Wei & Xu (2021) used the combined algorithm to obtain the subjective and objective evaluation weights. He then adopted the improved augmented Lagrange multiplier algorithm to get the combination optimization weights for a more effective evaluation. Li et al. (2019) used the Delphi method and AHP to construct the higher EI development level evaluation index system and conducted empirical research at Shanghai University. Wang (2008) used the AHP and grey theory to comprehensively evaluate EI.

In recent years, many scholars have applied the neural network in education. Lino, Rocha & Sizo (2019) applied BPNN to an automatic evaluation in a virtual teaching environment, optimized it by Bayesian regularization, and simulated expert scoring through different evaluation models. Zhike (2013) uses the immune genetic algorithm to improve the standard BPNN algorithm. Taking English teaching performance evaluation data from three universities as examples, the effectiveness and feasibility of the improved algorithm are verified. Huang (2013) applied the improved BPNN to the evaluation of basic EI, completed the modeling and implementation of the effectiveness evaluation of EI, and proved that the application of BPNN in the performance evaluation of basic EI has high stability. Peng (2012) used BPNN to build a relevant mathematical model, applied it to identifying education quality evaluation grades in ethnic colleges, and realized a more scientific and reasonable evaluation mode.

Overall design

The specific factors affecting the system planning of the UIC model include infrastructure, resource construction, IC and safeguard measures. Various factors play a role in the initial stage, development stage and optimization stage of the model. Each level of the model contains several key process areas. The process domain is the boundary range of the feature attribute set. The IC model’s current status can be identified by judging the internal process domain characteristics. The key process area is one of the essential bases for constructing the model index evaluation. It can make the index evaluation system reasonable and complete and meet the dynamic requirements. The four-quadrant method is used to study the relationship between the factors affecting the construction of the UIC and the correlation between each factor of the whole model, as shown in Fig. 1.

The infrastructure factors in Fig. 2, such as the construction of the multimedia network and the investment of various hardware resources, play a fundamental role. The main focus of UIC is resource construction, which includes creating and updating multiple educational resources, research resources for science, and evaluation indicators.

Architecture design

The evaluation system mainly includes a data acquisition, evaluation, and display layer. The system architecture is shown in Fig. 2.

The bottom layer of the system architecture is the data layer, which stores all kinds of data of the evaluation system, including evaluation index, index weight, process, result and fundamental data. The evaluation index data is to input the index system’s name and each index’s grade into the database. Index weight data refers to the proportion of each index in the overall weight. Process data refers to the data generated in the process of user operation. This part of the data eliminates the error caused by human subjectivity, which is the most important part of the data layer and the key data for information evaluation. Finally, result data refers to the data reflecting the IC, such as network bandwidth, consumable funds, etc., while the primary data include the user and organization table.

Data acquisition includes automatic and manual acquisition in the relevant layers. Automatic acquisition uses a docking interface to collect relevant evaluation data, which is easy to implement with open-source software such as Flume. In contrast, manual collection refers to data submitted by appropriate personnel and cannot be automatically acquired by the system, such as information disclosed on websites and in the media. Data cleaning is converting noisy data into normal data and preparing data for information evaluation.

The evaluation layer’s primary function is data-based evaluation and calculation. According to the evaluation algorithm, the MapReduce program is compiled, and a big data cluster calculates the development of indicators. The display layer mainly uses visualization technology to display the evaluation results of the system, including various statistical charts.

Evaluation index

All evaluation index documents issued by the superior departments play a guiding and exemplary role in the UIC. Still, there are more or fewer constraints among each index system. To fully reflect the achievements of IC, we need to implement three aspects of work: (1) establishing the basic framework model, organizing the index system and relationship, clarifying the framework, and standardizing the process; (2) an optimal combination method is found between the constraints and the corresponding evaluation indexes to balance the weight relationship of each scoring system to obtain the optimal solution, which reflects the unity of objective evaluation and subjective experience.

To build a good evaluation system, it is necessary to determine the weight ratio of each index. Big data tools should be used to screen the indicators of the evaluation system, and the correlation between indicators and the intrinsic correlation between indicator elements should be intensely mined. This article establishes an index evaluation system based on infrastructure, resource management, information management, and safeguard measures, which takes the four factors as the first level evaluation index. Then it refines each level index into several secondary indicators. The evaluation index system of UIC is shown in Fig. 3.

As the social, economic and network environments are dynamic. The model’s validity and dependability should be periodically assessed, along with the primary and secondary indicators and the weight proportion relationship between the index systems, to ensure that the information management model system is in step with the growing network trend.

Evaluation scheme

The comprehensive evaluation module measures the collected and cleansed data, and the calculation results will reflect the informatization level of college education management. This study uses the comprehensive index method to evaluate the education management of UIC. Due to the weak correlation between the indicators, the difference between the index values is not big, and the importance of each index is relatively uniform. The linear weighted model can be used to calculate the comprehensive index, as shown in Eq. (1): (1)${\displaystyle EDI=\sum _{i=1}^n{W}_i\left(\sum _{j=1}^m{W}_{ij}\left(\sum _{k=1}^r{W}_{ijk}{Z}_{ijk}\right)\right)}$

EDI represents the information level of college education management, which is directly proportional to the information level of UIC; n is the number of first-level core indicators of the UIC level; m is the number of i ^th class indicators of the UIC level; r is the index number of j ^th class index of the UIC level, representing the number of the final index; W_i is the weight of ith class index in the total index, and ${\sum }_{i=1}^n{W}_i=1$; W_ij represents the weight of the j ^th index in the i ^th index, and ${\sum }_{j=1}^n{W}_{ij}=1$; Z_ijk is the infinitely toughened value of the k ^th index of the j ^th index of the i ^th index, and ${\sum }_{k=1}^r{W}_{ijk}=1$.

First-level index evaluation

Firstly, the AHP method is used to determine the weight set $\omega =\left\{{\omega }_1,{\omega }_2,{\omega }_3,{\omega }_4\right\}$ of the first-level indicators of the information construction model, where ω₁ is the weight of infrastructure, ω₂ is the weight of resource management, ω₃ is the weight of information management and ω₄ safeguard measures. The Hadoop tool is used to study the original data of colleges related to the primary index to determine the importance weight. Four factors form the importance matrix of index weight (2)${\displaystyle H=\left[h_{ij}\right]=\left[h_{11}{h}_{12}{h}_{21}{h}_{22}\right]}$ where h₁₁ is information management, h₁₂ is infrastructure, h₂₁ is safeguard measures, h₂₂ is resource management.

Then determine the importance of each secondary evaluation index under the first-level evaluation index. Through the weight relationship between the first-level index and the second-level index, and calculate the maximum characteristic root λ of matrix H, then the consistency index η of UIC model can be expressed as: (3)${\displaystyle \eta =\frac{{\lambda }_{max}-n}{m-1}}$ where m is the number of matrix state vectors, n is the number of evaluation objects. When the consistency index value η < 0.1, it is considered that the design of the first-level index of the UIC evaluation system is reasonable.

Secondary index evaluation

System performance test

Taking five universities in a city with an established model as research samples (statistics time is from October 2018 to September 2021), the function and performance of each module of the model are verified. This article mainly tests the system performance from two aspects: the response time of the system and the number of users simultaneously. For a single program, the duration of entering the system and retrieving information is less than 5s to ensure a better user experience. The response time of 10 clients was counted, and the traditional evaluation model was introduced for comparison. The change curve of time response is shown in Fig. 4.

Figure 4 shows that it takes 2s to open a single program, 9s to open five programs, and 13s to open ten programs; While in the traditional static mode, they were 8s, 19s, and 23s, respectively. Compared to the other models, the efficiency of the proposed model is higher. In addition, the capacity of the IC evaluation system based on a combined algorithm is more robust.

Model validation

Results and discussion

The convergence of the PSO-BPNN model is shown in Fig. 5. The epochs of the BP algorithm in the iris dataset is 69 that of the standard PSO-BP algorithm is 59. Whereas it is 50 for the improved PSO-BPNN algorithm, which can be seen that the convergence speed of the model proposed in this article is faster when the convergence objectives are the same. The prediction effect of the model is shown in Fig. 6.

It can be seen that the improved PSO-BPNN algorithm has a better effect than BP and PSO-BP algorithms on the prediction output. In addition, the evaluation model of UIC based on the PSO-BPNN has a good result, and its fitting degree and trend are consistent with the expected value. This indicates that the generalization ability of the PSO-BPNN model is higher than that of the original BPNN model based on subjective and objective comprehensive evaluation data. When the amount of data is larger, the PSO-BPNN can be used to ensure the stability and accuracy of the evaluation. Still, it can also improve evaluation efficiency, which provides a new idea for evaluating the development level of university informatization.

Evaluation effect of UIC

Based on the first-level index weights of big data evaluation samples of a college in the past four years, this article evaluates the four aspects of infrastructure, resource management, information management, and safeguard measures. The results are shown in Fig. 7.

According to the comprehensive score obtained, combined with the known experience and achievements of UIC, it is clear that the investment in facility construction in the past four years has improved the nominal index to a certain extent but combined with the influence of other related factors, the total score has not achieved synchronous growth. In terms of information support, the score has been continuously improved due to the improvement of the operating environment and the development of curriculum construction. Still, its growth trend has slowed with the extension of time, which means that if we want to ensure the effect of IC, we must consider making long-term continuous adjustments or investments in the system, which is the focus of the future UIC. In addition, regarding information application and data resources, the overall improvement is not evident due to the weight control, which reflects the significance of the combined weight algorithm. Because of the adjustment of the combined weight of the support and maintenance system, the work and evaluation results show synchronous correlation changes, which further verifies that the evaluation model can effectively reflect the effectiveness of UIC.