Mathematical Models for Evaluation of the Higher Education System Functions with DEA Approach

The purpose of this research is to develop the Data Envelopment Analysis (DEA) methodology for modeling of the assessment of the regional higher education systems effectiveness. The importance and topicality of this study is based on the increasing role of universities in the economic development of regions and countries in recent decades as well as the need to develop approaches for assessing the university effectiveness, and using mathematical models and methods for these goals. The novelty of the research is the formation of the DEA model and its application to the analysis of regional higher education systems’ effectiveness. The hypothesis of uneven development of regional higher education systems was tested from the standpoint of functional approach; the higher education systems’ effectiveness has been calculated and the ranking of Russian regions was performed by different DEA models. As a result of the DEA modeling, a quantitative effectiveness assessment was carried out, and a set of Russian regions was ranked according to three basic university functions: education, science, and regional partnership. Conclusions about the level of effectiveness and development strategy of regional higher education systems in Russia have been drawn.


INTRODUCTION AND LITERATURE REVIEW
The importance of universities and their strengthened role in economic development, their contribution to the development of human capital, innovations and knowledge transfer make the research that determines and assesses the university effectiveness and their role in the economic development highly relevant.
In this context, the use of modern methods of mathematical and computer modeling, Data Envelopment Analysis (DEA) in particular, in order to obtain quantitative and qualitative nonparametric assessments of the higher education system performance appears to be an urgent research challenge.
The purpose of this study is the development and application of DEA methodology and tools for assessing the economic process effectiveness in decision-making in socioeconomic macrosystems, in particular, in modeling of the assessment of the regional higher education systems performance. For this purpose, the following tasks have been completed: -studying of the possibilities and limitations of DEA model applied in the evaluation of regional higher education systems; -defining a set of relevant indicators that reflect input resources and outputs of regional higher education systems; -adjusting and implementing a DEA model for the assessment of the effectiveness of regional higher education systems; -accumulating empirical data for the research: approbation of the developed model and comparative analysis of regional systems of higher education in 80 regions of the Russian Federation.

THEORETICAL ANALYSIS
In modern conditions, the competitiveness and sustainable development of the national economy is largely determined by the dynamics of regional systems and the effectiveness of interaction among their subjects, among which universities become a significant driver of economic growth [1][2][3].
The last decades saw profound systemic structural reforms in the Russian system of higher education. Management decision making aimed at "fine-tuning" of the efficient and optimal size of the higher education system, elimination of inefficiently functioning educational organizations, improvement of the quality and transparency of budget fund spending and applying the effectiveness principle in using funding sources is based on the evaluation of the university effectiveness.
In foreign and domestic research, considerable experience has been accumulated in the use of various specifications of DEA models in assessing the education effectiveness, there are a lot of intercountry comparisons of the university effectiveness, various aspects of the university effectiveness according to different input and output indicators are assessed, there are qualitative conclusions about factors that influence university effectiveness [4][5][6][7].
However, the nature of the federal structure of the Russian Federation, the vast territory, the heterogeneity of economic development make it necessary to assess the effectiveness and ranking of not only universities, but also the totality of universities in the region, subject to the influence of general institutional factors of the external environment. The policy of managing a balanced development of higher education in Russia should be based on regions, rather than on individual universities. The ranking of Russian regions in terms of the effectiveness of regional higher education systems has not received enough attention and these issues remain unresolved, highlighting the ongoing relevance of the present study, which is pioneer in this area.

DEA approach
Data Envelopment Analysis (DEA) is an area of operational research which is actively used to simulate and evaluate the effectiveness of organizations in various branches [5].

✸✺✷ ❮àó÷íûé îòäåë
As a nonparametric method DEA differs from parametric methods (conventional least square, maximum likelihood estimation and others) as it uses linear programming techniques to calculate the envelope representing the efficiency boundary. DEA does not require the definition of a functional form that is an important advantage of this approach. The DEA evaluates the effectiveness of a set of n peer entities called Decision-Making Units (DMUs) that convert multiple inputs to multiple outputs.
The set of objects is considered in the multidimensional space of input and output indicators, and a piecewise linear surface is constructed. The efficiency boundary (efficiency frontier) -which is the relative effectiveness of a particular element from the sample -is determined. Thus for each object in the DEA model the scalar value of the effectiveness is calculated.
The main efficiency measure of DEA for DMU is the ratio of the sum of DMU weighted outputs to the sum of its weighted inputs.
DMU is assumed to be effective in a DEA model if the object belongs to the frontier. DEA generates a set of reference objects based on the available data, and DMUs are rated as ineffective in dependence on their locations from the frontier.
The base concept of DEA applies mathematical programming to get the maximum ratio of efficiency of DMUk [9].
Technical efficiency T E k of DMUk can be represented as follows: where u r is the weight of output r, v i is the weight of input i, y rk is the quantity of output r of DMUk; x ik is the quantity of input i of DMUk. The following fractional model has to be solved for each DMUk: It can be converted to the linear program by two different approaches (input-oriented model, output-oriented model). For input-oriented model the weighted sums of inputs are minimized holding outputs constant. For output-oriented the weighted sums of outputs are maximized holding inputs constant. Input-oriented model in multiplier form is as follows: ➮çâ✳ Ñàðàò✳ óí✲òà✳ ❮îâ✳ ñåð✳ Ñåð✳ ❒àòåìàòèêà✳ ❒åõàíèêà✳ ➮íôîðìàòèêà✳ ✷✵✶✾✳ Ò✳ ✶✾✱ âûï✳ ✸ Output-oriented model is as follows: If we transform these models into a dual form, the envelopment form will be obtained: -input-oriented model (Dual equation) where 1 ϕ k and θ k represent the technical efficiency of DMUk, λ j represents the associated weighting of outputs and inputs of DMUj.
The envelopment form of a DEA model contains only s + m constraints rather than n + 1 constraints in the multiplier form, and it is more convenient to calculate.
There are several types of DEA based on the above basic model that vary by parameter of returns to scale including CRS (constant returns to scale), VRS (variable returns to scale), NIRS (non-increasing returns to scale), NDRS (non-decreasing returns to scale), GRS (generalized returns to scale) [10,11].
These DEA models are distinguished by different return to scale parameters. Constant returns to scale assumption is indicated in the constraints of the model. A measure of return to scale for DMUk is added in dual equations:

Indicators and data
The DEA method is a specific benchmarking tool that provides an opportunity to determine the most efficiently functioning objects. A DEA model allows to assess the relative effectiveness using mathematical programming methods.
Within the framework of DEA modeling, the regional educational system can be represented as a converter of its "input" parameters -resource costs, into the variable parameter of the "outcome" -the performance results: students entered the labor market, the commercialization of innovation and social spillover effects of the university functioning in the region.
The choice of indicators for DEA modeling in this study is based on a functional approach to assessing the effectiveness of regional higher education systems. The evaluation of the effectiveness of the regional system of higher education is done on the basis of its performance, which is expressed in the realization of its basic functions: the number of trained students, the effectiveness of researchers and scientists as well as the intellectual activity results, and also the number of partnerships in the regional economy that influence the economy of the region.
As a result, the indicators selected for the analysis were classified for the Input and Output items (Table 1).
➮íôîðìàòèêà ✸✺✺ ➮çâ✳ Ñàðàò✳ óí✲òà✳ ❮îâ✳ ñåð✳ Ñåð✳ ❒àòåìàòèêà✳ ❒åõàíèêà✳ ➮íôîðìàòèêà✳ ✷✵✶✾✳ Ò✳ ✶✾✱ âûï✳ ✸ To calculate the technical efficiency of the regional higher education systems based on the selected indicators, a sample of data was compiled for 830 universities from 80 regions of Russia in 2015 (the latest available data for the regions of the Russian Federation) [13,14]. As the results of the analysis, 31 regions were selected for further modeling by the criterion of the amount of funds received, that is, the amount of funding for regional higher education systems. The largest volumes of financing were established in the regions where significant investments were made in recent years in accordance with the state policy of development and reform of higher education. These regions have large federal, national, flagship universities and the Project 5-100 universities. These universities, as large actors of regional systems, make a significant contribution to the regional development, play a significant role in regional economy. From the point of view of the present study, the evaluation of their effectiveness is the most informative for analysis and indicative of the results.

Applying the DEA model
The proposed approach uses three different DEA models with the same set of inputs and different outputs. The models correspond to different areas of assessment of the university activities described in Table 2: educational activity (EDU), scientific researches (SCI), innovations and regional partnerships in the regional economy (IRP). A rough estimation rule of value n in a DEA model is to choose n max{m · s, 3(m + s)} [15]. The quantity of DMUs depends on the practical purposes of the research, but it should be noted that as the number of input and output variables increases, the number of effective entities increases. In this case, it is suggested to take the n = 31.
We have proposed an output-orientation model, because such DEA maximizes output for a given level of input. VRS model's orientation depends on researching objectives. The model which has been applied is NDRS type of returns to scale. Table 2 presents the computed values of technical efficiency of regional higher education systems for 31 regions of Russia in three different models in accordance with the functions of the higher education system.
The DEA model resulted in the ranking of the regions according to the technical efficiency indicator. This allowed to determine the homogeneity of the regions, to identify leaders and outsiders among the regions.
These regions have an effective structure and return to the funds invested in regional higher education systems; their experience requires detailed analysis, study and dissemination in benchmarking of other regions of Russia. Table 3 presents the final results of technical efficiency evaluation of regional higher education systems. When we compare the scores from models EDU and SCI, EDU and IRP, we can see significant differences. In terms of educational performance, 29% of regional higher education systems are effective, and there are no inefficient universities with θ < 0.5. In scientific research and innovation and regional partnership, the percentage of effective regional systems of higher education is much lower: 16%. Thus, while forming strategic directions for the development and reorganization of the Russian regional higher education systems, more attention should be paid to the research and innovation. Figure 1 shows diagram sticks 3D (visualisation method for plot of the Scatter type) for estimating the magnitude and dispersion of the obtained technical efficiency values for regions.
There are no leading regions in all three areas of development. It means that for all regions there are areas that require further improvement. At the same time, there is only a small number of outsiders in the indicated set of regions in three direction of assessment. Data visualization does not allow to conclude that there is a relationship among the technical efficiency indicators in three parameters by region. Figure 2 shows the technical efficiency values for each region from the source list. This will allow to compare of 16 regions that are effective in one or several areas in accordance with the university functions. For these regions, the value of technical efficiency equals 1 at least in one model.   For the leading regions presented in the diagram, there is a significant imbalance in the technical efficiency values in various areas. This leads to conclusion that it is necessary to develop a more rational strategy for the balanced development of higher education systems in such regions.
The use of the DEA methodology has made it possible to assess the performance efficiency of regional higher education systems and the achievement in outcomes and to draw the following conclusions.
➮íôîðìàòèêà ✸✺✾ ➮çâ✳ Ñàðàò✳ óí✲òà✳ ❮îâ✳ ñåð✳ Ñåð✳ ❒àòåìàòèêà✳ ❒åõàíèêà✳ ➮íôîðìàòèêà✳ ✷✵✶✾✳ Ò✳ ✶✾✱ âûï✳ ✸ Thus, the results of the region ranking in educational function efficiency and graduate employability turned out to be quite predictable: the given regions with their economic level are able to provide jobs for the graduates. These are industrially developed regions with well-balanced regional systems of higher education, where universities can provide training that meets the employers' needs. This is connected with the development of the region's economy.
Regions with relatively high efficiency of the innovation infrastructure development, business incubators, centers for common use of technologies, university affiliated small innovative enterprises: the Belgorod Region, the Ulyanovsk Region, the Republic of Bashkortostan, the Republic of Tatarstan, the Tomsk Region are recognized leaders of innovation in Russia. This indirect assessment of the effectiveness of the university research and innovation practice in terms of commercialization of their innovations is the evaluation of the university's effort and skill to build a partnership with stakeholders and innovative infrastructure organizations in the region.
The evaluation of the research efficiency in terms of publication activity of universities and regional higher education systems provides the following results: the leading positions are occupied by Stavropol and Dagestan. Regions with international ranking universities, with strong scholarly traditions in the field of fundamental and technical research, with high publication activity and high citation rates in international WoS and Sc databases, have significantly lower technical efficiency values in this area, for example, the Tomsk Region (θ = 0.34), the Novosibirsk Region (θ = 0.46), Tatarstan (θ = 0.44). These results show that the standard reporting indicators of publication activity need revision in a situation where the number of publications does not reflect their quality.

CONCLUSION
The development of DEA methods for assessing the effectiveness of the regional innovation system and the university will allow to study various aspects of regional university performance and create tools for such assessment.
Demonstration of the DEA methodology in such area has the essential value for further research. Regional higher education systems have been examined through DEA for three important parameters connected with different functions of the university. The DEA model makes it possible to obtain a quantitative estimation of the higher education system's efficiency, identify the leaders in the set under consideration, analyze the environment. As a result of monitoring, it is possible to formulate an optimization strategy for the regions with low performance indicators, focusing on the decisions of the leaders.