Evaluating University Academic Efficacy: Institutional Approach

Received September 17, 2019 Revised from October 15, 2019 Accepted November 29, 2019 Available online March 15, 2020 The aim of the article is to develop methodological approach to university academic performance evaluation on the basis of instruments of institutional economic theory. The novelty of this work is that unlike other methods authors’ method of quantitative evaluation of university academic performance allows for identifying the problems of institutional support of academic activities. During the research authors employed methods of system, logical and economic analysis; empirical information was processed with the help of statistical analysis and correlation analysis methods. As a result authors identified university academic efficacy institutions and suggested efficacy indicators for these institutions. Universities of the Ural Federal area were grouped according to quality parameters of academic efficacy institutions (that is presence of effective institutions, institutional traps, institutional malfunctions). JEL classification: I25, O22 DOI: 10.14254/1800-5845/2020.16-1.16


INTRODUCTION
Starting from the Decree of the Government of Russia of October 29, 2012 № 2006-р on the plan of activities for developing leading universities Russian universities increased their activities aimed at developing academic results. In order to ensure qualitative breakthrough in Russian universities' competitiveness level a project titled «5-100» was launched. Now we can speak of certain positive results of 5-100 project. For instance the period between 2012 -2018 witnessed an increase of publications in journals listed in academic citation database Web of Science from universities participating in the project. For example, the Ural Federal University and Novosibirsk State According to the opinion of Education Development Institute of HSE the most success in terms of achieving innovative character of education activities development is gained by those educational institutions that implement three types of processes:  developing educational technologies providing integration of project and research tasks into the teaching process;  developing projects, related to the development of different technologies, sectors and branches of economics;  conducting applied and fundamental research (Abankina et al., 2011).
Effective combination of all three processes in creating and renovating educational programs ensures their competitiveness. Researchers have witnessed two key trends in higher education system development. On the contrary, the Russian higher education is integrated into the international educational space. On the other hand, there is a strong regional orientation in the professional education in Russia that is an adaptation of personnel training and research systems and practices to peculiar features and needs of the region (Bain, 2003). These and other trends predetermine the need for a theoretical understanding of institutional transformations at the universities and formulating instruments for managing these transformations.

LITERATURE REVIEW
The research on transformation processes in higher education is based on the path dependence theory (Arthur, 1994). During the last decade, this theory was applied to the analysis of reform in the educational sphere by such foreign researchers as Strielkowski and Chigisheva (2018), Paradeise at al. (2009), Tortorella at al. (2020, Voda et al. (2019) and Gavurova et al. (2019). Dependence of institutions on the trajectory of their development explains many problems facing modern Russian higher education, including: imbalance in institutional interaction between regional labor markets and professional education institution and, consequently, imbalance of labor resources and real market requirements; discrepancy between personnel qualification requirements of employers and qualitative characteristics of the professional training of young specialists; high latent unemployment among young people.
Due to the complexity of this problem, many Russian research teams are busy solving problems related to the institutional transformation of the professional education system. A. Smolentzeva (2011) has conducted the analysis of higher education transformation using studying interaction mechanisms between institutions of higher learning and society in general (including state, business, regional and global community, etc.) in a comparative perspective. I. Abankina (2013) looks into the changes in conceptual approaches to education eco-nomics in the context of transition to broader resources (information, communication, intellectual, educational, symbolic, brand, etc.). The research team headed by Y. Kuzminova (2013) has conducted structural analysis results of which support the use of particular segment orientation and main University product characteristics as the foundation for the typology and subdivision of organization groups. The author also looks into trends and formulates aims for structural policy in the field of higher professional education.
There is an ongoing discussion among experts concerning the ultimate result of university activities and whether the diploma and academic degree can be considered as the one. One point of view, following A. Flexner (1994), is that universities should be considered as places of research and measured by their contribution to science. Another view, following R. Brown (2008) and many others, argues that the primary mission of a university be education. The third mission linked to public service is considered as important in a diverse democratic society and equally important to the other missions of a University (Checkoway, 2001) and S. Bush and L. Prather (2018).
Finally, there is the approach which considers every university a unique organization combining many missions (Marginson, 2007;Hasprova et al., 2017). Institutional analysis of university activities results can be based on resource dependence theory which states that on the one hand organizations depend on the environment, but the contrary can influence the environment they are functioning in (Pfeffer and Salancik, 1978).
According to A. Auzan (2013) besides qualified personnel training and research the modern university should be actively involved in forming "right" values and behavior of students.
Lately we see a sharp increase of research activities in the field of higher education management. The interest towards such research is based on the need for well-planned decisions in the process of reforming higher education and complexity of defining optimal economic and educational strategies of achieving competitiveness. However analysis of modern economic research demonstrates insufficient number of research works devoted to the topic of financing influence on university academic activities results. That is the reason behind this research devoted to the analysis of university academic activities efficacy.
The aim of the present research is developing a methodological approach to evaluating knowledge generation efficacy at the university that, unlike other known methods, allows for identifying the problems of institutional support of academic activities.
An important factor of modern university competitiveness and successful integration into modern knowledge -based economy is the implementation of academic activities management strategies.
Academic literature presents a multitude of methodological approaches to evaluating university academic efficacy. They can generally be grouped as follows:  Financial approach to evaluating university academic efficacy. It is a methodological approach based on evaluating the principles of financing and costs for obtaining academic results. Main indicators are financial expenditures on science and existing material and technical resources  Personnel approach to evaluating academic efficacy of universities. Methodical approach based on evaluating academic and research personnel quality and quantity. The main indicators are: number and academic degree of researchers, number of administrative and supporting staff, personnel training level (including acknowledgement indicator, covering membership in academies, councils and grant performance).  Innovative approach to evaluating university academic efficacy. Methodic approach based on evaluating innovation activities (including creation of own and the use of borrowed technologies).  Managerial approach to evaluating university academic performance. Methodic approach based on evaluating university management systems quality  Bibliometric approach to evaluating university academic performance. Methodical approach based on evaluating the following bibliometric indicators: number of publications in journals; citation indicator and Hirsch index; "publication load" of scientists; patents; co-authorship with foreign scientists.

METHODOLOGY
The institute can be defined as a combination of acting rules defining who has a right to make decisions concerning which actions are possible and which are not, which common rules will be used, which procedures should be followed, which information should be given or disclosed and how individuals will benefit from their actions.
Academic efficacy institution is an example of an economic institution.
Academic efficacy institution is a system of stable formal norms (rules) regulating interaction between two or more economic agents in an educational institution aimed at obtaining academic results and equipped with necessary executive mechanisms.
The idea of academic efficacy institution is in a stable long-term interaction between employees and organization aimed at obtaining academic results. First of all formal norms include labor agreements as well as various internal documents supporting relevant executive mechanisms.
The main types of academic results are: publications in Russian and foreign journals; monographs and publications in different non-periodicals; patents.
Different types of academic results are regulated by various documents, including execution and stimulation mechanisms. Therefore academic results institution consists of institutes, formed by norms regulating different publications, academic performance institution structure is presented in Figure 1. Within the framework of the present research authors use functional approach to evaluating efficacy of academic performance institution according to which evaluation is done in two dimensions:

Institution of publications in
 To which degree the institute fulfills the main functions that is achieving target efficacy parameters;  What is the balance between the size of academic activities financing and parameters characterizing these functions in educational institutions under research.
In order to evaluate performance of academic efficacy institutions universities use correlation analysis. Authors suggest the following ideas: -if the meaning of correlation coefficient is less than 0, that is negative we are dealing with an institutional trap. Institutional theory sees institutional trap as an ineffective stable norm (ineffective institution) of a self-sustaining nature;.
-if the meaning of correlation coefficient is from 0 to 0,6, it is an institutional malfunctionmalfunction of one of economic institutions predominantly of a qualitative nature;  if the meaning of a correlation coefficient is from 0,6 to 0,75 the institutions of academic efficacy is not working or we are dealing with institution development (creation);  if the meaning of correlation coefficient is more than 0,75the institute is effective.
With the aim of testing the method of evaluating university academic efficacy suggested by the author we conducted the analysis of academic efficacy institutions among the universities of the Ural Federal area. Informational basis of the research is derived from the series of annual informational and analytical collections of works "Academic potential of universities and academic institutions of the Ministry of Education and Science of the Russian Federation". Informational and analytical collections of works are a database of research, academic, technical and innovative activities of Russian universities for the period from 2009 to 2017. Indicators of state and development of higher education present state and potential are presented on the basis of annual reports on academic and research activities of the universities.

RESULTS
The first stage of research discovered connections between indicators characterizing academic results and overall amount of academic activities financing of universities under analysis. Results of the correlation analysis are presented in table 1.   Authors believe that significant differences between institution efficacy of these universities are explained either by internal policy of these universities (for example, Ural Federal University pays a lot of attention to performance indicator "foreign publications" which puts other academic efficacy indicators to a disadvantage), or by an initial stage of institutional structure formation.

GLOBAL UNIVERSITY RANKINGS
Many authors attempt to determine the importance of particular university ranking indicators from the perspective of global measurement systems. It seems that the most significant number of global rankings dominantly focus on the context of research success, which most consider to be a key indicator of the quality and competitiveness of universities.
Contemporary trends in the global economy, transformations in the labor market, and technological change (in fact: the knowledge economy) dominantly influence the basic directions of higher education development as well as scientific research. The significant growth in the number of universities has directly influenced the development of global systems and their ranking. Their primary goal is to provide the right information to all interested parties in the process of choosing higher education institutions. Global university ranking systems use a consistent set of indicators, primarily focused on measuring the success of the research process, through a variety of methodological and statistical techniques. Unlike the global ones, national ranking systems are more focused on educational and institutional parameters.
Regarding the perception of the research performance there are certain standpoints. Thus, for example, Vernon et al. (2018) point out the research performance from the aspect of scientific productivity, citation, and innovation. The research impact is viewed from two aspects: 1) from the citation, scientific production, and scientific excellence point of view, and 2) from the aspect of innovation, technology transfer, and technological impact. In addition, the university's global ranking systems also highlight some of the academ quality measures such as reputation, the environment quality, file characteristics, number of study programs, and the number of places on them, the relationship between the number of students and teaching staff, and the international mobility of students and teaching staff .
University ranking is particularly significant due to the fact that higher education has a strong impact on economic growth and income growth (Bloom et al., 2006, p. 16). The key factors of economic growth are knowledge, and the ability to create new technological and sustainable development. Higher education is extremely important for the economic competitiveness of every society, because higher education institutions generate knowledge and develop expertise and skills, enabling individuals to achieve their personal goals and become valuable members of society. Higher education is an infrastructure for future state-level social cohesiveness. It contributes to state competitiveness (Green et al., 2010) through human capital development, providing future returns to the economy, and increasing labor productivity.
According to F. Maringe and P. Gibbs (2009, p. 47) dynamic environment of the future higher education requires the development of new opportunities, such as: a more complex "educational product", a more pronounced social role of education institutions, and significant financial performances and competition. Due to all this, many authors attempt to model a way of gaining competitiveness in higher education. One of these models is shown in Figure 2. Also, as an author contribution, we have designed a positive impact of formal and informal institutions, and a negative impact of alternative institutions. The latter characterizes most post-socialist countries, with a very negative effect on the higher education quality. The main competitiveness indicator of higher education is the high quality. The term 'education quality' implies the following guidelines: (a) teaching quality (learning process design, teaching methodology); (b) academic staff quality; (c) study programmers quality; (d) equipment, maintenance and support rendered quality; (e) learning environment characteristics quality; (f) student characteristics; (g) university management quality, and (h) research quality. Therefore, the higher education quality is multidimensional, and its empowerment depends on many impacts, particularly on new forms of knowledge transfer supported by contemporary ICT tools. CONCLUSION Therefore during the conducted analysis the authors found out that the quality of university academic efficacy institutions should be defined using a set of performance criteria describing the correlation between the quantity and quality of academic publications and amount of financing. The use of institutional approach allows for defining narrow places in institutional support of university academic efficacy.
The research has identified 3 groups of universities: universities with the largest number of effective academic performance institutions (19% of the total number of Ural federal area universities); universities, which combine effective and ineffective (institutional malfunctions and traps) academic efficacy institutions-43% of the total number of universities; and universities with ineffective (institutional malfunctions and traps) academic performance institutions -38% of the total number of universities in the Ural Federal area.
The suggested method evaluating university academic efficacy can be used as instrument decision-making about improving academic results at the universities. The methodical approach makes it possible to compare research performance between different universities. Small alterations of the method allow for using it in the prosses of planning and monitoring research activity into the university.
Result of the authors' research demonstrates the following benefits of the suggested method for evaluating academic efficacy: -The assessing and monitoring the quality and cost-effectiveness of scientific productivity institutions; -The identifying and correcting institutional traps and dysfunctions arising in the course of scientific activity; -The optimizing the reallocation of financial resources that are designed to improve academic performance.
Therefore, the active competitive strategy of the university cannot be formulated without understanding the strong and weak points of the institutional environment for scientific activity. This paper contains the author 's method of assessing the quality and identifying weaknesses for academic efficiency institutions. The testing of the suggested method on the example of the Russian Ural universities showed its informativity and suitability for improving the decision-making efficiency regarding the institutional design of scientific activity at the universities.
The theoretical significance of this research is in the development of research productivity measurement theory using tools of institutional economics for further formulation of methodological recommendations on improving academic activities. The practical significance lies in the development of the analytical methods of studying institutional environment for the university 's scientific activity, as well as the optimization of the university science financing.