A DEA-TOPSIS-based approach for performance evaluation of Indian technical institutes

Article history: Received October 15, 2014 Accepted January 24, 2014 Available online February 21 2014 Since independence, India has been one of the few developing countries to invest extensively in both science and technical education. In India, technical education plays a pivotal role in human resource development while creating skilled manpower, increasing industrial productivity and enhancing the quality of life. If a technical institute means to be effective in developing learned and qualified engineers, then it would be useful to know the performance of that technical institution. However, measuring the performance of a technical institution has received very little attention because it is very difficult to measure its output. Thus, this paper focuses on assessing the performance of eight Indian Institutes of Technology (IITs) using a combined approach of data envelopment analysis (DEA) and technique of order preference by similarity to ideal solution (TOPSIS). In the first phase, DEA is applied to shortlist the efficient IITs having the desired characteristics from the stakeholders’ point of view, and TOPSIS method is then employed to rank those efficient IITs while also identifying the best performing IIT. It is observed that IIT Kharagpur outperforms all the considered IITs which exactly corroborates with the findings of the recently published surveys/reports. © 2014 Growing Science Ltd. All rights reserved.


Introduction
With 700 universities and 35,000 affiliated colleges enrolling more than 20 million students, Indian higher education is a large and complex system.One-sixth of all Indian students are enrolled in Engineering/Technology degrees.Technical education in India contributes a major share to its overall educational system, and plays a vital role in its social and economical development.India's technical education landscape is principally dominated by the Indian Institutes of Technology (IIT) and the Indian Institute of Science (IISc), with excellent faculties and facilities.Although the first IIT started functioning in Kharagpur in the year of 1956, the growth of IITs and IISc, with respect to both scalability and accountability, seemed to have gradually declined beginning in the late 1960's.In fact, no IIT or IISc was again put in place until 1995, which clearly indicates a failure on the part of the Indian government.
A large number of private engineering colleges with poor facilities and unqualified faculties thus came about after 1970, constituting nearly 75 percent of the total intake of engineering students.These colleges now number approximately 1,100 compared to 16 IITs and one IISc.Thus, technical education in India has developed a distorted image; graduating over 230,000 Bachelor's degree students, 20,000 Master's degree students and 1,000 Ph.D. degree students in 2006.By contrast, according to a December 2008 report from IIT Bombay, IITs and IISc have only 3% of this number at the Bachelor's level, 4% at the Master's level and 40% at the Ph.D. level.For a mere around 9,000 seats in all IITs, there are close to more than 1,000,000 applicants.Experience has shown that at least 10-15 percent of those applicants are of IIT quality and comparable to the students at major public universities in the U.S. The fact that a large number of non-IIT students turn out to be equally competent and innovative was largely ignored until about a decade ago.
Quality is a paradigm which would be successful only with a lot of inputs, like good students, good faculties and infrastructure.The faculty in a technical institute is extremely important, both in terms of adequate numbers and academic quality.The procedure for selection of faculty in IITs is stricter as compared to other institutions offering similar degrees.Unless there is a conscious effort within a technical institute to ensure quality, it would not happen.Former IIT students get greater respect from their peers, academia and industry in general.The IIT brand was reaffirmed when the U. S. House of Representatives passed a resolution honouring Indian Americans and specially graduates of IIT for their contributions to the American society.Similarly, China also recognised the value of IITs and planned to replicate the model.If a technical institution means to be effective in developing engineers and professionals, then it would be useful to know the performance of those institutions.However, measuring the performance of technical institutions has received very little attention compared with other industries because it is difficult to measure its output.Thus, it becomes extremely important to evaluate the IITs according to their performance in various fields of education.
Nationwide, most IITs are ranked above any other technical/engineering institutions in India in technical education domain.Top Engineering Colleges of 2012 ranked seven IITs in its top 10, including the top five positions.Although, there are now 16 IITs in India (Kharagpur, Bombay, Kanpur, Delhi, Guwahati, Roorkee, Madras, Bhubaneswar, Gandhinagar, Hyderabad, Jodhpur, Patna, Ropar, Indore, Mandi and Varanasi), in this paper, an attempt is made to assess the performance of eight IITs (Kharagpur, Bombay, Kanpur, Delhi, Guwahati, Roorkee, Madras and Varanasi) using a combined data envelopment analysis and technique of order preference by similarity to ideal solution (DEA-TOPSIS) approach.The remaining eight IITs are discarded from analysis because those have only started commencing their sessions from 2008 onwards, and no adequate information regarding number of research papers published in last five years (2009)(2010)(2011)(2012)(2013), placement of students for the year 2012, number of patents applied for and granted in last five years (2009)(2010)(2011)(2012)(2013) etc. are readily available from them.

Review of the past researches
McMillan and Datta (1998) assessed the relative efficiency of 45 Canadian universities employing DEA technique.Avkiran (2001) used DEA model to examine the relative efficiency of Australian universities, based on three performance models, i.e. overall performance, performance on delivery of educational services and performance on fee-paying enrolments.It was shown that the university sector was performing well on technical and scale efficiency, but there was room for improving performance on fee-paying enrolments.Abbott and Doucouliagos (2002) used DEA technique to derive estimates of the technical and scale efficiency of Victorian Technical and Further Education Institutes.Abbott and Doucouliagos (2003) used DEA model to estimate technical and scale efficiency of individual Australian universities, and observed that regardless of the output-input mix, Australian universities as a whole recorded high levels of efficiency relative to each other.
Johnes (2006 a ) applied DEA technique to 2547 Economics graduates from UK Universities in 1993 in order to assess teaching efficiency.Johnes (2006 b ) examined the possibility of measuring efficiency in the context of higher education and preferred the use of DEA technique for performance appraisal of more than 100 higher education institutions in England using data for the year 2000-01.Fandel (2007) presented a solution for a real process of redistributing funds for teaching and research among the universities in North Rhine-Westphalia in Germany according to some specific criteria.Kao and Hung (2008) applied DEA technique to assess the relative efficiency of the academic departments at National Cheng Kung University in Taiwan while considering total credit hours, publications and external grants as the outputs; and personnel, operating expenses and floor space as the inputs.Khan et al. (2008) applied DEA technique to assess the service quality of selected technical institutions in India.Debnath and Shankar (2009) examined the performance of 20 Indian B-Schools, and reported that the input and output measures did not seem to differ between the ten best run institutes and the next ten institutes in terms of scale efficiency.Tyagi et al. (2009) evaluated performance efficiencies of 19 academic departments of IIT Roorkee (India) using DEA technique.Katharaki and Katharakis (2010) estimated the efficiency of 20 public universities in Greece using DEA approach and identified that there had been a clear opportunity to increase research activity as well as research income.Employing DEA methodology, Agha et al. (2011) evaluated the relative technical efficiencies of academic departments at the Islamic University in Gaza.Operating expenses, credit hours and training resources were the inputs, while number of graduates, promotions and public service activities were considered as the outputs.Kounetas et al. (2011) applied DEA technique with six model variants to assess the research performance of academic departments in a Greek university.A Tobit model was also considered to study the impact of environmental effects on departmental efficiencies.Kuah and Wong (2011) presented a DEA model for jointly evaluating the relative teaching and research efficiencies of 30 universities.Sunitha and Duraisamy (2011) evaluated efficiency of a sample of engineering and polytechnic institutions in Kerala in India using DEA method.Das et al. (2012) focused on performance evaluation and ranking of seven IITs using fuzzy analytic hierarchy process (AHP) and complex proportional assessment (COPRAS) methods, and showed that the performance of two IITs would need considerable improvement.Wahab and Abdul Rahman (2012) measured the efficiency of zakat institutions in Malaysia during the period of 2003 to 2007, and showed that zakat institutions had mean technical efficiency of 80.6%.Das et al. (2013 a ) proposed a multi-criteria analysis-based performance evaluation model consisting of subjective and objective weight integrated approach and multi-objective optimization on the basis of ratio analysis (MOORA) method for performance evaluation of seven IITs.Das et al.  (2013 b ) illustrated an application of a structured approach to determine relative performance and ranking of seven IITs under multi-criteria environment.Das et al. (2012, 2013 a ,2013 b ) applied different multi-criteria decision-making tools to assess the performance of seven IITs based on six important criteria, i.e. faculty strength, student intake, number of Ph.D awarded, number of patents applied for, campus area and tuition fee per semester.Among those criteria, tuition fee per semester is the only non-beneficial attribute for which lower values are recommended.But presently, there are very marginal differences in the semester-wise course fees amongst different IITs.It is also revealed from the data provided by Das et al. (2012, 2013 a ).Das et  al. (2013 b ) also expressed those six criteria values using triangular fuzzy numbers which is also enough questionable.Thus, in this paper, a new set of eight criteria is taken into account for performance appraisal of eight IITs, considering the total expenses by those IITs for the year 2012 as the lone non-beneficial attribute.

IITs in India
The concept of the IITs originated even before India gained independence in 1947.After the end of the World War II and before India got independence, the Viceroy's executive council foresaw that the future prosperity of India would depend not so much on capital as on technology.Therefore, a proposal for setting up the Council of Scientific and Industrial Research was made.To man those laboratories, the US government was persuaded to offer hundreds of doctoral fellowships under the Technology Cooperation Mission (TCM) program.However, realizing that such steps could not help in the long run for development of India after it gained independence, concept of institutes that would train such work forces in the country itself was developed.This is believed to be the first conceptualization of IITs.
The 22 member committee from the Viceroy's executive council recommended the establishment of Higher Technical Institutions in the Eastern, Western, Northern and Southern regions of the country.The draft also urged speedy establishment of four institutions with the ones in the East and the West to be started immediately.The committee also felt that such institutes should not only produce undergraduates but should be engaged in research, producing research workers and technical teachers as well.They felt that the proportion of undergraduates and postgraduate students should be 2:1.On the ground Bengal had the highest concentration of engineering industries, the Committee suggested that an IIT might be set up in that state.In 1956, the Parliament of India passed an act known as the IIT (Kharagpur) Act declaring it as an Institute of National Importance.

Modelling for DEA
DEA technique was initiated in 1978 when Charnes, Cooper and Rhodes (CCR) demonstrated how to change a fractional linear measure of efficiency into a linear programming (LP) format (Charnes et al., 1978).As a result, the decision-making units (DMUs) can be assessed on the basis of multiple inputs (non-beneficial attributes) and outputs (beneficial attributes), even if the production function is unknown.DEA is a technique for measuring the relative efficiencies using multiple inputs and outputs with no a priori information regarding which inputs and outputs are the most important in determining an efficiency score.Charnes et al. (1994) described DEA as 'a mathematical programming model applied to observational data (which) provides a new way of obtaining empirical estimates of external relations, such as production functions and/or efficient production possibility surfaces that are a cornerstone of modern economics'.This non-parametric approach solves an LP formulation per DMU and the weights assigned to each linear aggregation are the results of the corresponding LP.The weights are chosen so as to show the specific DMU in as positive a light as possible, under the restriction that no other DMU, given the same weights, is more than 100% efficient.Consequently, a Pareto frontier is attained, marked by specific DMUs on the boundary envelope of the input-output variable space.This frontier is considered as a sign of relative efficiency, which has been achieved by at least one DMU.An efficient alternative possesses a relative efficiency score of one that indicates none of its outputs can be increased without either increasing one or more of its inputs or decreasing some of its other outputs.DEA has now become one of the fastest growing areas of operations research and management science (Mondal and Chakraborty, 2013).

The CCR model
In the 'ratio-form' of DEA, the so-called CCR model as originally introduced by Charnes, Cooper and Rhodes (Khouja, 1995), the ratio of total weighted output to total weighted input is used to measure the relative efficiency of a particular DMU.The CCR model is related to the reduction of the multiple output/multiple input situation (for each DMU) to that of a single 'virtual' output and 'virtual' input model.For a particular DMU, the ratio of this single virtual output to single virtual input provides a measure of efficiency which is a function of the corresponding multipliers.By comparing n units with s outputs denoted by y rk (for r = 1,...,s) and m inputs denoted by x ik (for i = 1,...,m), the efficiency measure for DMU k can be given by the following equation : where u r and v i are non-negative weights, u r is the weight associated with r th output (beneficial attribute) and v i is the weight associated with i th input (non-beneficial attribute).A second set of constraints requires that the same weights, when applied to all the DMUs, do not provide any unit with efficiency greater than one.This condition appears in the following set of constraints: The efficiency ratio ranges from 0 to 1, with DMU k being considered relatively efficient if it receives a score of 1.Thus, each unit will choose the weights so as to maximize self-efficiency, given the constraints.The result of DEA is the determination of the hyper-planes that define an envelope surface or Pareto frontier.The DMUs that lie on the surface determine the envelope and are deemed efficient, whilst those that do not are assumed inefficient.The formulation described above is a fractional non-convex programming problem which can be translated into a LP and can be subsequently solved relatively easily.A complete DEA solves n linear programs, one for each DMU.These linear programs can be expressed in two forms, i.e.(a) input-minimized, and (b) outputmaximized.In this paper, the output-maximized formulation of the CCR model is used, which can be given as below: where g k is the efficiency score of DMU k.

TOPSIS method
In this method, two artificial alternatives are hypothesized, i.e. a) ideal alternative which has the best level for all attributes (criteria) considered, and b) negative ideal alternative which has the worst attribute values.It selects the alternative that is the closest to the ideal solution and farthest from the negative ideal solution.TOPSIS method assumes that there is a decision (evaluation) matrix with m alternatives (options) and n criteria, and in this matrix, x ij denotes the performance of i th option with respect to j th criterion.Let J be the set of beneficial criteria (higher the better) and J' be the set of nonbeneficial criteria (lower the better).The implementation of TOPSIS method has the following procedural steps: Step 1: Construct the normalized decision matrix using the following equation: For beneficial criteria: For non-beneficial criteria: where r ij is the normalized value of x ij .
This normalization procedure is adopted to transform various attribute dimensions into nondimensional attributes to facilitate comparisons across criteria.Thus, normalization is done to bring all the criteria values between 0 and 1.
Step 2: Develop the weighted normalized decision matrix.
where w j is the priority weight (importance) of j th criterion.
Step 3: Determine the positive and the negative ideal solutions.
Positive ideal solution: Negative ideal solution: Step 4: Calculate the separation measures for each alternative.
The separation from the ideal solution is computed as below: Similarly, the separation from the negative ideal solution is estimated using the following equation: Step 5: Calculate the relative closeness to the ideal solution, C i * .
) ( where 0  C i *  1 and it is called the closeness coefficient.
Step 6: Select the alternative with C i * value closest to one.Ranking is done on the basis of descending order of the closeness coefficient values, i.e. the alternative with the highest closeness coefficient is assigned the topmost rank and the one with the lowest closeness coefficient is given the lowermost rank.

Performance evaluation of IITs
As mentioned earlier, in this paper, the performances of eight IITs, i.e.Kharagpur, Bombay, Kanpur, Delhi, Guwahati, Roorkee, Madras and Varanasi are appraised based on eight pivotal criteria while using the data for the year 2012.The considered eight criteria are campus area (CA) (in acres), number of international research papers published in last five years (2009-2013) (PP), total number of books and journals available in the library (BJ), placement of students for the year 2012 (PL) (in %), faculty strength (FS), students intake (SI), number of patents applied for and granted in the last five years (2009-2013) (PAG) and expenses for the year 2012 (in crores of Rs.) (EX).Among these, all the criteria except expenses for the year 2012 are beneficial in nature requiring higher values.Campus area is the total area covered by an individual IIT, including the area covered by all the departments, libraries, hostels (both girls and boys), gardens, auditoriums, playgrounds and other facilities.Papers published in last five years is the total number of all research papers published in the last five years, i.e. from 2009 to 2013 by the professors, associate professors, research students, postdoc fellows, postgraduate and undergraduate students in various international journals and reputed conferences.Number of books and journals available in the library is the total number of books, online and offline journals available in the Central Library of each of the IITs.The Central Library in each IIT has a Digital Library facility where online books and journals can be availed by the students.Placement criterion measures the percentage of recruitment of students from all disciplines in each IIT by several companies (MNCs, Government etc.) for the year 2012.Faculty strength denotes the total number of academic and non-academic staffs present in each IIT according to the annual reports of 2012 for each IIT.Students intake measures the total number of students got admitted in all disciplines of each considered IIT in 2012.Patents applied for and granted in last five years count the total number of patents (a scientific model or research work) applied for and granted in last five years (2009)(2010)(2011)(2012)(2013) in each IIT.Expenses criterion measures the value of total expenditures made by individual IITs in the year 2012.These expenditures are made out of various grants sanctioned to different IITs from the Central Government of India.Now, for performance evaluation of those eight considered IITs, the decision matrix of Table 1 is developed accumulating the related information from the individual websites of the IITs.Other websites, like www.scopus.com,www.wikipedia.cometc. also help in developing this decision matrix.Employing Eqs. ( 4) and ( 5), this decision matrix is then normalized in Table 2, thus transforming all the criteria values between 0 and 1.Using these normalized values and Eq. ( 3), the corresponding output-maximized LP formulations of the CCR model of DEA are developed and subsequently solved by LINDO software.The details of these LP formulations along with their optimal solutions for different IITs are given as below.
The objection function value (g) is 1.00000.
(b) Mathematical modelling for IIT, Bombay: The objection function value (g) is 1.00000.
The objection function value (g) is 1.00000.
The objection function value (g) is 0.261434.
The computed values of the efficiency scores for the eight IITs are provided in Table 3.It is observed that the efficiency scores of IIT Roorkee and IIT BHU are less than one, identifying them as the inefficient IITs according to DEA-based analysis.These two IITs are then discarded from the list of IITs while calculating the criteria weights and also for the TOPSIS method-based ranking of the IITs.Now, based on the information of the remaining six IITs and employing entropy method, the priority weights for different criteria are determined, as shown in Table 4.
It is observed that placement of students is the most important criterion, and number of patents applied for and granted in different IITs in last five years is the least important criterion.Table 5 exhibits the weighted normalized decision matrix obtained using Eq. ( 6).Finally, the corresponding separation measures and closeness coefficient values for the six IITs are calculated, as given in Table 6.From Figure 1, it can be concluded that based on DEA-TOPSIS method-based analysis, IIT Kharagpur is the best performing IIT, followed by IIT Bombay and IIT Madras.Actually, IIT Bombay and IIT Madras are almost equivalent in their performance.IIT Guwahati obtains the last rank and is the under-performer among the six IITs.The academic ambiance in IIT Kharagpur is the key factor which encourages faculties and students to strive for excellence.Also, its unique campus environment and culture motivate them to become an all-round performer which is reflected in the growth of the Institute itself at a macro level.It has extended its reach in several directions with programs in the streams of management, law, bio-energy, medical technology and in most of them, it is the first to do so.This has also helped IIT Kharagpur to compete better with other Asian universities of prominence.The universities have been ranked based on 13 performance indicators in teaching, research, knowledge transfer and international outlook.IIT Kharagpur has done relatively well on industry income or innovation, and has secured moderate scores in the teaching and research parameters.With the launch of Vision 2020, this Institute aims to join the league of World's top 20 science and technology-based institutions in the next 20 years by on research excellence, faculty excellence and industry linkages.

Conclusions
This paper mainly deals with the performance appraisal of eight leading IITs based on eight crucial criteria and the available information for the year 2012.DEA technique is first adopted to shortlist the efficient and better performing IITs, and then TOPSIS method is implemented to rank those efficient IITs and identify the best performing IIT.The derived results are quite in agreement with the findings of the recently published survey reports.It can be recommended that the Indian Government should put more endeavour for the promotion and betterment of the identified under-performing IITs.It becomes quite obvious from this paper that placement of students is given the maximum importance, followed by faculty strength and students intake, while number of patents applied for and granted in the last five years receives the least importance.The main findings from this paper would help the IITs in efficiently converting their inputs to outputs.As this combined DEA-TOPSIS approach has strong mathematical foundation, it can also be applied for assessing the performance of other educational institutions and manufacturing organizations.
When the pressure started to set up an IIT in the West, IIT Bombay got established in 1958.IIT Kanpur was established in 1959.Madras got the fourth IIT in 1959 itself as IIT Madras.The then Member of the Planning Commission proposed to set up an IIT at Delhi leading to the establishment of IIT Delhi in 1961.After the establishment of IIT in Delhi, there was a long gap in any notable development in the history of IITs.The IIT Guwahati campus started functioning in 1987.In the beginning of the 21st century, the Education Minister of India made University of Roorkee into an IIT, making IIT Roorkee the newest IIT but the oldest institution amongst the seven in 2001.In 2012, Banaras Hindu University was rechristened as IIT (BHU), Varanasi.Establishment of the IITs began with decision of the cabinet, which was announced by the Minister of Human Resource Development (MHRD) that the government planned to establish more IITs, Indian Institutes of Management (IIMs) and Central Universities across the country.Six IITs at Bhubaneswar, Gandhinagar, Hyderabad, Jodhpur, Patna and Ropar began functional from 2008, while other two at Indore and Mandi commenced their sessions from 2009.India Today's Best Engineering colleges of 2012 ranked seven IITs in its top 10, including the top four.Dataquest's India's Top Engineering Colleges 2011 (DQ-CMR Top T-Schools Survey 2011) ranked IITs in six of the top 10, including the top four.

Table 1
Decision matrix for performance evaluation of eight IITs DEA-TOPSIS method-based ranking of six IITsA recent survey that studied 111 engineering and technology schools in the country also said "IIT Kharagpur has topped the list for the third consecutive year".On a scale of 100, IIT Kharagpur has scored 81.9 points.According to another survey (https://www.iitkgp.org/index.php/news/kgp-inspotlight/952-iit-kgp-ranked-top-indian-educational-hub), it is ranked top among the Indian educational institutions in 2012-13.It has also been ranked one by Outlook India (http://en.wikipedia.wiki/Outlook_India) and by Hindustan Times (http://www.hindustantimes.com/India-new/NewDelhi/IIT-KGP-Roorkee-join-Bombay-in-global-ranking/Article1-939995.aspx) in 2012.IIT Madras has been ranked second by Dataquest in the year 2012 (http://en.wikipedia.org/wiki/Indian_Institute_of_Technology_Madras).