Skip to main content
SpringerLink
Log in

Measurement and determinants of academic research efficiency: a systematic review of the evidence

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

What is academic research efficiency and what determines the differences between scholars’ academic research efficiency? The literature on this topic has evolved exponentially during the last decades. However, the divergence of the approaches used, the differences in the bundles of outputs and inputs considered to estimate the efficiency frontiers, and the differences in the predictors of efficiency variability among scholars that are considered in prior studies, make it interesting to have an overview of the literature dedicated to this topic. Relying on a systematic review of empirical studies published between 1990 and 2012, this article proposes and discusses a framework which brings together a set of outputs and inputs related to academic research efficiency, and the individual, organizational, and contextual factors driving or hampering it. The ensuing results highlight several avenues which would help university administrators and policy makers to better foster academic research efficiency, and researchers to better channel their efforts in studying the phenomenon.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Source Becheikh et al. [69, p. 646]

Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. There are new promising developments regarding the non-parametric approach, especially the robust non-parametric methodology (Bonaccorsi et al. 2006; Cazals et al. 2002; Daraio and Simar 2005). This methodology permits the generation of indicators that are more robust against outliers and noise in data.

References

102 Selected articles of the systematic review

  • Abbott, M., & Doucouliagos, C. (2003). The efficiency of Australian universities: A data envelopment analysis. Economics of Education Review, 22(1), 89–97.

    Article  Google Scholar 

  • Abbott, M., & Doucouliagos, C. (2009). Competition and efficiency: Overseas students and technical efficiency in Australian and New Zealand universities. Education Economics, 17(1), 31–57.

    Article  Google Scholar 

  • Abramo, G., D’Angelo, C. A., & Pugini, F. (2008). The measurement of Italian universities research productivity by a non-parametric bibliometric methodology. Scientometrics, 76(2), 225–244.

    Article  Google Scholar 

  • Abramo, G., Cicero, T., & D’Angelo, C. A. (2011). A field-standardized application of DEA to national-scale research assessment of universities. Journal of Informetrics, 5(4), 618–628.

    Article  Google Scholar 

  • Agasisti, T., & Salerno, C. (2007). Assessing the cost efficiency of Italian universities. Education Economics, 15(4), 455–471.

    Article  Google Scholar 

  • Agasisti, T., & Johnes, G. (2009). Beyond frontiers: Comparing the efficiency of higher education decision-making units across more than one country. Educations Economics, 17(1), 59–79.

    Article  Google Scholar 

  • Agasisti, T., & Johnes, G. (2010). Heterogeneity and the evaluation of efficiency: The case of Italian universities. Applied Economics, 42(11), 1365–1375.

    Article  Google Scholar 

  • Agasisti, T., & Pérez-Esparrells, C. (2010). Comparing efficiency in a cross-country perspective: The case of Italian and Spanish state universities. Higher Education, 59(1), 85–103.

    Article  Google Scholar 

  • Agasisti, T., Dal Bianco, A., Landoni, P., Sala, A., & Salerno, M. (2011). Evaluating the efficiency of research in academic departments: An empirical analysis in an Italian region. Higher Education Quarterly, 65(3), 267–289.

    Article  Google Scholar 

  • Agasisti, T., Catalano, G., Landoni, P., & Verganti, R. (2012). Evaluating the performance of academic departments: An analysis of research-related output efficiency. Research Evaluation, 21(1), 2–14.

    Article  Google Scholar 

  • Agasisti, T., & Pohl, C. (2012). Comparing German and Italian public universities: Convergence or divergence in the higher education landscape? Managerial and Decision Economics, 33(2), 71–85.

    Article  Google Scholar 

  • Agrell, P. J., & Steuer, R. E. (2000). ACADEA—A decision support system for faculty performance reviews. Journal of Multi-Criteria Decision Analysis, 9(5), 191–204.

    Article  MATH  Google Scholar 

  • Athanassopoulos, A., & Shale, E. (1997). Assessing the comparative efficiency of higher education institutions in the UK by the means of data envelopment analysis. Education Economics, 5(2), 117–134.

    Article  Google Scholar 

  • Avkiran, N. K. (2001). Investigating technical and scale efficiencies of Australian universities through data envelopment analysis. Socio-Economic Planning Sciences, 35(1), 57–80.

    Article  Google Scholar 

  • Beasley, J. E. (1990). Comparing university departments. Omega, The International Journal of Management Science, 18(2), 171–183.

    Article  Google Scholar 

  • Beasley, J. E. (1995). Determining teaching and research efficiencies. The Journal of the Operational Research Society, 46(4), 441.

    Article  MATH  Google Scholar 

  • Bonaccorsi, A., & Daraio, C. (2003). A robust nonparametric approach to the analysis of scientific productivity. Research Evaluation, 12(1), 47–69.

    Article  Google Scholar 

  • Bonaccorsi, A., Daraio, C., & Simar, L. (2006). Advanced indicators of productivity of universities. An application of robust nonparametric methods to Italian data. Scientometrics, 66(2), 389–410.

    Article  Google Scholar 

  • Buzzigoli, L., Giusti, A., & Viviani, A. (2010). The Evaluation of university departments: A case study for Firenze. International Advances in Economic Research, 16(1), 24–38.

    Article  Google Scholar 

  • Chang, D. F., Wu, C. T., Ching, G. S., & Tang, C. W. (2009). An evaluation of the dynamics of the plan to develop first-class universities and top-level research centers in Taiwan. Asia Pacific Education Review, 10(1), 47–57.

    Article  Google Scholar 

  • Chang, T. Y., Chung, P. H., & Hsu, S. S. (2012). Two-stage performance model for evaluating the managerial efficiency of higher education: Application by the Taiwanese tourism and leisure department. Journal of Hospitality, Leisure, Sport and Tourism Education, 11(2), 168–177.

    Article  Google Scholar 

  • Chen, J. K., & Chen, I. S. (2011). Inno-Qual efficiency of higher education: Empirical testing using data envelopment analysis. Expert Systems with Applications, 38(3), 1823–1834.

    Article  MathSciNet  Google Scholar 

  • Cherchye, L., & Abeele, P. V. (2005). On research efficiency—A micro-analysis of Dutch university research in economics and business management. Research Policy, 34(4), 495–516.

    Article  Google Scholar 

  • Coccia, M. (2008). Measuring scientific performance of public research units for strategic change. Journal of Informetrics, 2(3), 183–194.

    Article  Google Scholar 

  • Çokgezen, M. (2009). Technical efficiencies of faculties of economics in Turkey. Education Economics, 17(1), 81–94.

    Article  Google Scholar 

  • Da Silva, E., Souza, G., Alves, E., & Dias Àvila, A. F. (1999). Technical efficiency of production in agricultural research. Scientometrics, 46(1), 141–160.

    Article  Google Scholar 

  • De Groot, H., McMahon, W. W., & Volkwein, J. F. (1991). The cost structure of American research universities. Review of Economics and Statistics, 73(3), 424–431.

    Article  Google Scholar 

  • Feng, Y. J., Lu, H., & Bi, K. (2004). An AHP/DEA method for measurement of the efficiency of R&D management activities in universities. International Transactions in Operational Research, 11(2), 181–191.

    Article  Google Scholar 

  • Flegg, A. T., Allen, D. O., Field, K., & Thurlow, T. W. (2004). Measuring the efficiency of British universities: A multi-period data envelopment analysis. Education Economics, 12(3), 231–249.

    Article  Google Scholar 

  • Glass, J. C., McKillop, D. G., & O’Rourke, G. (1997). Productivity growth in UK accountancy departments 1989–96. Financial Accountability and Management, 13(4), 313–330.

    Article  Google Scholar 

  • Glass, J. C., McKillop, D. G., & O’Rourke, G. (1998). A cost indirect evaluation of productivity change in UK universities. Journal of Productivity Analysis, 10(2), 153–175.

    Article  Google Scholar 

  • Glass, J. C., McKillop, D. G., & O’Rourke, G. (2002). Evaluating the productive performance of UK universities as cost-constrained revenue maximizers: An empirical analysis. Applied Economics, 34(9), 1097–1108.

    Article  Google Scholar 

  • Glass, J. C., McCallion, G., McKillop, D. G., Rasaratnam, S., & Stringer, K. S. (2006). Implications of variant efficiency measures for policy evaluations in UK higher education. Socio-Economic Planning Sciences, 40(2), 119–142.

    Article  Google Scholar 

  • Giménez, V. M., & Martìnez, J. L. (2006). Cost efficiency in the university: A departmental evaluation model. Economics of Education Review, 25(5), 543–553.

    Article  Google Scholar 

  • Groot, T., & Garcia-Valderrama, T. (2006). Research quality and efficiency—An analysis of assessments and management issues in Dutch economics and business research programs. Research Policy, 35(9), 1362–1376.

    Article  Google Scholar 

  • Guan, J., & Chen, K. (2012). Modeling the relative efficiency of national innovation systems. Research Policy, 41(1), 102–115.

    Article  MathSciNet  Google Scholar 

  • Halkos, G. E., Tzeremes, N. G., & Kourtzidis, S. A. (2012). Measuring public owned university departments’ efficiency: A bootstrapped DEA Approach. Journal of Economics and Econometrics, 55(2), 1–24.

    Google Scholar 

  • Hanke, M., & Leopoldseder, T. (1998). Comparing the efficiency of Austrian universities: A data envelopment analysis application. Tertiary Education and Management, 4(3), 191–197.

    Google Scholar 

  • Izadi, H., Johnes, G., Oskrochi, R., & Crouchley, R. (2002). Stochastic frontier estimation of a CES cost function: The case of higher education in Britain. Economics of Education Review, 2(1), 63–71.

    Article  Google Scholar 

  • Johnes, G., & Johnes, J. (1992). Apples and oranges-the aggregation problem in publication. Scientometrics, 25(2), 353–365.

    Article  Google Scholar 

  • Johnes, G. (1992). Performance indicators in higher education: A survey of recent work. Oxford Review of Economic Policy, 8(2), 19–34.

    Article  Google Scholar 

  • Johnes, G., & Johnes, J. (1993). Measuring the research performance of UK economics departments: An application of data envelopment analysis. Oxford Economic Papers, 45(2), 332–347.

    MATH  Google Scholar 

  • Johnes, J., & Johnes, G. (1995). Research funding and performance in U.K. university departments of economics: A Frontier analysis. Economics of Education Review, 14(3), 301–314.

    Article  Google Scholar 

  • Johnes, G. (1998). The costs of multi-product organizations and the heuristic evaluation of industrial Structure. Socio Economic Planning Science, 32(3), 199–209.

    Article  Google Scholar 

  • Johnes, J. (2006). Data envelopment analysis and its application to the measurement of efficiency in higher education. Economics of Education Review, 25(3), 273–288.

    Article  MATH  Google Scholar 

  • Johnes, G., & Salas-Velasco, M. (2007). The determinants of costs and efficiencies where producers are heterogeneous: The case of Spanish universities. Economics Bulletin, 4(15), 1–9.

    Google Scholar 

  • Johnes, G., Johnes, J., & Thanassoulis, E. (2008). An analysis of costs in institutions of higher education in England. Studies in Higher Education, 33(5), 527–549.

    Article  Google Scholar 

  • Johnes, J., & Yu, L. (2008). Measuring the research performance of Chinese higher education institutions using data envelopment analysis. China Economic Review, 19(4), 679–696.

    Article  Google Scholar 

  • Johnes, G., & Johnes, J. (2009). Higher education institutions’ costs and efficiency: Taking the decomposition a further step. Economics of Education Review, 28(1), 107–113.

    Article  Google Scholar 

  • Johnes, G., & Schwarzenberger, A. (2011). Differences in cost structure and the evaluation of efficiency: The case of German universities. Education Economics, 19(5), 487–499.

    Article  Google Scholar 

  • Joumady, O., & Ris, C. (2005). Performance in European higher education: A nonparametric production frontier approach. Education Economics, 13(2), 189–205.

    Article  Google Scholar 

  • Kantabutra, S., & Tang, J. C. S. (2010). Efficiency analysis of public universities in Thailand. Tertiary Education and Management, 16(1), 15–33.

    Article  Google Scholar 

  • Kao, C. A., & Hung, H. T. (2008). Efficiency analysis of university departments: An empirical study. Omega, The International Journal of Management Science, 36(4), 653–664.

    Article  Google Scholar 

  • Kao, C., & Lin, P. H. (2012). Efficiency of parallel production systems with fuzzy data. Fuzzy Sets and Systems, 198, 83–98.

    Article  MathSciNet  MATH  Google Scholar 

  • Katharaki, M., & Katharakis, G. (2010). A comparative assessment of Greek universities efficiency using quantitative analysis. International Journal of Educational Research, 49(4–5), 115–128.

    Article  Google Scholar 

  • Kempkes, G., & Pohl, C. (2008). Do institutions matter for university cost efficiency? Evidence from Germany. CESifo Economic Studies, 54(2), 177–203.

    Article  Google Scholar 

  • Kempkes, G., & Pohl, C. (2010). The efficiency of German universities—Some evidence from nonparametric and parametric methods. Applied Economics, 42(16), 2063–2079.

    Article  Google Scholar 

  • Khaneghah, G. M., Zolfalizadeh, M., & Barough, R. G. (2011). Evaluation of accounting educational departments efficiency. Interdisciplinary Journal of Contemporary Research Business, 3(2), 919–931.

    Google Scholar 

  • Kocher, M. G., Luptacik, M., & Sutter, M. (2006). Measuring productivity of research in economics: A cross-country study using DEA. Socio Economic Planning Sciences, 40(4), 314–332.

    Article  Google Scholar 

  • Köksal, G., & Nalçaci, B. (2006). The Relative efficiency of departments at a Turkish engineering college: A data envelopment analysis. Higher Education, 51(2), 173–189.

    Article  Google Scholar 

  • Korhonen, P., Tainio, R., & Wallenius, J. (2001). Value efficiency analysis of academic research. European Journal of Operational Research, 130(1), 121–132.

    Article  MATH  Google Scholar 

  • Kounetas, K., Anastasiou, A., Mitropoulos, P., & Mitropoulos, I. (2011). Departmental efficiency differences within a Greek university: An application of a DEA and Tobit analysis. International Transactions in Operational Research, 18(5), 545–559.

    Article  Google Scholar 

  • Kuo, J. S., & Ho, Y. S. (2008). The cost efficiency impact of the university operation fund on public universities in Taiwan. Economics of Education Review, 27(5), 603–612.

    Article  Google Scholar 

  • Leitner, K. H., Prikoszovits, J., Schaffhauser-Linzatti, M., Stowasser, R., & Wagner, K. (2007). The impact of size and specialisation on universities’ department performance: A DEA analysis applied to Austrian universities. Higher Education, 3(4), 517–538.

    Article  Google Scholar 

  • Leitner, K. H., Schaffhauser-Linzatti, M., Stowasser, R., & Wagner, K. (2005). Data envelopment analysis as method for evaluating intellectual capital. Journal of Intellectual Capital, 6(4), 528–543.

    Article  Google Scholar 

  • Li, Y., Chen, Y., Liang, L., & Xie, J. (2012). DEA models for extended two-stage network structures. Omega, The International Journal of Management Science, 40(5), 611–618.

    Article  Google Scholar 

  • Lu, W. M. (2012). Intellectual capital and university performance in Taiwan. Economic Modelling, 29(4), 1081–1089.

    Article  Google Scholar 

  • Madden, G., Savage, S., & Kemp, S. (1997). Measuring public sector efficiency: A study of economics departments at Australian universities. Education Economics, 5(2), 153–168.

    Article  Google Scholar 

  • Mamun, S. A. K. (2012). Stochastic estimation of cost frontier: evidence from Bangladesh. Education Economics, 20(2), 211–227.

    Article  Google Scholar 

  • Martin, E. (2006). Efficiency and quality in the current higher education context in Europe: An application of the data envelopment analysis methodology to performance assessment of departments within the University of Zaragoza. Quality in Higher Education, 12(1), 57–79.

    Article  Google Scholar 

  • Medin, E., Anthun, K. S., Hakkinen, U., Kittelsen, S. A. C., Linna, M., Magnussen, J., et al. (2011). Cost efficiency of university hospitals in the Nordic countries: A cross-country analysis. European Journal of Health Economics, 12(6), 509–519.

    Article  Google Scholar 

  • Melville, L., McMillan, M. L., & Chan, W. H. (2006). University efficiency: A comparison and consolidation of results from stochastic and non-stochastic methods. Education Economics, 149(1), 1–30.

    Google Scholar 

  • McMillan, M. L., & Datta, D. (1998). The relative efficiencies of Canadian universities: A DEA perspective. Canadian Public Policy, 24(4), 485–511.

    Article  Google Scholar 

  • Meng, W., Zhang, D., Qi, L., & Liu, W. (2008). Two-level DEA approaches in research evaluation. Omega, The International Journal of Management Science, 36(6), 950–957.

    Article  Google Scholar 

  • Mensah, Y. M., & Werner, R. (2003). Cost efficiency and financial flexibility in institutions of higher education. Journal of Accounting and Public Policy, 22(4), 293–323.

    Article  Google Scholar 

  • Moreno, A. A., & Tadepalli, R. (2002). Assessing academic department efficiency at a public university. Managerial and Decisions Economics, 23(7), 385–397.

    Article  Google Scholar 

  • Ng, Y. C., & Li, S. K. (2000). Measuring the research performance of Chinese higher education institutions: An application of data envelopment analysis. Education Economics, 8(2), 139–156.

    Article  Google Scholar 

  • Ng, Y. C., & Li, S. K. (2009). Efficiency and productivity growth in Chinese universities during the post-reform period. China Economic Review, 20(2), 183–192.

    Article  Google Scholar 

  • Olesen, O. B., & Petersen, N. C. (1995). Change-constrained efficiency evaluation. Management Science, 41(3), 442–457.

    Article  MATH  Google Scholar 

  • Rayeni, M. M., Vardanyan, G., & Saljooghi, F. H. (2010). The measurement of productivity growth in the academic departments using malmquist productivity index. Journal of Applied Sciences, 10(22), 2875–2880.

    Article  Google Scholar 

  • Robst, J. (2000). Do state appropriations influence cost efficiency in public higher education? Applied Economics Letters, 7(11), 715–719.

    Article  Google Scholar 

  • Rousseau, S., & Rousseau, R. (1997). Data envelopment analysis as a tool for constructing scientometric indicators. Scientometrics, 40(1), 45–56.

    Article  Google Scholar 

  • Rouyendegh, B. D., & Erol, S. (2010). The DEA—FUZZY ANP department ranking model applied in Iran Amirkabir University. Acta Polytechnica Hungarica, 7(4), 103–114.

    Google Scholar 

  • Sarafoglou, N., & Hayes, K. E. (1996). University productivity in Sweden: A demonstration and explanatory analysis for economics and business programs. The Annals Regional Science, 30(3), 285–304.

    Article  Google Scholar 

  • Sarrico, C. S., & Dyson, R. G. (2000). Using DEA for planning in UK universities—An institutional perspective. Journal of the Operational Research Society, 51(7), 789–800.

    MATH  Google Scholar 

  • Sav, T. G. (2012). Stochastic cost inefficiency estimates and rankings of public and private research and doctoral granting universities. Journal of Knowledge Management, Economics and Information Technology, 4(3), 11–29.

    Google Scholar 

  • Schubert, T. (2009). Empirical observations on new public management to increase efficiency in public research-boon or bane? Research Policy, 38(8), 1225–1234.

    Article  Google Scholar 

  • Sellers-Rubio, R., Mas-Ruiz, F. J., & Casado-Diaz, A. B. (2010). University efficiency: Complementariness versus trade-off between teaching, research and administrative activities. Higher Education, 64(4), 373–391.

    Article  Google Scholar 

  • Sinuany-Stern, Z., Mehrez, A., & Barboy, A. (1994). Academic departments efficiency via DEA. Computers & Operations Research, 21(5), 543–556.

    Article  MATH  Google Scholar 

  • Soares de Mello, J. C. C. B., Gomes, E. G., Meza, L. A., Soares de Mello, M. H. C., & Soares de Mello, A. J. R. (2006). Engineering post-graduate programmes: A quality and productivity analysis. Studies in Educational Evaluation, 32(2), 136–152.

    Article  Google Scholar 

  • Stevens, P. A. (2005). A stochastic frontier analysis of English and Welsh Universities. Education Economics, 13(4), 355–374.

    Article  Google Scholar 

  • Tagarelli, A., Trubitsyna, I., & Greco, S. (2004). Combining linear programming and clustering techniques for the classification of research centers. AI Communications, 17(3), 111–122.

    MathSciNet  MATH  Google Scholar 

  • Tauer, L. W., Fried, H. O., & Fry, W. E. (2007). Measuring efficiencies of academic departments within a college. Education Economics, 15(4), 473–489.

    Article  Google Scholar 

  • Taylor, B., & Harris, G. (2004). Relative efficiency among South African universities: A data envelopment analysis. Higher Education, 47(1), 73–89.

    Article  Google Scholar 

  • Thanassoulis, E., Kortelainen, M., Johnes, G., & Johnes, J. (2011). Costs and efficiency of higher education institutions in England: A DEA analysis. Journal of the Operational Research Society, 62(7), 1282–1297.

    Article  Google Scholar 

  • Thursby, J. G. (2000). What do we say about ourselves and what does it mean? Yet another look at economics department research. Journal of Economic Literature, 38(2), 383–404.

    Article  Google Scholar 

  • Tyagi, P., Yadav, S. P., & Singh, S. P. (2009). Relative performance of academic departments using DEA with sensitivity analysis. Evaluation and Program Planning, 32(2), 168–177.

    Article  Google Scholar 

  • Warning, S. (2004). Performance differences in German higher education: Empirical analysis of strategic groups. Review of Industrial Organization, 24(4), 393–408.

    Article  Google Scholar 

  • Wolszczak-Derlacz, J., & Parteka, A. (2011). Efficiency of European public higher education institutions: A two-stage multicountry approach. Scientometrics, 89(3), 887–917.

    Article  Google Scholar 

  • Worthington, A. C., & Higgs, H. (2011). Economies of scale and scope in Australian higher education. Higher Education, 61(4), 387–414.

    Article  Google Scholar 

  • Worthington, A. C., & Lee, B. L. (2008). Efficiency, technology and productivity change in Australian universities, 1998–2003. Economics of Education Review, 27(3), 285–298.

    Article  Google Scholar 

Other references

  • Abramo, G., & D’Angelo, C. A. (2011). National-scale research performance assessment at the individual level. Scientometrics, 86(2), 347–364.

    Article  Google Scholar 

  • Abramo, G., D’Angelo, C. A., & Rosati, F. (2013). Measuring institutional research productivity for the life sciences: The importance of accounting for the order of authors in the byline. Scientometrics, 97(3), 779–795.

    Article  Google Scholar 

  • Abramo, G., D’Angelo, C. A., & Rosati, F. (2014). Career advancement and scientific performance in universities. Scientometrics, 98(2), 891–907.

    Article  Google Scholar 

  • Aigner, D. J., Lovell, C. A. K., & Schmidt, P. J. (1977). Formulation and estimation of stochastic frontier production function models. Journal of Econometrics, 6(1), 21–37.

    Article  MathSciNet  MATH  Google Scholar 

  • Allison, P. D., & Long, J. S. (1990). Departmental Effects on Scientific Productivity. American Sociological Review, 55(4), 469–478.

    Article  Google Scholar 

  • Altbach, P. (2006). The dilemmas of ranking. International Higher Education, 42, 2–3.

    Google Scholar 

  • Amara, N., & Landry, R. (2012). Counting citations in the field of business and management: Why use Google Scholar rather than the Web of Science. Scientometrics, 93(3), 553–581.

    Article  Google Scholar 

  • Amara, N., Landry, R., & Halilem, N. (2015). What can university administrators do to increase the publication and citation scores of their faculty members? Scientometrics, 103(2), 489–530.

    Article  Google Scholar 

  • Antonio-García, M. T., López-Navarro, I., & Rey-Rocha, J. (2014). Determinants of success for biomedical researchers. A perception-based study in a health science research environment. Scientometrics, 101(3), 1747–1779.

    Article  Google Scholar 

  • Ballestero, E., & Maldonado, J. A. (2004). Objective measurement of efficiency: Applying single price model to rank hospital activities. Computers & Operations Research, 31(4), 515–532.

    Article  MATH  Google Scholar 

  • Banker, L. E., Charnes, A., Cooper, W., & Maindiratta, A. (1988). A comparison of DEA and Translog estimates of production frontiers using simulated observations from a known technology. In A. Dogramaci & R. Fare (Eds.), Application of modern production theory: Efficiency and productivity (pp. 33–55). Massachusets: Norwell.

    Chapter  Google Scholar 

  • Battese, G., & Coelli, T. (1995). A model for technical inefficiency effects in a stochastic frontier production function for panel data. Empirical Economics, 20(2), 325–332.

    Article  Google Scholar 

  • Bogenschneider, K., Olson, J. R., Linney, K. D., & Mills, J. (2000). Connecting research and policymaking: Implications for theory and practice from the family impact seminars. Family Relations, 49(3), 327–339.

    Article  Google Scholar 

  • Bonaccorsi, A., & Daraio, C. (2005). Exploring size and agglomeration effects on public research productivity. Scientometrics, 63(1), 87–120.

    Article  Google Scholar 

  • Booth, A., Papaioannou, D., & Sutton, A. (2012). Systematic approaches to a successful literature review. Los Angeles, CA: Sage.

    Google Scholar 

  • Bordons, M., & Zulueta, M. A. (1997). Comparison of research team activity in two biomedical fields. Scientometrics, 40(3), 423–436.

    Article  Google Scholar 

  • Bornmann, L. (2014). How are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literature. Research Evaluation, 23, 166–173.

    Article  Google Scholar 

  • Bozeman, B., & Corley, E. (2004). Scientists’ collaboration strategies: Implications for scientific and technical human capital. Research Policy, 33(4), 599–616.

    Article  Google Scholar 

  • Bozeman, B., Dietz, J. S., & Gaughan, M. (2001). Scientific and technical human capital: An alternative model for research evaluation. International Journal of Technology Management, 22(7), 716–740.

    Article  Google Scholar 

  • Bradley, S., Johnes, J., & Little, A. (2010). Measurement and determinants of efficiency and productivity in the further education sector in England. Bulletin of Economic Research, 62(1), 1–30.

    Article  Google Scholar 

  • Carayol, N., & Matt, M. (2006). Individual and collective determinants of academic scientists’ productivity. Information Economics and Policy, 18(1), 55–72.

    Article  Google Scholar 

  • Carrington, R., Coelli, T., & Rao, P. (2005). The performance of Australian universities: Conceptual issues and preliminary results. Economic Papers, 24(2), 145–163.

    Article  Google Scholar 

  • Cazals, C., Florens, J.-P., & Simar, L. (2002). Nonparametric frontier estimation: A robust approach. Journal of Econometrics, 106(1), 1–25.

    Article  MathSciNet  MATH  Google Scholar 

  • Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision-making units. European Journal of Operational Research, 2(6), 429–444.

    Article  MathSciNet  MATH  Google Scholar 

  • Coelli, T. J. (1996). Measurement and sources of technical efficiency in Australian coal-fired electricity generation. CEPA Working Paper 96/1. Department of Econometrics, University of New England, Armidale, NSW.

  • Coelli, T., Prasada Rao, D. S., & Battese, G. E. (1998). An Introduction to efficiency and productivity analysis. Boston: Kluwer Academic Publishers.

    Book  MATH  Google Scholar 

  • Cohen, E., Rhine, S. L., & Santos, M. C. (1989). Institutions of higher education as multi-product firms: Economies of scale and scope. The Review of Economics and Statistics, 71(2), 284–290.

    Article  Google Scholar 

  • Cohen, J. G., Sherman, A. E., Kiet, T. K., Kapp, D. S., Osann, K., Chen, L. M., et al. (2012). Characteristics of success in mentoring and research productivity—A case–control study of academic centers. Gynecologic Oncology, 125(1), 8–13.

    Article  Google Scholar 

  • Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997). Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine, 126(5), 379–380.

    Article  Google Scholar 

  • Cooper, H., & Hedges, L. (Eds.). (1994). Handbook of research synthesis. New York: Russell Sage Foundation.

    Google Scholar 

  • Cooper, R. G., Edgett, S. J., & Kleinschmidt, E. J. (2006). Portfolio management: Fundamental for new product success. The Product Development Institute, Working Paper 12.

  • Copper, W. W., Seiford, L. M., & Tone, K. (2000). Data Envelopment analysis: A comprehensive text with models, applications, references and DEA-solver software. Boston: Kluwer Academic Publishers.

    Google Scholar 

  • Corey Miller, J., Coble, K. H., & Lusk, J. L. (2013). Evaluating top faculty researchers and the incentives that motivate them. Scientometrics, 97(3), 519–533.

    Article  Google Scholar 

  • Dai, Y., Popp, D., & Bretschneider, S. (2005). Institutions and intellectual property: The influence of institutional forces on university patenting. Journal of Policy Analysis and Management, 24(3), 579–598.

    Article  Google Scholar 

  • Daraio, C., & Simar, L. (2005). Introducing environmental variables in nonparametric frontier models: A probabilistic approach. Journal of Productivity Analysis, 24(1), 93–121.

    Article  Google Scholar 

  • Deprins, D., Simar, L., & Tulkens, H. (1984). Measuring labor efficiency in post offices. In M. Marchand, P. Pestieau, & H. Tulkens (Eds.), The performance of public enterprises: Concepts and measurements. Amsterdam: North Holland.

    Google Scholar 

  • Despotis, D. K. (2002). Improving the discriminating power of DEA: Focus on globally efficient units. Journal of the Operational Research Society, 53(3), 314–323.

    Article  MathSciNet  MATH  Google Scholar 

  • Dietz, J. S., & Bozeman, B. (2005). Academic careers, patents, and productivity: Industry experience as scientific and technical human capital. Research Policy, 34(3), 349–367.

    Article  Google Scholar 

  • Dill, D. D., & Soo, M. (2005). Academic quality, league tables, and public policy: A cross-national analysis of university ranking systems. Higher Education, 49(4), 495–533.

    Article  Google Scholar 

  • Dyson, R. G., & Thanassoulis, E. (1988). Reducing weight flexibility in data envelopment analysis. The Journal of the Operational Research Society, 39(6), 563–576.

    Article  Google Scholar 

  • Etzkowitz, H., & Leydesdorff, L. (2000). The dynamics of innovation: From National Systems and ‘Mode 2’ to a Triple Helix of university-industry-government relations. Research Policy, 29(2), 109–123.

    Article  Google Scholar 

  • Finkelstein, M. J., Walker, E., & Chen, R. (2013). The American faculty in an age of globalization: Predictors of internationalization of research content and professional networks. Higher Education, 66(3), 325–340.

    Article  Google Scholar 

  • Ganley, A., & Cubbin, J. S. (1992). Public sector efficiency measurement: Application of data envelopment analysis. Amsterdam: Elsevier.

    Google Scholar 

  • Garcia, C. E., & Sanz-Menéndez, L. (2005). Competition for funding as an indicator of research competitiveness. Scientometrics, 64(3), 271–300.

    Article  Google Scholar 

  • Geuna, A., & Martin, B. R. (2003). University research evaluation and funding: An international comparison. Minerva, 41(4), 277–304.

    Article  Google Scholar 

  • Geuna, A. (2001). The changing rationale for European university research funding: Are there negative unintended consequences. Journal of Economic, 35(3), 607–632.

    Google Scholar 

  • Gonzalez-Brambila, C., & Veloso, F. M. (2007). The determinants of research productivity: A study of Mexican researchers. Research Policy, 36(7), 1035–1051.

    Article  Google Scholar 

  • Halme, M., Joro, T., Korhonen, P., Salo, S., & Wallenius, J. (1999). A value efficiency approach to incorporating preference information in data envelopment analysis. Management Science, 45(1), 103–115.

    Article  MATH  Google Scholar 

  • Harman, K. (2002). The research training experiences of doctoral students linked to Australian cooperative research centres. Higher Education, 44, 469–492.

    Article  Google Scholar 

  • Hasselback, J. R., & Reinstein, A. (1995). A proposal for measuring scholarly productivity of accounting faculty. Issues in Accounting Education, 10(2), 269–306.

    Google Scholar 

  • Heinze, T. P., Shapira, J., Rogers, D., & Senker, J. M. (2009). Organizational and institutional influences on creativity in scientific research. Research Policy, 38(4), 610–623.

    Article  Google Scholar 

  • Hemmings, B., & Kay, R. (2010). Journal ratings and the publications of Australian academics. Issues in Educational Research, 20(3), 234–243.

    Google Scholar 

  • Hemsley-Brown, J. V., & Sharp, C. (2003). The use of research to improve professional practice: A systematic review of the literature. Oxford Review of Education, 29(4), 449–471.

    Article  Google Scholar 

  • Hicks, D. (1999). The difficulty of achieving full coverage of international social science literature and the bibliometric consequences. Scientometrics, 44(2), 193–215.

    Article  Google Scholar 

  • Hopkins, K. D., Gollogly, L., Ogden, S., & Horton, R. (2002). Strange results mean it’s worth checking ISI data. Nature, 415, 732.

    Article  Google Scholar 

  • Horta, H., & Lacy, T. A. (2011). How does size matter for science? Exploring the effects of research unit size on academics’ scientific productivity and information exchange behaviors. Science and Public Policy, 38(6), 449–460.

    Article  Google Scholar 

  • Horta, H., Huisman, J., & Heitor, M. V. (2008). Does competitive research funding encourage diversity in higher education? Science and Public Policy, 35(3), 146–158.

    Article  Google Scholar 

  • Kademani, B. S., Kumar, V., Surwase, G., Sagar, A., Mohan, L., Kumar, A., et al. (2007). Research and citation impact of publications by the chemistry division at Bhabha atomic research centre. Scientometrics, 71(1), 25–57.

    Article  Google Scholar 

  • Kahanec, M., & Kralikova, R. (2011). Pulls of international student mobility. IZA Discussion paper series, No. 6233. http://ftp.iza.org/dp6233.pdf. Accessed 03 Apr 2015.

  • King, J. (1987). A review of bibliometric and other science indicators and their role in research evaluation. Journal of Information Science, 13(5), 261–276.

    Article  Google Scholar 

  • Korhonen, P. J., Soleimani-damaneh, M., & Wallenius, J. (2011). Ratio-based RTS determination in weight-restricted DEA models. European Journal of Operational Research, 215(2), 431–438.

    Article  MathSciNet  MATH  Google Scholar 

  • Korhonen, P. J., & Syrjänen, M. J. (2005). On the interpretation of value efficiency. Journal of Productivity Analysis, 24(2), 197–201.

    Article  Google Scholar 

  • Kumbhakar, S., & Lovell, C. A. K. (2000). Stochastic frontier analysis (1st ed.). New York: Cambridge University Press.

    Book  MATH  Google Scholar 

  • Kuosmanen, T. (1999). Data envelopment analysis of non-convex technology: With an application to finnish super league Pesis players. Working Paper 224, Helsinki School of Economics and Business Administration.

  • Landry, R., Saihi, M., Amara, N., & Ouimet, M. (2010). Evidence on how academics manage their portfolio of knowledge transfer activities. Research Policy, 39(10), 1387–1403.

    Article  Google Scholar 

  • Lariviere, V., Macaluso, B., Archambault, E., & Gingras, Y. (2010). Which scientific elites? On the concentration of research funds, publications and citations. Research Evaluation, 19(1), 45–53.

    Article  Google Scholar 

  • Latruffe, L., Davidova, S., & Balcombe, K. (2008). Application of a double bootstrap to investigation of determinants of technical efficiency of farms in Central Europe. Journal of Productivity Analysis, 29(2), 183–191. doi:10.1007/s11123-007-0074-2.

    Article  Google Scholar 

  • Leahey, E., & Cain, C. L. (2013). Straight from the Source: Accounting for Scientific Success. Social Studies of Science, 43(6), 927–951.

    Article  Google Scholar 

  • Lee, S., & Bozeman, B. (2005). The impact of research collaboration on scientific productivity. Social Studies of Science, 35(5), 673–702.

    Article  Google Scholar 

  • Lehmann, S., Jackson, A., & Lautrup, B. (2008). A quantitative analysis of indicators of scientific performance. Scientometrics, 76(2), 369–390.

    Article  Google Scholar 

  • Lewison, G., & Dawson, G. (1998). The effect of funding on the outputs of biomedical research. Scientometrics, 41(1–2), 17–27.

    Article  Google Scholar 

  • Lindsey, D. (1989). Using citation counts as a measure of quality in science: Measuring what is measurable rather than what’s valid. Scientometrics, 15(3–4), 189–203.

    Article  Google Scholar 

  • Link, A., & Siegel, D. (2005). Generating science-based growth: An econometric analysis of the impact of organizational incentives on university-industry technology transfer. The European Journal of Finance, 11(3), 169–181.

    Article  Google Scholar 

  • Link, A. N., Siegel, D. S., & Bozeman, B. (2007). An empirical analysis of the propensity of academics to engage in informal university technology transfer. Industrial and Corporate Change, 16(4), 641–655.

    Article  Google Scholar 

  • Littell, J. H. (2008). Evidence-based or biased? The quality of published reviews of evidence-based Practices. Children and Youth Services Review, 30(11), 1299–1317.

    Article  Google Scholar 

  • Lukman, R., Krajnc, D., & Glavic, P. (2010). University ranking using research, educational and environmental indicators. Journal of Cleaner Production, 18(7), 619–628.

    Article  Google Scholar 

  • Mazzarol, T., & Saoutar, G. (2002). Push-Pull factors influencing international student destination choice. The International Journal of Educational Management, 16(2), 82–90.

    Article  Google Scholar 

  • McManus, R. J., Wilson, S., Delaney, B. C., Fitzmaurice, D. A., Hyde, C. J., Tobias, R. S., et al. (1998). Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. British Medical Journal, 317, 1562–1563.

    Article  Google Scholar 

  • Meeusen, W., & van den Broeck, J. (1977). Efficiency estimation from Cobb-Douglas production functions with composed error. International Economic Review, 18(2), 435–444.

    Article  MATH  Google Scholar 

  • Meyer, M., Sinilainen, T., & Utecht, J. T. (2003). Towards hybrid Triple Helix indicators: A study of university-related patents and a survey of academic inventors. Scientometrics, 58(2), 321–350.

    Article  Google Scholar 

  • Moed, H. F. (2002). The impact factors debate: the ISI’s uses and limits. Nature, 415, 731–732.

    Article  Google Scholar 

  • Mok, V., & Yeung, G. (2005). Employee motivation, external orientation and the technical efficiency of foreign-financed firms in China: A stochastic frontier analysis. Managerial and Decision Economics, 26(3), 175–190.

    Article  Google Scholar 

  • Najman, J. M., & Hewitt, B. (2003). The validity of publication and citation counts for sociology and other selected disciplines. Journal of Sociology, 39(1), 63–81.

    Article  Google Scholar 

  • Neave, G. (2000). Diversity, differentiation and the market: The debate we never had but which we ought to have done. Higher Education Policy, 13(1), 7–21.

    Article  Google Scholar 

  • Nemoto, J., & Goto, M. (1999). Dynamic data envelopment analysis: Modeling intertemporal behavior of a firm in the presence of productive inefficiencies. Economics Letters, 64(1), 51–56.

    Article  MATH  Google Scholar 

  • Ouimet, M., Landry, R., Amara, N., & Belkhodja, O. (2006). What factors induce university researchers to transfer their research knowledge to users outside the scholarly community? Evidence from researchers in Canadian medical schools. Social Science and Medicine, 62(4), 964–976.

    Article  Google Scholar 

  • Petersen, N. C. (1990). Data envelopment analysis on a relaxed set of assumptions. Management Science, 36(3), 305–314.

    Article  MathSciNet  MATH  Google Scholar 

  • Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Oxford: Blackwell Publishing.

    Book  Google Scholar 

  • Portela, M. C. A. S., Borges, P. C., & Thanassoulis, E. (2003). Finding closest targets in non-oriented DEA models: The case of convex and non-convex technologies. Journal of Productivity Analysis, 19(2), 251–269.

    Article  Google Scholar 

  • Ramanathan, R. (2003). An introduction to data envelopment analysis. New Delhi: Sage Publications.

    Google Scholar 

  • Ravallion, M., & Wagstaff, A. (2011). On measuring scholarly influence by citations. Scientometrics, 88(1), 21–337.

    Article  Google Scholar 

  • Reid, M. B., Misky, G. J., Harrison, R. A., Sharpe, B., Auerbach, A., & Glasheen, J. J. (2012). Mentorship, productivity, and promotion among academic hospitalists. Journal of General Internal Medicine, 27(1), 23–27.

    Article  Google Scholar 

  • Rinia, E. J., Leeuwen, T. N., & Van Raan, A. F. J. (2002). Impact measures of interdisciplinary research in physics. Scientometrics, 53(2), 241–248.

    Article  Google Scholar 

  • Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27, 379–423.

    Article  MathSciNet  MATH  Google Scholar 

  • Stevens, J. M., & Bagby, J. W. (2001). Knowledge transfer from universities to business: Returns for all stakeholders. Organization, 8(2), 259–268.

    Article  Google Scholar 

  • Siegel, D. S., & Phan, P. (2005). Analyzing the effectiveness of university technology transfer: Implications for entrepreneurship education. Advances in the Study of Entrepreneurship, Innovation, and Economic Growth, 16, 1–38.

    Article  Google Scholar 

  • Siegel, D. S., Waldman, D. A., Atwater, L. E., & Link, A. N. (2004). Toward a model of the effective transfer of scientific knowledge from academicians to practitioners: Qualitative evidence from the commercialization of university technologies. Journal of Engineering and Technology Management, 21(1–2), 115–142.

    Article  Google Scholar 

  • Silkman, R. H. (1986). Measuring efficiency: An assessment of data envelopment analysis. San Francisco: Jossey-Bass.

    Google Scholar 

  • Simar, L., & Wilson, P. W. (1999). Of course we bootstrap DEA scores! But does it mean anything? Logic trumps wishful thinking. Journal of Productivity Analysis, 11(1), 93–97.

    Article  Google Scholar 

  • Simar, L., & Wilson, W. P. (1998). Sensitivity analysis of efficiency scores: How to bootstrap in nonparametric frontier models. Management Science, 44(1), 49–61.

    Article  MATH  Google Scholar 

  • Simar, L., & Wilson, W. P. (2004). Performance of the bootstrap for DEA estimators and iterating the principle. In W. W. Cooper, M. L. Seiford, & J. Zhu (Eds.), Handbook on data envelopment analysis (pp. 265–298). Boston: Kluwer Academic Publishers.

    Chapter  Google Scholar 

  • Soleimani-damaneh, M. (2013). An enumerative algorithm for solving nonconvex dynamic DEA models. Optimization Letters, 7(1), 101–115.

    Article  MathSciNet  MATH  Google Scholar 

  • Soleimani-damaneh, M., & Zarepisheh, M. (2009). Shannon’s entropy for combining the efficiency results of different DEA models: Method and application. Expert Systems with Applications, 36(3–1), 5146–5150.

    Article  Google Scholar 

  • Sooryamoorthy, R. (2014). Publication productivity and collaboration of researchers in South Africa: New empirical evidence. Scientometrics, 98(1), 531–545.

    Article  Google Scholar 

  • Stella, A., & Woodhouse, D. (2006). Ranking of higher education institutions. AUQA Occasional Publication no. 6, August. Melbourne: Australian Universities Quality Agency. http://www.auqa.edu.au/files/publications/ranking_of_higher_education_institutionsfinal.pdf. Accessed 6 Jan 2014.

  • Sueyoshi, T., & Sekitani, K. (2005). Returns to scale in dynamic DEA. European Journal of Operational Research, 161(2), 536–544.

    Article  MathSciNet  MATH  Google Scholar 

  • Thanassoulis, E. (2001). Introduction to the theory and application of data envelopment analysis: A foundation text with integrated software. Boston, MA: Kluwer Academic Publishers Norwell.

    Book  Google Scholar 

  • Transfield, D., Denyer, D., & Palminder, S. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14, 207–222.

    Article  Google Scholar 

  • Tulkens, H. (1993). On FDH analysis: Some methodological issues and applications to retail banking, courts and urban transit. Journal of Productivity Analysis, 4(1), 183–210.

    Article  Google Scholar 

  • Vaira, M. (2004). Globalization and higher education organizational change: A framework for analysis. Higher Education, 48(4), 483–510.

    Article  Google Scholar 

  • Valadkhani, A., & Ville, S. (2009). Discipline-specific forecasting of research output in Australian universities. Applied Economic Letters, 16(18), 1875–1880.

    Article  Google Scholar 

  • Van Bouwel, L., & Veugelers, R. (2013). The determinants of student mobility in Europe: The quality dimension. European Journal of Higher Education, 3(2), 172–190.

    Article  Google Scholar 

  • Van Raan, A. F. J. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133–143.

    Article  Google Scholar 

  • Wagner, J. M., & Shimshak, D. G. (2007). Stepwise selection of variables in data envelopment analysis: Procedures and managerial perspectives. European Journal of Operational Research, 180(1), 57–67.

    Article  MATH  Google Scholar 

  • Wang, X., Zhao, Y., Liu, R., & Zhang, J. (2013). Knowledge-transfer analysis based on co-citation clustering. Scientometrics, 97(3), 859–869.

    Article  Google Scholar 

  • Wong, Y. H. B., & Beasley, J. E. (1990). Restricting weight flexibility in data envelopment analysis. The Journal of the Operational Research Society, 41(9), 829–835.

    Article  MATH  Google Scholar 

  • Worthington, A. (2001). An empirical survey of frontier efficiency measurement techniques in education. Education Economics, 9(3), 245–268.

    Article  Google Scholar 

  • Worthington, A. (2004). Frontier efficiency measurement in healthcare: A review of empirical techniques and selected applications. Medical Care Research and Review, 61(2), 1–36.

    Article  Google Scholar 

  • Wu, J., Sun, J., Liang, L., & Zha, Y. (2011). Determination of weights for ultimate cross efficiency using Shannon entropy. Expert Systems with Applications, 38(5), 5162–5165.

    Article  Google Scholar 

  • Wu, J., Sun, J., & Liang, L. (2012). DEA cross-efficiency aggregation method based upon Shannon entropy. International Journal of Production Research, 50(23), 6726–6736.

    Article  Google Scholar 

Download references

Acknowledgements

The author would like to acknowledge financial assistance provided by the Social Sciences and Humanities Research Council of Canada (SSHRC) and The Fonds de recherche du Québec—Société et culture (FRQSC). I also would like to thank the two anonymous reviewers for their insightful comments and suggestions.

Author information

Authors and Affiliations

  1. Department of Management, Faculty of Business, Laval University, 2325 rue de l’Université, Québec City, QC, G1V 0A6, Canada

    Mehdi Rhaiem

Authors
  1. Mehdi Rhaiem
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehdi Rhaiem.

Appendix: Keywords chain used to identify relevant articles

Appendix: Keywords chain used to identify relevant articles

Keywords chain

((academic OR university* OR public OR center) AND (faculty OR faculties OR researcher OR scientist OR department OR unit) AND (research* OR scientific* OR publicat* OR citation* OR entrepreneur* OR commercial* OR Patent* OR Consult* OR spinoff* OR Spillover*) AND ((perform* OR productiv* OR efficiency* OR technical efficiency* OR inefficiency*) OR (DEA OR SER OR SFA OR Data Envelopment Analysis OR Stochastic efficiency OR Stochastic Frontier Analysis OR Bayesian Approach OR Malmquist Index parametric OR parametric OR nonparametric OR deterministic OR econometric frontier OR two-step procedure OR one-step procedure OR fully efficient frontier OR variable return to scale OR VRS OR Constant return to scale OR CRS)))

Note

1. Asterisk (*) use to find a root word plus all the words made by adding letters to the end of it.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rhaiem, M. Measurement and determinants of academic research efficiency: a systematic review of the evidence. Scientometrics 110, 581–615 (2017). https://doi.org/10.1007/s11192-016-2173-1

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-016-2173-1

Keywords

Search

Navigation