Skip to main content
SpringerLink
Log in

Efficiency Assessment of Schools Operating in Heterogeneous Contexts: A Robust Nonparametric Analysis Using PISA 2015

  • Chapter
  • First Online:
Data Science and Productivity Analytics

Part of the book series: International Series in Operations Research & Management Science ((ISOR,volume 290))

  • 1347 Accesses

Abstract

The present study proposes an international comparison of education production efficiency using cross-country data on secondary schools from different countries participating in PISA 2015. Given that the context in which schools are operating might be heterogeneous, we need to account for those divergences in the environmental conditions when estimating the efficiency measures of school performance. In this way, each school can be benchmarked with units with similar characteristics regardless of the country they belong to. For this purpose, we use a robust nonparametric approach that allows us to clean the effect of contextual factors previously to the estimation of efficiency measures. Since this approach needs smoothing in the conditional variables in the middle of the sample and not at the frontier (where the number of units is smaller), it seems to be a better option than other nonparametric alternatives previously developed in the literature to deal with the effect of external factors. Likewise, by using this novel approach, we will also be able to explore how those contextual factors might affect both the attainable production set and the distribution of the efficiencies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    See Worthington (2001) for an early review of this literature and Johnes (2015) or De Witte and López-Torres (2017) for updated revisions.

  2. 2.

    There are some exceptions using parametric methods (e.g., Deutsch et al. 2013; Sutherland et al. 2009).

  3. 3.

    While the conditional efficiency estimator of the FDH frontier was developed in Daraio and Simar (2005), the corresponding convex technology, i.e., the DEA estimator was established in Daraio and Simar (2007b).

  4. 4.

    Robust order-m frontiers were presented by Cazals et al. (2002) and Daraio and Simar (2005). Daouia and Simar (2007a, b) developed the alpha-quantile conditional efficiency estimators.

  5. 5.

    Florens et al. (2014) indicate that the rates of convergence are deteriorated by the smoothing in \(Z\) to get the different nonparametric estimators in the sense that \(n\) is to be replaced by \(n\mathop \prod \nolimits_{j = 1}^{{d_{z} }} h_{j}\) (being \(h_{j}\) the corresponding bandwidth for each unit) when product kernels are used for smoothing the \(d_{z}\) components of \(Z\) (see Jeong et al. 2010 for details).

  6. 6.

    In the case of \(X\) and \(Y\) were independent of the exogenous variables, the vectors \(\varepsilon_{x}\) and \(\varepsilon_{y}\) would directly be the standardized inputs and outputs (Florens et al. 2014).

  7. 7.

    Florens et al. (2014) propose a bootstrap based procedure to test the independence between the whitened inputs and outputs and the \(Z\)s. In this paper, we present the evidence of that independence as in Mastromarco and Simar (2017).

  8. 8.

    For a more detailed explanation of directional distance functions (DDF), see Färe and Grosskopf (2000), Simar and Vanhems (2012), Daraio and Simar (2014, 2016) or Daraio et al. (2019).

  9. 9.

    Mastromarco and Simar (2017, 2018) also apply this procedure after using the pre-whitening approach suggested by Florens et al. (2014). These works extend the method to a dynamic framework in which time (the database is of type data panel) plays a role as an additional exogenous variable.

  10. 10.

    In practice, this is equivalent to m = n (total number of units).

  11. 11.

    In the most recent waves of this survey, PISA also evaluates other innovative skills such as collaborative problem-solving or financial literacy.

  12. 12.

    Only 35 students for those countries where PISA assessment was administered in paper-based mode.

  13. 13.

    If 42 students within a school are selected, they do not provide as much “information” as 42 students randomly selected from all schools (Wu 2010).

  14. 14.

    For more detailed information about plausible values see Mislevy et al. (1992) or Wu (2005).

  15. 15.

    See Von Davier and Sinharay (2013) for further details.

  16. 16.

    However, some authors have highlighted that when test scores are used as proxies of educational outcomes, other dimensions of learning such as social skills, attitudes, personal maturity, or moral values are ignored, even though they are crucial for individual development (Levin 2012).

  17. 17.

    This is a common practice in several recent papers attempting to measure the efficiency of schools (e.g., Thieme et al. 2013; Agasisti 2014; Crespo-Cebada et al. 2014; Aparicio et al. 2017, 2018; Agasisti and Zoido 2018).

  18. 18.

    The original values of EDUSHORT and ESCS were rescaled to show positive values by adding up the minimum value to all the original values of the variables. This transformation does not alter the efficient frontier (or empirical production function), and hence the associated DEA model is translation invariant.

  19. 19.

    We only present the classification by using the order-m estimations because they are more robust and present a higher level of discrimination power. The original values have been transformed into values between 0 and 1 in order to facilitate their interpretation (higher values indicate higher levels of efficiency).

  20. 20.

    For comparative reasons, only average results in science are used, as this is the main competence assessed in PISA 2015 (OECD 2017). In any case, the three competencies assessed (science, mathematics, and reading) are highly correlated with each other.

  21. 21.

    The interpretation of the trend of the ratios for dummy variables could be confusing. Nonetheless, these graphs are available upon request.

References

  • Afonso, A., & St Aubyn, M. (2006). Cross-country efficiency of secondary education provision: A semi-parametric analysis with non-discretionary inputs. Economic Modelling, 23(3), 476–491.

    Article  Google Scholar 

  • Agasisti, T. (2014). The efficiency of public spending on education: An empirical comparison of EU countries. European Journal of Education, 49(4), 543–557.

    Article  Google Scholar 

  • Agasisti, T., & Zoido, P. (2018). Comparing the efficiency of schools through international benchmarking: Results from an empirical analysis of OECD PISA 2012 data. Educational Researcher, 47(6), 352–362.

    Google Scholar 

  • Agasisti, T., & Zoido, P. (2019). The efficiency of schools in developing countries, analysed through PISA 2012 data. Socio-Economic Planning Sciences. https://doi.org/10.1016/j.seps.2019.05.002. forthcoming.

    Article  Google Scholar 

  • Ammermüller, A., Heijke, H., & Woessmann, L. (2005). Schooling quality in Eastern Europe: Educational production during transition. Economics of Education Review, 24(5), 579–599.

    Article  Google Scholar 

  • Aparicio, J., Cordero, J. M., & Pastor, J. T. (2017). The determination of the least distance to the strongly efficient frontier in data envelopment analysis oriented models: Modelling and computational aspects. Omega, 71, 1–10.

    Article  Google Scholar 

  • Aparicio, J., Cordero, J. M., González, M., & López-Espin, J. J. (2018). Using non-radial DEA to assess school efficiency in a cross-country perspective: An empirical analysis of OECD countries. Omega, 79, 9–20.

    Article  Google Scholar 

  • Aristovnik, A., & Obadić, A. (2014). Measuring relative efficiency of secondary education in selected EU and OECD countries: The case of Slovenia and Croatia. Technological and Economic Development of Economy, 20(3), 419–433.

    Article  Google Scholar 

  • Badin, L., Daraio, C., & Simar, L. (2010). Optimal bandwidth selection for conditional efficiency measures: A data-driven approach. European Journal of Operational Research, 201(2), 633–640.

    Article  Google Scholar 

  • Badin, L., Daraio, C., & Simar, L. (2012). How to measure the impact of environmental factors in a nonparametric production model? European Journal of Operational Research, 223, 818–833.

    Article  Google Scholar 

  • Badin, L., Daraio, C., & Simar, L. (2019). A bootstrap approach for bandwidth selection in estimating conditional efficiency measures. European Journal of Operational Research, 277(2), 784–797.

    Article  Google Scholar 

  • Bogetoft, P., Heinesen, E., & Tranæs, T. (2015). The efficiency of educational production: A comparison of the Nordic countries with other OECD countries. Economic Modelling, 50, 310–321.

    Article  Google Scholar 

  • Bray, M., & Thomas, R. M. (1995). Levels of comparison in educational studies: Different insights from different literatures and the value of multilevel analyses. Harvard Educational Review, 65(3), 472–490.

    Article  Google Scholar 

  • Cazals, C., Florens, J. P., & Simar, L. (2002). Nonparametric frontier estimation: A robust approach. Journal of Econometrics, 106, 1–25.

    Article  Google Scholar 

  • Cazals, C., Fève, F., Florens, J. P., & Simar, L. (2016). Nonparametric instrumental variables estimation for efficiency frontier. Journal of Econometrics, 190(2), 349–359.

    Article  Google Scholar 

  • Cherchye, L., De Witte, K., Ooghe, E., & Nicaise, I. (2010). Efficiency and equity in private and public education: A nonparametric comparison. European Journal of Operational Research, 202(2), 563–573.

    Article  Google Scholar 

  • Clements, B. (2002). How efficient is education spending in Europe? European Review of Economics and Finance, 1(1), 3–26.

    Google Scholar 

  • Coco, G., & Lagravinese, R. (2014). Cronyism and education performance. Economic Modelling, 38, 443–450.

    Article  Google Scholar 

  • Cordero, J. M., Santín, D., & Simancas, R. (2017). Assessing European primary school performance through a conditional nonparametric model. Journal of the Operational Research Society, 68(4), 364–376.

    Article  Google Scholar 

  • Cordero, J. M., Cristobal, V., & Santín, D. (2018a). Causal inference on education policies: A survey of empirical studies using PISA, TIMSS and PIRLS. Journal of Economic Surveys, 32(3), 878–915.

    Article  Google Scholar 

  • Cordero, J. M., Polo, C., Santín, D., & Simancas, R. (2018b). Efficiency measurement and cross-country differences among schools: A robust conditional nonparametric analysis. Economic Modelling, 74, 45–60.

    Article  Google Scholar 

  • Creemers, B., & Kyriakides, L. (2008). The dynamics of educational effectiveness: A contribution to policy, practice and theory in contemporary schools. Abingdon, Oxon: Routledge.

    Google Scholar 

  • Crespo-Cebada, E., Pedraja-Chaparro, F., & Santín, D. (2014). Does school ownership matter? An unbiased efficiency comparison for regions of Spain. Journal of Productivity Analysis, 41(1), 153–172.

    Article  Google Scholar 

  • Daraio, C., & Simar, L. (2005). Introducing environmental variables in nonparametric frontier models: A probabilistic approach. Journal of Productivity Analysis, 24(1), 93–121.

    Article  Google Scholar 

  • Daraio, C., & Simar, L. (2007a). Advanced robust and nonparametric methods in efficiency analysis. Springer, New York: Methodologies and Applications.

    Google Scholar 

  • Daraio, C., & Simar, L. (2007b). Conditional nonparametric frontier models for convex and non-convex technologies: A unifying approach. Journal of Productivity Analysis, 28, 13–32.

    Article  Google Scholar 

  • Daraio, C., & Simar, L. (2014). Directional distances and their robust versions: Computational and testing issues. European Journal of Operational Research, 237(1), 358–369.

    Article  Google Scholar 

  • Daraio, C., & Simar, L. (2016). Efficiency and benchmarking with directional distances: A data-driven approach. Journal of the Operational Research Society, 67(7), 928–944.

    Article  Google Scholar 

  • Daraio, C., Simar, L., & Wilson, P. W. (2015). Testing the “separability” condition in two-stage nonparametric models of production, LEM Working Paper Series 2015/21.

    Google Scholar 

  • Daraio, C., Simar, L., & Wilson, P. W. (2018). Central limit theorems for conditional efficiency measures and tests of the ‘separability’ condition in non-parametric, two-stage models of production. Econometrics Journal, 21(2), 170–191.

    Article  Google Scholar 

  • Daraio, C., Simar, L., & Wilson, P. W. (2019). Fast and efficient computation of directional distance estimators. https://doi.org/10.1007/s10479-019-03163-9. forthcoming.

    Article  Google Scholar 

  • David, R., Teddlie, C., & Reynolds, D. (2000). The international handbook of school effectiveness research. Psychology Press.

    Google Scholar 

  • De Jorge, J., & Santín, D. (2010). Determinantes de la eficiencia educativa en la Unión Europea. Hacienda Pública Española, 193, 131–155.

    Google Scholar 

  • De Witte, K., & López-Torres, L. (2017). Efficiency in education: A review of literature and a way forward. Journal of the Operational Research Society, 68(4), 339–363.

    Article  Google Scholar 

  • Deutsch, J., Dumas, A., & Siber, J. (2013). Estimating an educational production function for five countries of Latin America on the basis of the PISA data. Economics of Education Review, 36, 245–262.

    Article  Google Scholar 

  • Dufrechou, P. A. (2016). The efficiency of public education spending in Latin America: A comparison to high-income countries. International Journal of Educational Development, 49, 188–203.

    Article  Google Scholar 

  • Färe, R., & Grosskopf, S. (2000). Theory and application of directional distance functions. Journal of Productivity Analysis, 13(2), 93–103.

    Article  Google Scholar 

  • Florens, J., Simar, L., & van Keilegom, I. (2014). Frontier estimation in nonparametric location-scale models. Journal of Econometrics, 178, 456–470.

    Article  Google Scholar 

  • Giambona, F., Vassallo, E., & Vassiliadis, E. (2011). Educational systems efficiency in European Union countries. Studies in Educational Evaluation, 37(2), 108–122.

    Article  Google Scholar 

  • Giménez, V., Prior, D., & Thieme, C. (2007). Technical efficiency, managerial efficiency and objective-setting in the educational system: An international comparison. Journal of the Operational Research Society, 58(8), 996–1007.

    Article  Google Scholar 

  • Giménez, V., Thieme, C., Prior, D., & Tortosa-Ausina, E. (2017). An international comparison of educational systems: A temporal analysis in presence of bad outputs. Journal of Productivity Analysis, 47(1), 83–101.

    Article  Google Scholar 

  • Gustafsson, J. E. (2008). Effects of international comparative studies on educational quality on the quality of educational research. European Educational Research Journal, 7(1), 1–17.

    Article  Google Scholar 

  • Hanushek, E. A. (1979). Conceptual and empirical issues in the estimation of educational production functions. Journal of Human Resources, 14, 351–388.

    Article  Google Scholar 

  • Hanushek, E. A. (2003). The failure of input-based schooling policies. The Economic Journal, 113(485), 64–98.

    Article  Google Scholar 

  • Hanushek, E. A., & Kimko, D. D. (2000). Schooling, labor-force quality, and the growth of nations. American Economic Review, 90(5), 1184–1208.

    Article  Google Scholar 

  • Hanushek, E. A., & Woessmann, L. (2014). Institutional structures of the education system and student achievement: A review of cross-country economic research. In R. Strietholt, W. Bos, J. E. Gustafsson, & M. Rosen (Eds.), Educational policy evaluation through international comparative assessments (pp. 145–176). Waxmann Verlag.

    Google Scholar 

  • Henry, K. L. (2007). Who’s skipping school: Characteristics of truants in 8th and 10th grade. The Journal of School Health, 77, 29–35.

    Article  Google Scholar 

  • Jimerson, S. R. (2001). Meta-analysis of grade retention research: Implications for practice in the 21st century. School Psychology Review, 30(3), 420–437.

    Google Scholar 

  • Jeong, S., Park, B., & Simar, L. (2010). Nonparametric conditional efficiency measures: Asymptotic properties. Annals of Operations Research, 173, 105–122.

    Article  Google Scholar 

  • Johnes, J. (2015). Operational research in education. European Journal of Operational Research, 243(3), 683–696.

    Article  Google Scholar 

  • Le Donné, N. (2014). European variations in socioeconomic inequalities in students’ cognitive achievement: The role of educational policies. European Sociological Review, 30(3), 329–343.

    Article  Google Scholar 

  • Levin, H. (1974). Measuring the efficiency in educational production. Public Finance Quarterly, 2, 3–24.

    Article  Google Scholar 

  • Levin, H. M. (2012). More than just test scores. Prospects, 42(3), 269–284.

    Article  Google Scholar 

  • Mastromarco, C., & Simar, L. (2017). Cross-section dependence and latent heterogeneity to evaluate the impact of human capital on country performance. Discussion Paper UCL-Université Catholique de Louvain, 2017/30.

    Google Scholar 

  • Mastromarco, C., & Simar, L. (2018). Globalization and productivity: A robust nonparametric world frontier analysis. Economic Modelling, 69, 134–149.

    Article  Google Scholar 

  • Mislevy, R. J., Beaton, A. E., Kaplan, B., & Sheehan, K. M. (1992). Estimating population characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29(2), 133–161.

    Article  Google Scholar 

  • OECD. (2009). PISA data analysis manual, SPSS (2nd ed.). PISA: OECD Publishing, Paris.

    Google Scholar 

  • OECD. (2016). PISA 2015 Technical Report. PISA: OECD Publishing, Paris.

    Google Scholar 

  • OECD. (2017). PISA 2015 assessment and analytical framework: Science, reading, mathematic, financial literacy and collaborative problem solving (revised ed.). Paris: PISA, OECD Publishing.

    Google Scholar 

  • O’Donnell, C., Rao, D., & Battese, G. (2008). Metafrontier frameworks for the study of firm-level efficiencies and technology ratios. Empirical Economics, 37(2), 231–255.

    Article  Google Scholar 

  • Rasch, G. (1960/1980). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research (Expanded edition 1980). Copenhagen: The University of Chicago Press.

    Google Scholar 

  • Simar, L., Vanhems, A., & Van Keilegom, I. (2016). Unobserved heterogeneity and endogeneity in nonparametric frontier estimation. Journal of Econometrics, 190(2), 360–373.

    Article  Google Scholar 

  • Simar, L., & Vanhems, A. (2012). Probabilistic characterization of directional distances and their robust versions. Journal of Econometrics, 166(2), 342–354.

    Article  Google Scholar 

  • Simar, L., & Wilson, P. W. (2007). Estimation and inference in two-stage, semi-parametric models of production processes. Journal of Econometrics, 136(1), 31–64.

    Article  Google Scholar 

  • Simar, L., & Wilson, P. W. (2011). Two-stage DEA: Caveat emptor. Journal of Productivity Analysis, 36(2), 205.

    Article  Google Scholar 

  • Sutherland, D., Price, R., & Gonand, F. (2009). Improving public spending efficiency in primary and secondary education. OECD Journal: Economic Studies, 2009(1), 1–30.

    Google Scholar 

  • Tauchmann, H. (2012). Partial frontier efficiency analysis. Stata Journal, 12(3), 461–478.

    Article  Google Scholar 

  • Thieme, C., Giménez, V., & Prior, D. (2012). A comparative analysis of the efficiency of national education systems. Asia Pacific Education Review, 13(1), 1–15.

    Article  Google Scholar 

  • Thieme, C., Prior, D., & Tortosa-Ausina, E. (2013). A multilevel decomposition of school performance using robust nonparametric frontier techniques. Economics of Education Review, 32, 104–121.

    Article  Google Scholar 

  • Todd, P. E., & Wolpin, K. I. (2003). On the specification and estimation of the production function for cognitive achievement. The Economic Journal, 113(485), 3–33.

    Article  Google Scholar 

  • Verhoeven, M., Gunnarsson, V., & Carcillo, S. (2007). Education and health in G7 countries: Achieving better outcomes with less spending (No. 2007-2263). International Monetary Fund.

    Google Scholar 

  • Von Davier, M., & Sinharay, S. (2013). Analytics in international large-scale assessments: Item response theory and population models. In L. Rutkowski, M. Von Davier, & D. Rutkowski (Eds.), Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis (pp. 155–174). London: CRS Press.

    Google Scholar 

  • Willms, J. D., & Smith, T. (2005). A manual for conducting analyses with data from TIMSS and PISA. Report prepared for UNESCO Institute for Statistics.

    Google Scholar 

  • Woessmann, L. (2003). School resources, educational institutions and student performance: The international evidence. Oxford Bulletin of Economics and Statistics, 65(2), 117–170.

    Article  Google Scholar 

  • Worthington, A. C. (2001). An empirical survey of frontier efficiency measurement techniques in education. Education Economics, 9(3), 245–268.

    Article  Google Scholar 

  • Wu, M. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31(2–3), 114–128.

    Article  Google Scholar 

  • Wu, M. (2010). Measurement, sampling, and equating errors in large-scale assessments. Educational Measurement: Issues and Practice, 29(4), 15–27.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Extremadura, Badajoz, Spain

    Jose Manuel Cordero, Cristina Polo & Rosa Simancas

Authors
  1. Jose Manuel Cordero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cristina Polo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rosa Simancas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jose Manuel Cordero .

Editor information

Editors and Affiliations

  1. School of Management, University of Bradford, Bradford, UK

    Vincent Charles

  2. Center of Operations Research, University Miguel Hernandez, Elche, Alicante, Spain

    Juan Aparicio

  3. Foisie Business School, Worcester Polytechnic Institute, Worcester, MA, USA

    Joe Zhu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Cordero, J.M., Polo, C., Simancas, R. (2020). Efficiency Assessment of Schools Operating in Heterogeneous Contexts: A Robust Nonparametric Analysis Using PISA 2015. In: Charles, V., Aparicio, J., Zhu, J. (eds) Data Science and Productivity Analytics. International Series in Operations Research & Management Science, vol 290. Springer, Cham. https://doi.org/10.1007/978-3-030-43384-0_9

Download citation

Publish with us

Policies and ethics

Search

Navigation