Skip to main content
SpringerLink
Log in

How to Increase PhD Completion Rates? An Impact Evaluation of Two Reforms in a Selective Graduate School, 1976–2012

  • Published:
Research in Higher Education Aims and scope Submit manuscript

Abstract

Graduate and doctoral schools around the world struggle to shorten the long time to degree and to prevent high dropout rates. While most of previous research studied individual determinants of PhD completion, we analyze the impact of two structural reforms of the doctoral program on thesis completion at a selective European graduate school. Exploiting a unique PhD dataset covering 30 entry cohorts, we identify reform effects on PhD outcomes using an interrupted time-series regression design. We find that the first reform improved timely completion rates by between 10 and 15 percentage points (according to the specific outcome), whereas the second reform increased completion rates by between 9 and 20 percentage points. Additionally, each reform reduced dropout rates by 7 percentage points. The results are robust to various sensitivity checks. At the end, we discuss lessons learned for those in charge of graduate schools and/or PhD programs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. A commonly heard light-bulb joke about PhD programs. For a recent report, see “The Long-Haul Degree”. By Patricia Cohen in the New York Times, April 16, 2010. Accessible online at http://www.nytimes.com/2010/04/18/education/edlife/18phd-t.html?pagewanted=all.

  2. This number normally includes the time to finish a masters’ degree.

  3. Some exceptions exist in the United States, notably Ehrenberg et al. (2007) and Groen et al. (2008), which evaluated the impact of the Andrew W. Mellon Foundation’s Graduate Education Initiative (GEI) in the 1990s that provided funding to improve the quality of PhD programs in humanities and related social sciences.

  4. EUI Strategic Review, November 1991, “Beyond Maintenance”. Although this review was published later than the year 1990, the recommendations would affect students starting in the 1989/90 student cohort.

  5. “EUI Strategic Review. November 2001 Enhancing and Enlarging. The Future EUI”. The main results of the review were put in place for the student cohort that had started in the year 2000.

  6. See for instance the position paper issued by the Coimbra group on doctoral education, a leading coalition of research universities. http://www.coimbra-group.eu/uploads/2010-2011/DoctoralProgrammesPositionPaper.pdf.

  7. Our results do not change substantively when we apply the time-restrictions more sharply (i.e. 4 years is 48 months, 5 years is 60 months, and 6 years is 72 months).

  8. Unfortunately we were not able to retrieve the departmental assessment of candidates in the selection procedure. We are therefore not able to include any measures of academic ability in our list of covariates. Nevertheless, since the process is so selective, the variance in academic ability may not be all that big.

  9. Table 4 in the Appendix reports the detailed results of the autocorrelation tests, which were performed using the user-written command –actest– (Baum and Schafer 2013) in Stata 14.1 (StataCorp 2015).

  10. The models were estimated using the user-written command –itsa– (Linden 2015, 2017) in Stata 14.1 (StataCorp 2015).

  11. More precisely, Y t is the predictive margin of the outcome of interest, which correspond to the covariate-adjusted predicted probability of graduating within 4, 5 and 6 years and to dropout from the program. More details are provided below.

  12. One might argue that interpreting test statistics makes little sense as we have total population data of EUI students from 1976 to 2006. However, the students who effectively studied at the EUI might be seen at as a sample from a universe of students who are eligible for the EUI. Elements of chance in the recruitment resulting from arbitrariness in applying and admitting then creates a mechanism of chance sampling from that super-population, that is, if recruitment were repeated hypothetically different student populations may have been created. Thus, one could interpret standard errors as measures for the variability of the hypothetical distribution of coefficients that could have been produced by other potential recruitment outcomes under the assumption that the null hypothesis is true. Nonetheless, we do not over-interpret test statistics in our setting and use them only in a heuristic way.

  13. For checking robustness, we applied a more flexible local linear regression to our data, using optimal bandwidth (Imbens and Kalyanaraman 2012) and lower degree polynomials (Gelman and Imbens 2014). Results (available from the authors upon request) were substantially very similar to the models presented in the article.

  14. Detailed results from the logistic regression models from which the predictive margins were derived are available from the authors upon request.

  15. Figure 3 shows the (non-adjusted) observed values of the four outcomes under study and the corresponding adjusted predictive margins. Table 6 reports the ITSA performed on the observed, non-adjusted, data.

  16. Indeed, the coefficients for REF1 × TIME reported in Table 3, which measure the difference in the slopes after and before the first reform, are not statistically significant.

  17. This finding comes from the 2012 Alumni Survey conducted by the EUI on a sample of 945 former PhD students in order to investigate their following occupational placement.

References

  • Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105(490), 493–505.

    Article  Google Scholar 

  • Abadie, A., Diamond, A., & Hainmueller, J. (2015). Comparative politics and the synthetic control method. American Journal of Political Science, 59(2), 495–510.

    Article  Google Scholar 

  • Abedi, J., & Benkin, E. (1987). The effects of students’ academic, financial, and demographic variables on time to the doctorate. Research in Higher Education, 27(1), 3–14.

    Article  Google Scholar 

  • Baird, L. L. (1990). Disciplines and doctorates: The relationships between program characteristics and the duration of doctoral study. Research in Higher Education, 31(4), 369–385.

    Article  Google Scholar 

  • Ballarino, G., & Colombo, S. (2010). Occupational outcomes of PhD graduates in Northern Italy. Italian Journal of Sociology of Education, 2(2), 149–171.

    Google Scholar 

  • Baum, C. F., & Schafer, M. E. (2013). actest: Stata module to perform Cumby-Huizinga general test for autocorrelation in time series. Statistical Software Components s457668, Boston College Department of Economics. http://ideas.repec.org/c/boc/bocode/s457668.html.

  • Beekhoven, S., De Jong, U., & Van Hout, H. (2002). Explaining academic progress via combining concepts of integration theory and rational choice theory. Research in Higher Education, 43(5), 577–600.

    Article  Google Scholar 

  • Bernal, J. L., Cummins, S., & Gasparrini, A. (2017). Interrupted time series regression for the evaluation of public health interventions: A tutorial. International Journal of Epidemiology, 46(1), 348–355.

    Google Scholar 

  • Bhaskaran, K., Gasparrini, A., Hajat, S., Smeeth, L., & Armstrong, B. (2013). Time series regression studies in environmental epidemiology. International Journal of Epidemiology, 42(4), 1187–1195.

    Article  Google Scholar 

  • Booth, A. L., & Satchell, S. E. (1995). The hazards of doing a PhD: An analysis of completion and withdrawal rates of British PhD students in the 1980s. Journal of the Royal Statistical Society. Series A (Statistics in Society), 158(2), 297–318.

    Article  Google Scholar 

  • Bowen, W. G., & Rudenstine, N. L. (1992). In pursuit of the Ph.D. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Breen, R., & Goldthorpe, J. H. (1997). Explaining educational differentials: Towards a formal rational action theory. Rationality and society, 9(3), 275–305.

    Article  Google Scholar 

  • Byrne, J., Jørgensen, T., & Loukkola, T. (2013). Quality assurance in doctoral education—results of the ARDE project. Brussels: European University Association.

    Google Scholar 

  • Council of Graduate Schools. (2010). Ph.D. completion and attrition: Policies and practices to promote student success. Washington DC: CGS.

    Google Scholar 

  • Crosbie, J. (1993). Interrupted time-series analysis with brief single-subject data. Journal of Consulting and Clinical Psychology, 61, 966–974.

    Article  Google Scholar 

  • Cumby, R. E., & Huizinga, J. (1992). Investigating the correlation of unobserved expectations: Expected returns in equity and foreign exchange markets and other examples. Journal of Monetary Economics, 30(2), 217–253.

    Article  Google Scholar 

  • De Valero, Y. F. (2001). Departmental factors affecting time-to-degree and completion rates of doctoral students at one land-grant research institution. Journal of Higher Education, 72(3), 341–367.

    Article  Google Scholar 

  • Ehrenberg, R. G., Jakubson, G. H., Groen, J. A., So, E., & Price, J. (2007). Inside the black box of doctoral education: What program characteristics influence doctoral students’ attrition and graduation probabilities? Educational Evaluation and Policy Analysis, 29(2), 134–150.

    Article  Google Scholar 

  • Ehrenberg, R. G., & Mavros, P. G. (1995). Do doctoral students’ financial support patterns affect their times-to-degree and completion probabilities? Journal of Human Resources, 30(3), 581–609.

    Article  Google Scholar 

  • Frasier, H. (2013). An analysis of institutional characteristics that contribute to extended time to doctoral degree. Unpublished doctoral dissertation, University of Maryland, College Park.

  • Gelman, A., & Imbens, G. (2014). Why high-order polynomials should not be used in regression discontinuity designs. NBER Working Paper No. 20405. doi:10.3386/w20405.

  • Golde, C. M. (2005). The role of the department and discipline in doctoral student attrition: Lessons from four departments. The Journal of Higher Education, 76(6), 669–700.

    Article  Google Scholar 

  • Gottman, J. M. (1981). Time-series analysis. A comprehensive introduction for social scientists. New York: Cambridge University Press.

    Google Scholar 

  • Graubard, B. I., & Korn, E. L. (1999). Predictive margins with survey data. Biometrics, 55(2), 652–659.

    Article  Google Scholar 

  • Greenland, S. (2004). Model-based estimation of relative risks and other epidemiologic measures in studies of common outcomes and in case-control studies. American Journal of Epidemiology, 160(4), 301–305.

    Article  Google Scholar 

  • Groen, J. A. (2012). Time to the doctorate and labor demand for new PhD recipients. Cornell Higher Education Research Institute, mimeo.

  • Groen, J. A., Jakubson, G. H., Ehrenberg, R. G., Condie, S., & Liu, A. Y. (2008). Program design and student outcomes in graduate education. Economics of Education Review, 27(2), 111–124.

    Article  Google Scholar 

  • Groenvynck, H., Vandevelde, K., & Van Rossem, R. (2013). The PhD track: Who succeeds, who drops out? Research Evaluation, 22(4), 199–209.

    Article  Google Scholar 

  • Hansen, W. L. (1990). Educating and training new economics PhDs: How good a job are we doing? American Economic Review, 80(2), 437–450.

    Google Scholar 

  • Heckman, J., & Hotz, V. J. (1989). Choosing among alternative nonexperimental methods for estimating the impact of social programs: The case of manpower training. Journal of the American Statistical Association, 84(408), 862–874.

    Article  Google Scholar 

  • Hovdhaugen, E. (2011). Do structured study programmes lead to lower rates of dropout and student transfer from university? Irish Educational Studies, 30(2), 237–251.

    Article  Google Scholar 

  • Imbens, G., & Kalyanaraman, K. (2012). Optimal bandwidth choice for the regression discontinuity estimator. The Review of Economic Studies, 79(3), 933–959.

    Article  Google Scholar 

  • Linden, A. (2015). Conducting interrupted time-series analysis for single-and multiple-group comparisons. Stata Journal, 15(2), 480–500.

    Google Scholar 

  • Linden, A. (2017). A comprehensive set of postestimation measures to enrich interrupted time-series analysis. Stata Journal, 17(1), 73–88.

    Google Scholar 

  • Lott, J. L., Gardner, S., & Powers, D. A. (2009). Doctoral student attrition in the STEM fields: An exploratory event history analysis. Journal of College Student Retention: Research, Theory and Practice, 11(2), 247–266.

    Article  Google Scholar 

  • Lovitts, B. E. (2001). Leaving the ivory tower: The causes and consequences of departure from doctoral study. Lanham: Rowman & Littlefield.

    Google Scholar 

  • Lunneborg, C. E., & Lunneborg, P. W. (1973). Doctoral study attrition in psychology. Research in Higher Education, 1(4), 379–387.

    Article  Google Scholar 

  • Mangematin, V. (2000). PhD job market: Professional trajectories and incentives during the PhD. Research Policy, 29(6), 741–756.

    Article  Google Scholar 

  • McKnight, S., McKean, J. W., & Huitema, B. E. (2000). A double bootstrap method to analyze linear models with autoregressive error terms. Psychological Methods, 5, 87–101.

    Article  Google Scholar 

  • National Science Foundation. (2013). Doctorate Recipients from U.S. Universities 2012. Special Report NSF 14-305. Arlington, VA. Retrieved September 3, 2017 from http://www.nsf.gov/statistics/sed/digest/2012.

  • Price, J. (2006). Does a spouse slow you down? Marriage and graduate student outcomes. Cornell University, School of Industrial and Labor Relations site. Retrieved September 3, 2017 from http://digitalcommons.ilr.cornell.edu/workingpapers/147.

  • Rodriguez-Planas, N. (2012). Longer-term impacts of mentoring, educational services, and learning incentives: Evidence from a randomized trial in the United States. American Economic Journal: Applied Economics, 4(4), 121–139.

    Google Scholar 

  • Seagram, B. C., Gould, J., & Pyke, S. W. (1998). An investigation of gender and other variables on time to completion of doctoral degrees. Research in Higher Education, 39(3), 319–335.

    Article  Google Scholar 

  • Shadish, W.R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Wadsworth Cengage learning.

  • Siegfried, J. J., & Stock, W. A. (2001). So you want to earn a Ph.D. in economics? How long do you think it will take? Journal of Human Resources, 36(2), 364–378.

    Article  Google Scholar 

  • Smallwood, S. (2004). Doctor dropout. The Chronicle of Higher Education, 16th January. Retrieved December, 2015.

  • StataCorp. (2015). Stata Statistical Software: Release 14. College Station: StataCorp LP.

    Google Scholar 

  • Stock, W. A., Finegan, T. A., & Siegfried, J. J. (2006). Attrition in economics Ph.D. programs. The American Economic Review, 36(2), 458–466.

    Article  Google Scholar 

  • Taljaard, M., McKenzie, J. E., Ramsay, C. R., & Grimshaw, J. M. (2014). The use of segmented regression in analysing interrupted time series studies: An example in pre-hospital ambulance care. Implementation Science, 9(1), 77.

    Article  Google Scholar 

  • Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (Rev ed.). Chicago: University of Chicago Press.

    Google Scholar 

  • Van de Schoot, R., Yerkes, M. A., Mouw, J. M., & Sonneveld, H. (2013). What took them so long? Explaining PhD delays among doctoral candidates. PLoS ONE, 8(7), e68839.

    Article  Google Scholar 

  • Van der Haert, M., Arias Ortiz, E., Emplit, P., Halloin, V., & Dehon, C. (2014). Are dropout and degree completion in doctoral study significantly dependent on type of financial support and field of research? Studies in Higher Education, 39(10), 1–25.

    Google Scholar 

  • Van Ours, J. C., & Ridder, G. (2003). Fast track or failure: A study of the graduation and dropout rates of PhD students in economics. Economics of Education Review, 22(2), 157–166.

    Article  Google Scholar 

  • Wagner, A. K., Soumerai, S. B., Zhang, F., & Ross-Degnan, D. (2002). Segmented regression analysis of interrupted time series studies in medication use research. Journal of Clinical Pharmacy and Therapeutics, 27(4), 299–309.

    Article  Google Scholar 

  • Wao, H. O., & Onwuegbuzie, A. J. (2011). A mixed research investigation of factors related to time to the doctorate in education. International Journal of Doctoral Studies, 6, 115–134.

    Article  Google Scholar 

  • Williams, R. (2012). Using the margins command to estimate and interpret adjusted predictions and marginal effects. Stata Journal, 12(2), 308–331.

    Google Scholar 

  • Wright, T., & Cochrane, R. (2000). Factors influencing successful submission of PhD theses. Studies in Higher Education, 25(2), 181–195.

    Article  Google Scholar 

  • Zhou, E. & Okahana, H. (2016). The role of department supports on doctoral completion and time-to-degree. Journal of College Student Retention: Research, Theory & Practice. doi:10.1177/1521025116682036

Download references

Acknowledgements

We would like to thank the European University Institute for providing us with the data and Ken Hulley for precious guidance on the information contained in the dataset. We would like to thank the participants to our seminar at the EUI’s Economics Department (2015) and at the COMPIE Conference at the Catholic University of Milan (2016) for useful comments. We would also like to express our gratitude to Hans-Peter Blossfeld, Andrea Ichino, Michael Grätz, Hannes Kröger, Loris Vergolini, and Nadir Zanini, as well as two reviewers who provided stimulating remarks on a previous version of this article. All remaining errors are our own.

Author information

Author notes

  1. Authors are in alphabetical order. All authors contributed equally to the paper.

Authors and Affiliations

  1. World Bank, G7-700, 1776 G ST NW, Washington, DC, 20006, USA

    Koen Geven

  2. Department of Sociology, School of Social Sciences and Philosophy, Trinity College Dublin, The University of Dublin, Dublin 2, Ireland

    Jan Skopek

  3. Department of Sociology and Social Research, University of Trento, Via Verdi 26, 38122, Trento, Italy

    Moris Triventi

Authors
  1. Koen Geven
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Skopek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Moris Triventi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Moris Triventi.

Appendix

Appendix

See Tables 4, 5 and 6 and Fig. 3.

Table 4 Cumby–Huizinga test for autocorrelation in the time-series
Full size table
Table 5 Predictive margins for the four outcomes from the ITSA models
Full size table
Table 6 ITSA: results on the reforms effects using observed yearly aggregated outcomes (without controlling for compositional changes)
Full size table
Fig. 3
figure 3

Comparison of observed data and adjusted predictive margins of the outcomes under study. Note Predictive margins represent the expected probability of an outcome for a certain year (entry cohort) adjusted for covariates

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Geven, K., Skopek, J. & Triventi, M. How to Increase PhD Completion Rates? An Impact Evaluation of Two Reforms in a Selective Graduate School, 1976–2012. Res High Educ 59, 529–552 (2018). https://doi.org/10.1007/s11162-017-9481-z

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11162-017-9481-z

Keywords

Search

Navigation