Skip to main content
SpringerLink
Log in

An Introduction to the PISA 2015 Questionnaire Field Trial: Study Design and Analysis Procedures

  • Chapter
  • First Online:
Assessing Contexts of Learning

Part of the book series: Methodology of Educational Measurement and Assessment ((MEMA))

Abstract

Theoretical considerations (see Chaps. 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, and 19 of this volume) led to the development of a great amount of questionnaire material, which was localized to fit the contexts of more than 70 countries and economies, implemented in a study with about 120,000 students across these countries, and analyzed to yield indicators on how well questions functioned across and within countries. Such a study, and its results, depends on design and methodological considerations and on decisions made during preparation and implementation. This chapter summarizes decisions regarding the design and analysis procedures of the PISA 2015 field trial, and the resulting data that were used to develop the PISA 2015 main survey’s context assessment. Furthermore, this chapter provides strategies and reasoning related to the choice of questionnaire material that was deemed to be suitable to assess learning contexts in different countries equally well.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Teacher aides, teachers who were also the principal of the school, substitute or occasional teachers who did not teach at this school for more than 6 weeks, teachers who only teach adults, teachers on long-term leave, pedagogical or health and social support staff, were not included in this definition.

  2. 2.

    Sampling weights are used to relate the selected sample back to the population it was drawn from, and thus can correct for potential sampling biases. By definition, such weights are the inverse of the probability of being included in the sample, relative to the sampling design.

References

  • Adams, R. (Ed.). (2002). PISA 2000 technical report. Paris: OECD.

    Google Scholar 

  • Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., Skolits, G. J., & Esquival, S. L. (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research and Evaluation, 18(6), 1–12.

    Google Scholar 

  • Educational Testing Services & German Institute for International Educational Research. (2014). PISA 2015 field trial analysis plan for questionnaire data. Paris: OECD.

    Google Scholar 

  • German Institute for International Educational Research. (2014). Report on questionnaire experiments and methodological questions studied in the Field Trial for PISA 2015. Paris: OECD.

    Google Scholar 

  • Gideon, L. (2012). Handbook of survey methodology for the social sciences. New York: Springer.

    Book  Google Scholar 

  • Green, J. L., Camilli, G., & Elmore, P. B. (2006). Handbook of complementary methods in education research. Mahwah: Lawrence Erlbaum Associates; Published for the American Educational Research Association.

    Google Scholar 

  • International Consortium PISA 2015. (2012a). PISA 2015 field trial school co-ordinator’s manual: CY6_OPS_FTCombinedCBAPBASchoolCo-ordinator’s Manual_1_v12. Paris: OECD. Unpublished report.

    Google Scholar 

  • International Consortium PISA 2015. (2012b). PISA 2015 technical standards. CY6_GEN_PISA2015TechnicalStandards_1.docx. Paris: OECD. Unpublished report.

    Google Scholar 

  • International Consortium PISA 2015. (2012c). Progress report on questionnaire development and design: EDU/PISA/GB(2012)28. Paris: OECD. Unpublished report.

    Google Scholar 

  • International Consortium PISA 2015. (2013). PISA 2015 survey design: EDU/PISA/GB(2012)27. Paris: OECD. Unpublished report.

    Google Scholar 

  • International Consortium PISA 2015. (2014). PISA 2015 field trial analysis plan for questionnaire data: EDU/PISA/GB(2014)3. Paris: OECD. Unpublished report.

    Google Scholar 

  • Jude, N. (2016). The assessment of learning contexts in PISA. In S. Kuger, E. Klieme, N. Jude, & D. Kaplan (Eds.), Assessing contexts of learning: An international perspective. Dordrecht: Springer.

    Google Scholar 

  • Kaplan, D., & Kuger, S. (2016). The methodology of PISA: Past, present, and future. In S. Kuger, E. Klieme, N. Jude, & D. Kaplan (Eds.), Assessing contexts of learning: An international perspective. Dordrecht: Springer.

    Google Scholar 

  • Klieme, E., & Kuger, S. (2016). PISA 2015 context questionnaires framework. In OECD (Ed.), PISA 2015 assessment and analytical framework (pp. 101–127). Paris: OECD Publishing.

    Google Scholar 

  • Kuger, S., & Klieme, E. (2016). Dimensions of context assessment. In S. Kuger, E. Klieme, N. Jude, & D. Kaplan (Eds.), Assessing contexts of learning: An international perspective. Dordrecht: Springer.

    Chapter  Google Scholar 

  • OECD. (2009a). Creating effective teaching and learning environments: First results from TALIS. Paris: OECD Publishing. http://www.oecd.org/edu/school/43023606.pdf. Accessed 25 Apr 2016.

    Book  Google Scholar 

  • OECD. (2009b). PISA data analysis manual: SPSS (2nd ed.). Paris: OECD.

    Book  Google Scholar 

  • OECD. (2013). Education at a glance 2013. Paris: OECD Publishing.

    Google Scholar 

  • OECD. (2014). PISA 2012 technical report. Paris: OECD.

    Google Scholar 

  • Schermelleh-Engel, K., Moosbrugger, H., & MĂĽller, H. (2003). Evaluating the fit of structural equation models: Test of significance and descriptive goodness-of-fit measures. Methods of Psychological Research-Online, 8(2), 23–74.

    Google Scholar 

  • von Davier, M., Gonzalez, E., & Mislevy, R. J. (2009). What are plausible values and why are they useful? In M. von Davier & D. Hastedt (Eds.), IERI Monograph series: Issues and methodologies in large-scale assessment (Vol. 2, pp. 9–36). Princeton: IER Institute, Education Testing Services.

    Google Scholar 

  • Warm, T. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427–450.

    Article  Google Scholar 

  • Wu, M. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31, 114–128.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department for Educational Quality and Evaluation, German Institute for International Educational Research (DIPF), Frankfurt, Germany

    Susanne Kuger, Nina Jude & Eckhard Klieme

  2. Department of Educational Psychology, University of Wisconsin-Madison, Wisconsin, USA

    David Kaplan

Authors
  1. Susanne Kuger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nina Jude
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eckhard Klieme
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Kaplan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Susanne Kuger .

Editor information

Editors and Affiliations

  1. Department for Educational Quality and Evaluation, German Institute for International Educational Research (DIPF) , SchloĂźstraĂźe 29 60486, Frankfurt, Hessen, Germany

    Susanne Kuger

  2. Department for Educational Quality and Evaluation SchloĂźstraĂźe 29 60486, German Institute for International Educational Research (DIPF), SchloĂźstraĂźe 29 60486, Frankfurt, Hessen, Germany

    Eckhard Klieme

  3. Department for Educational Quality and Evaluation, German Institute for International Educational Research (DIPF), SchloĂźstraĂźe 29 60486, Frankfurt, Hessen, Germany

    Nina Jude

  4. Department of Educational Psychology, University of Wisconsin-Madison , 1025 West Johnson Street Madison, Wisconsin, USA

    David Kaplan

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Kuger, S., Jude, N., Klieme, E., Kaplan, D. (2016). An Introduction to the PISA 2015 Questionnaire Field Trial: Study Design and Analysis Procedures. In: Kuger, S., Klieme, E., Jude, N., Kaplan, D. (eds) Assessing Contexts of Learning. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-319-45357-6_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-45357-6_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-45356-9

  • Online ISBN: 978-3-319-45357-6

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation