Skip to main content

Advertisement

SpringerLink
Log in

Development and use of a test instrument to measure biology teachers’ content knowledge (CK) and pedagogical content knowledge (PCK)

  • Published:
Educational Assessment, Evaluation and Accountability Aims and scope Submit manuscript

Abstract

Research on teachers’ professionalism and professional development has increased in the last two decades. A main focus of this line of research has been the cognitive component of teacher professionalism, i.e., professional knowledge. Most of the previous studies on teacher knowledge—such as the Learning Mathematics for Teaching (LMT) (Hill et al. 2004), the Professional Competence of Teachers, Cognitively Activating Instruction, and Development of Students´ Mathematical Literacy (COACTIV) (Baumert et al. 2010), and the Mathematics Teaching in the 21st Century (MT21) (Schmidt et al. 2007) studies—have been conducted in the field of mathematics teachers’ pedagogical content knowledge (PCK) and content knowledge (CK). There have been few comparable studies conducted with science teachers, especially biology teachers. To fill the gap, this study examines the development and use of instruments to measure biology teachers’ CK and PCK. In particular, this study describes a method to develop reliable, objective, and valid instruments measuring teachers’ CK and PCK in four steps by the use of empirical data of students. Additionally, the study explores whether CK and PCK might be measured as separate knowledge categories by using a paper-and-pencil test. This paper presents a theoretical model that guides test development and provides steps to develop and validate the instruments. Details are also provided regarding the computation of the Rasch scale score measures for 158 biology teachers. The results indicate that the instruments measured teachers’ CK and PCK in an objective, valid, and reliable way. This suggests that the new instruments can be used in combination with classroom observations to examine teaching quality and further its relation to student learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Abell, S. (2007). Research on science teachers’ knowledge. In S. Abell & N. Lederman (Eds.), Handbook of research on science education (pp. 1105–1149). Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Adams, R. J., & Wu, M. (2002). PISA 2000. Technical Report. Programme for International Student Assessment. Paris: OECD.

    Google Scholar 

  • Alexander, P. A., Schallert, D. L., & Hare, V. C. (1991). Coming to terms: how researchers in learning and literacy talk about knowledge. Review of Educational Research, 61(3), 315–343.

    Google Scholar 

  • Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561–573.

    Article  Google Scholar 

  • Aryadoust, S. V. (2009). Mapping Rasch-based measurement onto the argument-based validity framework. Rasch Measurement Transactions, 23(1), 1192–1193.

    Google Scholar 

  • Ball, D., & Bass, H. (2003). Toward a practice-based theory of mathematical knowledge for teaching. In E. Stimmt & B. Davis (Eds.), Proceedings of the 2002 Annual Meeting of the Canadian Mathematics Education Study Group (pp. 3–14). Edmonton, Alberta, Canada: AB.

  • Barba, R., & Rubba, P. A. (1993). Expert and novice, earth and space science: teachers’ declarative, procedural and structural knowledge. International Journal of Science Education, 15(3), 273–282.

    Article  Google Scholar 

  • Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., et al. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal, 47(1), 133–180.

    Article  Google Scholar 

  • Baxter, J. A., & Lederman, N. G. (1999). Assessment and measurement of pedagogical content knowledge. In J. Gess-Newsome & N. Lederman (Eds.), Examining pedagogical content knowledge: the construct and its implications for science education (pp. 147–161). Dordrecht: Kluwer.

    Google Scholar 

  • Berry, A., Loughran, J., & van Driel, J. (2008). Revisiting the roots of pedagogical content knowledge. International Journal of Science Education, 30(10), 1271–1279.

    Article  Google Scholar 

  • Blömeke, S., Felbrich, A., Müller, C., Kaiser, G., & Lehmann, R. (2008a). Effectiveness of teacher education: state of research, measurement issues and consequences for future studies. ZDM: The International Journal on Mathematics Education, 40(5), 719–734.

    Article  Google Scholar 

  • Blömeke, S., Kaiser, G., Lehmann, R., & Schmidt, W. H. (2008b). Introduction to the issue on empirical research on mathematics teachers and their education. ZDM Mathematics Education, 40, 715–717.

    Article  Google Scholar 

  • Blömeke, S., Seeber, S., Lehmann, R., Kaiser, G., Schwarz, B., Felbrich, A., et al. (2008c). Messung des fachbezogenen Wissens angehender Mathematiklehrkräfte [Measurement of the subject-specific knowledge of future mathematics teachers]. In S. Blömeke, G. Kaiser, & R. Lehmann (Eds.), Professionelle Kompetenz angehender Lehrerinnen und Lehrer. Wissen, Überzeugungen und Lerngelegenheiten deutscher Mathematikstudierender und -referendare (pp. 49–88). Münster: Waxmann.

    Google Scholar 

  • Blömeke, S., Bremerich-Vos, A., Haudeck, H., Kaiser, G., Nold, G., Schwippert, K., et al. (Eds.). (2011). Kompetenzen von Lehramtsstudierenden in gering strukturierten Domänen: Erste Ergebnisse aus TEDS-LT [Competencies of future teachers in seldom structured domains: first results of TEDS-LT]. Münster: Waxmann.

    Google Scholar 

  • Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: fundamental measurement in the human sciences. Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Boone, W. J., & Rogan, J. (2005). Rigour in quantitative analysis: The promise of Rasch analysis techniques. African Journal of Research in SMT Education, 9(1), 25–38.

    Google Scholar 

  • Boone, W. J., & Scantlebury, K. (2006). The role of Rasch analysis when conducting science education research utilizing multiple-choice tests. Science Education, 90, 253–269.

    Google Scholar 

  • Boone, W., Townsend, S., & Staver, J. (2011). Using Rasch Theory to Guide the Practice of Survey Development and Survey Data Analysis in Science Education and to Inform Science Reform Efforts: An Exemplar Utilizing STEBI Self-efficacy Data. Science Education, 95(2), 258–290.

  • Borowski, A., Neuhaus, B. J., Tepner, O., Wirth, J., Fischer, H. E., Leutner, D., et al. (2010). Professionswissen von Lehrkräften in den Naturwissenschaften (ProwiN): Kurzdarstellung des BMBF-Projekts [Professional knowledge of science teachers: outline of the BMBF project]. Zeitschrift für Didaktik der Naturwissenschaften, 16, 341–349.

    Google Scholar 

  • Bromme, R. (1997). Kompetenzen, Funktionen und unterrichtliches Handeln der Lehrer [Competencies, functions and enactment of teachers in lessons]. In F. E. Weinert (Ed.), Enzyklopädie der Psychologie (Psychologie des Unterrichts und der Schule, Vol. 3, pp. 177–212). Göttingen: Hogrefe.

    Google Scholar 

  • Carey, S., Evans, R., Honda, M., Jay, E., & Unger, C. (1989). An experiment is when you try it and see if it works: a study of grade 7 students’ understanding of the construction of scientific knowledge. International Journal of Science Education, 11(Special), 514–529.

    Article  Google Scholar 

  • Clement, J. (2000). Model based learning as a key research area for science education. International Journal of Science Education, 22(9), 1041–1053.

    Article  Google Scholar 

  • de Jong, T., & Ferguson-Hessler, M. G. M. (1996). Types and qualities of knowledge. Educational Psychologist, 31(2), 105–113.

    Article  Google Scholar 

  • Fenstermacher, G. D. (1994). The knower and the known: the nature of knowledge research on teaching. In L. Darling-Hammond (Ed.), Review of research in education 20e (pp. 3–56). Washington, DC: AERA.

    Google Scholar 

  • Field, A. (2009). Discovering statistics using SPSS (3rd ed.). London: Sage.

    Google Scholar 

  • Fischer, H. E., Borowski, A., & Tepner, O. (2012). Professional knowledge of science teachers. In B. J. Fraser, K. Tobin, & C. McRobbie (Eds.), Second international handbook of science education. New York: Springer.

    Google Scholar 

  • Gagatsis, A., & Kyriakides, L. (2000). Teachers’ attitudes towards their pupils’ mathematical errors. Educational Research and Evaluation, 6(1), 24–58.

    Article  Google Scholar 

  • Gagné, R. M., Briggs, L. J., & Wagner, W. W. (1988). Principles of instructional design (3rd ed.). New York: Holt, Rinehart, and Winston.

    Google Scholar 

  • Gess-Newsome, J. (1999). Secondary teachers’ knowledge and beliefs about subject matter and their impact on instruction. In J. Gess-Newsome & N. Lederman (Eds.), PCK and science education (pp. 51–94). Dordrecht: Kluwer.

    Google Scholar 

  • Gess-Newsome, J., & Lederman, N. G. (1995). Biology teachers’ perceptions of subject matter structure and its relationship to classroom practice. Journal of Research in Science Teaching, 32(3), 301–325.

    Article  Google Scholar 

  • Gonzalez, E., & Rutkowski, L. (2010). Practical approaches for choosing multiple-matrix sample designs. IEA-ETS Research Institute Monograph, 3, 125–156.

    Google Scholar 

  • Grossman, P. L. (1990). The making of a teacher: teacher knowledge and teacher education. New York: Teachers College Press.

    Google Scholar 

  • Hammann, M., Phan, T. T. H., Ehmer, M., & Grimm, T. (2008). Assessing pupils’ skills in experimentation. Journal of Biological Education, 42(2), 66–72.

    Article  Google Scholar 

  • Hashweh, M. (2005). Teacher pedagogical constructions: a reconfiguration of pedagogical content knowledge. Teachers and Teaching: Theory and Practice, 11(3), 273–292.

    Article  Google Scholar 

  • Hill, H. C. (2007). Mathematical knowledge of middle school teachers: implications for the No Child Left Behind policy initiative. American Educational Research Journal, 29(2), 95–114. doi:10.3102/0162373707301711.

    Google Scholar 

  • Hill, H. C., Schilling, S. G., & Ball, D. L. (2004). Developing measures of teachers’ mathematics knowledge for teaching. The Elementary School Journal, 105(1), 11–30.

    Article  Google Scholar 

  • Hill, H. C., Rowan, B., & Ball, D. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42(2), 371–406.

    Article  Google Scholar 

  • Hill, H. C., Ball, D. L., Blunk, M., Geoffney, I. M., & Rowan, B. (2007). Validating the ecological assumption: the relationship of measure scores to classroom teaching and student learning. Measurement: Interdisciplinary Research and Perspectives, 5(2), 107–118.

    Article  Google Scholar 

  • Hill, H. C., Loewenberg Ball, D., & Schilling, S. G. (2008). Unpacking pedagogical content knowledge: conceptualizing and measuring teacher’s topic-specific knowledge of students. Journal for Research in Mathematics Education, 39(4), 372–400.

    Google Scholar 

  • Jüttner, M. (in press). Entwicklung, Evaluation und Validierung eines Fachwissenstests und eines fachdidaktischen Wissenstests für die Erfassung des Professionswissens von Biologielehrkräften. [Development, evaluation and validation of a content knowledge test and a pedagogical content knowledge test measuring biology teachers’ knowledge]. (Doctoral Thesis, Biology Education, LMU Munich, Germany).

  • Jüttner, M., & Neuhaus, B. J. (2012). Development of items for a pedagogical content knowledge-test based on empirical analysis of pupils’ errors. International Journal of Science Education, 34(7), 1125–1143. doi:10.1080/09500693.2011.606511.

    Google Scholar 

  • Jüttner, M., Spangler, M., & Neuhaus, B. J. (2009). Zusammenhänge zwischen den verschiedenen Bereichen des Professionswissens von Biologielehrkräften. [Connectedness between different categories of biology teachers‘ professional knowledge]. In D. Krüger, A. Upmeier zu Belzen, S. Hof, K. Kremer, & J. Mayer (Hrsg.), Erkenntnisweg Biologiedidaktik 8 (pp. 69–82). Gießen: Universitätsdruckerei Kassel.

  • Klieme, E., Neubrand, M., & Lüdtke, O. (2001). Mathematische Grundbildung: Testkonzeption und Ergebnisse [Mathematical basic education: test conception and results]. In Deutsches PISA-Konsortium (Ed.), Basiskompetenzen von Schülerinnen und Schülern im internationalen Vergleich (pp. 139–190). Opladen: Leske.

    Google Scholar 

  • KMK (Sekretariat der Ständigen Konferenz der Kultusminister der Länder in der Bundesrepublik Deutschland). (2005). Bildungsstandards im Fach Biologie für den Mittleren Schulabschluss (Jahrgangsstufe 10) (Beschluss der Kultusministerkonferenz vom 16.12.2004) [National educational standards for the subject of biology concerning the 10th grade]. München: Luchterhand.

    Google Scholar 

  • Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: an overview. Theory Into Practice, 41(4), 212–218.

    Article  Google Scholar 

  • Krauss, S., Baumert, J., & Blum, W. (2008). Secondary mathematics teachers’ pedagogical content knowledge and content knowledge: validation of the COACTIV constructs. ZDM: The International Journal on Mathematics Education, 40(5), 873–892.

    Article  Google Scholar 

  • Lederman, N. G., Gess-Newsome, J., & Latz, M. S. (1994). The nature and development of preservice science teachers’ conceptions of subject matter and pedagogy. Journal of Research in Science Teaching, 31(2), 129–146.

    Article  Google Scholar 

  • Lee, E., & Luft, J. A. (2008). Experienced secondary science teachers’ representation of pedagogical content knowledge. International Journal of Science Education, 30(10), 1343–1363.

    Article  Google Scholar 

  • Linacre, J. M. (2009). A user’s guide to Winsteps/Ministep: Rasch-model computer programs. Retrieved from http://www.winsteps.com/a/winsteps.pdf.

  • Magnusson, S., Krajcik, J., & Borko, H. (1999). Nature, sources, and development of pedagogical content knowledge for science teaching. In J. Gess-Newsome & N. Lederman (Eds.), Examining pedagogical content knowledge: the construct and its implications for science education (pp. 95–132). Dordrecht: Kluwer.

    Google Scholar 

  • Merrill, P. R. (1987). Job and task analysis. In R. M. Gagné (Ed.), Instructional technology: foundations (pp. 141–173). Hillsdale, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Neuweg, G. H. (2011). Das Wissen der Wissensvermittler [The knowledge and the broadcaster of knowledge]. In E. Terhart, H. Bennewitz, & M. Rothland (Eds.), Handbuch der Forschung zum Lehrerberuf (pp. 451–477). Münster: Waxmann.

    Google Scholar 

  • Paris, S. G., Lipson, M. Y., & Wixson, K. K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8, 293–316.

    Article  Google Scholar 

  • Park, S., Jang, J.-Y., Chen, Y.-C., & Jung, J. (2010). Is pedagogical content knowledge (PCK) necessary for reformed science teaching? Evidence from an empirical study. Research in Science Education, 41(2), 245–260. doi:10.1007/s11165-009-9163-8.

  • Park, S., & Oliver, S. J. (2008). Revisiting the conceptualisation of pedagogical content knowledge (PCK): PCK as a conceptual tool to understand teachers as professionals. Research in Science Education, 38(3), 261–284.

    Google Scholar 

  • Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests (Copenhagen, Danish Institute for Educational Research). Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Rowan, B., Chiang, F.-S., & Miller, R. J. (1997). Using research on employees’ performance to study the effects of teachers on students’ achievement. Sociology of Education, 70(4), 256–284.

    Article  Google Scholar 

  • Schilling, S. G., & Hill, H. C. (2007). Assessing measures of mathematical knowledge for teaching: a validity argument approach. Measurement: Interdisciplinary Research and Perspectives, 5(2–3), 70–80.

    Article  Google Scholar 

  • Schmelzing, S., van Driel, J., Jüttner, M., Brandenbusch, S., Sandmann, A., & Neuhaus, B.J. (2013). Development, evaluation, and validation of a paper-and-pencil test for measuring two components of biology teachers’ pedagogical content knowledge concerning the “cardiovascular system”. International Journal of Science and Mathematics Education. doi:10.1007/s10763-012-9384-6.

  • Schmidt, W. H., Tatto, M. T., Bankov, K., Blömeke, S., Cedillo, T., Cogan, L., et al. (2007). The preparation gap: teacher education for middle school mathematics in six countries (report no. MT21). East Lansing, MI: Michigan State University Center for Research in Mathematics and Science Education.

  • Schwab, J. J. (1978). Science, curriculum, and liberal education: selected essays. Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Schwarz, B., Wissmach, B., & Kaiser, G. (2008). “Last curves not quite correct”: diagnostic competences of future teachers with regard to modelling and graphical representations. ZDM: The International Journal on Mathematics Education, 40(5), 777–790.

    Article  Google Scholar 

  • Shulman, L. (1986). Those who understand: knowledge growth in teaching. Educational Researcher, 15(2), 4–14.

    Google Scholar 

  • Shulman, L. S. (1987). Knowledge and teaching of the new reform. Harvard Educational Review, 57, 1–22.

    Google Scholar 

  • Tatto, M. T. et al. (2007). Teacher education and development study in mathematics (TEDS-M 2008). Conceptual framework: field trial. East Lansing, MI: College of Education, Michigan State University.

  • Tepner, O., Borowski, A., Fischer, H. E., Jüttner, M., Kirschner, S., Leutner, D., et al. (2012). Modell zur Entwicklung von Testitems zur Erfassung des Professionswissens von Lehrkräften in den Naturwissenschaften [Model for the development of test items measuring science teachers’ professional knowledge]. Zeitschrift für die Didaktik der Naturwissenschaften, 18, 7–28.

    Google Scholar 

  • van Driel, J., Verloop, N., & de Vos, W. (1998). Developing science teachers’ pedagogical content knowledge. Journal of Research in Science Teaching, 35(6), 673–695.

    Article  Google Scholar 

  • Wayne, A. J., & Youngs, P. (2003). Teacher characteristics and student achievement gains: a review. Review of Educational Research, 73(1), 89–122.

    Article  Google Scholar 

  • Wright, B. D., & Stone, M. H. (1979). Best test design. Chicago, IL: MESA Press.

    Google Scholar 

Download references

Acknowledgments

The presented project is funded by the Federal Ministry of Education and Research in Germany; it is a cooperation project embedded in the framework program of ‘empirical research in education’ (01JH0904).

The full details of this study (including figures, tables, and diagrams) were reported in a German dissertation in Jüttner in press.

Author information

Authors and Affiliations

  1. Biology Education, Faculty of Biology, Ludwig-Maximilians-University of Munich, Winzererstr. 45/II, 80797, Munich, Germany

    Melanie Jüttner & Birgit J. Neuhaus

  2. Miami University, Oxford, OH, USA

    Williame Boone

  3. University of Iowa, Iowa City, IA, USA

    Soonhye Park

Authors
  1. Melanie Jüttner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Williame Boone
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Soonhye Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Birgit J. Neuhaus
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Melanie Jüttner.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jüttner, M., Boone, W., Park, S. et al. Development and use of a test instrument to measure biology teachers’ content knowledge (CK) and pedagogical content knowledge (PCK). Educ Asse Eval Acc 25, 45–67 (2013). https://doi.org/10.1007/s11092-013-9157-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11092-013-9157-y

Keywords

Search

Navigation