Skip to main content
Search
Search
    Quick Search anywhere
    Quick search in Citations
Advanced Search
Skip main navigationClose Drawer MenuOpen Drawer Menu
Previous article
Next article
Original Article

Assessing Professional Vision in Teacher Candidates

Approaches to Validating the Observer Extended Research Tool

Published Online:March 06, 2015https://doi.org/10.1027/2151-2604/a000200

Abstract

In this study, we present an approach to validating the video-based Observer Extended Research Tool, which empirically captures prospective teachers’ professional vision in a standardized yet contextualized way. We extended the original Observer tool with the aim of providing a reliable, efficient measure for subpopulations in different consecutive phases of teacher education (university and induction phase). Therefore, we expand the measure to include a broader spectrum of knowledge about effective teaching by drawing on a cognitive process-oriented teaching and learning model while at the same time the number of test items is shortened to ensure a economically manageable assessment tool. In the validation study, we tested the extent to which the extension meets the criteria of context validity, reliability, and sensitiveness for different subpopulations. The participants were 317 preservice teachers and teacher candidates who worked with the Observer Extended Research Tool. Measurement quality was investigated using methods of item response theory. Our results confirm that the Observer Extended Research Tool provides a reliable measure of description, explanation, and prediction as aspects of professional vision within and across different subpopulations in teacher education.

References

  • Adams, R. , & Carstensen, C. H. (2002). Scaling outcomes. In R. Adams, & M. Wu (Eds.), Pisa 2000. Technical Report (pp. 149–162). Paris, France: OECD. First citation in articleGoogle Scholar

  • Baumert, J. , Kunter, M. , Blum, W. , Brunner, M. , Voss, T. , Jordan, A. , … Yi-Miau, T. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal, 47, 133–180. First citation in articleCrossref, Google Scholar

  • Berliner, D. C. (1991). Perceptions of student behavior as a function of expertise. Journal of Classroom Interaction, 26, 1–8. First citation in articleGoogle Scholar

  • Berliner, D. C. (2001). Learning about learning from expert teachers. International Journal of Educational Research, 35, 463–482. doi: 10.1016/S0883-0355(02)00004-6 First citation in articleCrossref, Google Scholar

  • Berliner, D. C. , Stein, P. , Sabers, D. S. , Clarridge, P. B. , Cushing, K. S. , & Pinnegar, S. (1988). Implications of research on pedagogical expertise and experience in mathematics teaching. In D. A. Grouws, & T. J. Cooney (Eds.), Perspectives on research on effective mathematics teaching (pp. 67–95). Reston, VA: National Council of Teachers of Mathematics. First citation in articleGoogle Scholar

  • Blömeke, S. , Gustafsson, J.-E. , & Shavelson, R. (2015). Beyond dichotomies: Competence viewed as a continuum. Zeitschrift für Psychologie, 223, doi: 10.1027/2151-2604/a000194. First citation in articleGoogle Scholar

  • Blömeke, S. , Kaiser, G. , Lehmann, R. , König, J. , Döhrmann, M. , Buchholtz, C. , & Hacke, S. (2009). TEDS-M: Messung von Lehrerkompetenzen im internationalen Vergleich [TEDS-M: Assessment of teacher competencies in international comparison]. In O. Zlatkin-Troitschanskaia, K. Beck, D. Sembill, R. Nickolaus, & R. Mulder (Eds.) Lehrprofessionalität. Bedingungen, Genese, Wirkungen und ihre Messung (pp. 181–210). Weinheim, Germany: Beltz. First citation in articleGoogle Scholar

  • Bolhuis, S. (2003). Towards process-oriented teaching for self-directed lifelong learning: A multidimensional perspective. Learning and Instruction, 13, 327–347. doi: 10.1016/s0959-4752(02)00008-7 First citation in articleCrossref, Google Scholar

  • Bond, T. G. , & Fox, C. M. (2001). Applying the Rasch Model. Mahwah, NJ: Erlbaum. First citation in articleCrossref, Google Scholar

  • Borko, H. (2004). Professional development and teacher learning: Mapping the terrain. Educational Researcher, 33, 3–15. doi: 10.3102/0013189x033008003 First citation in articleCrossref, Google Scholar

  • Brouwer, N. (2010). Determining long term effects of teacher education. In P. Peterson, E. Baker, & B. McGaw (Eds.), International encyclopedia of education, Vol. 7, (pp. 503–510). Oxford, UK: Elsevier. First citation in articleGoogle Scholar

  • Cochran-Smith, M. , & Zeichner, K. M. (2005). Studying teacher education: The report of the AERA Panel on Research and Teacher Education. Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Darling-Hammond, L. (2006). Assessing teacher education: The usefulness of multiple measures for assessing program outcomes. Journal of Teacher Education, 57, 120–138. doi: 10.1177/0022487105283796 First citation in articleCrossref, Google Scholar

  • Darling-Hammond, L. (2010). Teacher education and the American future. Journal of Teacher Education, 61, 35–47. doi: 10.1177/0022487109348024 First citation in articleCrossref, Google Scholar

  • Darling-Hammond, L. , & Bransford, J. D. (Eds.). (2005). Preparing teachers for a changing world: What teachers should learn and be able to do. San Francisco, CA: Jossey-Bass. First citation in articleGoogle Scholar

  • Döhrmann, M. , Kaiser, G. , & Blömeke, S. (2012). The conceptualisation of mathematics competencies in the international teacher education study TEDS-M. ZDM – International Journal on Mathematics Education, 44, 325–340. First citation in articleCrossref, Google Scholar

  • Fraser, B. J. , Walberg, H. J. , Welch, W. W. , & Hattie, J. A. (1987). Syntheses of educational productivity research. International Journal of Educational Research, 11, 145–252. First citation in articleCrossref, Google Scholar

  • Goodwin, C. (1994). Professional vision. American Anthropologist, 96, 606–633. doi: 10.1525/aa.1994.96.3.02a00100 First citation in articleCrossref, Google Scholar

  • Grossman, P. , Compton, C. , Igra, D. , Ronfeldt, M. , Shahan, E. , & Williamson, P. W. (2009). Teaching practice: A cross-professional perspective. Teachers College Record, 111, 2055–2100. First citation in articleCrossref, Google Scholar

  • Grossman, P. , & McDonald, M. (2008). Back to the future: Directions for research in teaching and teacher education. American Educational Research Journal, 45, 184–205. doi: 10.3102/0002831207312906 First citation in articleCrossref, Google Scholar

  • Hammerness, K. , Darling-Hammond, L. , & Shulman, L. S. (2002). Toward expert thinking: How curriculum case writing prompts the development of theory-based professional knowledge in student teachers. Teaching Education, 13, 219–243. First citation in articleCrossref, Google Scholar

  • Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analysis relating to achievement. New York, NY: Routledge. First citation in articleGoogle Scholar

  • Hill, H. C. , Rowan, B. , & Ball, D. L. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42, 371–406. doi: 10.3102/00028312042002371 First citation in articleCrossref, Google Scholar

  • Jahn, G. , Prenzel, M. , Stürmer, K. , & Seidel, T. (2011). Varianten einer computergestützten Erhebung von Lehrerkompetenzen [Variants of computer-based teacher competency assessment]. Unterrichtswissenschaft, 39, 136–153. First citation in articleGoogle Scholar

  • Jahn, G. , Stürmer, K. , Seidel, T. , & Prenzel, M. (2014). Professionelle Unterrichtswahrnehmung von Lehramtsstudierenden: Eine Scaling-up Studie des Observe-Projekts [Professional teaching perception of student teachers: A scaling-up study from the Observe project]. Zeitschrift für Entwicklungspsychologie und pädagogische Psychologie, 46, 171–180. First citation in articleLink, Google Scholar

  • Kane, M. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 64, 425–461. doi: 10.3102/00346543064003425 First citation in articleCrossref, Google Scholar

  • Kersting, N. (2008). Using video clips of mathematics classroom instruction as item prompts to measure teachers’ knowledge of teaching mathematics. Educational and Psychological Measurement, 68, 845–861. First citation in articleCrossref, Google Scholar

  • Kunter, M. , Baumert, J. , & Köller, O. (2007). Effective classroom management and the development of subject-related interest. Learning and Instruction, 17, 494–509. First citation in articleCrossref, Google Scholar

  • OECD . (2005). PISA 2003 technical report. Paris, France: OECD. First citation in articleCrossref, Google Scholar

  • Penfield, R. D. , & Algina, J. (2006). A generalized DIF effect variance estimator for measuring unsigned differential test function in mixed format tests. Journal of Educational Measurement, 43, 295–312. First citation in articleCrossref, Google Scholar

  • Putnam, R. T. , & Borko, H. (2000). What do new views of knowledge and thinking have to say about research on teacher learning? Educational Researcher, 29, 4–15. First citation in articleCrossref, Google Scholar

  • Rauch, P. , & Hartig, J. (2010). Multiple-choice versus open-ended responses formats of reading test items: A two-dimensional IRT analysis. Psychological Test and Assessment Modeling, 52, 354–379. First citation in articleGoogle Scholar

  • Santagata, R. , & Angelici, G. (2010). Studying the impact of the lesson analysis framework on preservice teachers’ abilities to reflect on videos of classroom teaching. Journal of Teacher Education, 61, 339–349. doi: 10.1177/0022487110369555 First citation in articleCrossref, Google Scholar

  • Santagata, R. , & Guarino, J. (2010). Using video to teach future teachers to learn from teaching. ZDM – International Journal on Mathematics Education, 43, 133–145. doi: 10.1007/s11858-010-0292-3 First citation in articleCrossref, Google Scholar

  • Seidel, T. (2012). Implementing competence assessment in university education. Empirical Research in Vocational Education and Training, 4, 91–94. First citation in articleGoogle Scholar

  • Seidel, T. , Blomberg, G. , & Renkel, A. (2013). Instructional strategies for using video in teacher education. Teaching and Teacher Education, 34, 56–65. doi: 10.1016/j.tate.2013.03.004 First citation in articleCrossref, Google Scholar

  • Seidel, T. , Blomberg, G. , & Stürmer, K. (2010a). Observer: A video-based tool to diagnose teachers’ professional vision. Unpublished instrument. Retrieved from ww3.unipark.de/uc/observer_engl/demo/kv/ First citation in articleGoogle Scholar

  • Seidel, T. , Blomberg, G. , & Stürmer, K. (2010b). “Observer” – Validation of a video-based instrument for measuring the perception of professional education. In E. Klieme, D. Leutner, & M. Kenk (Eds.), Kompetenzmodellierung – Zwischenbilanz des DFG-Schwerpunktprogramms und Perspektiven des Forschungsansatzes [Competence modeling – Summary and perspectives of the research approach] [Special issue]. Zeitschrift für Pädagogik (Suppl. 56), 296–306. First citation in articleGoogle Scholar

  • Seidel, T. , & Prenzel, M. (2007). Wie Lehrpersonen Unterricht wahrnehmen und einschätzen – Erfassung pädagogisch-psychologischer Kompetenzen bei Lehrpersonen mit Hilfe von Videosequenzen [Teachers’ perceptions and evaluations in the classroom – assessment of teachers’ pedagogical and psychological competencies with the aid of video sequences] In M. Brenzle, I. Gogolin, & H.-H. Krüger (Eds.), Kompetenzdiagnostik [Special issue]. Zeitschrift für Erziehungswissenschaft (Suppl. 8), 201–218. First citation in articleGoogle Scholar

  • Seidel, T. , & Shavelson, R. J. (2007). Teaching effectiveness research in the past decade: The role of theory and research design in disentangling meta-analysis results. Review of Educational Research, 77, 454–499. doi: 10.3102/0034654307310317 First citation in articleCrossref, Google Scholar

  • Seidel, T. , & Stürmer, K. (2014). Modeling and measuring the structure of professional vision in pre-service teachers. American Educational Research Journal, 51, 739–771. doi: 10.3102/0002831214531321 First citation in articleCrossref, Google Scholar

  • Shavelson, R. J. (2012). Assessing business-planning competence using the Collegiate Learning Assessment as a prototype. Empirical Research in Vocational Education and Training, 4, 77–90. First citation in articleGoogle Scholar

  • Sherin, M. G. (2001). Developing a professional vision of classroom events. In T. Wood, B. S. Nelson, & J. Warfield (Eds.), Bexond classical pedagogy: Teaching elementary school mathematics. Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Sherin, M. G. (2007). The development of teachers’ professional vision in video clubs. In R. Goldman, R. Pea, B. Barron, & S. J. Derry (Eds.), Video research in the learning sciences (pp. 383–395). Mahwah, NJ: Erlbaum. First citation in articleGoogle Scholar

  • Sherin, M. G. , & van Es, E. (2009). Effects of video club participation on teachers’ professional vision. Journal of Teacher Education, 60, 20–37. doi: 10.1177/0022487108328155 First citation in articleCrossref, Google Scholar

  • Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1–22. First citation in articleCrossref, Google Scholar

  • Star, J. R. , & Strickland, S. K. (2008). Learning to observe: Using video to improve preservice mathematics teachers’ ability to notice. Journal of Mathematics Teacher Education, 11, 107–125. doi: 10.1007/s10857-007-9063-7 First citation in articleCrossref, Google Scholar

  • Stürmer, K. , Könings, K. D. , & Seidel, T. (2013). Declarative knowledge and professional vision in teacher education: Effect of courses in teaching and learning. British Journal of Educational Psychology, 83, 467–483. doi: 10.1111/j.2044-8279.2012.02075.x First citation in articleCrossref, Google Scholar

  • van Es, E. , & Sherin, M. G. (2002). Learning to notice: Scaffolding new teachers’ interpretations of classroom interactions. Journal of Technology and Teacher Education, 10, 571–596. First citation in articleGoogle Scholar

  • van Es, E. , & Sherin, M. G. (2008). Mathematics teachers’ “learning to notice” in the context of a video club. Teaching and Teacher Education, 24, 244–276. doi: 10.1016/j.tate.2006.11.005 First citation in articleCrossref, Google Scholar

  • Voss, T. , Kunter, M. , & Baumert, J. (2011). Assessing teacher candidates’ general pedagogical/psychological knowledge: Test construction and validation. Journal of Educational Psychology, 103, 952–969. doi: 10.1037/a0025125 First citation in articleCrossref, Google Scholar

  • Wiens, P. D. , Hessberg, K. , LoCasale-Crouch, J. , & DeCoster, J. (2013). Using a standardized video-based assessment in a university teacher education program to examine preservice teachers knowledge related to effective teaching. Teaching and Teacher Education, 33, 24–33. doi: 10.1016/j.tate.2013.01.010 First citation in articleCrossref, Google Scholar

  • Wu, M. L. , Adams, R. J. , & Wilson, M. R. (1997). ConQuest: Multi-Aspect Test Software. Camberwell, Australia: Australian Council for Education Research. First citation in articleGoogle Scholar