Abstract
The new paradigm approaches to technology-enhanced feedback (TEF) have piqued the interest of educators seeking to shift away from feedback monologic processing and toward feedback dialogic exchange. The way teachers demonstrate constructive TEF behaviours for their students is essential to the development of students’ feedback literacy. However, there is a general lack of multidimensional theoretical conceptualisation for understanding, and instrumentation for measuring teacher TEF literacy. To fill this gap in the literature, this study designed and empirically validated a teacher TEF literacy scale (TTLS) using exploratory factor analysis and confirmatory factor analysis on two distinct samples of teacher participants chosen at random. A total of 632 school teachers from throughout most of China, with a variety of subject backgrounds, participated in this study. The results show that the TTLS’ three-factor structure is valid and reliable. To provide supplementary evidence of structural validity, the status quo of TEF literacy among primary and secondary school teachers in China was tested, and a discipline variation in teacher TEF literacy was identified, as hypothesised.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets used and analysed during the current study are available from the corresponding author on reasonable request. Please contact author for data requests.
References
Borup, J., West, R. E., & Thomas, R. (2015). The impact of text versus video communication on instructor feedback in blended courses. Educational Technology Research and Development, 63, 161–184. https://doi.org/10.1007/s11423-015-9367-8
Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698–712. https://doi.org/10.1080/02602938.2012.691462
Brookhart, S. M. (2017). How to give effective feedback to your students (2nd ed.). ASCD.
Brown, T. A. (2006). Confirmatory factor analysis for applied research. The Guilford Press.
Carless, D. (2020). From teacher transmission of information to student feedback literacy: Activating the learner role in feedback processes. Active Learning in Higher Education. https://doi.org/10.1177/1469787420945845
Carless, D., & Winstone, N. (2020). Teacher feedback literacy and its interplay with student feedback literacy. Teaching in Higher Education. https://doi.org/10.1080/13562517.2020.1782372
Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Sage.
Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245–276.
Crook, A., Mauchline, A., Maw, S., Lawson, C., Drinkwater, R., Lundqvist, K., Orsmond, P., Gomez, S., & Park, J. (2012). The use of video technology for providing feedback to students: Can it enhance the feedback experience for staff and students? Computers & Education, 58(1), 386–396. https://doi.org/10.1016/j.compedu.2011.08.025
De Smul, M., Heirweg, S., Van Keer, H., Devos, G., & Vandevelde, S. (2018). How competent do teachers feel instructing self-regulated learning strategies? Development and validation of the teacher self-efficacy scale to implement self-regulated learning. Teaching and Teacher Education, 71, 214–225. https://doi.org/10.1016/j.tate.2018.01.001
Debuse, J. C. W., & Lawley, M. (2014). Benefits and drawbacks of computer-based assessment and feedback systems: Student and educator perspectives. British Journal of Educational Technology, 47(2), 294–301. https://doi.org/10.1111/bjet.12232
Deneen, C. C., Brown, G. T. L., & Carless, D. (2017). Students’ conceptions of eportfolios as assessment and technology. Innovations in Education and Teaching International, 55(4), 487–496. https://doi.org/10.1080/14703297.2017.1281752
ElShaer, A., Casanova, D., Freestone, N. S., & Calabrese, G. (2020). Students’ perceptions of the value of electronic feedback—Does disciplinary background really matter? British Journal of Educational Technology, 51(2), 590–606. https://doi.org/10.1111/bjet.12881
Espasa, A., Mayordomo, R. M., Guasch, T., & Martinez-Melo, M. (2019). Does the type of feedback channel used in online learning environments matter? Students’ perceptions and impact on learning. Active Learning in Higher Education. https://doi.org/10.1177/1469787419891307
Esterhazy, R. (2018). What matters for productive feedback? Disciplinary practices and their relational dynamics. Assessment and Evaluation in Higher Education, 43(8), 1302–1314. https://doi.org/10.1080/02602938.2018.1463353
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39–50.
Geiser, C. (2013). Data analysis with Mplus. Guilford.
George, D., & Mallery, P. (2003). SPSS for windows step by step: A simple guide and reference. 11.0 update (4th ed.). Allyn & Bacon.
Golke, S., Dörfler, T., & Artelt, C. (2015). The impact of elaborated feedback on text comprehension within a computer-based assessment. Learning and Instruction, 39, 123–136. https://doi.org/10.1016/j.learninstruc.2015.05.009
Haelermans, C., Ghysels, J., & Prince, F. (2015). A dataset of three educational technology experiments on differentiation, formative testing and feedback. British Journal of Educational Technology, 46(5), 1102–1108. https://doi.org/10.1111/bjet.12334
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Multivariate data analysis: Pearson new international edition. Pearson.
Han, Y., & Xu, Y. (2019). The development of student feedback literacy: The influences of teacher feedback on peer feedback. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2019.1689545
Hardy, M., & Totman, S. (2016). Teaching an old game new tricks: Long-term feedback on a re-designed online role play. British Journal of Educational Technology, 48(6), 1260–1272. https://doi.org/10.1111/bjet.12498
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487
Haughney, K., Wakeman, S., & Hart, L. (2020). Quality of feedback in higher education: A review of literature. Education Sciences, 10(3), 60. https://doi.org/10.3390/educsci10030060
Henderson, M., & Phillips, M. D. (2014). Technology enhanced feedback on assessment. In T. Sweeney & S. Urban (Eds.), Annual meeting of the Australian council for computers in education (pp. 284–294). Australian Council for Computers in Education (ACCE).
Hepplestone, S., Holden, G., Irwin, B., Parkin, H. J., & Thorpe, L. (2011). Using technology to encourage student engagement with feedback: A literature review. Research in Learning Technology. https://doi.org/10.3402/rlt.v19i2.10347
Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1(1), 104–121.
Istenič, A. (2021). Online learning under COVID-19: Re-examining the prominence of video-based and text-based feedback. Educational Technology Research and Development, 69(1), 117–121. https://doi.org/10.1007/s11423-021-09955-w
Kaiser, H. (1960). The application of electronic computers to factor analysis. Educational and Measurement, Psychological Measurement, 20, 141–151.
Kaplan-Rakowski, R. (2020). Addressing students’ emotional needs during the COVID-19 pandemic: A perspective on text versus video feedback in online environments. Educational Technology Research and Development, 69, 133–136. https://doi.org/10.1007/s11423-020-09897-9
Kline, R. B. (2010). Principles and practice of structural equation modeling (3rd ed.). Guilford.
Lau, W. W. F., & Yuen, A. H. K. (2014). Developing and validating of a perceived ICT literacy scale for junior secondary school students: Pedagogical and educational contributions. Computers & Education, 78, 1–9. https://doi.org/10.1016/j.compedu.2014.04.016
Lee, I. (2021). The development of feedback literacy for writing teachers. TESOL Quarterly. https://doi.org/10.1002/tesq.3012
Li, S., Zhao, J., Shi, G., Tan, Y., Xu, H., Chen, G., Lan, H., & Lin, Z. (2019). Chinese grammatical error correction based on convolutional sequence to sequence model. IEEE Access, 7, 72905–72913. https://doi.org/10.1109/access.2019.2917631
Lowenthal, P. R. (2021). Video feedback: Is it worth the effort? A response to Borup et al. Educational Technology Research and Development, 69, 127–131. https://doi.org/10.1007/s11423-020-09872-4
Makransky, G., Mayer, R., Nøremølle, A., Cordoba, A. L., Wandall, J., & Bonde, M. (2019). Investigating the feasibility of using assessment and explanatory feedback in desktop virtual reality simulations. Educational Technology Research and Development, 68(1), 293–317. https://doi.org/10.1007/s11423-019-09690-3
Malhotra, N. K. (2010). Marketing research: An applied orientation (6th ed.). Pearson.
Mapplebeck, A., & Dunlop, L. (2019). Oral interactions in secondary science classrooms: A grounded approach to identifying oral feedback types and practices. Research in Science Education. https://doi.org/10.1007/s11165-019-9843-y
Mathieson, K. (2012). Exploring student perceptions of audiovisual feedback via screencasting in online courses. American Journal of Distance Education, 26(3), 143–156. https://doi.org/10.1080/08923647.2012.689166
Ministry of Education. (2016). Seize the opportunity to accelerate and create a new situation in education informatization work-speech by Vice Premier Liu Yandong at the National Video Conference on Education Informatization Work [in Chinese]. http://wwwgov.cn/xinwen/201611/01/content_5127061.htm
Munshi, C., & Deneen, C. (2018). Technology-enhanced feedback. In A. Lipnevich & J. Smith (Eds.), The Cambridge handbook of instructional feedback. Cambridge Handbooks in Psychology (pp. 335–356). Cambridge University Press. https://doi.org/10.1017/9781316832134.017
Nunnally, J. C. (1978). Psychometric theory. McGraw-Hill.
Oinas, S., Vainikainen, M.-P., & Hotulainen, R. (2018). Is technology-enhanced feedback encouraging for all in Finnish basic education? A person-centered approach. Learning and Instruction, 58, 12–21. https://doi.org/10.1016/j.learninstruc.2018.05.002
Pitt, E., & Winstone, N. (2020). Towards technology enhanced dialogic feedback. The Enabling Power of Assessment. https://doi.org/10.1007/978-3-030-41956-1_7
Rakoczy, K., Pinger, P., Hochweber, J., Klieme, E., Schütze, B., & Besser, M. (2019). Formative assessment in mathematics: Mediated by feedback’s perceived usefulness and students’ self-efficacy. Learning and Instruction, 60, 154–165. https://doi.org/10.1016/j.learninstruc.2018.01.004
Rogers, A. P., Reagan, E. M., & Ward, C. (2020). Preservice teacher performance assessment and novice teacher assessment literacy. Teaching Education. https://doi.org/10.1080/10476210.2020.1840544
Ryan, T. (2021). Designing video feedback to support the socioemotional aspects of online learning. Educational Technology Research and Development, 69, 137–140. https://doi.org/10.1007/s11423-020-09918-7
Siemens, G. (2004). Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2. https://jotamac.typepad.com/jotamacs_weblog/files/Connectivism.pdf
Sirkiä, T., & Haaranen, L. (2017). Improving online learning activity interoperability with Acos server. Software: Practice and Experience, 47(11), 1657–1676. https://doi.org/10.1002/spe.2492
Timothy, T., Seng Chee, T., Chwee Beng, L., Ching Sing, C., Joyce Hwee Ling, K., Wen Li, C., & Horn Mun, C., (2010). The self-directed learning with technology scale (SDLTS) for young students: An initial development and validation. Computers & Education, 55(4), 1764–1771. https://doi.org/10.1016/j.compedu.2010.08.001.
Tong, J., Bickmeier, R. M., & Rogelberg, S. G. (2020). A comparison of frequency- and agreement-based response formats in the measurement of burnout and engagement. International Journal of Environmental Research and Public Health, 17(2), 543. https://doi.org/10.3390/ijerph17020543
Turner, W., & West, J. (2013). Assessment for “digital first language” speakers: Online video assessment and feedback in higher education. International Journal of Teaching and Learning in Higher Education, 25(3), 288–296.
van der Kleij, F., Adie, L., & Cumming, J. (2019). Feasibility and value of using a GoPro Camera and iPad to study teacher-student assessment feedback interactions. In B. Veldkamp & C. Sluijter (Eds.), Theoretical and practical advances in computer-based educational measurement. Methodology of educational measurement and assessment. Springer. https://doi.org/10.1007/978-3-030-18480-3_18
Wenger, E. (1998). Communities of practice: Learning, meaning and identity. Cambridge University Press. https://doi.org/10.1017/CBO9780511803932
West, J., & Turner, W. (2016). Enhancing the assessment experience: Improving student perceptions, engagement and understanding using online video feedback. Innovations in Education and Teaching International, 53(4), 400–410. https://doi.org/10.1080/14703297.2014.1003954
Winstone, N. E., Balloo, K., & Carless, D. (2020). Discipline-specific feedback literacies: A framework for curriculum design. Higher Education. https://doi.org/10.1007/s10734-020-00632-0
Winstone, N., Boud, D., Dawson, P., & Heron, M. (2021). From feedback-as-information to feedback-as-process: A linguistic analysis of the feedback literature. Assessment & Evaluation in Higher Education. https://doi.org/10.1080/02602938.2021.1902467
Wisniewski, B., Zierer, K., & Hattie, J. (2020). The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2019.03087
Woo, M. M., Chu, S. K. W., & Li, X. (2013). Peer-feedback and revision process in a wiki mediated collaborative writing. Educational Technology Research and Development, 61(2), 279–309. https://doi.org/10.1007/s11423-012-9285-y
Xu, Y., & Carless, D. (2017). “Only true friends could be cruelly honest”: Cognitive scaffolding and social-affective support in teacher feedback literacy. Assessment & Evaluation in Higher Education, 42(7), 1082–1094. https://doi.org/10.1080/02602938.2016.1226759
Yan, S., & Yang, Y. (2020). Education Informatization 2.0 in China: Motivation, framework, and vision. ECNU Review of Education. https://doi.org/10.1177/2096531120944929
Acknowledgement
The first author would like to thank the National Natural Science Foundation of China (No.42071183) for their support to this research.
Funding
This work was supported by the International Joint Research Project of Huiyan International College, Faculty of Education, Beijing Normal University [Grant Number ICER201902].
Author information
Authors and Affiliations
Contributions
This research is conducted by a team of researchers. All authors contributed to the study conception and design. YY: conceptualization, data curation, methodology, visualization, writing—original draft. ZL: methodology, data analysis, writing, investigation. YD: funding acquisition, investigation, project administration, supervision. PMK: Supervision, writing—review and editing. YW: investigation. all authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
There is no conflict of interest in this article from any of the authors.
Ethical approval
APA ethical standards were followed in the conduct of the study, and we received approval from the Beijing Normal University institutional review board.
Consent to participate
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yang, Y., Luo, Z., Dong, Y. et al. Towards a new paradigm: the development and validation of a scale to explore technology-enhanced feedback literacy among primary and secondary school teachers. Education Tech Research Dev 71, 391–413 (2023). https://doi.org/10.1007/s11423-022-10168-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-022-10168-y