Skip to main content
SpringerLink
Log in

Does matching peers at finer-grained levels of prior performance enhance gains in task performance from peer review?

  • Published:
International Journal of Computer-Supported Collaborative Learning Aims and scope Submit manuscript

Abstract

Online peer feedback has proven to be practically useful for instructors and to be useful for learning, especially for the feedback provider. Because students can vary widely in skill level, some research has explored matching reviewer and author by performance level. However, past research on the impacts of reviewer matching has found little effect but used a simple binary high–low approach, which may mask the relative benefits of performance matching. In the current study, we leveraged a large dataset involving three large biology courses implementing multiple assignments with online peer feedback. This large dataset enabled dividing students into four levels of relative task performance to tease apart the relative contributions of providing and receiving feedback within the 16 different author–reviewer performance pairings. The results reveal that changes in task performance over assignments attributable to reviewing experiences vary by these finer-grained prior performance distinctions. In particular, providing to students at the same performance level appears to be beneficial, and receiving feedback from students at the same level is helpful except for very low-performing students. A simulation was used to examine the combined effects of receiving and providing under different algorithms for assigning reviewers to documents. The simulations suggest a matching algorithm will produce overall better outcomes than a random assignment algorithm for students at each of the four performance levels.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The data presented in this study are available upon request from the corresponding author. The data are not publicly available for reasons of privacy.

References

  • Abrache, M. A., Bendou, A., & Cherkaoui, C. (2021). Clustering and combinatorial optimization based approach for learner matching in the context of peer assessment. Journal of Educational Computing Research, 59(6), 1135–1168.

    Article  Google Scholar 

  • Adachi, C., Tai, J. H. M., & Dawson, P. (2018). Academics’ perceptions of the benefits and challenges of self and peer assessment in higher education. Assessment & Evaluation in Higher Education, 43(2), 294–306.

    Article  Google Scholar 

  • Alemdag, E., & Yildirim, Z. (2022). Effectiveness of online regulation scaffolds on peer feedback provision and uptake: A mixed methods study. Computers & Education, 188, 104574.

    Article  Google Scholar 

  • Ariawan, S. (2018). The effectiveness of cooperative learning method (student team achievement divisions) in Christian education. International Journal of Education and Curriculum Application, 1(3), 45–50.

    Article  Google Scholar 

  • Brown, G. T., Peterson, E. R., & Yao, E. S. (2016). Student conceptions of feedback: Impact on self-regulation, self-efficacy, and academic achievement. British Journal of Educational Psychology, 86(4), 606–629.

    Article  Google Scholar 

  • Buttaro, A., Jr., & Catsambis, S. (2019). achievement grouping in the early grades: Long-term consequences for educational equity in the United States. Teachers College Record, 121(2), 1–50.

    Article  Google Scholar 

  • Casserly, A. M., Tiernan, B., & Maguire, G. (2019). Primary teachers’ perceptions of multi-grade classroom grouping practices to support inclusive education. European Journal of Special Needs Education, 34(5), 617–631.

    Article  Google Scholar 

  • Chang, C. Y., Lee, D. C., Tang, K. Y., & Hwang, G. J. (2021). Effect sizes and research directions of peer assessments: From an integrated perspective of meta-analysis and co-citation network. Computers & Education, 164, 104123.

    Article  Google Scholar 

  • Chien, S. Y., Hwang, G. J., & Jong, M. S. Y. (2020). Effects of peer assessment within the context of spherical video-based virtual reality on EFL students’ English-Speaking performance and learning perceptions. Computers & Education, 146, 103751.

    Article  Google Scholar 

  • Cho, K., & MacArthur, C. (2011). Learning by reviewing. Journal of Educational Psychology, 103(1), 73–84.

    Article  Google Scholar 

  • Cho, Y. H., & Cho, K. (2011). Peer reviewers learn from giving comments. Instructional Science, 39(5), 629–643.

    Article  Google Scholar 

  • Dawson, P., Henderson, M., Mahoney, P., Phillips, M., Ryan, T., Boud, D., & Molloy, E. (2019). What makes for effective feedback: Staff and student perspectives. Assessment & Evaluation in Higher Education, 44(1), 25–36.

    Article  Google Scholar 

  • Day, I. N. Z., Saab, N., & Admiraal, W. (2022). Online peer feedback on video presentations: Type of feedback and improvement of presentation skills. Assessment & Evaluation in Higher Education, 47(2), 183–197.

    Article  Google Scholar 

  • De Guerrero, M. C., & Villamil, O. S. (2000). Activating the ZPD: Mutual scaffolding in L2 peer revision. The Modern Language Journal, 84(1), 51–68.

    Article  Google Scholar 

  • Francis, B., Taylor, B., & Tereshchenko, A. (2019). Reassessing Achievement Grouping: Improving Practice for Equity and Attainment. Routledge.

  • Giannoukos, I., Lykourentzou, I., Mpardis, G., Nikolopoulos, V., Loumos, V., & Kayafasa, E. (2010). An adaptive mechanism for author-reviewer matching in online peer assessment. In M. Wallace, I. E. Anagnostopoulos, P. Mylonas & M. Bielikova (Eds.), Semantics in adaptive and personalized services: Methods, tools, and applications (pp. 109–126). Springer-Verlag.

  • Gobet, F., & Sala, G. (2023). Cognitive training: A field in search of a phenomenon. Perspectives on Psychological Science, 18(1), 125–141.

    Article  Google Scholar 

  • Hastings, E. M., Krishna Kumaran, S. R., Karahalios, K., & Bailey, B. P. (2022, February). A learner-centered technique for collectively configuring inputs for an algorithmic team formation tool. In L. Merkle & M. Doyle (Eds.), Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 1 (pp. 969–975). ACM.

  • Hattie, J. (2009). Visible Learning: A Syntheses of over 800 Meta-Analysis Relating to Achievement. Routledge.

    Google Scholar 

  • Hattie, J. (2012). Visible Learning for Teachers: Maximising Impact on Learning. Routledge.

    Book  Google Scholar 

  • Henry, T. R. (2013, March). Creating effective student groups: An introduction to groupformation.org. In T. Camp & P. Tymann (Eds.), Proceeding of the 44th ACM technical symposium on Computer science education (pp. 645–650). ACM.

  • Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111–127.

    Article  Google Scholar 

  • Hollifield, J. (1987). Ability grouping in elementary schools. ERIC Clearinghouse on Elementary and Early Childhood Education.

  • Hsia, L. H., Huang, I., & Hwang, G. J. (2016). Effects of different online peer-feedback approaches on students’ performance skills, motivation and self-efficacy in a dance course. Computers & Education, 96, 55–71.

    Article  Google Scholar 

  • Hsiao, I. H., & Brusilovsky, P. (2011). The role of community feedback in the student example authoring process: An evaluation of annotex. British Journal of Educational Technology, 42(3), 482–499.

    Article  Google Scholar 

  • Huisman, B., Admiraal, W., Pilli, O., van de Ven, M., & Saab, N. (2018). Peer assessment in MOOCs: The relationship between peer reviewers’ achievement and authors’ essay performance. British Journal of Educational Technology, 49(1), 101–110.

    Article  Google Scholar 

  • Huisman, B., Saab, N., van Driel, J., & van den Broek, P. (2017). Peer feedback on college students’ writing: Exploring the relation between students’ achievement match, feedback quality and essay performance. Higher Education Research & Development, 36(7), 1433–1447.

    Article  Google Scholar 

  • Huisman, B., Saab, N., van den Broek, P., & van Driel, J. (2019). The impact of formative peer feedback on higher education students’ academic writing: A meta-analysis. Assessment & Evaluation in Higher Education, 44(6), 863–880.

    Article  Google Scholar 

  • Kanika, Chakraverty, S., Chakraborty, P., & Madan, M. (2022). Effect of different grouping arrangements on students’ achievement and experience in collaborative learning environment. Interactive Learning Environments, 12, 1–13.

  • Kellogg, R. T., & Whiteford, A. P. (2009). Training advanced writing skills: The case for deliberate practice. Educational Psychologist, 44(4), 250–266.

    Article  Google Scholar 

  • Kim, M. K., & Ketenci, T. (2019). Learner participation profiles in an asynchronous online collaboration context. The Internet and Higher Education, 41, 62–76.

    Article  Google Scholar 

  • Kim, S., Lin, T., Chen, J., Logan, J., Purtell, K. M., & Justice, L. M. (2020). Influence of teachers’ grouping strategies on children’s peer social experiences in early elementary classrooms. Frontiers in Psychology, 11, 587170.

  • Klein, C., Diazgranados, D., Salas, E., Le, H., Burke, C. S., Lyons, R., et al. (2009). Does team building work? Small Group Research, 40(2), 181–222.

    Article  Google Scholar 

  • Li, H., Bialo, J. A., Xiong, Y., Hunter, C. V., & Guo, X. (2021). The effect of peer assessment on non-cognitive outcomes: A meta-analysis. Applied Measurement in Education, 34(3), 179–203.

    Article  Google Scholar 

  • Li, H., Xiong, Y., Hunter, C. V., Guo, X., & Tywoniw, R. (2020). Does peer assessment promote student learning? A meta-analysis. Assessment & Evaluation in Higher Education, 45(2), 193–211.

    Article  Google Scholar 

  • Li, H., Xiong, Y., Zang, X., Kornhaber, M. L., Lyu, Y., Chung, K. S., & Suen, H. K. (2016). Peer assessment in the digital age: A meta-analysis comparing peer and teacher ratings. Assessment & Evaluation in Higher Education, 41(2), 245–264.

    Article  Google Scholar 

  • Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525–536.

    Article  Google Scholar 

  • Lou, Y., Abrami, P. C., Spence, J. C., Poulsen, C., Chambers, B., & d’Apollonia, S. (1996). Within-class grouping: A meta-analysis. Review of Educational Research, 66(4), 423–458.

    Article  Google Scholar 

  • Lundstrom, K., & Baker, W. (2009). To give is better than to receive: The benefits of peer review to the reviewer’s own writing. Journal of Second Language Writing, 18(1), 30–43.

    Article  Google Scholar 

  • Matthews, M. S., Ritchotte, J. A., & McBee, M. T. (2013). Effects of schoolwide cluster grouping and within-class achievement grouping on elementary school students’ academic achievement growth. High Achievement Studies, 24(2), 81–97.

    Google Scholar 

  • McKeen, H. (2019). The impact of grade level flexible grouping on math achievement scores. Georgia Educational Researcher, 16(1), 48–62.

    Article  Google Scholar 

  • Min, H. T. (2016). Effect of teacher modeling and feedback on EFL students’ peer review skills in peer review training. Journal of Second Language Writing, 31, 43–57.

    Article  Google Scholar 

  • Oppermann, E., Brunner, M., & Anders, Y. (2019). The interplay between preschool teachers’ science self-efficacy beliefs, their teaching practices, and girls’ and boys’ early science motivation. Learning and Individual Differences, 70, 86–99.

    Article  Google Scholar 

  • Patchan, M. M., & Schunn, C. D. (2015). Understanding the benefits of providing peer feedback: how students respond to peers’ texts of varying quality. Instructional Science, 43(5), 591–614.

  • Patchan, M. M., Schunn, C. D., & Clark, R. J. (2018). Accountability in peer assessment: examining the effects of reviewing grades on peer ratings and peer feedback. Studies in Higher Education, 43(12), 2263−2278.

  • Ramachandran, L., Gehringer, E. F., & Yadav, R. K. (2017). Automated assessment of the quality of peer reviews using natural language processing techniques. International Journal of Artificial Intelligence in Education, 27(3), 534–581.

    Article  Google Scholar 

  • Sala, G., & Gobet, F. (2020a). Working memory training in typically developing children: A multilevel meta-analysis. Psychonomic Bulletin & Review, 27(3), 423–434.

    Article  Google Scholar 

  • Sala, G., & Gobet, F. (2020b). Cognitive and academic benefits of music training with children: A multilevel meta-analysis. Memory & Cognition, 48(8), 1429–1441.

    Article  Google Scholar 

  • Saleh, M., Lazonder, A. W., & De Jong, T. (2005). Effects of within-class achievement grouping on social interaction, achievement, and motivation. Instructional Science, 33(2), 105–119.

    Article  Google Scholar 

  • Salehi, N., & Bernstein, M. S. (2018). Hive: Collective design through network rotation. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 151:1–26.

  • Schmid, U., & Finzel, B. (2020). Mutual explanations for cooperative decision making in medicine. KI-Künstliche Intelligenz, 34(2), 227–233.

    Article  Google Scholar 

  • Schunn, C., Godley, A., & DeMartino, S. (2016). The reliability and validity of peer review of writing in high school AP English classes. Journal of Adolescent & Adult Literacy, 60(1), 13–23.

  • Seifert, T., & Feliks, O. (2019). Online self-assessment and peer-assessment as a tool to enhance student-teachers’ assessment skills. Assessment & Evaluation in Higher Education, 44(2), 169–185.

    Article  Google Scholar 

  • Shang, H. F. (2022). Exploring online peer feedback and automated corrective feedback on EFL writing performance. Interactive Learning Environments, 30(1), 4–16.

    Article  Google Scholar 

  • Sheppard, C., Manalo, E., & Henning, M. (2018). Is achievement grouping beneficial or detrimental to Japanese ESP students’ English language proficiency development? English for Specific Purposes, 49, 39–48.

    Article  Google Scholar 

  • Shooshtari, Z. G., & Mir, F. (2014). ZPD, tutor; peer scaffolding: Sociocultural theory in writing strategies application. Procedia-Social and Behavioral Sciences, 98, 1771–1776.

    Article  Google Scholar 

  • Slavin, R. E. (1993). achievement grouping in the middle grades: Achievement effects and alternatives. The Elementary School Journal, 93(5), 535–552.

    Article  Google Scholar 

  • Steenbergen-Hu, S., Makel, M. C., & Olszewski-Kubilius, P. (2016). What one hundred years of research says about the effects of achievement grouping and acceleration on K–12 students’ academic achievement: Findings of two second-order meta-analyses. Review of Educational Research, 86(4), 849–899.

    Article  Google Scholar 

  • Storch, N. (2018). Written corrective feedback from sociocultural theoretical perspectives: A research agenda. Language Teaching, 51(2), 262–277.

    Article  Google Scholar 

  • Strijbos, J. W., & Sluijsmans, D. (2010). Unravelling peer assessment: Methodological, functional, and conceptual developments. Learning and Instruction, 20, 265–269.

    Article  Google Scholar 

  • Sunahase, T., Baba, Y., & Kashima, H. (2019). Probabilistic modeling of peer correction and peer assessment. In C. F. lynch, A. Merceron, M. Desmarais & R. Nkambou (Eds.), Proceedings of the 12th International Conference on Educational Data Mining (pp. 426–431). ERIC.

  • Tseng, S. C., & Tsai, C. C. (2007). On-line peer assessment and the role of the peer feedback: A study of high school computer course. Computers & Education, 49(4), 1161–1174.

    Article  Google Scholar 

  • Tsivitanidou, O. E., Constantinou, C. P., Labudde, P., Rönnebeck, S., & Ropohl, M. (2018). Reciprocal peer assessment as a learning tool for secondary school students in modeling-based learning. European Journal of Psychology of Education, 33(1), 51–73.

    Article  Google Scholar 

  • Tsui, A. B. (2011). Teacher education and teacher development. Handbook of Research in Second Language Teaching and LearniNg, 2, 21–39.

    Google Scholar 

  • Urdan, T., & Kaplan, A. (2020). The origins, evolution, and future directions of achievement goal theory. Contemporary Educational Psychology, 61, 101862.

    Article  Google Scholar 

  • Vanhorn, S., Ward, S. M., Weismann, K. M., Crandall, H., Reule, J., & Leonard, R. (2019). Exploring active learning theories, practices, and contexts. Communication Research Trends, 38(3), 5–25.

    Google Scholar 

  • van Popta, E., Kral, M., Camp, G., Martens, R. L., & Simons, P. R. J. (2017). Exploring the value of peer feedback in online learning for the provider. Educational Research Review, 20, 24–34.

    Article  Google Scholar 

  • Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes, (M. Cole, V. John-Steiner, S. Scribner & E. Souberman, Eds. and trans.). Harvard University Press.

  • Wang, D. Y., Lin, S., & Sun, C. T. (2007). Diana: A computer-supported heterogeneous grouping system for teachers to conduct successful small learning groups. Computers in Human Behavior, 23(4), 1997–2010.

    Article  Google Scholar 

  • Wang, Y. Q., & Sun, F. Q. (2018, May). How to choose an appropriate reviewer assignment strategy in peer assessment system? Considering fairness and incentive. In L. Wang, D. Askarany & M. Pawlak (Eds.), 4th Annual International Conference on Management, Economics and Social Development (ICMESD 2018) (pp. 603–608). Atlantis Press.

  • Warwick, P., & Maloch, B. (2003). Scaffolding speech and writing in the primary classroom: A consideration of work with literature and science pupil groups in the USA and UK. Reading, 37(2), 54–63.

    Article  Google Scholar 

  • Wen, M., Maki, K., Dow, S., Herbsleb, J. D., & Rose, C. (2017). Supporting virtual team formation through community-wide deliberation. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 533–538.

  • Wooley, R. S. (2007). The effects of web-based peer review on student writing (Doctoral dissertation, Kent State University).

  • Wu, Y., & Schunn, C. D. (2020). From feedback to revisions: Effects of feedback features and perceptions. Contemporary Educational Psychology, 60, 101826.

  • Wu, Y., & Schunn, C. D. (2021a). From plans to actions: A process model for why feedback features influence feedback implementation. Instructional Science, 49(3), 365–394.

  • Wu, Y., & Schunn, C. D. (2021b). The effects of providing and receiving peer feedback on writing performance and learning of secondary school students. American Educational Research Journal, 58(3), 492–526.

  • Wu, Y., & Schunn, C. D. (2023a). Passive, active, and constructive engagement with peer feedback: A revised model of learning from peer feedback. Contemporary Educational Psychology, 73, 102160.

  • Wu, Y., & Schunn, C. D. (2023b). Assessor writing performance on peer feedback: Exploring the relation between assessor writing performance, problem identification accuracy, and helpfulness of peer feedback. Journal of Educational Psychology, 115(1), 118–142.

  • Xiong, Y., & Schunn, C. D. (2021). Reviewer, essay, and reviewing-process characteristics that predict errors in web-based peer review. Computers & Education, 166, 104146.

  • Yang, M., Badger, R., & Yu, Z. (2006). A comparative study of peer and teacher feedback in a Chinese EFL writing class. Journal of Second Language Writing, 15(3), 179–200.

    Article  Google Scholar 

  • Yim, S., & Warschauer, M. (2017). Web-based collaborative writing in L2 contexts: Methodological insights from text mining. Language Learning & Technology, 21(1), 146–165.

    Google Scholar 

  • Yu, S., & Hu, G. (2017). Can higher-proficiency L2 learners benefit from working with lower-proficiency partners in peer feedback? Teaching in Higher Education, 22(2), 178–192.

    Article  Google Scholar 

  • Zhang, X., & McEneaney, J. E. (2020). What is the influence of peer feedback and author response on Chinese University students’ English writing performance? Reading Research Quarterly, 55(1), 123–146.

    Article  Google Scholar 

  • Zong, Z., Schunn, C. D., & Wang, Y. (2021). What aspects of online peer feedback robustly predict growth in students’ task performance?. Computers in Human Behavior, 124, 106924.

Download references

Author information

Authors and Affiliations

  1. School of Law and Economics, China University of Political Science and Law, No. 25 Xitucheng Road., Haidian District, Beijing, 100088, People’s Republic of China

    Zheng Zong

  2. Learning Research & Development Center, University of Pittsburgh, Pittsburgh, PA, USA

    Christian D. Schunn

Authors
  1. Zheng Zong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian D. Schunn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zheng Zong.

Ethics declarations

Conflict of interest disclosure

No conflict of interest exists in the submission of this manuscript, and all authors approve the manuscript for publication. We would like to declare on behalf of all co-authors that the work described is original research that has not been published previously, and not under consideration for publication elsewhere, in whole or in part. All the authors listed have approved the manuscript that is enclosed.

Ethics approval

The study was approved by the University’s Human Research Protection Office, and we did not use any data for analysis that contained student names.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A

Table 5 Mean and standard deviations for each variable within each achievement pairing, along with maximum observed values on each variable
Full size table

Appendix B

Table 6 Within each performance level, Pearson intercorrelations among predictors and the outcome variable
Full size table

,

Table 7 In each performance, Pearson intercorrelations among predictors and the outcome variable
Full size table

Appendix C Stata code for regressions

  • Regressions in Table 3: bys performance-level: reg Z-Score Z-ScoreJ-1 Very-low Length providedJ-1 Low Length providedJ-1 High Length providedJ-1 Very-high Length providedJ-1 Total Length receivedJ-1 i.Course Round, beta

  • Regressions in Table 4: bys performance-level: reg Z-Score Z-ScoreJ-1 Very-low Length receivedJ-1 Low Length receivedJ-1 High Length receivedJ-1 Very-high Length receivedJ-1 Total Length providedJ-1 i.Course Round, beta

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zong, Z., Schunn, C.D. Does matching peers at finer-grained levels of prior performance enhance gains in task performance from peer review?. Intern. J. Comput.-Support. Collab. Learn 18, 425–456 (2023). https://doi.org/10.1007/s11412-023-09401-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11412-023-09401-4

Keywords

Search

Navigation