Abstract
Working in groups is an important collaboration activity in the educational context, where a variety of factors can influence the prediction of the teams’ performance. In the pertinent bibliography, several machine learning models are available for delivering predictions. In this sense, the main goal of the current research is to assess 28 different machine learning models, including a Deep Neural Network (DNN) which is structured by 4 hidden layers, for predicting teams’ performance. Additionally, both data analysis and optimization of input data are also explored for their effectiveness in the improvement of the models’ performance. One key finding of the present study is that the XGBoost model succeeded better prediction results, and its precision and robustness were found to be higher, compared to the other models. Additionally, data optimization was shown to be an essential procedure, since the prediction accuracy of all the models, and specifically, that of the XGBoost, improved and found to be 96% during the first phase that of the process, and 94% during the second phase that of the product. Similarly, after applying the hyperparameter tuning and data optimization, the prediction accuracy of the DNN was also improved and found to be 89.94% and 86.16%, during the same two phases. Finally, for interpreting the output of the ML models, in terms of features’ importance, the Shapley Additive Explanations framework (SHAP) was used.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ali, A. A., Dafoulas, G. A., & Augusto, J. C. (2019). Collaborative educational environments incorporating mixed reality technologies: A systematic mapping study. IEEE Transactions on Learning Technologies, 12(3), 321–332.
Almotiri, J., Elleithy, K., Elleithy, A. (2017) Comparison of autoencoder and principal component analysis followed by neural network for e-learning using handwritten recognition. In: 2017 IEEE Long Island Systems, Applications and Technology Conference (LISAT) (pp. 1–5)
Ancona, D. G., & Caldwell, D. F. (1992). Demography and design: Predictors of new product team performance. Organization Science, 3(3), 321–341.
Aouine, A., Mahdaoui, L., Moccozet, L. (2019) A workflow-based solution to support the assessment of collaborative activities in e-learning. The International Journal of Information and Learning Technology
Bourkoukou, O., El Bachari, E., El Boustani, A.: Building effective collaborative groups in e-learning environment. In: International Conference on Advanced Intelligent Systems for Sustainable Development (pp. 107–117). Springer (2019)
Brooks, C. M., & Ammons, J. L. (2003). Free riding in group projects and the effects of timing, frequency, and specificity of criteria in peer assessments. Journal of Education for Business, 78(5), 268–272.
Calma, A., Kottke, D., Sick, B., Tomforde, S. (2017) Learning to learn: Dynamic runtime exploitation of various knowledge sources and machine learning paradigms. In: 2017 IEEE 2nd International Workshops on Foundations and Applications of Self* Systems (FAS* W) (pp. 109–116). IEEE
Chang, Y. S., Chen, Y. S., & Chiang, C. W. (2019). The differences in pleasing value and learning performance among different groups using mobile augmented reality system for cultural environment learning. Multimedia Tools and Applications, 78(4), 4965–4986.
Chin, K.Y., Ko-Fong, L., Chen, Y.L. (2019) Effects of a ubiquitous guide-learning system on cultural heritage course students’ performance and motivation. IEEE Transactions on Learning Technologies
Chis, A. E., Moldovan, A. N., Murphy, L., Pathak, P., & Muntean, C. H. (2018). Investigating flipped classroom and problem-based learning in a programming module for computing conversion course. Journal of Educational Technology & Society, 21(4), 232–247.
Chopade, P., Khan, S.M., Edwards, D., von Davier, A. (2018) Machine learning for efficient assessment and prediction of human performance in collaborative learning environments. In: 2018 IEEE International Symposium on Technologies for Homeland Security (HST) (pp. 1–6). IEEE
Dlamini, M., Leung, W.S. (2018) Evaluating machine learning techniques for improved adaptive pedagogy. In: 2018 IST-Africa Week Conference (IST-Africa) (pp. Page–1). IEEE
El Hammoumi, O., Benmarrakchi, F., Ouherrou, N., El Kafi, J., El Hore, A. (2018) Emotion recognition in e-learning systems. In: 2018 6th International Conference on Multimedia Computing and Systems (ICMCS) (pp. 1–6)
Elliott, C.H., Radke, S.C. (2018) A focus on contribution towards product and performance in collaborative design. International Society of the Learning Sciences, Inc.[ISLS]
Ferdiana, R. (2016). Software engineering education learning process for professional developers. Journal of e-Learning and Knowledge Society, 12(2). https://www.learntechlib.org/p/173460/
Giannakas, F., Troussas, C., Voyiatzis, I., Sgouropoulou, C. (2021). A deep learning classi cation framework for early prediction of team-based academic performance. Applied Soft Computing 106, 107355. https://doi.org/10.1016/j.asoc.2021.107355.
Hanko, G. (2016) Increasing competence through collaborative problem-solving: Using insight into social and emotional factors in children’s learning. Routledge
Hirshfield, L., & Koretsky, M. D. (2018). Gender and participation in an engineering problem-based learning environment. Interdisciplinary Journal of Problem-Based Learning, 12(1), 2.
Holmes, M., Latham, A., Crockett, K., & O’Shea, J. D. (2018). Near real-time comprehension classification with artificial neural networks: Decoding e-learner non-verbal behavior. IEEE Transactions on Learning Technologies, 11(1), 5–12.
Huizenga, J., Admiraal, W., ten Dam, G., & Voogt, J. (2019). Mobile game-based learning in secondary education: Students’ immersion, game activities, team performance and learning outcomes. Computers in Human Behavior, 99, 137–143.
Kerrigan, S., Feng, S., Vuthaluru, R., Ifenthaler, D., Gibson, D. (2019) Network analytics of collaborative problem-solving. ADIS International Conference Cognition and Exploratory Learning in Digital Age (pp. 43–50)
Khodke, P.A., Tingane, M.G., Bhagat, A.P., Chaudhari, S.P., Ali, M.S. (2016) Neuro fuzzy intelligent e-learning systems. In: 2016 Online International Conference on Green Engineering and Technologies (IC-GET) (pp. 1–7)
Kingma, D., Ba, J. (2015) Adam: A method for stochastic optimization. In: International Conference on Learning Representations
Kokku, R., Sundararajan, S., Dey, P., Sindhgatta, R., Nitta, S., Sengupta, B. (2018) Augmenting classrooms with ai for personalized education. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 6976–6980). IEEE
Li, G., Wang, Y. (2018) Research on leamer’s emotion recognition for intelligent education system. In: 2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC) (pp. 754–758)
Lundberg, S.M., Lee, S.I. (2017) A unified approach to interpreting model predictions. In: Advances in neural information processing systems (pp. 4765–4774)
Marie-Sainte, S. L., Alalyani, N., Alotaibi, S., Ghouzali, S., & Abunadi, I. (2018). Arabic natural language processing and machine learning-based systems. IEEE Access, 7, 7011–7020.
Martínez-Cerdá, J. F., Torrent-Sellens, J., & González-González, I. (2018). Promoting collaborative skills in online university: comparing effects of games, mixed reality, social media, and other tools for ict-supported pedagogical practices. Behaviour & Information Technology, 37(10–11), 1055–1071.
Matazi, I., Bennane, A., Messoussi, R., Touahni, R., Oumaira, I., Korchiyne, R. (2018) Multi-agent system based on fuzzy logic for e-learning collaborative system. In: 2018 International Symposium on Advanced Electrical and Communication Technologies (ISAECT) (pp. 1–7). IEEE
Mengoni, P., Milani, A., Li, Y. (2018) Clustering students interactions in elearning systems for group elicitation. In: International Conference on Computational Science and Its Applications (pp. 398–413). Springer
Neill, C. J., DeFranco, J. F., & Sangwan, R. S. (2017). Improving collaborative learning in online software engineering education. European Journal of Engineering Education, 42(6), 591–602.
Nguyen, V., Dang, H. H., Do, N. K., & Tran, D. T. (2016). Enhancing team collaboration through integrating social interactions in a web-based development environment. Computer Applications in Engineering Education, 24(4), 529–545.
Noguera, I., Guerrero-Roldan, A. E., & Maso, R. (2018). Collaborative agile learning in online environments: Strategies for improving team regulation and project management. Computers & Education, 116, 110–129.
Petkovic, D., Sosnick-Pérez, M., Huang, S., Todtenhoefer, R., Okada, K., Arora, S., Sreenivasen, R., Flores, L., Dubey, S. (2014) Setap: Software engineering teamwork assessment and prediction using machine learning. In: 2014 IEEE Frontiers in Education Conference (FIE) Proceedings (pp. 1–8). IEEE
Petkovic, D., Sosnick-Pérez, M., Okada, K., Todtenhoefer, R., Huang, S., Miglani, N., Vigil, A. (2016) Using the random forest classifier to assess and predict student learning of software engineering teamwork. In: 2016 IEEE Frontiers in Education Conference (FIE) (pp. 1–7). IEEE, https://archive.ics.uci.edu/ml/datasets/Data+for+Software+Engineering+Teamwork+Assessment+in+Education+Setting. Accessed Nov 2020
Poondej, C., & Lerdpornkulrat, T. (2019). Gamification in e-learning: A moodle implementation and its effect on student engagement and performance. Interactive Technology and Smart Education, 17(1), 56–66.
Qi, C. (2019). Social media usage of students, role of tie strength, and perceived task performance. Journal of Educational Computing Research, 57(2), 385–416.
Saito, T., Watanobe, Y. (2018) Learning path recommender system based on recurrent neural network. In: 2018 9th International Conference on Awareness Science and Technology (iCAST) (pp. 324–329)
Sharma, K., Pappas, I., Papavlasopoulou, S., Giannakos, M. (2019) Towards automatic and pervasive physiological sensing of collaborative learning. International Society of the Learning Sciences (ISLS)
Siregar, C., Pane, M.M., Ruman, Y.S. (2018) The virtual team performance in solving teamwork conflict problems. In: Proceedings of the 2018 International Conference on Distance Education and Learning (pp. 1–5)
Swiecki, Z., Shaffer, D.W. (2018) Toward a taxonomy of team performance visualization tools. In: Proceedings of the 2018 International Conference of the Learning Sciences
Troussas, C., Krouska, A., Sgouropoulou, C. (2020). Collaboration and fuzzy-modeled personalization for mobile game-based learning in higher education. Computers & Education 144, 103698.
Wang, C., Fang, T., & Gu, Y. (2020). Learning performance and behavioral patterns of online collaborative learning: Impact of cognitive load and affordances of different multimedia. Computers & Education, 143, 103683.
Wen, M., Maki, K., Dow, S., Herbsleb, J. D., & Rose, C. (2017). Supporting virtual team formation through community-wide deliberation. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 1–19.
Zacharis, N. Z. (2016). Predicting student academic performance in blended learning using artificial neural networks. International Journal of Artificial Intelligence and Applications, 7(5), 17–29.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Rights and permissions
About this article
Cite this article
Giannakas, F., Troussas, C., Krouska, A. et al. Multi-technique comparative analysis of machine learning algorithms for improving the prediction of teams’ performance. Educ Inf Technol 27, 8461–8487 (2022). https://doi.org/10.1007/s10639-022-10900-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-022-10900-4