ABSTRACT
Little is known about the usability and cognitive load experienced by students using online asynchronous discussion tools in higher education. Given that the use of both generic chat tools and specialised learning technology forum tools continue to increase, it is important for educators and designers/developers to understand the user experience of these tools. This research evaluates a text-based chat tool, Discord, and a text-based forum tool, Ed Discussion, from a usability and cognitive load perspective to complement the findings from the educational literature. We have identified usability issues that plague such tools related to learning curves, engagement, and effectiveness. Our findings can be used by educators to assist in choosing the right tools for their context and also make recommendations to designers of educational discussion tools.
- Abbas, N., , Online chat and chatbots to enhance mature student engagement in higher education. International Journal of Lifelong Education, 2022. 41(3): p. 308-326.Google Scholar
Cross Ref
- 2. Tang, Y. and K.F. Hew, Effects of using mobile instant messaging on student behavioral, emotional, and cognitive engagement: a quasi-experimental study. International Journal of Educational Technology in Higher Education, 2022. 19(1): p. 3.Google ScholarCross Ref
- Lucho, S., G.B. Gardini, and J.M.E. Bueno. Teaching web development courses using flipped classroom and Discord: a two-year experience in the Peruvian context during the COVID-19 pandemic. in 2023 IEEE World Engineering Education Conference (EDUNINE). 2023. IEEE.Google ScholarCross Ref
- Santiago, A. and C. Mattos, From classroom education to remote emergency education: transformations in a dialogical pedagogy proposal:. Dialogic Pedagogy: An International Online Journal, 2023. 11(1): p. DT1-DT21.Google ScholarCross Ref
- Heinrich, E., H. Thomas, and E.R. Kahu, An exploration of course and cohort communication spaces in Discord, Teams, and Moodle. Australasian Journal of Educational Technology, 2022: p. 107-120.Google Scholar
- Kahu, E.R., H.G. Thomas, and E. Heinrich, ‘A sense of community and camaraderie’: Increasing student engagement by supplementing an LMS with a Learning Commons Communication Tool. Active Learning in Higher Education, 2022: p. 146978742211276.Google Scholar
- Arifianto, M. and I. Izzudin, Students’ acceptance of discord as an alternative online learning media. International Journal of Emerging Technologies in Learning (iJET), 2021. 16(20): p. 179-195.Google ScholarCross Ref
- Mora-Jimenez, L.D., , User experience in communication and collaboration platforms: a comparative study including discord, microsoft teams, and zoom, in Information Technology and Systems: Proceedings of ICITS 2022. 2022, Springer. p. 52-61.Google ScholarCross Ref
- Uong, T.G.T., D.K. Nguyen, and H.N. Nguyen, Teachers’ Feedback on Using Discord as an Online Learning Platform. International Journal of TESOL & Education, 2022. 2(4): p. 84-104.Google ScholarCross Ref
- 10. Ling, G., Learning with your Buddies: an investigation of community based UX design learning on Discord. 2022.Google Scholar
- Tang, Y. and K.F. Hew, Examining the utility and usability of mobile instant messaging in a graduate-level course: A usefulness theoretical perspective. Australasian Journal of Educational Technology, 2019. 35(4).Google Scholar
- Hameed, I., , Social media usage and academic performance from a cognitive loading perspective. On the Horizon, 2022.Google Scholar
- Van Nuland, S.E., R. Eagleson, and K.A. Rogers, Educational software usability: Artifact or Design? Anatomical sciences education, 2017. 10(2): p. 190-199.Google ScholarCross Ref
- Van Greunen, D. and J. Wesson, Formal usability testing of interactive educational software: A case study. Usability: Gaining a Competitive Edge, 2002: p. 161-176.Google Scholar
- MacFarlane, S., G. Sim, and M. Horton. Assessing usability and fun in educational software. in Proceedings of the 2005 conference on Interaction design and children. 2005.Google ScholarDigital Library
- Costabile, M.F., On the usability evaluation of e-learning applications. in Proceedings of the 38th Annual Hawaii International Conference on System Sciences. 2005. IEEE.Google ScholarDigital Library
- Ardito, C., Usability of e-learning tools. in Proceedings of the working conference on Advanced visual interfaces. 2004.Google ScholarDigital Library
- Ardito, C., Towards guidelines for usability of e-learning applications. in User-Centered Interaction Paradigms for Universal Access in the Information Society: 8th ERCIM Workshop on User Interfaces for All, Vienna, Austria, June 28-29, 2004, Revised Selected Papers 8. 2004. Springer.Google ScholarCross Ref
- Gunesekera, A.I., Y. Bao, and M. Kibelloh, The role of usability on e-learning user interactions and satisfaction: A literature review. Journal of Systems and Information Technology, 2019.Google Scholar
- Sweller, J., Cognitive load theory, in Psychology of learning and motivation. 2011, Elsevier. p. 37-76.Google Scholar
- . Sweller, J., Cognitive Load Theory: Recent Theoretical Advances, in Cognitive Load Theory, J.L. Plass, R. Brünken, and R. Moreno, Editors. 2010, Cambridge University Press: Cambridge. p. 29-47.Google ScholarCross Ref
- Brignell, C., , Fostering purposeful engagement by building staff-student communities. Student Engagement in Higher Education Journal, 2022. 4(2).Google Scholar
- Ranga Auaha Ako (Learning and Teaching Design Team). Comparison - Piazza / Ed Discussion / Canvas Discussions. Available from: https://teachwell.auckland.ac.nz/resources/engagement/communication/#comparison.Google Scholar
- . Bygstad, B., , From dual digitalization to digital learning space: Exploring the digital transformation of higher education. Computers & Education, 2022. 182: p. 104463.Google ScholarDigital Library
- Loncar, M., N.E. Barrett, and G.-Z. Liu, Towards the refinement of forum and asynchronous online discussion in educational contexts worldwide: Trends and investigative approaches within a dominant research paradigm. Computers & Education, 2014. 73: p. 93-110.Google Scholar
- Thomas, G. and S. Thorpe, Enhancing the facilitation of online groups in higher education: a review of the literature on face-to-face and online group-facilitation. Interactive Learning Environments, 2019. 27(1): p. 62-71.Google ScholarCross Ref
- Fehrman, S. and S.L. Watson, A Systematic Review of Asynchronous Online Discussions in Online Higher Education. American Journal of Distance Education, 2021. 35(3): p. 200-213.Google ScholarCross Ref
- De Lima, D.P.R., , What to expect, and how to improve online discussion forums: the instructors’ perspective. Journal of Internet Services and Applications, 2019. 10(1): p. 22.Google ScholarCross Ref
- Griffin, L. and J. Roy, A great resource that should be utilised more, but also a place of anxiety: student perspectives on using an online discussion forum. Open Learning: The Journal of Open, Distance and e-Learning, 2022. 37(3): p. 235-250.Google ScholarCross Ref
- Manca, S., Snapping, pinning, liking or texting: Investigating social media in higher education beyond Facebook. The Internet and Higher Education, 2020. 44: p. 100707.Google Scholar
- Vladoiu, M. and Z. Constantinescu. Learning During COVID-19 Pandemic: Online Education Community, Based on Discord. in 2020 19th RoEduNet Conference: Networking in Education and Research (RoEduNet). 2020. IEEE.Google ScholarCross Ref
- Hernández-Lara, A.B., A. Perera-Lluna, and E. Serradell-López, Game learning analytics of instant messaging and online discussion forums in higher education. Education + Training, 2021. 63(9): p. 1288-1308.Google Scholar
- Squires, D. and J. Preece, Usability and learning: evaluating the potential of educational software. Computers & education, 1996. 27(1): p. 15-22.Google ScholarCross Ref
- Chu, A., , Usability of Learning Moment: Features of an E-learning Tool That Maximize Adoption by Students. West J Emerg Med, 2019. 21(1): p. 78-84.Google Scholar
- Kakasevski, G., Evaluating usability in learning management system moodle. in ITI 2008 - 30th International Conference on Information Technology Interfaces. 2008.Google ScholarCross Ref
- Althobaiti, M.M. and P. Mayhew. Assessing the Usability of Learning Management System: User Experience Study. in E-Learning, E-Education, and Online Training. 2016. Cham: Springer International Publishing.Google ScholarCross Ref
- Abuhlfaia, K. and E.d. Quincey, The usability of E-learning platforms in higher education: a systematic mapping study, in Proceedings of the 32nd International BCS Human Computer Interaction Conference. 2018, BCS Learning & Development Ltd.: Belfast, United Kingdom. p. Article 7.Google Scholar
- Abuhlfaia, K. and E. De Quincey, The Usability of E-learning Platforms in Higher Education: A Systematic Mapping Study. 2018. 1-13.Google Scholar
- Masood, M. and A. Musman, The Usability and its Influence of an e-Learning System on Student Participation. Procedia - Social and Behavioral Sciences, 2015. 197: p. 2325-2330.Google ScholarCross Ref
- Kubacz-Szumska, J. and O. Szumski, Cloud Communications During the Pandemic From the Perspective of Collaboration Platforms. Problemy Zarządzania, 2021. 19(3/2021 (93): p. 138-149.Google ScholarCross Ref
- Cuijpers, J., An empirical evaluation of video conferencing systems used in industry, academia, and entertainment. in Companion of the ACM/SPEC International Conference on Performance Engineering. 2021.Google ScholarDigital Library
- Sidpra, J., , Sustaining education in the age of COVID-19: a survey of synchronous web-based platforms. Quantitative Imaging in Medicine and Surgery, 2020. 10(7): p. 1422.Google ScholarCross Ref
- Keller, S., J. Pirker, and E. List. Digital Communication Tools in Private and Professional Environments. in International Conference on Interactive Collaborative Learning. 2022. Springer.Google Scholar
- Dayana, Y.E., O.M. Andre, and L. Andrade-Arenas. Design of the Discord application as an E-learning tool at the University of Sciences and Humanities. in LACCEI International Multi-Conference for Engineering, Education, and Technology. 2020.Google Scholar
- Shi, Z., G. Luo, and L. He, Mobile-assisted Language Learning Using WeChat Instant Messaging. International Journal of Emerging Technologies in Learning (iJET), 2017. 12(02): p. pp. 16-26.Google Scholar
- Young, M.S., , State of science: mental workload in ergonomics. Ergonomics, 2015. 58(1): p. 1-17.Google Scholar
- Paas, F.G., Training strategies for attaining transfer of problem-solving skill in statistics: a cognitive-load approach. Journal of educational psychology, 1992. 84(4): p. 429.Google Scholar
- Hart, S.G. and L.E. Staveland, Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research, in Advances in Psychology, P.A. Hancock and N. Meshkati, Editors. 1988, North-Holland. p. 139-183.Google ScholarCross Ref
- Marquart, G., C. Cabrall, and J. de Winter, Review of Eye-related Measures of Drivers’ Mental Workload. Procedia Manufacturing, 2015. 3: p. 2854-2861.Google Scholar
- Bagley, J. and L. Manelis, Effect of awareness on an indicator of cognitive load. Perceptual and Motor Skills, 1979. 49(2): p. 591-594.Google ScholarCross Ref
- Korbach, A., R. Brünken, and B. Park, Measurement of cognitive load in multimedia learning: a comparison of different objective measures. Instructional science, 2017. 45: p. 515-536.Google ScholarCross Ref
- Brooke, J., SUS-A quick and dirty usability scale. Usability evaluation in industry, 1996. 189(194): p. 4-7.Google Scholar
- Lewis, C. and J. Rieman, Task-centered user interface design. A practical introduction, 1993.Google Scholar
- Tonsen, M., C.K. Baumann, and K. Dierkes, A high-level description and performance evaluation of pupil invisible. arXiv preprint arXiv:2009.00508, 2020.Google Scholar
- Klingner, J., Measuring cognitive load during visual tasks by combining pupillometry and eye tracking. 2010: Stanford University.Google Scholar
- Bangor, A., P. Kortum, and J. Miller, Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies, 2009. 4(3): p. 114-123.Google Scholar
- Fuhl, W., , Pupil detection for head-mounted eye tracking in the wild: an evaluation of the state of the art. Machine Vision and Applications, 2016. 27: p. 1275-1288.Google ScholarDigital Library
- 58. Goldberg, J.H. and A.M. Wichansky, Eye tracking in usability evaluation: A practitioner's guide, in the Mind's Eye. 2003, Elsevier. p. 493-516.Google ScholarCross Ref
- Poole, A., Ball. LJ (2005). Eye tracking in human-computer interaction and usability research: current status and future prospects. Encyclopedia of Human-Computer Interaction: p. 211-219.Google Scholar
- Kwak, S.K. and J.H. Kim, Statistical data preparation: management of missing values and outliers. Korean journal of anesthesiology, 2017. 70(4): p. 407-411.Google ScholarCross Ref
- . McHugh, M.L., Interrater reliability: the kappa statistic. Biochem Med (Zagreb), 2012. 22(3): p. 276-82.Google Scholar
- McCrum-Gardner, E., Which is the correct statistical test to use? British Journal of Oral and Maxillofacial Surgery, 2008. 46(1): p. 38-41.Google ScholarCross Ref
- . Stephen, D. and A. SAZ, Cochran's Q with pairwise McNemar for dichotomous multiple responses data: A practical approach. Int J Eng Technol, 2018. 7(3): p. 4-6.Google Scholar
- Watson, A.B. and J.I. Yellott, A unified formula for light-adapted pupil size. Journal of vision, 2012. 12(10): p. 12-12.Google ScholarCross Ref
- Nielsen, J. and T.K. Landauer, A mathematical model of the finding of usability problems, in Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems. 1993, Association for Computing Machinery: Amsterdam, The Netherlands. p. 206–213.Google ScholarDigital Library
- Dumas, J.S. and J.C. Redish, A Practical Guide to Usability Testing. 1999: Intellect Books.Google Scholar
Index Terms
- Usability of Chat and Forum Discussion Tools in Higher Education
Recommendations
Student and Instructor Satisfaction with E-learning Tools in Online Learning Environments
This study utilized Expectancy Confirmation Theory ECT and Technology Acceptance Model TAM to examine satisfaction of students and instructors toward online learning tools and resources in online learning environments. The study findings indicate that ...
Online formative assessment in higher education: A review of the literature
As online and blended learning has become common place educational strategy in higher education, educators need to reconceptualise fundamental issues of teaching, learning and assessment in non traditional spaces. These issues include concepts such as ...
Effects of Competitive E-Learning Tools on Higher Education Students: A Case Study
Over the last few years, most of the attempts to introduce active learning methodologies in the classroom have made use of Information and Communication Technology (ICT). Many of these efforts have been directed to collaborative scenarios used in remote,...
Comments