Using Augmented Reality to Support Collaborative Problem Solving in Computer Debugging Practice

Augmented reality (AR) is a visualization technology allowing virtual information to be imposed on physical objects. The contiguity of two-channel representations can enhance understanding of information by reducing extraneous cognitive load resulted by aligning information from different sources. For example, in chemistry, AR can be used to visualize the reaction of catalysts and provides relevant information like molecules and formulas during a lab experiment in real time. AR also makes user interaction with the content more natural compared to traditional computer-based interfaces, e.g., a user can easily share the content with others by pointing a new device to the same target. There has been much research exploring how these AR affordances can support learning of abstract concepts, invisible or unobservable phenomena, and improve interaction. However, the use of AR is less explored in learning computer programming which actually shares similar challenges: the integration of programming logic and the produced outcome is invisible and the interaction between students is usually limited due to the restricted access to computer keyboards and mice. To fill this gap, this study is aimed to explore how AR can support programming problem solving and group collaboration by multimodal analytics including program snapshots and group conversation. I developed a mobile app, Ogmented, which is able to present collaborative programming tasks by AR visualization and meanwhile collect student's conversation during the tasks for multimodal behavior analysis. The result of a pilot study provides an insight into student's reaction on Ogmented and provides a path to further investigation.