Sentiment analysis of students’ attitudes toward mobile learning activities

In this research, students’ sentiments and emotions embedded in their learning journals are analyzed to understand their attitudes to mobile-based lessons as they progress during an English as a Foreign Language (EFL) course. Sentiment Analysis (SA) was utilized to extract emotions and sentiment throughout students’ learning experience, as expressed in their weekly online learner journals. The sentiment scores were generated from four sentiment dictionaries with different scales. The findings suggest that overall, the students had a positive sentiment and emotions toward mobile learning, consisting of anticipation, trust, joy, and surprise. The strongest negative emotion was fear, which may be explained by anxiety surrounding communication in a foreign language.


Introduction
Emotional obstacles impede learning (Pekrun, Goetz, Titz, & Perry, 2002;Zeidner, 2014), as students typically learn and perform better when they experience positive sentiments about a subject and learning context. In this research, students' sentiments and emotions embedded in their learning journals are analyzed to understand their attitudes to mobile-based EFL. SA was utilized to extract emotions and sentiment during the course, as expressed in students' weekly online learner journals. Sentiment scores were generated from four sentiment dictionaries with different scales, including syuzhet, bing, afinn, and nrc. The findings suggest that overall, the students had positive sentiment and emotions toward mobile learning as defined by these terms: anticipation, trust, joy, and surprise. The strongest negative emotion was fear, which may be explained by anxiety surrounding communication in a foreign language. SA has been effective for evaluating mobile pedagogical affordances (Bano, Zowghi, & Kearney, 2017). However, the focus has been on consumer sentiment toward mobile social networks, with little coverage of mobile learning environments (Abdulsalami et al., 2017;Hew, Hu, Qiao, & Tang, 2020;Martin, Ortigosa, & Carro, 2012;Rani & Kumar, 2017).

Method
The study design was a case study adopted for one academic year to gain a deeper understanding of the outcomes of completing collaborative learning activities through mobile devices by four Japanese university undergraduate EFL classes on translation. Each class formed one case study group with between five and eight members. All factors remained constant across the groups, and participation in the study was voluntary. The groups were comprise of: • Group 1: five female and two male students; • Group 2: eight female students; • Group 3: six female students; and • Group 4: six female students.
The data collection for each group was identical and consisted of each student submitting an open-ended e-journal at the end of each week in the students' L1, Japanese, with comments on any use of mobile devices for homework activities. These e-journals were then translated into English by the researcher before the analysis. While translation is always a limitation in research, there were no significantly complex word meanings that could strongly influence the result. However, this is a topic that may be discussed further in an extended version of this paper. The SA was performed using the R package syuzhet (Manning et al., 2014) for sentiment scores and emotion classification. All other text mining was done through the tm R package (Feinerer & Hornik, 2019). Sentiments are classified as positive, neutral, or negative; and numerical. The syuzhet package was used for generating sentiment scores and has four sentiment dictionaries with different scales, including syuzhet, bing, afinn, and nrc. As explained in Mhatre (2020, n.p.), sentiment scores using the syuzhet take the form of a decimal range from -1 (most negative) to +1(most positive), bing is a binary scale with -1 indicating negative and +1 indicating positive sentiment, and similarly, afinn is an integer scale ranging from -5 to +5. However, the nrc emotion lexicon (Mohammad & Turney, 2013) is a list of English words and their associations with eight basic emotions (anger, fear, anticipation, trust, surprise, sadness, joy, and disgust) and two sentiments (negative and positive) manually annotated through crowdsourcing.

Results and discussion
The e-journals' data set of 3,800 English words was a word frequency table.
The next two frequent words of interest are 'phone' and 'mobile', and they may refer to the same object -mobile phone. The word root most associated with both words is 'busi', at 74% for 'phone' and 63% for 'mobil', suggesting that they associate this device to some extent as a tool. The word 'phone' correlates with no negative words, but the positive word roots are 'happi' (67%), 'advantage' (57%), 'comfort' (57%), 'thank' (49%), 'great' (34%), 'nice' (33%), and 'good' (28%). Also, the correlation of 'street' (57%), 'technolog' (57%), and 'walk' (56%) may be explained by the requirement that students collect examples during their everyday lives outside of school to complete the homework activities. Likewise, 'mobil' is correlated with 'happi' (54%), 'amaz' (36%), 'conv' (convenient) (36%), 'bett' (36%), 'benefit' (36%), 'comfort' (36%), 'nice' (29%), and the negative 'negat' (36%). These results strongly indicate that the students have a generally positive view of their mobile phone in these homework activities. Table 1 includes the syuzhet package-generated sentiment scores using the four sentiment dictionaries, and the normalized scores used for comparison. The nrc function, in line with Mhatre (2020), "returns a data frame with each row representing a sentence [and] ten columns, one for each of the eight emotions and one column for positive sentiment valence and one for negative sentiment valence" (n.p.). Figure 1 shows the number of instances of words in the text associated with these eight emotions. Positive words associated with 'anticipation' occur 65 times and may be explained by the students' comments that they had never done this type of learning activity using mobile phones. The next positive words relate to 'trust' (40), then 'joy' (18), and 'surprise' (11). The strongest negative emotion words relate to 'fear' (38), which may be explained by their general fear of communicating online in their second language, as expressed in their journals. This is followed by the negative emotion of 'sadness' (9), 'anger' (6), and 'disgust' (4). Figure 2 compares the number of emotion words as a percentage of the total of meaningful words. The positive emotion 'anticipation' accounts for over 30% of all meaningful words in the e-journals. Moreover, 'trust' is second, at over 20% of meaningful words. They are followed by the negative emotion 'fear', also at 20%. Overall, the words associated with positive emotions that the software considers meaningful account for over 65% of total words.

Conclusions
The findings suggest that students in this study held positive sentiment on the use of mobile devices for collaborative activities, as shown by the syuzhet, bing, and afinn results. The nrc result indicates that these sentiments can be best described as 'anticipation', 'trust', 'joy', and 'surprise'. At the same time, they had the emotion of 'fear' for the activities, which could be related to the anxiety of communicating in a foreign language. However, this requires more research to clarify the possible connection.