Next Article in Journal
Future Mathematics Teachers’ Perceptions towards Inclusion in Secondary Education: University of Granada
Next Article in Special Issue
Leading Edge Use of Technology for Teacher Professional Development in Indian Schools
Previous Article in Journal
Primary School Teachers’ Perspectives on Emergency Remote Teaching of Mathematics: Challenges and Opportunities for the Post-COVID-19 Era
Previous Article in Special Issue
University Students’ Experiences and Reflections of Technology in Their Transition to Online Learning during the Global Pandemic
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

WhatsApp Use in a Higher Education Learning Environment: Perspective of Students of a Malaysian Private University on Academic Performance and Team Effectiveness

by
Cheng Ean (Catherine) Lee
*,
Huei Huei Chern
and
Dzafran Adris Azmir
Department of Communication, School of Arts, Sunway University, Subang Jaya 47500, Selangor, Malaysia
*
Author to whom correspondence should be addressed.
Educ. Sci. 2023, 13(3), 244; https://doi.org/10.3390/educsci13030244
Submission received: 14 December 2022 / Revised: 8 February 2023 / Accepted: 8 February 2023 / Published: 25 February 2023
(This article belongs to the Special Issue Leading Edge Technologies Ensuring Education)

Abstract

:
The mobile instant messaging application, WhatsApp Messenger (WhatsApp), has become a popular form of communication among adolescents, especially university students, and it has increasingly been used as a tool in collaborative learning in higher education. The use of WhatsApp for education to facilitate ubiquitous learning has been practised worldwide due to its popularity and potential to support teaching and learning processes derived from the diffusion of mobile technology and empowered by the use of smartphones. This study investigates the impact of the use of WhatsApp in a higher education learning environment on students’ perceived academic performance and team effectiveness. A convergent parallel mixed-methods research design was adopted with data collected through a self-administered online survey and two focus group interviews with students of a private university in the Sunway City, Malaysia. The findings of this study present insights into the popularity of WhatsApp among university students and that students use it for social and educational purposes due to its perceived ease of use and usefulness in enhancing academic performance and team effectiveness. Although WhatsApp is recognised as a rich and powerful collaborative tool for students with a positive impact on academic performance, it has a limited impact on the cohesion and openness of team effectiveness.

1. Introduction

The evolution of digital technologies with renowned technological innovations such as computers, laptops, tablets, smartphones and the Internet have impacted our daily lives. The world Internet usage and population statistics show that 69% of the total world population are Internet users with 2917 million users in the Asia region in June 2022 [1]. The digital technologies and the Internet are used by adolescents as a form of communication, information exchange and to support education for teaching and learning processes.
In this digital age, among young people, social media such as Facebook, Instagram, TikTok and WhatsApp are becoming more prevalent and gradually changing the way they interact and socialise with others. WhatsApp is one of the most popular applications used by young people as a way of communicating with friends, family and strangers, and it enables users to have multimedia downloads and text messaging without having to pay [2]. WhatsApp has grown in popularity among adolescents because of its distinctive features of easy communication via text or voice messages, pictures, videos and video calls between two persons or groups, which makes communication easier, faster, cheaper and more enjoyable [3]. Currently, WhatsApp has 2 billion users over 180 countries as a free messaging and video calling application that works across mobiles and desktops on an Internet connection with no subscription fees [4].
As well as the use of WhatsApp for personal and social communication, scholars have rapidly adopted it as a great tool for pedagogy to facilitate ubiquitous learning and that it has potential for supporting teaching and learning processes. [5] argued that even though WhatsApp is a relatively new social media application, higher education institutions have adopted it for communicating upcoming events and other campus news to students; it facilitates synchronous and asynchronous communication to enhance closer social interactivity, as well as support learning approaches in enhancing creativity, critical thinking, collaborative skills and critical reflection. In addition, there are evidences that propose the utilisation of mobile technologies such as Facebook and WhatsApp plays an effective role in influencing the academic performance and habits of students in the university [6,7,8,9].
In its usage to support education, WhatsApp provides affordances that allow interactive collaboration, knowledge sharing and effective communication among student teams that contribute to building a sense of community in higher education. Students engage in teams and/or learning communities in higher education such as student support groups, assignment groups and teams in clubs or societies. According to [10], learning communities are “a learning atmosphere, a context providing a supportive system from which sustainable learning processes are gained through a dialogue and collaborative construction of knowledge by acquiring, generating, analysing and structuring information” (p. 273). The phenomenon of using mobile technologies such as WhatsApp for communication among students and team members in higher education contexts enables students to learn and interact in real time through the exchange of text messages, images, videos and voice notes [11].
A recent systematic review on the educational use of WhatsApp shows the advantages and disadvantages or difficulties in the educational use of WhatsApp [12]. The review concluded that although WhatsApp was not designed for the educational environment, its affordances and advantages can be shown to be “an adequate educational and support resource in teaching and learning at all educational stages… acknowledged better learning results by students who use the app… use the app for extracurricular purposes… promoting group, cooperative, or collaborative work” (p. 9). A mixed result in the empirical literature of the usage, advantages and disadvantages or difficulties in the educational use of WhatsApp shows inconclusive evidence on its uses and impacts on students’ academic performance and team effectiveness. In addition, students are central in all teaching and learning activities and when the paradigm shift from the post-COVID-19 pandemic “caused teachers and students to undergo many unplanned and unprecedented changes in terms of teaching and learning” [13] (p. 1690), gaining student perception was pertinent. Thus, this study aims to examine the perception of students of a private university in Malaysia on their uses and the impact of WhatsApp on academic performance and team effectiveness.

2. Literature Review

The WhatsApp Messenger (WhatsApp) was established by Jan Koum and Brian Acton in 2009 [14] and supports forms of different communication styles, from voice calls, video calls and unlimited text messaging, to the sharing of videos, images, audio recordings and documents [4]. WhatsApp receives overwhelming responses from users all around the globe which has revolutionised social information exchanges, and it has been ranked as the most popular application across 109 countries, representing 58% of the 187 involved in a survey conducted by Business Insider [15]. The literature shows the trend of using WhatsApp as an educational tool in ensuring education, teaching and learning processes and collaborative learning, as well as for interaction between students and lecturers, and among students.
WhatsApp is a universal form of communication tool for younger generations [16] to interact and communicate with their families, friends or colleagues at work. [17] revealed that university students use WhatsApp for four key reasons: to communicate with friends and families, to encourage the discussion of thoughts and ideas among peers, to initiate conversations with acquaintances in a more comfortable environment and to nurture and cultivate their social atmosphere. In addition to social communication, WhatsApp shapes individualised learning among students, allowing them to learn at their own pace [18]. An effective use of WhatsApp cultivates collaboration among peers, boosts their sense of community and allows valuable knowledge sharing in an educational context [19]. WhatsApp improves the talents and learning experiences of students, develops students’ self-identities and allows them to express their thoughts freely [20].
With the increase in the ownership of smartphones by university students and the ease of use and affordability of WhatsApp in the higher education learning environment, many studies recognise that WhatsApp is of “potential for ubiquitous learning, and the students have a good attitude to this app as a teaching and learning platform” [12] (p. 8). Results from 192 papers obtained from the Google Scholar database show the educational use of WhatsApp resulted in: (1) enhanced students’ academic performance, (2) an improvement in efficiency, time planning and organisation, active learning, decision making and motivation, (3) high levels of satisfaction among students and teachers who used the application and (4) its suitability as an educational and support resource in teaching and learning processes at all educational stages [12].
The technological and pedagogical affordances of WhatsApp enable users to perform desired tasks such as sending text, image and voice messages, making or answering voice calls, as well as performing teaching and learning activities such as further enhancing interaction, collaboration and learning [5]. The usage of WhatsApp does impact students’ academic performance depending on the degree of the application use [14]. Studies have shown the benefits and advantages of university students’ usage of WhatsApp in learning processes, for instance, WhatsApp eases communication through its variety of features such as sharing of documents, audio recordings, videos and images, which is deemed beneficial for information and knowledge exchange among students [17]. In addition, WhatsApp enables students to create a group chat with up to 512 users in a single group for discussion purposes [21]. One researcher has an optimistic viewpoint with regards to WhatsApp in that it allows the easy transformation of ideas and thoughts into words onto the screens of smartphones, hence developing a new generation of young writers who have the tendency to write more as compared to any other generation, which in return help to improve and enhance students’ academic performance [22].
Conversely, there are also disadvantages and drawbacks of WhatsApp usage for students’ learning processes. Some researchers claimed that the use of WhatsApp in education brings in more negative than positive impacts [14]; for example, the fact that 21st century students scrolling and communicating through their smartphones using WhatsApp is slowly becoming a common phenomenon of addiction [23]; and the noneducational, inappropriate and unethical use of smartphone applications leading to an undesirable habit that might eventually affect their academic performance negatively [24]. In addition, adverse impacts such as lack of concentration, poor time management and procrastination were found in cases of students’ use of WhatsApp for leisure rather than for academic purposes [16]. Researchers have highlighted that WhatsApp may possibly impose a negative impact on students’ academic writing with the use of informal Internet English being used as a part of the English Language class as students face difficulties in overlooking punctuation and vowels and suffer from distinguishing formal and informal writing [25]. A mixed-methods study by [26] on the use of WhatsApp by business students of a public university in Malaysia concluded that WhatsApp Messenger usage had a negative impact on the academic performance of university students because the incoming messages from WhatsApp took up most of their concentration span and study time, which further led to the promotion of the act of procrastination and lack of concentration during lectures, along with the interference of students’ perception of formal and informal English writing [26]. Therefore, the presence of evidence that supports the fact that WhatsApp usage has a great effect on student academic performance remains contentious. This necessitates the examination of the impact of WhatsApp usage on students’ perceived academic performance in the Malaysian higher education learning context.
As well as its impact on academic performance, the group-based feature of WhatsApp postulates the collaborative characteristics of the application, which enable students to create groups to exchange messages and facilitate group communication [11] that warrant its incorporation in student teams within clubs and societies in higher education. A study discovered that university students view WhatsApp as an important tool to facilitate interactive collaboration and socialisation within student teams [27]. A team is a group of individuals who interact interdependently and come together voluntarily to achieve a certain outcome or accomplish particular tasks [28]. There are three types of teams: face-to-face, virtual and hybrid. Hybrid teams are an amalgam of face-to-face and virtual team designs and communicate face-to-face and virtually, depending upon the needs and convenience of team members [29].
Researchers stress the lack of quantity and depth in studies regarding hybrid teams describing their characteristics, and the performance of hybrid teams remains unclear [28,29]. Hybrid teams offer an interesting case to study due to their novelty, adaptability to incorporate communication technologies to facilitate their collaborative function [28]. University students may engage in hybrid teams either in temporary ad hoc teams such as assignment groups or, saliently, in more permanent long-standing teams such as clubs and societies. Despite the merits of WhatsApp discovered in previous studies in terms of it engendering collaborative learning and team work, the uses and impacts of WhatsApp in the context of hybrid team collaboration and team effectiveness have not been explored in the Malaysian context.
Team cohesion and team openness are two elements that may affect the team effectiveness of student hybrid teams. [30] investigated the moderating role of experience with instant messaging on team cohesion, team openness and team effectiveness through data obtained from 365 virtual team members at a southern university in America through a quantitative survey. The findings were manifold: team cohesion plays a positive and significant role on team effectiveness in a virtual setting; team openness is enhanced by the respondent’s experience with instant messaging such as a team member utilising openness to facilitate higher effectiveness; and the role of instant messaging allows leaders to more effectively build virtual teams and provide effective training and support. [30] concluded that cohesive virtual teams enable team members to share knowledge via instant messaging that allow them to be more effective in the given task, and team effectiveness was higher if team members were willing to communicate in a more open manner via instant messaging. Two past studies indicated that this lack of cohesion shows that team members are less receptive to interpersonal issues and the emotions of others, inhibiting the team’s emotional intelligence [31,32]. Members who are less participative in collaboration may not perform well in a team through the usage of WhatsApp [32], thus, drawing from the literature on WhatsApp and team cohesion, this study aims to explore the use of WhatsApp in impacting team cohesion in the Malaysian higher education learning environment.
The usability and positive effects of WhatsApp in education as a valuable educational and collaborative tool, coupled with being the most popular application for communication and teaching and learning among higher education students in the Middle East and Asia, particularly Malaysia, who are a trendsetter in WhatsApp studies [11]) means that the interest of university students and their understanding of how WhatsApp can be adopted for learning and team effectiveness remains a domain worth examining. This study provides insights on the investigation into how students perceive the uses and impacts of WhatsApp on academic performance and team effectiveness that will benefit students, lecturers and university administrators.

3. Research Questions

In line with the aim of this study, the two research questions are:
  • What is the perception of students of a private university in Malaysia on their uses and impact of WhatsApp on academic performance?
  • How does the use of WhatsApp impact the effectiveness of student hybrid teams?

4. Methodology

4.1. Research Approach

This study is carried out using the convergent parallel mixed-methods research design with data collected through a self-administered online survey (quantitative research) and two focus group interviews (qualitative research) with students of a private university in the Sunway City, Malaysia. A convergent parallel design entails that the researcher concurrently conducts the quantitative and qualitative elements in the same phase of the research process, weights the methods equally, analyses the two components independently and interprets the results together [33]. This enables researchers “to triangulate the methods by directly comparing the quantitative statistical results and qualitative findings… for the purpose of corroboration and validation” [34] (p. 123). In this study, as shown in Figure 1, two datasets derived from an online survey and two focus group interviews were obtained and analysed separately.
The quantitative approach provides an emphasis on numerical or statistical analysis of data collected through surveys, polls and questionnaires [35] and uses a structure-based research to further observe the aspect of the attitudes, opinions and behaviours of the respondents [36]. An online survey questionnaire was used to examine the uses and impact of WhatsApp on university students’ academic performance. The online survey questionnaire is a quantitative data collection method, particularly used to comprehend and evaluate the perceptions and viewpoints of all respondents of the study [37].
The qualitative approach emphasises the study of the whole rather than variables, to identify patterns or common themes from the collected data [38]. In the qualitative stage, focus group interviews were used. Focus group interviews offer the benefit of shared insight on a topic with large amounts of information gained in a short period of time with a multiplicity of views that are also influenced by the dynamics of the group [39]. The participants of the focus group interview were recruited using purposive sampling, in which the participants were students of a private university at Sunway City, Malaysia, and participated in hybrid teams of clubs or societies over six months to a year or more at the time of research. The interview comprised of ten main questions related to WhatsApp usage in student teams, opinions on team cohesion and team openness, as well as impact of WhatsApp on team effectiveness.

4.2. Data Collection Procedure

The survey questionnaire consisted of five close-ended questions adapted from the study by [26] on frequency of WhatsApp usage and time spent on WhatsApp per day and from [18] and [40] on frequency of time spent for academic–related activities through WhatsApp, types of academic–related activities using WhatsApp and student perceptions towards the impact of WhatsApp usage on academic performance.
University ethics approval was obtained before the study was implemented. Prior to collecting the quantitative data, the potential respondents were given the consent form to inform them about the aim of the research, confidentiality of the data and ethical issues. Before responding to the survey, the respondents made it clear that their participation was completely voluntary and anonymous, with findings aggregated. The study involved university students of a private university in Malaysia through the convenience sampling method. Convenience sampling is a nonprobability sampling technique depending on data collection from respondents who are conveniently available to participate in the online survey questionnaire; however, it is highly vulnerable to bias to the extent that is beyond the control of the researcher [41]. The online survey was distributed through Google Forms using a social media platform such as Facebook. After the data were collected, descriptive statistics were performed.
A total of 12 participants (four males, eight females) participated in the focus group interviews with two groups of six participants from two clubs: an international student organisation that focuses on leadership and employment development (Group A), and a student organisation dedicated to the development of leadership and skills among scholars within the university (Group B). The participants were informed about the research through the participant information sheet and they voluntarily participated. To keep the participants’ identities confidential, each participant is represented with P1 to P12.

5. Findings and Discussion

5.1. Quantitative Research Findings

The link to the online survey was distributed to the students of a private university in Sunway City Malaysia through Facebook. A total of 126 responses were received and recorded on Google Forms and the usable responses were 99 (78.6%). The quantitative data were analysed using IBM Statistical Package for the Social Sciences (SPSS) with the standard statistical descriptive analysis of frequencies, percentages and means. Each survey item was calculated to provide information about the frequency of WhatsApp usage, time spent on WhatsApp per day and for academic-related activities per week, uses of WhatsApp for academic-related activities, as well as the perception of students towards WhatsApp usage and its impact on academic performance. The following sections present the findings of the quantitative research.
To address the research question on university students’ uses of WhatsApp, two survey items were analysed using frequency and percentage. Table 1 shows that almost all of the respondents use WhatsApp several times a day (97.0%) and only three respondents use WhatsApp once a day. This finding indicates that students of this study are frequent users of WhatsApp.
In terms of time spent on WhatsApp, Table 2 shows that 42.4% of the respondents spent 3–5 h per day, followed by 34.3% of respondents who spent 1–2 h per day and 19.2% of respondents who spent over 8 h per day. This finding indicates that the students are considered heavy users of WhatsApp.
Lastly, three questions from the survey were analysed using frequency, percentage and mean about the time spent on WhatsApp for academic–related activities per week (Table 3), types of academic–related activities conducted by students using WhatsApp (Table 4) and students’ perception towards WhatsApp usage and its impact on academic performance (Table 5).
Table 3 shows the time spent on WhatsApp for academic–related activities per week by university students. The majority of respondents (71.7%) spent less than 5 h per week on WhatsApp for academic–related activities, followed by 23.2% of respondents who spent 5 to 10 h per week, four respondents (4%) spent more than 20 h per week and only one respondent (1%) spent 15 to 20 h per week on WhatsApp for academic–related activities. This finding indicates the students are willing to invest about 20% of their time throughout a week on WhatsApp for academic purposes.
Table 4 displays the academic–related activities conducted by university students on WhatsApp. The findings show that 33 respondents (33.3%) used WhatsApp for group discussion among peers, followed by 30 respondents who used WhatsApp to share resources (30.3%). The other two activities less frequently used with WhatsApp were informing an educational agenda (20.2%) and conducting group studies (16.2%). This indicates that students in this study used WhatsApp for group discussion, sharing resources and an educational agenda, and conducting group studies as part of academic-related activities.
Finally, in examining the perceptions of students of a private university in Malaysia on the uses and impact of WhatsApp on their academic performance, Table 5 displays the descriptive statistics of means and standard deviation (SD). The findings revealed that the respondents agreed on six statements: that WhatsApp helped pass any information discussed easily (M = 4.34, SD = 0.70), allowed them to academically engage with peers and lecturers (M = 4.11, SD = 0.84), was a good online platform to facilitate group projects (M = 4.03, SD = 0.90) and favoured faster knowledge sharing (M = 4.03, SD = 0.75), as well as their attitude towards their interaction using WhatsApp messaging being understandable (M = 4.02, SD = 0.75) and clear (M = 4.00, SD = 0.82).
Nevertheless, the findings also reveal the respondents’ stance of moderate agreement that the usage of WhatsApp destroys spelling and the grammatical construction of sentences (M = 3.62, SD = 1.20) and leads to a lack of concentration during lectures (M = 3.55, SD = 1.00).
The remaining findings show the students’ slight disagreement on the negative impact of WhatsApp on academic performance. The respondents have a neutral to slight disagreement that WhatsApp distracts them from adhering to their private study timetable (M = 2.37, SD = 1.10), completing their assignments (M = 2.32, SD = 1.02), results in difficulty in balancing online activities (M = 2.27, SD = 1.06) and they procrastinate their academic work due to WhatsApp (M = 2.26, SD = 1.01). This indicates that the students are in favour of using WhatsApp for academic purposes because of its features and it does not negatively affect their academic performance.

5.2. Quantitative Research Discussion

The results from the quantitative survey revealed that 97% of the students are heavy users of WhatsApp as they log on to WhatsApp several times per day for a span of 3–5 h for social and academic purposes. This finding is consistent with [14] that revealed WhatsApp as one of the most popular mobile messaging applications frequently used by university students because it is convenient and easy to use. When the respondents were asked about the impact of WhatsApp on students’ academic performance, 71% of the respondents use WhatsApp less than 5 h a week for academic–related activities such as group discussions (33%), resources sharing (30%), informing an educational agenda (20%) and for group studies (16%). This finding echoes the study by [17], which claimed that WhatsApp and its unique features were beneficial when it was used for the exchanging of information and knowledge among students. Specifically, the finding of this study on the habit of students in a Malaysian university using WhatsApp for learning purposes was consistent with prior studies conducted in the Malaysian context [8,9,18].
However, the findings of this study differ from the study by [16], which claimed adverse impacts of the use of WhatsApp such as poor time management, procrastination and a lack of concentration because, in their study, the majority of the students used WhatsApp for leisure purposes instead of academic purposes. In Table 5, the mean scores show that the respondents of this study have a neutral to disagreeing response in terms of WhatsApp distracting them from adhering to their private study timetable (M = 2.37, SD = 1.10) and completing their assignments (M = 2.32, SD = 1.02), and that it results in difficulty in balancing online activities (M = 2.27, SD = 1.06) and procrastination about their academic work due to WhatsApp (M = 2.26, SD = 1.01). This can be interpreted that the students of a private university at Sunway City, Malaysia, have high self-discipline and they are able to manage their time well in using WhatsApp for academic-related activities, which brings a positive impact to their academic performances.

5.3. Qualitative Research Findings and Discussion

The qualitative data collected from two focus group sessions were audio and video recorded with a smartphone and video camera. The two sessions conducted with 12 participants (six participants in each group) took 61 and 67 min, respectively, in a meeting room at the private university campus. The focus group interviews were transcribed and manually analysed through three phases of a thematic analysis method. During the first phase, answers for each question were summarised into codes and themes. Next, participants’ responses were analysed and compared, and finally, the thematic analysis method ends with the compilation of themes that answer the research question on the uses and impact of WhatsApp on the effectiveness of student hybrid teams.
Table 6 displays the demographic profiles of the focus group participants. The participants comprised of four males and eight females, and they were adequately homogenous in their studied programme and their year and semester of study. All six participants of Group A and two participants of Group B possess leadership positions and this opened the opportunities to explore the differences in the usage of WhatsApp between leaders and team members.
Firstly, the three questions asked of the focus group participants were their frequency of WhatsApp usage, purposes of usage and the features of WhatsApp used. Participants said that WhatsApp is used daily in both teams for collaborating; however, the frequency varies according to the team’s needs and norms. Participants from Group A were found to use it to collaborate more consistently as P4 asserted: “When it comes to operations, we communicate with WhatsApp daily, like every day”. P6 said that due to their weekly meetings, it is a necessity to use WhatsApp more frequently. This consistency of using WhatsApp in a frequent manner may be explained by how the club operates—it is guided by professional principles to perform consistently when the members communicate. For Group B, the members echoed P4 and P6 on the frequent use of WhatsApp in their team communication. P8 said that they use WhatsApp “every hour, every week”. This finding illustrates the pervasiveness of WhatsApp among students for collaboration within teams. The frequency of WhatsApp usage is affected by the workload of the members or if they are engaged in a project. For example, both P6 and P10 saw an increase in frequency when their teams were engaged in projects.
In terms of the purpose of usage, WhatsApp is used by both groups for five reasons: (1) ubiquity of usage, (2) accessibility to communicate and collaborate, (3) efficiency in collaboration, (4) inclusivity in collaboration and (5) communicating matters of low complexity. P3, P8 and P11 support the availability of WhatsApp as a factor to use in their clubs because “everyone has WhatsApp” (P8) and “It’s the most common messaging app” (P11). This finding corresponds with the strong presence of WhatsApp as the most popular application [15]. The next reason for WhatsApp usage is the accessibility, which allows team members to communicate and collaborate with each other. According to P3, WhatsApp is available for use at most times and with little difficulty stating, “it’s very accessible to everybody and it’s quick”. P10 claimed that WhatsApp is accessible as it creates an opportunity to collaborate with others when they are geographically dispersed, relating to the principal nature and needs of how geographically dispersed teams rely upon computer-mediated communication to collaborate [28].
The efficiency in collaboration referred to by members of Groups A and B denotes WhatsApp’s ability to streamline collaborative processes and minimise the time required for collaboration through rapid feedback. P4 believes that WhatsApp streamlines how team members collaborate by removing the necessity to use other systems or means to communicate information. Both P4 and P6 stated that the application reduces the time needed to collaborate by allowing team members to receive quick responses and feedback on conversations, decisions and consensus and providing an effective alternative to slower and less utilised channels. The use of WhatsApp due to the rapidity of feedback is preferred by P2 as she made a comparison to the inadequacies of how email as an asynchronous channel requires a period of time to reply and that team members do not check their email often, to create the optimal response time for collaboration that WhatsApp could. In contrast, P11 did not support rapid feedback as a reason for WhatsApp usage due to how the message response time is greatly contingent on how rapidly team members themselves can reply, nullifying rapid feedback for efficiency in collaboration.
The fourth reason for WhatsApp’s use in teams is that it fosters inclusive and transparent collaboration. P2 views that WhatsApp is able to effectively keep members in the loop and promote shared understanding and transparency. According to P6, WhatsApp can also transcend between teams through a larger encompassing group and eases collaboration in ways that would be otherwise difficult or inconvenient, stating “team members would know what the other member is doing, we can collaborate easier”. Finally, participants of Groups A and B reasoned that WhatsApp is used because they require it to collaborate and communicate matters of low complexity or ambiguity, where WhatsApp is the most convenient means to do so. Fulfilling tasks of low complexity or ambiguity do not require face-to-face interaction, which would be inconvenient for hybrid teams. According to P5 and P7, tasks of lower magnitudes of importance or complexity such as task delegation or updating team members produce little interpretation in meanings; therefore, WhatsApp is considered an appropriate option to be used.
The participants identified four features of WhatsApp that were considered most salient for teams’ collaboration, which are: group chats, emojis, read receipts and quote message/reply. Group chats are used in most of the communication among team members for distributing information and gathering opinions. P7 claimed that information can be efficiently distributed to all members through group chats, which allows Group B to gather opinions from its members through their responses posted in the group chat, and P4 expressed that multiple group chats are made within the club based on different portfolios or position. Emojis are another feature frequently used by both group members to enrich text-based communication on WhatsApp by adding or creating the tone of the conversation, to facilitate language deficiencies and support relationship building among members during collaboration. Consequently, P3 stated “emojis underscore the layered tone”, which allows other team members to have a better understanding of the tone of his messages, while P8 asserted emojis are used in order to express better. Notably, for team members who have a poorer command of language to articulate meanings or inability to extensively articulate feelings through words, emojis are used to easily convey those feelings.
Two other features that are frequently used by the participants are read receipts and quote message/reply functions. Read receipts are markers that indicate the state of a message’s delivery and reception [42]. This feature is used by both teams’ members to enforce accountability and responsiveness among members by keeping track of the read receipt status on their messages. The President of Group A (P1) reasoned: “when we send texts to anyone, we expect replies from them so we constantly focus on the read receipts”. P9 further stated that members can gain insight on the precise timing on when messages are delivered and how long it takes for recipients to reply, which creates accountability. In the process of team collaboration, P11 believed that its use allows members to reprimand others who are not timely or are unhappy with messages. Lastly, message quoting in a reply is effectively used to respond to specific messages with precision and to reduce message spamming in group chats. P3 posited that this feature of message/reply quoting reduces spamming in group chats as quoted messages abridge the original referred message, which occupies less space in a chat screen. Responding to messages with clarity and specificity is important to reduce confusion in group chats by minimising the clutter of lengthy and repeated messages, as according to P11:
“especially in a group chat, once people start replying nonstop, you see a whole bunch of messages keep going and suddenly out of a sudden, at the end of the day you open it, it’s the only time you’re free to read them, you’ll see a flood of messages… this feature allows you just to reply to that message so that people know what you are replying to, know what context you’re answering”.
In answering the remaining questions about participants’ opinions on team cohesion, team openness and the impact of WhatsApp usage on team effectiveness, three participants from Group A (P3, P4, P5) and one participant from Group B (P10) believe that their association with the team is harmed due to WhatsApp or they do not necessarily feel that they are a part of their team. Specifically, P4 stated that team members disassociate themselves from Group A when using WhatsApp due to the overwhelming volume of messages that they are faced with in the group chat; P5 further mentioned they do not feel a part of the team because there is a lack of obligation for members to communicate online due to the lack of a physical presence that produces the necessary pressure to interact; P3 expressed his feelings of disassociation from the team as he found members to be more irritating on WhatsApp than face-to-face stating “I feel withdrawn sometimes and disengaged… there are spammers, sharers, people talk a lot and people who keep sharing pictures… sometimes it gets annoying”; and P10 claimed feeling distanced from the team when conversations in groups do not apply to her as discussions within these group chats are pushed to all members. Overall, the participants of Group A felt more strongly than Group B on the ill effects of WhatsApp on team cohesion. The lack of cohesion means that team members are less likely to share team knowledge that allows them to be more effective in a particular task, thus limiting the ability of members to benefit the team’s performance [30,32].
The affordances of WhatsApp enable users to perform desired communication tasks to further enhance interaction, collaboration and learning [5] and create groups to exchange messages and facilitate group communication [11] within student hybrid teams. The participants of both groups believe that all members are encouraged to openly express their opinions using WhatsApp and being involved in decision making improves the team’s effectiveness due to the willingness of team members to share information and be more predisposed to learn and see other members’ views. P3 concurred that it is easier for members to express their opinions to each other without regards to their positions in the club through WhatsApp, while P10 believed that her team culture on using WhatsApp has naturally caused members to express themselves openly, thus, increasing team effectiveness.
Finally, the participants of the focus group interviews were asked to comment on the impact of WhatsApp usage on team effectiveness. In general, the participants of both groups believed that WhatsApp greatly contributes to the effectiveness of the teams. P1, P2, P3 and P4 of Group A viewed WhatsApp as having a positive impact on their team effectiveness by streamlining team operations into a single platform; increasing personal convenience through its accessibility of usage; maintaining relationships among team members when they are separated in time and distance; and aiding team planning. Similarly, P9 and P10 of Group B stated that WhatsApp positively impacts their team effectiveness because it promotes collaboration among team members by easing access to other collaborative systems such as Google Drive and centralises communication into one place. However, there were some criticisms of the use of WhatsApp in which P4 argued that teams find it difficult to separate socialisation from productivity, detracting from the communication activities’ collaborative purposes. This mixed usage of socialisation and productivity is explained by the repurposed nature of WhatsApp for collaboration [22]. In addition, P1 believed that communication in WhatsApp is damaging team effectiveness as messages may be untrue in their intents and emotions in terms of messages possibly misleading and manipulating team members’ understanding of the meanings.

6. Conclusions

This study investigated the usage of WhatsApp and its impact on academic performance and team effectiveness among students at a private university in Sunway City Malaysia. Through a convergent parallel mixed-methods research design with data collected through a self-administered online survey and two focus group interviews, it answered the two research questions: What is the perception of students of a private university in Malaysia on their uses and impact of WhatsApp on academic performance? and How does the use of WhatsApp impact the effectiveness of student hybrid teams?
It is important to focus on how and why young people such as university students use instant messaging or mobile technology and it is relevant to understand how these tools can be incorporated into teaching and learning processes and student teams. Specifically, the first research question was answered through the quantitative data analysis. The findings from the quantitative survey revealed that the university students reflected a positive and optimistic attitude to incorporating the use of WhatsApp for academic purposes due to its distinctive features and the agreement that WhatsApp brings a positive impact to their academic performance such as sharing information for discussion (M = 4.34) and being academically engaged with peers and lecturers (M = 4.11) as well as facilitating group projects and knowledge sharing (M = 4.03). It is evident that the students of this study spent 3–5 h on average per day using WhatsApp (42%) and less than 5 h per week for academic–related activities (71%). They were optimistic and positive about incorporating the use of WhatsApp for academic purposes, primarily due to the fact that they can use it anywhere, anytime, on all platforms and it supports an unlimited length of messages, audios, videos, images and documents in an individual chatroom or even a group chatroom, which can accommodate up to 512 members in one group. This further supports the trend of using WhatsApp as an educational tool in ensuring education, teaching and learning processes and collaborative learning, as well as for interaction between students and lecturers, and among students within hybrid student teams.
Data collected from the qualitative research brought insights on the best practices of using WhatsApp in student hybrid teams, addressing the gap of the literature in the usage of WhatsApp in student hybrid teams through team cohesion, team openness and team effectiveness. The qualitative data analysis answered the second research question of this study. The findings show the use of WhatsApp in hybrid teams due to its ubiquity, accessibility, efficiency, inclusivity and for communicating matters of lower complexity. The teams utilise group chats, emojis, read receipts and quote message/reply to facilitate communication more effectively. Nevertheless, communicating and collaborating through WhatsApp was not able to impact the cohesion among team members and, in turn, the usage negatively impacts the overall team effectiveness with participants having feelings of disassociation and being disengaged from the team as well as finding teams members to be more irritating on WhatsApp. The findings indicated that the student teams of this study perceive WhatsApp to be effective in general, despite inconsistencies in their responses regarding team cohesion and openness.
In a nutshell, both research questions were answered in which the responses of both the quantitative and qualitative research show a positive and optimistic attitude incorporating the use of WhatsApp for academic purposes and team effectiveness due to its distinctive features, and the agreement that WhatsApp brings a positive impact to academic performance and facilitates effective communication within student teams. The quantitative research findings corroborate the qualitative research findings on the heavy usage of WhatsApp among university students and its effectiveness in student–lecturer and student–student interactions and team communication. Yet, the qualitative data findings show inconsistencies regarding team cohesion and openness.
The study has two limitations. The study collected the data from self-reporting, namely, a self-administered online survey and focus group interviews on the uses and impacts of WhatsApp on academic performance and team effectiveness. However, the study can set the initial direction for further studies on assessing the impact of WhatsApp usage on students’ cumulative grade point average and team performance. The second limitation is that all the respondents were from a private university at Sunway City. Hence, any generalisation for students of the higher education learning environment in Malaysia requires similar studies in other universities including public, private and foreign university campuses. Nevertheless, the findings of this study are relevant and apply to the private university in the Klang Valley Malaysia. Despite the limitations, this study makes significant contributions to the body of knowledge as one of the early attempts to understand the uses and impacts of WhatsApp as an educational tool in ensuring education and within hybrid student teams in terms of academic performance and team effectiveness.

Author Contributions

Conceptualization, C.E.L., H.H.C. and D.A.A.; methodology, H.H.C. and D.A.A.; writing—original draft preparation, C.E.L.; writing—review and editing, C.E.L.; visualization, H.H.C. and D.A.A.; supervision, C.E.L. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Research ethics approval was obtained from Sunway University Malaysia before data collection.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy issue.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Internet World Stats. World Internet Users and 2022 Population Stats. 2022. Available online: http://www.internetworldstats.com/stats.htm (accessed on 5 December 2022).
  2. Bhatt, A.; Arshad, M. Impact of WhatsApp on youth: A Sociological Study. IRA–Int. J. Manag. Soc. Sci. 2016, 4, 376–386. [Google Scholar] [CrossRef]
  3. Gul, H.; Firat, S.; Sertcelik, M.; Gul, A.; Gurel, Y.; Kilic, B.G. Problematic WhatsApp use among adolescents: Linking fear of missing out and psychiatric symptoms. Psychiatry Behav. Sci. 2021, 11, 96–103. [Google Scholar] [CrossRef]
  4. About WhatsApp. Available online: https://www.whatsapp.com (accessed on 5 September 2022).
  5. Yeboah, D.; Nyagorme, P. Students’ acceptance of WhatsApp as teaching and learning tool in distance higher education in sub-Sharan Africa. Cogent. Educ. 2022, 9, 2077045. [Google Scholar] [CrossRef]
  6. Aiyende, O.; Omojola, O. Influence of social media on the academic performance of students: A study of University of Port Harcourt, Nigeria. Samaru J. Inf. Stud. 2021, 21, 13–23. [Google Scholar]
  7. Dror, I.E. Technology enhanced learning: The good, the bad, and the ugly. Pragmat. Cogn. 2008, 16, 215–223. [Google Scholar]
  8. Lee, C.E.; Leow, S.; Kong, X.J. The use of mobile technologies for learning in higher education: Students’ readiness. In Proceedings of the 6th International SEARCH Conference 2019, Kuala Lumpur, Malaysia, 27–28 June 2019; pp. 107–127. [Google Scholar]
  9. Lee, C.E.; Teh, P.L. Educational use of Facebook by undergraduate students in Malaysia higher education: A case study of a private university. Soc. Media Technol. 2016, 1, 1–8. [Google Scholar] [CrossRef]
  10. Carlen, U.; Jobring, O. The rationale of online learning communities. Int. J. Web Based Communities 2005, 1, 272–295. [Google Scholar] [CrossRef]
  11. Udenze, S.; Oshionebo, B. Investigating ‘Whatsapp’ for collaborative learning among undergraduates. Etkilesim 2020, 5, 24–50. [Google Scholar] [CrossRef]
  12. Suarez-Lantaron, B.; Deocano-Ruiz, Y.; Garcia-Perales, N.; Castillo-Reche, I.S. The educational use of WhatsApp. Sustainability 2022, 14, 10510. [Google Scholar] [CrossRef]
  13. Indiran, D.; Ismail, H.H.; Rashid, R.A. Exploring opportunities and challenges of using WhatsApp in teaching reading: A Malaysian rural primary school context. Creat. Educ. 2022, 13, 1689–1709. [Google Scholar] [CrossRef]
  14. Yeboah, J.; Ewur, G.D. The impact of Whatsapp Messneger usage on students’ performance in tertiary institutions in Ghana. J. Educ. Pract. 2014, 5, 157–164. [Google Scholar]
  15. Meola, A. WhatsApp Is the Most Popular Chat App in More than Half the World. 27 May 2016. Available online: http://www.businessinsider.com/whatsapp-is-the-most-popular-chat-app-in-more-than-half-the-world-2016-5?IR=T&r=US&IR=T (accessed on 13 December 2022).
  16. Kibona, L.; Mgaya, G. Smartphones’ Effects on Academic Performance of Higher Learning Students. J. Multidiscip. Eng. Sci. Technol. 2015, 2, 777–784. [Google Scholar]
  17. Bouhnik, D.; Deshen, M. WhatsApp goes to school: Mobile instant messaging between teachers and students. J. Inf. Technol. Educ. Res. 2014, 13, 217–231. Available online: http://www.jite.org/documents/Vol13/JITEv13ResearchP217-231Bouhnik0601.pdf (accessed on 13 December 2022). [CrossRef]
  18. Lee, C.Y. Adoption of WhatsApp instant messaging among students in Ipoh higher education institutions. 2016. Available online: http://woulibrary.wou.edu.my/thesesproject/MED2016_CYLEE.pdf (accessed on 15 December 2022).
  19. Minocha, S. A case study-based investigation of students’ experiences with social software tools. New Rev. Hypermedia Multimed. 2009, 15, 245–265. [Google Scholar] [CrossRef]
  20. Konetes, G.; McKeague, M. The effects of social networking sites on the acquisition of social capital among college students: A pilot study. Glob. Media J. 2011, 11, 1–10. Available online: https://www.globalmediajournal.com/open-access/the-effects-of-social-networking-sites-on-the-acquisition-of-social-capital-among-college-students-a-pilot-study.pdf (accessed on 15 December 2022).
  21. The Economic Times. WhatsApp Will Soon Allow Up to 1024 Participants to Join Same Group Chat. 11 October 2022. Available online: https://economictimes.indiatimes.com/magazines/panache/whatsapp-will-soon-allow-up-to-1024-participants-to-join-same-group-chat/articleshow/94782713.cms (accessed on 15 December 2022).
  22. Alsaleem, B.I.A. The Effect of “WhatsApp” Electronic Dialogue Journaling on Improving Writing Vocabulary Word Choice and Voice of EFL Undergraduate Saudi Students. In Proceedings of the 21st Century Academic Forum Conference Proceedings–2014, Cambridge, MA, USA, 21–23 August 2014; Available online: http://www.21caf.org/uploads/1/3/5/2/13527682/alsaleem-hrd-conference_proceedings.pdf (accessed on 13 December 2022).
  23. Eum, Y.; Park, J.; Yim, J. Teenager’s Addiction to Smart Phones and Its Integrated Therapy Method. Adv. Sci. Technol. Lett. 2016, 132, 116–122. [Google Scholar] [CrossRef]
  24. Iorliam, A.; Ode, E. The Impact of Social Network Usage on University Students Academic Performance: A Case Study of Benue State University Makurdi, Nigeria. Int. J. Comput. Sci. Eng. 2014, 6, 275–279. Available online: http://www.enggjournals.com/ijcse/doc/IJCSE14-06-07-099.pdf (accessed on 15 December 2022).
  25. Sweeny, S.M. Writing for the instant messaging and text messaging generation: Using new literacies to support writing instruction. J. Adolesc. Adult Lit. 2010, 54, 121–130. [Google Scholar] [CrossRef]
  26. Hashim, N.A.; Abdullah, N.L.; Isa, R.M.; Janor, H. WhatsApp Messenger Application among Business Students in Malaysia—An Exploration. J. Pers. Pelajar 2015, 18, 128–139. [Google Scholar]
  27. Echenique, E.E.; Molias, L.M.; Bullen, M. Students in higher education: Social and academic uses of digital technology. RUSC. Univ. Knowl. Soc. J. 2015, 12, 25–37. [Google Scholar] [CrossRef]
  28. Berry, G.R. Enhancing effectiveness on virtual teams: Understanding why traditional team skills are insufficient. J. Bus. Commun. 2011, 48, 186–206. [Google Scholar] [CrossRef]
  29. Cheng, X.; Yin, G.; Azadegan, A.; Kolfschoten, G. Trust evolvement in hybrid team collaboration: A longitudinal case study. Group Decis. Negot. Group Decis Negot 2015, 25, 267–288. [Google Scholar] [CrossRef]
  30. Carlson, J.R.; Carlson, D.; Hunter, E.M.; George, J.F. Virtual effectiveness: Investigating the moderating role experience with computer-mediated communication on the impact of team cohesion and openness. J. Organ. End User Computing. 2013, 25, 1–18. [Google Scholar] [CrossRef]
  31. Beam, M.M. Emotional Intelligence and Team Cohesiveness. Thesis Submitted to the Graduate College of Marshall University. 2012. Available online: https://mds.marshall.edu/cgi/viewcontent.cgi?article=1304&context=etd (accessed on 18 December 2022).
  32. Hansen, D.E. Cohesion in online student teams versus traditional teams. J. Mark. Educ. 2016, 38, 37–46. [Google Scholar] [CrossRef]
  33. Creswell, J.W.; Palno-Clark, V.L. Designing and Conducting Mixed Methods Research; Sage: Thousand Oaks, CA, USA, 2018. [Google Scholar]
  34. Demir, S.B.; Pismek, N. A convergent parallel mixed-methods study of controversial issues in social studies classes: A clash of ideologies. Educ. Sci. Theory Pract. 2018, 18, 119–149. [Google Scholar]
  35. Babbie, E.R. The Practice of Social Research, 12th ed.; Wadsworth Cengage: Belmont, CA, USA, 2010. [Google Scholar]
  36. Mirzaee, A. 3 Types of Survey Research, Fluid Surveys. 2014. Available online: http://fluidsurveys.com/university/3-types-survey-research-use-can-benefit-organization/ (accessed on 18 November 2022).
  37. Sincero, S.M. Surveys and Questionnaires–Guide. 2014. Available online: https://explorable.com/surveys-and-questionnaires (accessed on 18 November 2022).
  38. Taylor, S.J.; Bogdan, R.; DeVault, M.L. Introduction to Qualitative Research Methods: A Guidebook and Resource; Wiley: Hoboken, NJ, USA, 2016. [Google Scholar]
  39. Gibbs, A. Social Media Update: Focus Group. 1997. Available online: https://sru.soc.surrey.ac.uk/SRU19.html (accessed on 18 November 2022).
  40. Mistar, I.B.; Embi, M.A. Students’ perception on the use of WhatsApp as a learning tool in ESL classroom. J. Educ. Soc. Sci. 2016, 4, 96–104. [Google Scholar]
  41. Saunders, M.; Lewis, P.; Thornhill, A. Research Methods for Business Students, 5th ed.; Pearson Education Limited: London, UK, 2009. [Google Scholar]
  42. Loi, E. 26 WhatsApp Features You Didn’t Know You Had, Let Alone Could Use. 30 March 2016. Available online: http://www.stuff.tv/sg/features/19-whatsapp-features-you-might-not-be-aware (accessed on 5 September 2022).
Figure 1. The research process using the convergent parallel mixed-methods research design (adopted from [34]).
Figure 1. The research process using the convergent parallel mixed-methods research design (adopted from [34]).
Education 13 00244 g001
Table 1. Students’ Frequency of WhatsApp Usage (N = 99).
Table 1. Students’ Frequency of WhatsApp Usage (N = 99).
FrequencyPercentage (%)
Several times per day9697.0
Once a day33.0
Total99100.0
Table 2. Students’ Time Spent on WhatsApp Per Day (N = 99).
Table 2. Students’ Time Spent on WhatsApp Per Day (N = 99).
FrequencyPercentage (%)
1–2 h3434.3
3–5 h4242.4
6–7 h44.0
Over 8 h1919.2
Total99100.0
Table 3. Students’ Time Spent on WhatsApp for Academic–Related Activities Per Week (N = 99).
Table 3. Students’ Time Spent on WhatsApp for Academic–Related Activities Per Week (N = 99).
FrequencyPercentage (%)
Less than 5 h7171.7
5–10 h2323.2
15–20 h11.0
More than 20 h44.0
Total99100.0
Table 4. Academic–Related Activities Using WhatsApp.
Table 4. Academic–Related Activities Using WhatsApp.
FrequencyPercentage (%)
Group discussion3333.3
Group studies1616.2
Sharing resources3030.3
Informing educational agenda2020.2
Total99100.0
Table 5. Descriptive Statistics of Students’ Level of Agreement on the Uses and Impact of WhatsApp on Academic Performance.
Table 5. Descriptive Statistics of Students’ Level of Agreement on the Uses and Impact of WhatsApp on Academic Performance.
NMeanSD
1. WhatsApp helped me to pass any information discussed easily994.340.702
2. WhatsApp allows me to academically engage with peers and lecturers994.110.844
3. WhatsApp is a good online platform to facilitate group projects994.030.897
4. WhatsApp favoured faster knowledge sharing994.030.749
5. My interaction in WhatsApp messaging was understandable994.020.714
6. My interaction in WhatsApp messaging was clear994.000.821
7. WhatsApp destroys spellings and grammatical construction of sentence993.621.193
8. WhatsApp leads to lack of concentration during lectures993.550.993
9. WhatsApp distracts me from adhering to my private study timetable992.371.093
10. WhatsApp distracts me from completing my assignments992.321.018
11. WhatsApp results in difficulty in balancing online activities992.271.058
12. I will procrastinate my academic work due to WhatsApp992.261.006
Valid N (listwise)99
Table 6. Focus Group Interviewees’ Demographic Profile.
Table 6. Focus Group Interviewees’ Demographic Profile.
ParticipantGenderClub/SocietyPositionProgrammeYear of Study
P1MaleGroup APresidentACCA2
P2FemaleGroup AVice PresidentAccounting2
P3MaleGroup ADirectorACCAFinal
P4MaleGroup ADirectorFinance2
P5FemaleGroup ADirectorActuarial2
P6FemaleGroup AVice PresidentAccounting2
P7FemaleGroup BSecretaryActuarial2
P8FemaleGroup BVice PresidentActuarial2
P9FemaleGroup BMemberActuarial2
P10FemaleGroup BMemberAccounting2
P11MaleGroup BMemberActuarial2
P12FemaleGroup BMemberAccounting2
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lee, C.E.; Chern, H.H.; Azmir, D.A. WhatsApp Use in a Higher Education Learning Environment: Perspective of Students of a Malaysian Private University on Academic Performance and Team Effectiveness. Educ. Sci. 2023, 13, 244. https://doi.org/10.3390/educsci13030244

AMA Style

Lee CE, Chern HH, Azmir DA. WhatsApp Use in a Higher Education Learning Environment: Perspective of Students of a Malaysian Private University on Academic Performance and Team Effectiveness. Education Sciences. 2023; 13(3):244. https://doi.org/10.3390/educsci13030244

Chicago/Turabian Style

Lee, Cheng Ean (Catherine), Huei Huei Chern, and Dzafran Adris Azmir. 2023. "WhatsApp Use in a Higher Education Learning Environment: Perspective of Students of a Malaysian Private University on Academic Performance and Team Effectiveness" Education Sciences 13, no. 3: 244. https://doi.org/10.3390/educsci13030244

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop