Time to evaluate: the students’ perspective of an online MA in CALL programme

Despite the critical importance of Computer Assisted Language Learning (CALL) Teacher Education (CTE) programmes, L2 practitioners often report that there are not many such onsite/online programmes, and the ones that exist may not directly meet their specific needs. Moreover, there seems to be a lack of systematic evaluation studies of such courses. As a result, problems in such courses cannot be diagnosed, and they are left unresolved. This paper presents a study undertaken to evaluate an online Master’s of Arts (MA) in CALL programme in the Republic of Cyprus. The investigation was based on a conceptual multidimensional e-learning evaluation model rated by the students. Data were collected from 25 graduate students via an online anonymous survey. This focussed on the participants’ perceptions of the value of the following aspects: (1) their engagement; (2) the course and its organisation, teaching mode, and materials; (3) course strengths; (4) course aspects most helpful to learning; and (5) course aspects which were obstacles to learning. This paper discusses the findings and offers some first recommendations for further improvement to the MA in CALL programme.


Introduction
In the last decades, substantial research has been dedicated to CTE. The research of Hubbard and Levy (2006), Torsani (2016), Hauck and Kurek (2017), and Jordano de la Torre (2019) are indicative. However, a brief review in the area of CTE programme evaluation reveals that this area still "warrants considerably more professional -and international -attention" (Hubbard & Levy, 2006, p. ix). This paper presents a study undertaken to evaluate an online MA in CALL programme in the Republic of Cyprus. The investigation was based on a conceptual multidimensional e-learning evaluation model rated by the students.

Method
The present investigation concentrates on the evaluation of an online MA in CALL programme (running since September 2015) from the students' viewpoint. The paper presents the results of the research study implemented at the end of the academic year 2018-2019.

Participants
Nineteen students (80% of the total programme population over a period of four years) from Cyprus University of Technology, ranging between 22 to 60 years old, participated in this study. The grade point average mean of females was 2.80, while that of males was 2.52, out of a possible 4.00.

A conceptual multidimensional e-learning evaluation model
The investigation proposed a conceptual multidimensional e-learning evaluation model focussing on: (1) their engagement; (2) the course and its organisation, teaching mode, and materials; (3) course strengths; (4) course aspects most helpful to learning; and (5) course aspects which were obstacles to learning

Procedure
The data collection was conducted between May and June 2019. The online survey was distributed to 25 students. Nineteen participants submitted a signed consent form and completed the online survey anonymously.

Instrument and data collection
The online survey contained questions on the eight modules: The survey consisted of 58 questions and supported quantitative and qualitative elements. Likert scale closed-ended questions were applied. Respondents were presented with a set of verbal answer options -Disagree (D), Strongly Disagree (SD), Neutral (N), Agree (A), Strongly Agree (SA) -to cover a range of opinions, and 32 open-ended questions. Open-ended questions were also included and covered three aspects: (1) course strengths; (2) course aspects most helpful to learning; and (3) course aspects which were obstacles to learning.

Quantitative data
A key research issue was to investigate the merits and shortcomings of the MA course. Participants were asked to evaluate their engagement in the process (Table 1). High numbers show that students believed to have put in a considerable amount of effort (Q1), and are highly consistent when it comes to attending webinars (Q3). However, numbers tend to drop in Q2 and Q4, regarding students' punctuality and consistency in completing their weekly tasks. While students feel they make great efforts to evolve as students and are present when there is immediate contact with their peers and instructors during online webinars, maintaining consistency and punctuality in their autonomous learning proved a challenge.
The study investigated structure and materials used in order to evaluate the quality of the course and its content. Table 2 displays data per module. The students' evaluations of the structure and material used throughout the MA programme offered insights on whether the planning and construction of the modules are well-presented, efficient, and beneficial to students' learning. The responses complement the previous segment of student engagement, as the structure and materials used in each module, to an extent, explain the positive outcomes of Table 1. In the current segment, the only section where students felt indifferent was in Q4, where they were asked about their assignments. This outcome was anticipated after the analysis of the first segment, as the students' greatest challenge was the consistency and punctuality of their assignments. This comes back to the nature of the online MA and the fact that students are encouraged to work autonomously and construct their learning. Their autonomous skills demand greater development, as students need further practice in self-training to reach the peak of their productivity.

Qualitative data
The qualitative component uncovered students' thoughts and opinions. The openended questions focussed on (1) strengths of the course, (2) parts that aided students' learning, and (3) parts that were obstacles to their learning of each module. Themes re-occurred for all modules and emerged from the data coding. The most common themes are described in Table 3 below. The students found the postgraduate programme materials, webinars, guidance, and content highly beneficial and they have advanced their CALL understanding and practice. Six stated they did not identify any parts of the programme hindering their learning. However, four found the workload heavy; three found meeting deadlines difficult. One found continuous reflective journal writing tiring; another found it sometimes exhausting. One also had an issue with feedback promptness.
One interesting outcome is that all modules have been similarly evaluated sharing the majority of the same positives and challenges.

Conclusions
There is a need to identify the extent to which programmes meet students' needs. The present paper reflects the effort to evaluate the MA in CALL from the students' perspective. The results were mostly positive, both in terms of student input and in terms of the programme evaluation. However, further investigation into the workload, weekly activities, and time limit issues are necessary.
Disclaimer: Research-publishing.net does not take any responsibility for the content of the pages written by the authors of this book. The authors have recognised that the work described was not published before, or that it was not under consideration for publication elsewhere. While the information in this book is believed to be true and accurate on the date of its going to press, neither the editorial team nor the publisher can accept any legal responsibility for any errors or omissions. The publisher makes no warranty, expressed or implied, with respect to the material contained herein. While Researchpublishing.net is committed to publishing works of integrity, the words are the authors' alone.
Trademark notice: product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.
Copyrighted material: every effort has been made by the editorial team to trace copyright holders and to obtain their permission for the use of copyrighted material in this book. In the event of errors or omissions, please notify the publisher of any corrections that will need to be incorporated in future editions of this book.