Exploring mobile learning success factors

This paper is a comparative account and analysis of three mobile Web 2.0 projects instigated within a tertiary learning environment during 2008. Following the successful instigation of a mobile Web 2.0 project in the third year of a Bachelor of Product Design course during semester one, similar projects were initiated in semester two within the first-year and second-year Bachelor of Product Design courses. A common methodology for supporting and facilitating mobile Web 2.0 projects was used for all three projects. The projects were designed to explore the potential of mobile Web 2.0 tools to enhance both the formal and informal teaching and learning environments with a focus upon mobile blogging (moblogging). A comparison of student and teaching staff feedback from each of the three projects provided a basis for identifying and illustrating critical success factors within similar m-learning scenarios. Critical success factors identified include: the importance of the pedagogical integration of the technology into the course assessment, lecturer modelling of the pedagogical use of the tools, the need for regular formative feedback from lecturers to students, and the appropriate choice of mobile devices and software to support the pedagogical model underlying the course.


Introduction
The structure of the paper is as follows: beginning with an overview of the m-learning concepts underpinning the research, followed by a short summary of associated mlearning research, the project's main research questions and an introduction to the research project.The methodology section outlines the project participants, data collection instruments, and the pedagogical design of the project.The results section comprises a comparative analysis of the participants' activity and feedback, which is then used to illustrate one of the critical pedagogical success factors that have been identified across the research projects between 2006 and 2008.How these then informed subsequent m-learning project iterations in 2009 is outlined.

Defining m-learning
Sharples proposes a form of Laurillard's conversational framework, excluding the teacher, to define mobile learning by its contextual and informal learning characteristics: *Email: tcochrane@unitec.ac.nz "The processes of coming to know through conversations across multiple contexts amongst people and personal interactive technologies" (Sharples, Taylor, and Vavoula 2007, 225).However, a key element in the conversational framework is the dialogue between teacher and student.In contrast to Sharples, Taylor, and Vavoula (2007), Laurillard (2007) emphasises the teacher's input in mobile environments through good pedagogic design that facilities continuity between the face-to-face and remote peer learning contexts.Her definition of mobile learning incorporates the critical pedagogical design input of the teacher: "M-learning, being the digital support of adaptive, investigative, communicative, collaborative, and productive learning activities in remote locations, proposes a wide variety of environments in which the teacher can operate" (Laurillard 2007, 172).
It is the potential for mobile learning to bridge pedagogically designed learning contexts, facilitate learner-generated contexts, and content (both personal and collaborative), while providing personalisation and ubiquitous social connectedness, that sets it apart from more traditional learning environments.Mobile learning, as defined in this paper, involves the use of wireless-enabled mobile digital devices (wireless mobile devices [WMDs]) within and between pedagogically designed learning environments or contexts.From an activity theory perspective, WMDs are the tools that mediate a wide range of learning activities and facilitate collaborative learning environments (Uden 2007).

Mobile Web 2.0
The term Web 2.0 was coined in 2005 (O'Reilly 2005) as a way of characterising the emerging interactive, user-centred web-based tools that were revolutionising the way the Internet was conceptualised and used.Many educators have harnessed Web 2.0 tools for creating engaging student-centred learning environments.This appropriation of Web 2.0 tools within a social-constructivist pedagogy facilitates what has been termed 'Pedagogy 2.0' (McLoughlin and Lee 2008).Web 2.0 supports student mediarich content creation and sharing via free, easily personalisable interfaces.This research project is interested in appropriating the benefits of Web 2.0 and Pedagogy 2.0 anywhere anytime using mobile Web 2.0 (Web 2.0 services that are formatted for use with mobile devices and often make use of mobile-specific affordances, such as GPS tagging, and built-in cameras) and WMDs.

Situating the research
This paper explores how the introduction of mobile Web 2.0 technologies into a Bachelor of Product Design programme have impacted, disrupted and transformed the established teaching and learning paradigms.The introduction and integration of mlearning within traditional face-to-face and portfolio-based tertiary education have facilitated a change in the role and understanding of the nature of teaching of the lecturer and of the students' prior experience of learning.The goal of the research and project has been to move pedagogical approaches in tertiary education from instructivist pedagogies to a social-constructivist pedagogy (Vygotsky 1978) and to facilitate a context bridging collaborative learning environment.Disruptive technologies (Sharples 2000(Sharples , 2001(Sharples , 2005;;Stead 2006) are those technologies that challenge established systems and thinking, requiring change, and are thus viewed by many as a threat to the status quo.Disruptive technologies democratise education environments challenging the established power relations between lecturers and students.Their disruptive nature forces a rethink of pedagogical strategies and relationships in education.
The decision to try a new technology, experiment with its capabilities, and then finally integrate its use into one's daily workflow and personal experience is a complicated process that has been described as adoption and appropriation.Carroll et al. define technology appropriation as "the way that users evaluate and adopt, adapt and integrate a technology into their everyday practices" (2002,58).Theoretical explorations of technology adoption and appropriation include: the Diffusion of Innovation (Rogers 2003), originally published in 1962 but still used as a foundational framework; Adaptive Structuration Theory (DeSanctis and Poole 1994); and the Technology Acceptance Model (Davis 1989).The main criticism of these approaches to technology adoption, and particularly in the case of mobile Web 2.0, is the incredibly fast rate of change of these innovations like a continual innovation 'wave', an adopter must learn to 'surf' in a constantly changing environment.
While Web 2.0 tools are characterised by user-generated content and social networking, mobile devices add the extra dimension of user-generated contexts."The intrinsic nature of mobile technologies is to offer digitally-facilitated site-specific learning, which is motivating because of the degree of ownership and control" (Laurillard 2007, 157).Herrington and Herrington (2007, 4) note that "Despite the significant potential of mobile technologies to be used as powerful learning tools in higher education, their current use appears to be predominantly within a didactic, teacher-centred paradigm, rather than a more constructivist environment".
M-learning and Web 2.0 technologies have been identified as emerging tools to enhance teaching and learning (Anderson 2007;Becta 2007;McFarlane, Roche, and Triggs 2007;McLoughlin and Lee 2008;New Media Consortium 2007, 2008;Sharples, Milrad et al. 2007;Traxler 2007;Trinder et al. 2008), but are not usually explicitly linked together.A list of current m-learning projects can be found on the International Association for Mobile Learning (2008) website.Few studies have yet to explicitly bridge both the formal and informal learning contexts within mainstream tertiary education.

Research questions
The research summarised herein is part of a wider research project investigating the potential of mobile Web 2.0 for enhancing tertiary education through a series of action research projects in a variety of disciplines.This paper focuses on the effect of mobile Web 2.0 upon the pedagogical development of one of these projects (Bachelor of Product Design).
The wider research questions are as follows: ( The exploration of identified critical pedagogical success factors is used to answer Research Questions (1) and (2) in particular.This paper focuses upon exploring the validity and implications of these critical pedagogical success factors against the three Bachelor of Product Design m-learning projects in 2008, rather than upon the wider research questions.
The intended learning outcomes of these projects for students are as follows: • Developing critical reflective skills.
• Developing an online e-portfolio.
• Developing a potentially worldwide peer support and critique network.
• Learning how to maximise technology to enhance the learning environment across multiple contexts.
Following the enthusiastic feedback from both the lecturers and the students involved in the 2008 semester one, third-year project, course lecturers requested similar projects be established in the first-year and second-year Bachelor of Product Design courses for semester two 2008.A similar methodology for facilitating and supporting the m-learning scenarios was used, with a few variables between the three courses, which allows for comparative analysis of potential success factors as illustrated by the results and participant feedback from the three projects.
The projects investigated how a smartphone could be used to enhance almost any aspect of the course, but focus particularly on their collaboration and communication capabilities.The project used the smartphone within a wide range of activities aligned with the projects underlying social-constructivist pedagogy.Each of the trials explicitly used a social-constructivist pedagogy by focusing upon students forming teams to create a team project, usually involving real clients external to the classroom, fostering peer critique and review via commenting on each other's blog posts and Vox's 'neighbourhood' social network facility, and by using instant messaging to create a context-independent learning community.The focus was on student-generated content, not on content delivery from lecturers to students.Course lecturers were encouraged to create a learning environment where regular formative feedback is posted as lecturer comments on students' blogs, and to use instant messaging to be able to respond to students' questions whenever they are available online (as well as in the classroom).Figure 1 shows a concept map developed to graphically illustrate the links between multiple learning contexts, and Web 2.0 technologies that the smartphones afford.Lecturers within the Product Design programme have identified several pedagogical benefits from the integration of mobile Web 2.0 within the BDesign programme.These will be explored further in the discussion section of this paper.
The integration of mobile web 2.0 has facilitated a shift away from the default Atelier 'private method' of instruction to a new more fluid and dynamic pedagogical method.This project has deliberately disrupted the timetabled instructivist studio learning that is frequently used and placed the student group in a social constructivist framework.(BDesign third-year lecturer) Based on the experiences gathered from eight mobile learning trials over the past three years (Cochrane 2005(Cochrane , 2006(Cochrane , 2007(Cochrane , 2008a)), several critical success factors were identified as emergent themes from the various m-learning case studies undertaken.The opportunity to compare three m-learning implementations within the same course enables comparative analysis of these critical success factors.These are explored in the feedback and results of the three m-learning projects described in this paper.
• The level of pedagogical integration of the technology into the course criteria and assessment.• The level of lecturer modelling of the pedagogical use of the tools.
• The use of regular formative feedback from both lecturers and student peers.
• Appropriate choice of mobile devices and software.
• Technological and pedagogical support.
These success factors are measured against the following: • The level of student engagement and satisfaction achieved -as evidenced in evaluative surveys and focus group feedback.• The level of mobile blogging (moblogging) achieved by students in the courses.
• Lecturer and student reflective feedback.

Methodology
All participants signed ethics consent and acceptable use agreements for the purposes of the research.Students were informed of the option of withdrawing from the research project at any time, and assessed activities could be achieved with or without the WMDs.An overview of each of the three projects and their participants is given in Tables 1, 2 and 3.
Each project was guided and supported by weekly 'technology sessions'(effectively establishing Communities of Practice) facilitated by a 'technology steward'   use and integration of mobile Web 2.0 in their own daily workflows and to provide regular formative feedback to students via posts on their blogs and other media.

Data collection
A participatory action research methodology (McLoughlin and Lee 2007; Wadsworth 1998) was used, creating a reflective research environment that continually sought to improve the student learning outcomes based on regular student and lecturer feedback.Therefore reflective events were scheduled throughout the projects to capture participant feedback, including the following: (1) Pre-trial surveys of lecturers and students, to establish current practice and expertise.
(2) Post-trial surveys and focus groups, to measure the impact of the wireless mobile computing environment, and the implementation of the guidelines.(3) Lecturer and student reflections via their own blogs during the trial.The blog is also an online e-portfolio facilitating the collection of rich media resources capturing critical incidents and providing a dynamic journal of student projects and lecturer input (both formative and summative).These were also useful for triangulating the data captured via the surveys and focus group questions.
Course lecturers were asked to reflect on the impact of mobile Web 2.0 at several points throughout the trial, and used a variety of media to capture their reflections, including: posts to their blogs, VODCasts (video recordings uploaded to their blogs and YouTube), paper surveys, discussions and brainstorms with the researcher.Staff reflections were focused on the aspect of pedagogical transformation.
Students were also asked to record (as VODCasts) their reflections on the project at the middle and the end of each project. 2 These VODCasts authentically capture the learning journeys and experiences of the participants, and became a favourite reflective tool for many of the students.The VODCasts provide triangulation of longitudinal data for evaluation that is often lacking in m-learning projects (Sharples 2009).Sharples (2009) promotes the use of video for capturing critical incidents for analysis, while the study funded by the Joint Information Systems Committee (2007) into students' use of technology found that students' experiences of using webcams for recording video reflections varied (Jefferies and Hyde 2009).

Results
The third-year course started the m-learning project using supplied Nokia N80 smartphones and bluetooth folding keyboards in semester one 2008.Course lecturers were provided with Nokia N95 smartphones and bluetooth folding keyboards prior to the beginning of the academic semester.The project was enthusiastically adopted by both the lecturers and their students.With additional research funding made available late in semester one, the students were all updated to Nokia N95 smartphones.The enthusiastic response to the project by the third-year lecturers and students led to an application for internal research funding to widen the project to encompass all three years of the Bachelor of Product design programme.Hence an additional project was also established in the second-year course, brokered by the third-year lecturers with the second-year lecturers and students.Following this, the Apple iPhone was released in New Zealand in July 2008, providing an opportunity to investigate its potential as a WMD for m-learning within the first-year course.However, while the third-year course fully integrated the use of m-learning into the course and the lecturers' workflow and social lives, this did not end up being the case in the second-year and first-year projects.The second-year project became entirely optional and carried no assessment, with the second-year lecturers effectively ignoring the project, thus leaving the researcher facilitating weekly m-learning communities of practice with the students only while the lecturers saw it as an opportunity for a free period.Nevertheless, the second-year students engaged with the project and enjoyed the experience (see student feedback below).The first-year project was limited to a single paper within the course, and while the lecturer engaged with the process and weekly m-learning communities of practice they did not integrate the use of the iPhone into their daily workflow or social life.The implications of this are explored further in the discussion.Figure 2 indicates that participants in the three projects had similar previous experiences of mobile and Web 2.0 technologies.While most participants were to some extent consumers of Web 2.0 media, the majority were not involved in regularly creating Web 2.0 content (e.g.regularly blogging, uploading videos to YouTube, etc.).The Product Design course has established an ethos of student-owned laptops; participant access to wireless laptops was therefore relatively high, and cellphone ownership almost ubiquitous.Instant messaging usage was lower than expected, although this may be more to do with use within a learning context rather than social use.Table 4 presents a comparative overview of the three groups of student feedback gathered from the final survey at the end of each project.The third-year students indicate a significantly higher level of satisfaction with the m-learning project and its impact on their learning environment.Virtually all of the students enjoyed the experience and saw significant benefits from it.The most obvious feedback differences involve the impact of the m-learning projects on the development of learning communities and communication between students and lecturers.First-year students were dissatisfied with the iPhone's inability to record video and the low quality of the builtin camera.These affordances were perceived as invaluable for recording students' design processes and steps.

Staff feedback
The following is an example reflection on the impact of the mobile Web 2.0 project from one of the third-year lecturers: We have seen increased engagement -students are sharing on a more regular basis the things they are doing and uncovering with each other and with staff.I can engage with the students even when I am not in the studio via a variety of media.Embedding assessment is fundamental -because of the time involved in producing these eportfolios and blogs you would not get the uptake or seriousness without it being an assessed deliverable.Without the mobile devices (as in 2007) blogging was confined to the studio using laptops, so mobile blogging has changed the nature and engagement level!(Third-year lecturer) No comments were supplied from second-year lecturers, as they declined to participate.The first-year lecturer focused upon the integration of Web 2.0 technologies in the course assessment, and initially regarded the mobile affordances as an interesting addition rather than integral.However, by the end of the project the first-year lecturer was far more positive about the potential pedagogical affordances of mobile technology in the course.

Student feedback
Students were asked to describe what they used the smartphones for (beyond moblogging for assignments).Their answers displayed a wide variety of integration of the smartphone's capabilities into their daily learning, work, and social lives.Figures 3  and 4 show a graphical representation of the types of activities that third-year and second-year students used their N95 smartphones for.As indicated, third-year students focused more on tools that integrated the use of the smartphone into their learning contexts than second-year students, who tended to focus more upon the social connectivity capabilities of the smartphones.

Level of pedagogical integration of the technology into course criteria and assessment
Third-year lecturers identified assessment integration as a key issue: We ran a 2007 project that did not carry an assessment weighting and the uptake was lower than for this 2008 project where assessment of the blog was embedded.It makes sense that students want to receive credit for doing something that takes time, focus and commitment.(Third-year lecturer) Third-year and first-year students enjoyed using their blogs and mobile devices as part of their courses.While moblogging was seen as a relatively time-intensive activity, students saw many benefits from changing traditionally paper-based journal-type assessment activities into collaborative multimedia e-portfolios.Students also appreciated the level of context flexibility that the WMDs provided, with many students blogging from home or sites of research rather than having to be in a face-to-face studio environment.The ability of mobile Web 2.0 tools to facilitate a context-independent, 'virtual' studio (entitled the Nomadic Studio by the lecturers) is an area that lecturers are keen to experiment more with in 2009.Students used the mobile Web 2.0 technologies to blog their assignment posts from virtually any context.As an example, four of the third-year students decided to go on a mid-term 'research' trip to the snowfields of Queenstown, officially to test their prototype snow-kite harness designs.However, two of these students were scheduled to present their NPC (New Product Development) research to the class that week.These students therefore recorded their NPC class presentations on their N95 smartphones, and uploaded the virtual presentations to their Vox blogs for the rest of the class and the course tutor to view and comment on their presentations, in almost real time.To 'prove' they were in Queenstown, they also blogged mobile videos of their campervan and Queenstown scenery.The various m-learning trials undertaken have illustrated that pedagogical integration of m-learning into a course/curriculum requires a paradigm shift on behalf of the lecturers involved, and this takes significant time.Hameed and Shah (2009) describe this process as a 'cultural re-alignment'.Many of the identified m-learning scenarios were serendipitous rather than planned by the lecturers during 2008.Students also require significant time to gain the skills required to maximise the potential of new and emerging Web 2.0 tools -as our pre-trial surveys indicated, few students were already using these tools for their own content creation before the trial.Based upon these experiences, in order to achieve an explicit move to a social-constructivist learning environment using mobile Web 2.0 tools during 2009, a staged and scaffolded approach has been adopted.This staged approach allows the bridging of the PAH (Pedagogy, Andragogy, Heutagogy) continuum (Luckin et al. 2008), and the embedding of mobile Web 2.0 affordances that support each stage.
Therefore the integration of m-learning (mobile Web 2.0) across the three years of the Bachelor of Product Design programme in 2009 is structured as presented in Table 5.
In response to this, the 2009 mobile Web 2.0 projects were timetabled to provide explicit time and support for lecturers to learn the use and daily integration of the mobile devices before beginning the projects with their students.A smaller subset of mobile Web 2.0 affordances was also decided upon to keep the learning curve from being as steep as in the 2008 projects.Taking some broad framework ideas from the recent Wollongong m-learning projects (Herrington et al. 2008), lecturers participating in the projects were required to fulfil several commitments (as below), and the projects were rolled out over two semesters: beginning with the continuation and expansion of established projects in semester one, with new projects focusing initially on lecturer professional development during semester one, followed by student implementation in semester two of 2009.
Participant (lecturers) requirements for 2009: (1) Participation in a weekly Community of Practice.
(3) Development of m-learning activities based on social-constructivist pedagogy for implementation with students.(4) Implement a semester-long m-learning project with students.
(5) Publish a research output based on the project; for example, as a study paper at a conference, or in a journal, or presentation at a symposium to other staff.(6) Ethics consent for researchers' anonymous use of data.

Conclusions
As a limited sample case study, the presented findings and conclusions cannot be used to generalise too broadly, but serve as illustrative principles, strategies used, lessons learned and the results achieved that may help to inform future mobile Web 2.0 projects in tertiary education.The student engagement, flexibility of learning contexts, and the quality of student moblogging established through the mobile Web 2.0 projects within the Bachelor of Product Design course have led to the establishment of the integration of mobile Web 2.0 across the entire curriculum in 2009, and wider mobile Web 2.0 projects within the institution for 2009.A proposal for purchasing 1 Figure 1.

Figure 2 .
Figure 2. Participants' previous use of technology.

Figure 2 .
Figure 2. Participants' previous use of technology.

Figure 3 .
Figure 3. Year Two student mobile usage.Figure 4. Year Three student mobile usage.

Figure 4 .
Figure 3. Year Two student mobile usage.Figure 4. Year Three student mobile usage.

Figure 3 .
Figure 3. Year Two student mobile usage.

Figure 4 .
Figure 4. Year Three student mobile usage.

Table 1 .
Outline of BDesign third-year mobile project.
Course: Bachelor of Product Design, third-year class.

Table 2 .
Outline of BDesign second-year mobile project.
Course: Bachelor of Product Design, second-year class.

Table 3 .
Outline of BDesign first year mobile project.
Course: Bachelor of Product Design, first-year class.

Table 5 .
Scaffolding the rollout of mobile web 2.0 throughout the Product Design course.