Delivering the first internationally accessible Massive Online Open Course (MOOC) on suicide prevention: A case study and insights into best practice

To date, little guidance exists on how to design safe and effective online programming on sensitive and/or controversial topics. Massive online open courses (MOOCs) represent a unique opportunity for delivering inclusive and accessible teaching to international learner audiences. This paper provides an insight into designing and delivering the first internationally accessible MOOC on suicide prevention in the global context in 2019-highlighting insights into best practice as well as pertinent challenges. The results from two runs of this MOOC indicate that there appears to be a global demand for education on suicide prevention. Our practice suggests that new knowledge on extremely sensitive topics such as suicide can be safely and effectively delivered through a MOOC to an international community of learners. Learner safety needs to be carefully considered when developing and delivering online learning. Thorough and careful moderation is essential to ensure that learners engage safely and sensitively with the content and with one another. The involvement of diverse stakeholders, including people with lived experience, in the MOOC design is recommended to enhance the authenticity, inclusiveness and rigour of the curriculum.

teaching practice to successfully deliver the teaching of various topics in virtual learning environments without compromising pedagogic integrity and quality, or student engagement and wellbeing (Dhawan, 2020).
The authors received funding to develop a three-week MOOC on suicide prevention, which showcased the expertise within the Global Mental Health Master's Programme team and the [academic department name omitted], and addressed a gap in the online learning provision. The MOOC, 'Understanding Suicide and Suicide Prevention Strategies in a Global Context', ran in March and in September 2019. This innovative MOOC aimed to raise awareness of mental health issues and self-harm, and was the first internationally accessible online course that explored the challenges of suicide prevention in the global setting. An expert team supported the resource development and included people with lived experience of mental health difficulties and self-harm. The resource was endorsed by a range of key organisations including Samaritans and Breathing Space. The MOOC was launched with five organisational endorsements and one accreditation. Altogether, the two runs of the MOOC attracted over 3600 learners from across the globe.
Developing a MOOC on suicide and suicide prevention: Intended learning outcomes, structure and design principles Self-harm and suicide represent a global public health emergency with far-reaching clinical, socioeconomic and health equity implications for individuals, families and communities (Hawton, Saunders & O'Connor, 2012). More than 800,000 individuals are estimated to lose their lives to suicide every year, with millions more likely affected by self-harm and suicidal behaviours directly and indirectly (WHO, 2014). Implementing evidence-based and sensitive public education, anti-stigma initiatives and media reporting have been widely endorsed cross-cutting strategies for reducing suicide risk globally (O'Connor & Pirkis, 2016). In addition to saving lives through linking those engaged in selfharm ideation or behaviours with vital professional and community supports, awareness-raising is instrumental to creating effective and appropriate health services, stigma-free societies and sustainable involvement from stakeholders (WHO, 2018). Nevertheless, the inadequate awareness of suicide risk and reduction approaches represents a persistent challenge to the elimination of stigma and the prevention of suicide and self-harm (Kinchin & Doran, 2017).
Furthermore, the concerning surge of mental health difficulties such as depression, anxiety and suicidal ideation among university students in the U.K. and internationally demands concerted and innovative efforts to increase the mental health literacy and the suicide literacy of this population (Royal College of Psychiatrists, 2011;Institute for Public Policy Research, 2017;Hubble & Bolton, 2020;Gorczynski, Sims-Schouten, Hill & Wilson, 2017). This includes the development of accessible learning resources on mental health promotion, self-care and suicide prevention. Mental health literacy has been defined as 'a tool for personal and collective empowerment' that encompasses knowledge about mental health difficulties, their aetiology, treatment and prevention, including help-seeking, as well as the knowledge and attitudes to overcome stigma and offer and accept appropriate support options (Gorczynski et al., 2017, p. 715;Jorm, 2012). Relatedly, suicide literacy entails knowledge, beliefs and attitudes regarding the risk factors, sign and symptoms and treatment and prevention of suicide (Chan, Batterham, Christensen & Galletly, 2014). Suicide literacy has been recognised as an important component of mental health education curricula (Chan et al., 2014;Oliffe, Hannan-Leith, Ogrodniczuk, Black, Mackenzie, Lohan & Creighton, 2016).
The MOOC development team were aware that there was lack of accessible free online learning materials focused on suicide and suicide prevention which lead to the decision to create the MOOC on this topic. The team's background in Global Mental Health made it appropriate for the resources to tackle cultural variations, and to be internationally applicable and available. The course aimed to help learners gain a broader understanding of suicide as a worldwide issue, to recognise and understand multi-cultural variations within their own community and to understand how to analyse global suicide rates and patterns. There were opportunities to explore common risk factors along with the social and cultural factors that can influence suicidal behaviour. Suicide prevention strategies were introduced and their implementation in communities-evaluated.
The specific intended learning outcomes were to (a) explain some of the risk factors for suicide in the global context; (b) reflect on the impact of suicide at a personal and societal level; (c) explore the role of stigma in the context of suicide; and (d) produce examples of different strategies used to reduce the risk of suicide. The MOOC was intended for all healthcare professionals, including both medical and allied health professionals, who are looking to enhance their understanding of suicide as a global mental health issue, as well as to anyone interested in health research.
It is the current authors' belief that effective mental health education and training must be designed to challenge stigma and empower learners and trainees to open informed and constructive conversations about the topic of mental health in schools, in medical institutions, at the workplace and in the larger society. The MOOC was developed to further this ambitious goal and address the wide knowledge gap in the field of suicide and suicide prevention.
While the MOOC technology was deemed suitable for realising those aims, it was clear to the authors that a collaborative, multidisciplinary, culturally and ethically aware and digitally literate approach to course development was required to maximise the pedagogic potential of this learning technology for the teaching of such a sensitive topic as suicide. An ethical, inclusive and evidence-based approach to developing the MOOC curriculum was undertaken. The curriculum featured topics such as suicide prevalence, the cultural context of self-harm and suicide, risk and protective factors, stigma, the legality of suicide, and prevention strategies. The MOOC development team included a consultant psychiatrist, three clinical psychologists, an established researcher specialising in suicide research, a digital education specialist, persons with lived experience and Doctoral researchers. The course content was reviewed by a range of clinical experts and people with lived experience to ensure risks to learner safety were minimised. The contributions from a diverse team of researchers, clinicians and learning technologists enhanced the academic rigour and accessibility of the curriculum. All course developers screened content to minimise risk. The curriculum was also reviewed by a range of external organisations that support vulnerable individuals to optimise its safety and appropriateness. Quiz: Myths or facts.
Self-care activity: Take a little time to think about a song which inspires, motivates, uplifts, makes you smile or that has helped you through a difficult time.
Article: Suicide prevalence and cultural influences in Japan.
Case study and video: The lived experience of mental health difficulties and suicide.
Week 2 Case study and video: Being affected by suicide-related loss.
Video: Experiences of stigma.

Week 3: Suicide prevention both at a local and global level
Disease prevention

Suicide prevention
Stigma Role play video: Safety planning.
Quiz: Myths or facts.
Video: Suicide-specific prevention interventions.
Self-care activity: This activity encourages you to focus on positive aspects of your life. Reflect back on a memory that makes you smile and try to visualise the feelings involved and details of what it was you were doing in that memory. If you feel able to, please share any hobby or activity which helps you to relax.
Discussion: Suicide prevention strategies.
The content was also reviewed by two service-users, who offered additional feedback. Persons with lived experience were involved not only in the course review but also in the creation of learning content. Specifically, video learning resources were created that included persons with lived experience sharing their first-person accounts of dealing with mental health difficulties, self-harm and suicide. Those contributors recommended that the learning content should reflect the challenges that people face when confronted with suicide but also underscore the centrality of hope, optimism and resilience. Those recommendations were incorporated into the MOOC design.
The curriculum featured contributors and case studies from countries such as Scotland, Bulgaria and Brazil, which strengthened its cultural inclusivity. Also, the inclusion of audio-visual materials ensured the learning content was accessible to learners with diverse linguistic abilities. The pedagogic utility of integrating audio-visual materials into the teaching of sensitive and/or controversial topics has been demonstrated by prior research (Dalton, 2010;Lowe & Jones, 2010).

Challenging unhelpful beliefs about suicide and prioritising learner safety
A decision was made to exclude any direct references to suicide methods within the content to minimise any risk of creating suicide spikes. This decision was reached after consulting and applying World Health Organization (WHO) guidelines on the safe and responsible reporting on suicide (WHO, 2008;. As well as avoiding any discussions of specific means of suicide, the course content avoided language that sensationalised or glamorised suicide and used destigmatising language (such as 'to die by suicide' instead of using the verb 'commit'). In addition, the course development team scrutinised the stock images of choice to ensure they did not normalise suicide or depict any potential methods of self-harm.
The course content featured short videos of personal accounts by several contributors, in which they narrated about how their lives had been affected by self-harm and suicide. Their stories featured accounts of personal losses and struggles but also emphasised the importance of hope, recovery and personal growth, social support and empathy.
In accordance with the aforementioned guidance, self-help resources and information on where learners can seek help were embedded throughout the three-week MOOC content. Learners were also frequently encouraged to take breaks and consider engaging in self-care activities. Self-care activities included reflection, mindfulness, relaxation and managing stress (See 'Table 1', for examples). The self-care activities were strategically placed throughout the course in efforts to reduce any 'content fatigue' that learners might experience when exposed to sensitive content (Dalton, 2010). They offered opportunities to 'switch off' for a moment by focusing on the positive aspects of one's life or by simply doing nothing or having 'me' time. Embedding well-being resources and self-care activities within the curriculum was motivated by an ethic of care that all educators should uphold and that is especially crucial to learning outcomes and student well-being in courses that address sensitive issues (Dalton, 2010).
At the beginning of the course, learners were also asked to familiarise themselves with the online learning platform's universal Code of Conduct. The Code of Conduct emphasised kindness and respect in the interactions among users, as well as being supportive and constructive when participating in group discussions. The Code explicitly cautioned users against using offensive or inflammatory language or personal attacks. The Code stated the learning platform's commitment to creating an inclusive learning environment, in which learners from all ages, nationalities, knowledge, ability, experience, religions and cultural backgrounds are equally respected and heeded. Importantly, the Code also cautioned that communication via online media may lead to misunderstandings and that learners should 'give other people the benefit of the doubt' upon interpreting each other's written contributions. The authors felt the Code offered adequate guidance and a useful framework against which to model the moderation strategy for this MOOC (See 'Moderation, contingency planning and learner engagement' below).
It was also important to manage learners' expectations as to the nature and purpose of the online course. This was partly due to the high likelihood of the course attracting learners who may be bereaved by suicide or have other lived experience of selfharm and/or mental health difficulties. In the first week of the course, it was highlighted that 'this module is an educational resource, it is not a therapeutic tool', and learners were strongly encouraged to seek help and consult the collated wellbeing resources in cases when they might be experiencing distressing or suicidal thoughts. The following text was communicated, followed by weblinks to a comprehensive set of international support options: 'If you're feeling distressed, in a state of despair or suicidal, it's important to tell someone. We do not provide a treatment service or advice for those in crisis. We urge you to seek help from your doctor, a key worker or family and friends. You can also contact helpline services where there are people you can talk to in confidence and, if you prefer, without revealing your identity.' Also, at the end of each learning activity, learners were provided with a 'Wellbeing Resource' link containing such information.
The commitment of the course team to encourage constructive and positive conversations about suicide prevention was strongly emphasised, and learners were asked to discuss the topics covered in the course in a sensitive and responsible manner.
WHO guidelines have also cautioned against oversimplifying the topics of suicide and self-harm. Also, as recommended in the Samaritan Media Guidelines for Reporting Suicide (Guardian News and Media Limited, 2013, p. 8), '[i]t is important not to brush over the complex realities of suicide and its devastating impact on those left behind.'. By including the first-hand accounts of persons with lived experience as vital components of the learning content, the MOOC development team endeavoured to emphasise the complexity of suicide and self-harm, including the complex and diverse reactions affected people may have following such events. Furthermore, the complex causality possibly accounting for self-harming and suicidal ideation and behaviour was emphasised throughout the course. Specifically, socio-demographic factors, substance use, mental illness, exposure to violence and trauma, cultural attitudes and stigma, among others, were highlighted as likely risk factors.
In addition, the opening week contained a 'myths or facts' questionnaire that asked learners to appraise the veracity of common claims regarding suicide and self-harm. This exercise aimed to challenge popular misconceptions about suicide such as that all people who have attempted suicide have mental illness, or that a person who has attempted suicide and survived will not make any future attempts.
Effort was also made to highlight a range of evidence-based interventions to reduce the risk of suicide (See 'Table  1'). Throughout the MOOC, a range of prevention strategies were highlighted, including safety planning, restricting the means for self-harm, stigma reduction, crisis interventions and others. To provide a contextualised example of one specific intervention, two members of the MOOC development team video recorded a role-play demonstrating a safety planning session. The video outlines how to open conversations about vulnerability, explores approaches to identifying risk factors and supports the person in recording independent and social distractions that minimise risk of engagement with suicide behaviours. This, together with the testimonies offered by persons with lived experience, reflected the desire to embed authentic learning in the MOOC curriculum (Iucu & Marin, 2014).
Lastly, within the course content, it is acknowledged that risk factors are not always pre-requisites for suicidal behaviour, and learners were encouraged to critically consider the role of individual, socio-economic and socio-cultural factors that dynamically and non-deterministically shape suicidal tendencies.

Moderation, contingency planning and learner engagement
The online platform which hosted the MOOC obligated the activation of discussion areas within each learning step, with the exception of quizzes, which meant that approximately 60 independent discussion threads were enabled. While learner discussions offer invaluable opportunities for reflective and collaborative learning, their inclusion also presented a degree of risk in terms of learner safety. Careful moderation of the discussions and apt contingency planning were necessary to minimise the risk of learners' becoming exposed to insensitive or otherwise inappropriate content that could be not only offensive but also potentially retraumatising. This posed a series of pedagogical and practical challenges.
The best-practice guidelines on responding to suicidal content online by the National Suicide Prevention Alliance were applied when moderating the course. This meant that systematic and consistent rules were applied to screening, 'flagging' and removing any comments made by learners that explicitly discussed methods of suicide or engaged in discussions in an openly hostile and/or insensitive manner. Learners also had the option to 'flag' other learners' comments that they deemed inappropriate or unsafe, which triggered a standard review procedure by the online platform's moderators and managers.
A stepped approach to offering student support at times of crises and moderating student discussions was put in place and was developed collaboratively by the MOOC development team and the online learning platform's staff. Discussion posts deemed to be 'problematic' (unsafe and/or extremely insensitive, and/or offensive, for example, mentioning specific methods of self-harm) were 'flagged' by trained moderators who were part of the course development team. A template response was used to 'reply' to the post explaining the reason for the comment being 'flagged' and advising the student to delete or edit their post. A sample response was: 'I'm a member of XXX's team for this online course. You aren't in trouble, but because your comment included reference to unsafe behaviour, we will flag it and the moderators will hide it'.
The platform's moderators then revisited the 'flagged' comment and, if appropriate, hid the comment. An online 'problem behaviours form' was used to document potentially unsafe content, as well as any concerns about learner safety. Learners also had the option to 'flag' any posts which caused them concern or which they did not feel were compatible with the platform's Code of Conduct. These instances were reviewed and addressed by the host platform moderators.
In cases when the moderators had concerns about the wellbeing and safety of any learner based on a post or a series of posts, the moderators encouraged the learner to seek support. The sample response used was: 'We've seen your comment and we're concerned about you. If you need to seek support, please access the Wellbeing Resources in Step 1.5. If you're not sure you can keep yourself safe, please contact your doctor, go to hospital or call the emergency services. Please take care.' In cases when the moderators had a serious concern about an immediate danger to a learner's safety, an emergency procedure was implemented, which involved 'replying' to the post by advising the student to seek assistance from emergency services and escalating the issue by contacting an emergency response team set-up to respond out-of-hours in cases of emergencies. Fortunately, this form of escalation was not required.
Altogether, more than 4,400 comments were made during the two runs of the MOOC, of which 62 (or 1.4%) were flagged and 'hidden' due to unsafe content. Common reasons include discussions of suicide methods, SPAM, personal attacks, and sexually explicit or violent content. The manager of the hosting platform's moderators observed that "for the size of the cohort and the length of time the comments were available for, it (the proportion of flagged comments) was high." Unfortunately, we are unable to compare those data with comparable data from other MOOCs as, to our knowledge, such information is not easily available.
The moderation of the learner discussions embedded in most learning activities was resource-intensive. This was due to the high number of total learner comments, the large number of independent discussion threads and the continuous nature of moderation over the three-week period. A rota was created to ensure discussions were screened multiple times a day and potentially harmful content-flagged and removed, if appropriate. The different time zones that learners may be occupying presents a challenge to the effective continuous moderation process. In this MOOC, a minimum of three moderators each reviewed every discussion thread at least twice daily-documenting and actioning any issues encountered accordingly. This meant a minimum of six daily reviews, which ensured that most problematic comments were addressed promptly-therefore minimising their potential negative impact on individual learners as well as the learning environment. Naturally, different team members preferred to review the comments at different times of the day-including early mornings and late evenings. As a result, reviews typically occurred regularly within a window of 18 hours each day. Also, the hosting platform had a 24-hour moderation policy. Furthermore, the lead educators had provided their contact details to the moderators for 24-hour contact in case of serious incidents. The authors believe those arrangements satisfied the need for continuous and efficient moderation.
A collaborative approach was used in reaching decisions to edit or remove learner comments, which involved the host platform's moderators. The team were committed to ensuring a fair and flexible yet consistent and unambiguous approach to moderation. The team acknowledged that disagreement and a diversity of perspectives are not only to be expected but often conducive to constructive and insightful collaborative learning (Dalton, 2010). It was, therefore, important to clearly distinguish between disagreement based on respectfully presented ideas and confrontation that is personal, offensive, inflammatory and otherwise unhelpful. It was also important that any 'flagging' of comments or other moderator involvement should be perceived by the learners as well-justified and consistent rather than as forms of unwarranted censorship or of 'muting' or disciplining learners who express their emotions (Dalton, 2010). Because the topic of suicide may carry personal significance to many learners, and due to the inherent difficulties of unambiguously interpreting written comments, care was taken to ensure any such moderation was appropriate beyond doubt.
For instance, one learner was identified by two moderators as being rather reactive towards other learners in their comments, but not to the extent of overt 'trolling'. The MOOC team did not want to overtly discipline the individual but felt it was necessary to acknowledge that the learner's comments might cause distress or agitation in other learners-thus creating a hostile learning environment. The team collaborated with the host platform moderators to emphasise the importance sensitivity when posting comments, and then directed the learner to review the recommendations relating to sensitive discussion of the topic.
Finally, educators should be mindful of the potentially emotionally taxing effects of reading personal stories of suicide and ensure moderators are supported adequately by, for instance, organising regular debrief meetings. The lead educators of this MOOC set up a WhatsApp group for all moderators to facilitate daily, ongoing and informal interactions as a means of debriefing, peer support and anonymised discussions of any moderation challenges. All moderators had professional research and teaching experience in suicide and mental health and, therefore, felt well-equipped to make informed decisions about any moderation dilemmas encountered.

MOOC evaluation and learner experiences
To evaluate learners' experiences with, and perceptions about, this MOOC, the authors drew upon three sources of data: user analytics auto-generated by the host platform; learners' responses to a standard end-of-course questionnaire; and, finally, learners' comments about the course made within the Discussion sections of the various learning activities.
The MOOC had a global reach with learners from across six continents and 136 countries including Nigeria, Brazil, Kenya, Myanmar, Indonesia, China, Romania, Peru, Oman, and Ecuador. Some of the countries with higher representation levels included the UK (51%), Australia (5%), the United States (4%), Mexico (3%), Canada (2%), India (2%), China (2%), Russia (1%) and Saudi Arabia (1%). Those data demonstrate a diversity of learner representation, and, possibly, the inclusivity and the accessibility of the course design and curriculum. When compared to the average ages of students enrolled on other MOOCs hosted by the same platform, the average age of the cohorts of students enrolled on this MOOC was considerably older (See Table 2 above). Those data seem to indicate an older student cohort-perhaps due to the higher number of lifelong learners and professionals choosing to complete this MOOC compared with other MOOCs.
Learners were also asked to complete an end-of-course evaluation questionnaire. Almost all students indicated that they had gained new knowledge by completing the course (See Table 3). Around half of the students indicated that they had applied their learning during the course run and three-quarters had shared their learning with other people during this period. The overwhelming majority of learners also felt the course met their expectations. Those responses seem to indicate the practical relevance of the MOOC, as well as the high quality of its learning resources.
As the course had been accredited by the Royal College of Psychiatrists (RCGP), general practitioners (GPs) were invited to complete a brief survey to allow the course team to feedback engagement experiences to the RCGP. Over the two runs 25 GPs completed the survey. Twenty-four (96%) indicated that they had found the course helpful for their day-to-day work; twenty-two (88%) felt more confident in opening up conversations with patients about feelings of suicide during their clinics; and twenty-four (96%) said they would recommend the course to GP colleagues. Yes-76% (290) Yes-76% (143) Did the course meet your expectations? Yes-94% (291) Yes-93% (144) Several learners also offered free-text comments as part of the evaluation questionnaire. They appeared to recognise the sensitive approach undertaken in the delivery of the MOOC, as well as its accessibility and novelty: 'Excellent sharing of data and articles, and for tackling a difficult subject.' 'A subject of huge importance sensitively handled.' 'Thoughtful and compassionate, relevant material.' 'Easy to understand, brief and impactful.' 'Stimulating material, relevant to my current work role demands.' 'An excellent and thought-provoking course.' Notably, however, one learner found it hard to engage with the materials due to their distressing nature: 'The content is good, I am struggling though as I find it distressing.' This indicates the need to continue to offer support options and self-help resources to learners and continue to be responsive to their needs. It also demonstrates the need for further evaluation of the impact of the learning materials on student well-being.
Following the conclusion of the second run of the MOOC, the MOOC team concluded that, although the pedagogic value of learner discussions at the end of each learning activity was clear, the required level of ongoing moderation during the threeweek course run was unsustainable. Therefore, a decision was made to remove the option for learner discussions, and instead, offer the course as a permanently open resource providing an educationally focused place to learn about suicide and suicide prevention. This open resource was launched in November 2020 and student feedback and engagement will continue to be monitored, which will inform decisions about the progression of this course.

Recommendations for best practice in teaching sensitive topics via MOOCs
This case study evidences the utility of the MOOC as a promising educational technology for the safe and accessible teaching of sensitive topics. Recommendations for best practice are summarised below:  Create or adapt, and embed, an adequate, fair and accessible Code of Conduct to serve as a framework for protecting learner safety and assessing the appropriateness of learner comments.
 Prioritise the physical and emotional safety of both learners and educators by embedding wellbeing resources including self-care activities and links to useful support services; continuous moderation of group discussions and regular team meetings (wellbeing checks); and screening the curriculum for potentially harmful content.
 Enhance the authenticity of the curriculum by involving persons with lived experience in the curriculum design and mindfully and sensitively incorporating personal stories into the learning content.
 Manage learners' expectations by highlighting the educational (as opposed to therapeutic) nature of the course and signpost learners to support services.
 Wherever possible, challenge stereotypical views on the topic(s) of interest and highlight multiple perspectives by consulting with diverse stakeholders.
 Explicitly tackle the potentially adverse consequences of the use of insensitive language and encourage the use of non-judgemental, empathetic, inclusive and trauma-aware language.
 If deemed appropriate and helpful, incorporate a limited number of strategically placed group discussion activities strictly focusing on the learning objectives.
 Devise and enact a contingency plan to escalate potentially dangerous discussion posts and enhance the learner experience.