Large Scale Blended Learning Design in an Interprofessional Undergraduate Course in Norway: Context Description and Supervisors’ Perspective

Background: Society’s demands for better coordination of services for children, young people and their families, are increasing. Many professionals learn about collaboration in their speciﬁc professions without having the opportunity to collaborate across study programmes while undergoing pre-service training. Although interprofessional learning (IPL) has been suggested to achieve better services, better outcomes and reduced costs, IPL studies have mainly been restricted to health care. Aim: to describe the transformation of an IPL course with plenary activities into an IPL blended learning course approach (no plenary activities) in conjunction with a quadrupling of the number of students, and to reﬂect on the possibilities and limitations related to structural and outcome aspects. Methods: Design: exploratory cross-sectional study. Students (n=1,401) from teacher educations, health and social care study programmes worked in 196 IPL groups using a combination of digital learning resources and face-to-face discussions, during two days of seminars and through a group submission assignment. Supervisors (n = 39, response rate 61%) evaluated the course delivery through an online questionnaire. Results: Three quarters of the supervisors were highly satisﬁed with the use of ‘student-active learning methods rather than lectures’, and the IPL blended learning approach. They ranked the students’ learning outcomes from the face-to-face activities on campus twice as high as that of the assignment, and three times higher than that of the syllabus. Nearly curriculums in professional studies are strictly regulated, this study may be useful for educators in higher education extending IPL beyond health care. qualitative study found that a wide range of about in terms its collaborative and and its in helping to achieve outcomes linked to IPL The role of the learning and the role of the from role. Future investigate and best possible outcomes


Introduction
Despite high awareness, strong legal status and a generally high level of health and social services, challenges remain before fundamental rights for all children can be secured, even in a rich country like Norway (The Ombudsman for Children, 2019). Children in vulnerable situations face the highest risk of breaches in the fulfilment of their rights (Ministry of Children and Equality, 2017). In Norwegian legal terms, a 'child' means every person under 18 years of age. The term 'young people' is often used for people up to 25 years of age. Challenges relating to modern childhood include obesity, fractured families, and issues related to use of the internet. Interprofessional collaboration (IPC) has thereby been extended beyond health and social care professions to also include professions such as teaching and child welfare (Whiteley et al., 2014;Ministry of Children and Equality, 2017;Norwegian Society of Pediatricians, 2017;Fukkink and van Verseveld, 2019;Strunk et al., 2019).
Interprofessional learning (IPL) is defined by the World Health Organization (WHO) as being present when students from two or more professions learn about, from and with each other in order to prepare them for IPC and to improve collaboration and quality of care (WHO, 2010). IPL is recognised as being essential in the education of future professionals in order to equip them with skills, knowledge and attitudes that enable them to work together in meeting the needs of end users of services (Reeves et al., 2016). Although collaboration is a large part of the job for professionals working with children, youths and their families, many professionals learn about collaboration in their specific disciplines without actually having the opportunity to collaborate across disciplines while undergoing training (Strunk et al., 2019).
The implementation of IPL is a requirement for all Norwegian health and social care education programmes from the academic year 2020/2021 (Ministry of Education and Research, 2017a). The learning outcome descriptors should be in line with the National Qualifications Framework for Lifelong Learning, and on completion of their education, graduates should have achieved the two following shared learning outcomes: 'be able to interact across disciplines, professions, and sectors, and across establishments and levels, and should be able to initiate such interaction' and 'have knowledge of children and young people and, as practitioners, should be able to take care of their needs in terms of treatment and/or services, ensure their participation, and safeguard their rights'. Despite shared learning outcomes, Norwegian institutions are free to choose their learning, feedback and assessment methods if the final knowledge and skills, as defined above, have been achieved. In teacher education programmes, priority is given to three interdisciplinary themes: democracy and citizenship, sustainable development, and public health and well-being (Ministry of Education and Research, 2015). School is an important arena for promoting mental and physical health in children and young people (Hood et al., 2017). However, to the best of our knowledge no large-scaled IPL study has been published including students from both teacher educations, and health and social care study programmes. There is a therefore knowledge gap concerning potential differences in learning outcomes from IPL courses that include students not only from health and social care programmes, but also from teacher educations.

The pilot course
A pilot course was offered to 350 undergraduate students in Spring semester 2018 (Foss et al., 2018). Students attained the study programmes Early Childhood Education and Care, Primary and Lower Secondary Teacher Education, Physiotherapy, Mensendieck Physiotherapy, Nursing, Social Work, Child Welfare and Occupational Therapy. Students from the involved educations was presented for the same teaching, tasks, and syllabus. The pilot course was organised as a combination of traditional lectures (5 hours and 15 minutes in total, with each lecture lasting 30 minutes) and IPL groups (Table 1). After two days of seminars, the students were tasked with individually conducting observations or conversations with a child, collaborating at three 2-hour meetings, and submitting an IPL group assessment. Each IPL group consisted of a maximum of eight students representing study programmes in the fields of teacher education, health and social care. The following work requirements had to be approved: participation in a seminar over 2 days and individually conducted observation or conversation with a child in advance of collaboration in IPL student groups. The group work was completed by handling in a digital assignment through the learning management system (LMS) Canvas. The aim of the group assignment was to link the seminar days, coursework tasks and the syllabus. The students could choose between four different digital formats; written text, Pecha Kucha, digital story and podcast. Each IPL group elected a group member who submitted the group work through LMS Canvas. Staff, master students and professionals from the workplace were trained as supervisors through courses and information meetings. The supervisors assessed and approved/failed the assignments and gave individual feedback to their IPL groups. The assessment was a formative assessment (assessment for learning), and since there was no exams or grades, the course requirement was purposely easy to pass. The aim of the group work and the group assignment was 'Learning for practice' . The total provisional workload was 40 hours.  (Foss et al., 2018); 3 Divided into groups comprising eight students from health, social care and teacher educations study programmes

The blended learning course
The first delivery of the IPL course in the blended learning format was offered to 1401 students in Spring semester 2019 (Table 1). The course was made mandatory to bachelor students taking health, social care and teacher educations study programmes at OsloMet: Early Childhood Education and Care, Primary and Lower Secondary Teacher Education, Physiotherapy, Mensendieck Physiotherapy, Nursing, Social Work, Child Welfare, and Occupational Therapy at OsloMet. The students were divided into 196 pre-determined IPL groups in autumn 2018 with a maximum of 8 students/group. The students preparedness for IPL ahead of the course (Almendingen, Molin and Saltyte-Benth, 2021) and the students course evaluation (unpublished data) will be published elsewhere.
Due to the nature of the implementation, the IPL course description was broad, allowing for adjustments to be made after the pilot. The pilot showed that conversations with children proved to not be feasible due to privacy regulations and the large number of students. Instead, the students had to learn about observation as a method through watching video clips. Moreover, the students would only meet once after the seminar days, if needed.
A case-based blended learning approach in small groups is considered a useful strategy for facilitating IPL since it integrates components of traditional face-to-face discussions with online learning (Curran et al., 2008;Bryan and Volchenkova, 2016;Lillejord et al., 2018). Student-active learning is used as an approach to actively engage the students in the learning process through collaboration and discussion rather than having them passively receive information from their instructors (Lillejord et al., 2018). A socio-cultural constructivist approach is said to facilitate the move from teaching as content delivery to more student-active learning with an increased learning outcome (Vygotsky, 1978;Lillejord et al., 2018). In line with this, it was decided that a blended learning, small group approach would be used in the 2019 delivery, provided through the LMS application.
The revised primary learning objectives (Table 1) and the specific learning objectives related to the individual topics (Table 2), were developed in dialogue with the different stakeholders in autumn 2018. All enrolled students were invited to submit suggestions for the revised 2019 course through a web link embedded in LMS (autumn 2018) in order to improve the learning outcomes and increase student satisfaction. All digital learning material was embedded in LMS together with the student tasks to be carried out. Some digital learning material were produced by staff and colleagues from related fields of work, while others were embedded from elsewhere, such as free online public videos located on YouTube (total time spent watching embedded videos: one hour and 18 minutes, each video should ideally not last for more than 4 minutes).
In the case-based IPL discussions, it was emphasised that the students should 'play their future professional role' and take note of each other's perspectives. Table 3 illustrates how a free online video published on YouTube was used as learning material (Table 3). This particular video shows a dialogue between professionals after a mother has tried to commit suicide, and highlights relevant laws and ethics (Ministry of Children and Equality, 2017). Such case-based discussions did not have a 'correct answer' but were designed to challenge the students to question their own knowledge and motivate them to seek new understanding. This is a student-centred form of teaching, where the students' learning needs are at the centre. The idea is to build knowledge for the future, and the immediate purpose is to create engagement among the students. Some of the IPL activities included a critical component where the IPL groups were asked to discuss a session through, for example, a 'one-minute paper', which prompted the students to reflect on their learning activities. The blended learning design was consciously structured so that it directed the students to complete learning activities that would achieve the learning objectives (Ministry of Education and Research, 2017b).  : The video clip 2 describes a situation that forms the basis for interprofessional collaboration. Think as you would in your future professional situation, and reflect on the following: • How should you cooperate as a team to ensure that the child in the video clip feels looked after, and can thus contribute to and influence decisions that affect his everyday life? • Which ethical dilemmas does this video clip thematise? 1 This is a copy of a page module as it was presented to the students in the learning management system; 2 The casebased video shows a dialogue among professionals after a mother has tried to commit suicide, and has been developed by the Family Centre for Relatives (http://parorendeprogrammet.no/om-oss) and published on YouTube; 3 Used as a starting point for interprofessional interaction as part of a , 45 minutes session entitled "Children as relatives" Separate Canvas modules entitled 'Seminar Day 1' and 'Seminar Day 2' were published on the respective seminar days (January 2019), thus ensuring that the students had the same opportunity to prepare for receiving the content. Each page in Canvas had learning goals related to the particular topic, a short introductory text, digital learning material and student activation tasks (an example is shown in Table 3). The pages had easily recognisable symbols indicating activities. Icebreaker activities were also provided to allow students to introduce themselves and become more comfortable learning from, with and about each other (WHO, 2010). In December 2018, digital learning content was published through LMS Canvas containing the syllabus, additional learning material and practical information. The groups started work on their group assignment on the second seminar day after lunch and could either submit the assignment at the end of that day, or later if necessary. Students had learnt about observation of children and young people during the seminar. Students should for their assignment choose one of four different video examples of children. The group assignment could be submitted in the form of an academic text (max. 3,000 words), a podcast (max. 10 minutes, mp3 format) or a video (max. 10 minutes, mp4 format). The supervisors visited the IPL groups during the second day of the seminars. Supervisors (n = 64) were recruited from staff (n = 36), master programme students (n = 8) and professionals working in the field (n = 20). They were offered supervision courses and information meetings. Because some supervisors were unable to attend the seminars, facilitators (n = 10) visited their IPL groups. All groups had to receive a visit by a supervisor, and supervisors who could not participate during the second day of seminars were asked to meet with their IPL groups after the seminar days.
Interaction between the IPL groups and their supervisors was complemented by email and/or telephone contact. No other major changes were made from the pilot delivery (Foss et al., 2018).

Campus logistics
To put the seminar days into effect, 122 rooms had to be made available in nine buildings at the Pilestredet campus during the first week of January 2019. The nursing students were based at the Kjeller campus (20 km outside Oslo city), while the others were based at the Pilestredet campus in Oslo city. Due to the geographical distance between campuses, transportation by bus was provided for the students from the Kjeller campus. Staff were posted at all entrances on the first day of the seminars in order to help the students with practical issues. Approximately 100 people (staff, hosts, facilitators, supervisors and others) were involved with different aspects of the seminar days.

Supervisor outcome survey
At the end of the course, supervisors were contacted by email and invited to participate in an anonymous online, specially prepared course evaluation. The questionnaire developed was based on earlier questionnaire-based quantitative research using an anonymous self-administrated web survey (University of Oslo, 2020). 'Nettskjema' is a Norwegian tool for designing and conducting online surveys with features that are customised for research purposes. It is easy to use, and the respondents can submit answers from a browser on a computer, mobile phone or tablet. The questionnaire was given feedback on by university colleagues and revised accordingly. The survey included questions focusing on academic content, the use of the blended learning approach in small groups and practical aspects of the course. The question was worded 'how much do you agree or disagree with the following statements, where 0 means "never" and 5 means "to a great extent," i.e. the responses were scored on a Likert scale from 0 to 5. Two reminders were sent.

Data analysis
Data from the supervisor survey were presented as frequencies and percentages with groups of participants compared by a χ 2 -test. All tests were two-sided and results with p-values below 0.05 were considered statistically significant. No adjustment for multiple hypothesis testing was applied as the study was exploratory. Statistical analyses were performed in SPSS v25 (Statistical Package for Social Science).

Ethics
All participants gave their informed consent to participating in the study. Participation was voluntary and anonymous. The questionnaire did not include questions about personal health information or sensitive data. Gender was not included due to the low number of male students. The quantitative data were collected through an anonymous web survey specifically designed to meet Norwegian privacy requirements (University of Oslo, 2020). Data protection was approved by the Norwegian Centre for Research Data (NSD) (reference number: 741649).

Supervisors' evaluation
The response rate was 61% (n=39). Most of the respondents (56.4%) were between 41 and 60 years of age and had backgrounds in the fields of health and social care (43.6%) and teacher educations (56.4%) ( Table 4). Most (69.2%) had supervised 2-3 IPL groups. The majority (69.2%) had met the students at the seminar, whereas 5.1% had not had any contact with them. Slightly more than half (51,3%) reported having had digital contact with the students, and 15.4% had met the students after the seminar days. More than half (53.8%) had been supervisors in the pilot course, 56.4% had completed the supervisor course, 41.0% were present at the information meeting, and nearly 90% had studied both the curriculum and the digital content in LMS.  More than 60% of the supervisors considered the seminar days (both the digital content and the group discussions) to give the best learning outcomes (Table 5). In comparison, only 15.4% reported that the extra group meeting contributed to the best learning outcomes, and only 17.9% reported that the syllabus resulted in the best learning outcomes.  In order to see what the supervisors were most satisfied with, we merged the two highest categories (score 4 and 5) in the items in Table 6, and the following ranking was achieved: 82.0% on 'how the academic content corresponds with the work methods on the seminar days and the assignment for submission'; 81.0% on 'giving individual feedback'; 77.8% on 'student-active learning methods rather than lectures'; 67.8% on 'timetable management'; 66.6% on 'the assignment's relevance to professional practice'; 66.7% on 'cooperation in the IPL groups' and 'the assessment criteria'; 65.7% on 'the content of syllabus'; 63.9% on 'the academic content of the seminar days'; 59.4% on 'the workload'; and 56.7% on 'the level of difficulty'.
To the question 'Do you think the academic content (both the seminar days and the assignment for submission) is relevant to professional practice?' 92.3% of the supervisors answered 'yes' and 2.6%, 'no'. A large majority (92.3%) assessed the academic level of the submissions to be 'as expected' (data not shown). means "to a great extent"': 'All in all, how satisfied are you as a supervisor with'; Unsure = These are not included in the calculation of percentages

Structural aspects
Through the present course, OsloMet has provided a large-scale complex blended learning course for the first time.
The transformation of a traditional IPL course with plenary lectures for 350 students into an approach using blended learning with no plenary activities for 1410 students went well. The transition from two days of seminar with passive learning in large auditoriums into two days of seminar with student active learning in 196 IPL groups placed in 122 different group rooms on campus was therefore manageable on the structural level.
Although the start-up requirements should not be underestimated, the provision of future large scaled courses at OsloMet will be less demanding and more sustainable because the necessary procedures have been established at the structural level. This implies that we solved dominant issues related to previously recognised IPL barriers that originate from the discipline-oriented organisation of education, practical and logistical challenges, and the fact that cross-faculty activities require non-standard administrative and extra financial arrangements (WHO, 2010;Reeves et al., 2016). The involvement of different stakeholders most probably contributed to the present results. The strong support from the management and staff in connection with both the transformation and course provision was crucial, and financial, administrative and academic support made the boundaries easier to cross.
The educational trajectories of health, social care and teacher educations harbour different disciplinary professional identity, culture, tradition, syllabus (Borg and Drange, 2019) etc., all of which may act as barriers for shared learning in IPL courses. OsloMet seized the opportunity to make this IPL course a part of the response to the new shared requirements for health and social care education (Ministry of Education and Research, 2017a). Whether these reforms and pre-service IPL will achieve better services, better outcomes and reduced costs remains to be determined.
Technologically enhanced learning has been widely adopted in IPL . The combination of reusable digital learning materials and face-to-face IPL group discussions on campus turned out to be the preferable structural approach at our University. Achieving digital transformation requires deep changes to both structures and traditions, which takes time (Lillejord et al., 2018). No advanced digital functions or tools that could have jeopardized this first-time course provision were applied. It should be noted that pre-service technologically enhanced learning cannot replace practice-based learning.

Blended learning and learning outcomes
The present study indicate that the supervisors showed high acceptance of the blended learning approach. Three quarters of the supervisors were satisfied with the use of 'student-active learning methods rather than lectures' in the IPL course. The self-administrated timetable management via LMS was positively evaluated, as well as the digital content.
Overall, the supervisors responded that the students achieved the best learning outcomes from the face-to-face activities on the seminar days. The seminar days mainly consisted of face-to-face discussion in the IPL-group. Although supervisors were positive regarding giving individual feedback on the submission assignment, they ranged the learning outcomes from the submission of assignment in the mid-range. Some (20.5%) of the supervisors had arranged to meet their IPL groups after the seminar days in order to help them to finish the submission assignment. The learning outcomes were reported to be weak for this additional group meeting. Reading the syllabus is an individual task, and the learning outcomes were presumed to be weak. In general, it is difficult to generalise IPL outcomes from courses Shrader et al., 2017;Lindqvist et al., 2019). However, these supervisors clearly favoured the face-to-face IPL discussions as compared to other components of the blended learning course.

Formative assessment
The present formative assessment criteria were broad, because the confusion of ideas between different professions is known to be a barrier in some contexts (Green and Johnson, 2015). It is very difficult to assess whether the present supervisors gave comparable tutoring, assessment and feedback, or in other words, whether their supervision varied according to their own professional background and experiences. The IPL course's greatest strength, its interprofessional expertise, can therefore also be a challenge because the diversity of perspectives can lead to differences of opinion regarding how supervision, assessment and individual feedback should be provided. Formative assessment has been found to play a vital role in development and delivery of IPL (Morison and Stewart, 2005;Wagner and Reeves, 2015;Barr et al., 2016). Assessment approaches for IPL are varied, and best practices have not been identified Shrader et al., 2017). More extensive use of built-in tools in Canvas, such as SpeedGrader with Rubrics, would allow the evaluators (supervisors and/or students) to use assessment criteria, systematised through rubrics, in their assessments. One previous study reported that common learning activities and assessment methods could be developed and common standards set, however that study was restricted to health professionals (Morison and Stewart, 2005).

Relevance
There is a considerable use of health and social care services that address children, young people and their families. The vast majority of the supervisors responded that the course was relevant to professional practice. This is a very positive finding since there is a need for better IPC in kindergartens/schools and the health and welfare services (Anderson, 2013;Power et al., 2013;Hood et al., 2017;Martinussen et al., 2017;GDB 2017SDG Collaborators (2017; Fukkink and van Verseveld, 2019).
Some of the present students will not work with children and young people as pupils/students/patients/clients in their future jobs. Students often respond more positively, and more readily see relevance, when they are learning with professionals with whom they anticipate working after qualifying . However, parents may be hospitalized, imprisoned, etc., and thus all candidates probably will meet children and young people as relatives. Norwegian law states that health care professionals should ensure that such children and young people should receive adequate information and follow-up. Children whose parents are suffering from substance abuse mental illness, or somatic disease are at risk of developing psychosocial problems, psychiatric diseases, and somatic diseases (Jakobsen and Christiansen, 2011;Gullbrå et al., 2014;Knutsson et al., 2016). Society's demands for better coordination of services for children and young people are increasing. This course is therefore relevant to all students taking teacher education, health and social care degrees.
Several of the participating study programmes are among the dominant study programmes in Norway. The results may thus also be relevant to other higher education institutions since curriculums in professional studies are strictly regulated. The experiences described here may thus have wider societal relevance. One qualitative study found that university teachers have a wide range of views about IPL, in terms of its potential to enhance collaborative practice and care, and its role in helping students to achieve outcomes linked to IPL (Lindqvist et al., 2019). The role of the 'blended learning supervisor' and the role of the 'interprofessional supervisor' most probably differ from the 'traditional supervisor' role. Future studies should investigate aspects of IPL supervision and the IPL pedagogy style in order to achieve the best possible learning outcomes for the students.

Strengths and limitations
This study has both strengths and limitations. Firstly, the number of respondents was low. However, the response rate was high, and supervisors from all the involved study programmes contributed. Self-selection bias may threaten internal validity, but the diversity in our sample enhances the robustness of the findings. Another limitation is the use of self-reported data. However, the master students and professionals working in the field could be considered external peer reviewers. Strengths include the anonymous sampling of responses and the high number of IPL groups. It is not possible to make firm conclusions based on one single exploratory cross-sectional study from one university. We are not aware of any other published studies that describe the delivery of a large scaled blended IPL course that include students from both teacher education, and health and social care study programmes.

Conclusion
Although further development of digital solutions and methodologies, pedagogics, methods for co-construction, evaluation forms and professional ethics are all needed, this first time delivery of the IPL course in the blended learning format went well. Supervisors were overall positive to the blended learning course delivery, and to use of student active learning methods rather than plenary lectures. The face-to-face IPL group activities was favoured over the rest of the course. Supervisors agreed that students received relevant pre-service training on working with children, young people and their families. Since curriculums in professional studies are strictly regulated, this study may be useful for educators in higher education extending IPL beyond health care.

Take Home Messages
Society's demands for better coordination of services for children are increasing. Interprofessional collaboration should there be extended beyond health and social care professions to also include professions such as teaching Students from teacher educations, health and social care programmes attained a new blended learning interprofessional course Supervisors agreed that the course was relevant for interprofessional practice when working with children, young people and their families Supervisors ranked the students' learning outcome from the face-to-face activities the highest Blended learning with student active learning methods is a feasible approach even in the context of large-scale interprofessional learning