Introduction

Ethics is a core requirement both in research and in higher education (HE) institutions (Anderson et al. 2013). While research ethics can be field specific, this paper refers to it as a transversal competency needed across the fields and contexts, i.e. in research, teaching, collegial relations, and society (aligned with Macfarlane et al. 2014). In this context, an individual must make value judgments about what the best or right thing to do is at any given moment and situation. The importance of research ethics could be summarised by ten Have’s statement (2021, p 1):”Within a university setting, ethics education invites students to participate in a specific professional community, grasping what is valuable in being a professional, emphasizing moral obligations towards colleagues and society, and shaping personal character”. This statement could also be transferred to academics and institutional leaders, and applied in the entire higher education system.

In the light of research ethics competencies becoming more imperative in HE context, it is even more striking that research ethics content is kept in the periphery of general academic education, as pointed out by Beever et al. (2021). How can a culture of integrity be developed if only a fraction of courses contains an ethics component (fewer than 5% of courses) and this number has barely increased in the last decade (Beever et al. 2021)? This calls for alternative pedagogical strategies to enhance research ethics education.

Among the effective pedagogical strategies for ethics education, several authors have highlighted the value of ethical dilemmas and cases (Fisher and Kuther 1997; Zucchero 2008; Dahm 2015; Avci 2017), and their discussion in peer groups (Jordan 2013). There are examples of online learning materials on ethics (e.g. Goldin, Ashley & Pinkus 2001; Lu et al. 2010). Still, they are often field-specific (e.g., focusing on engineering or medicine) and more focused on developing the foundational research ethics competencies rather than providing tools for gradually developing competencies to guide others and making decisions in novel contexts. Previous research (Lu et al. 2010; Furberg 2016) also shows that computer-mediated training of transversal competencies may provide good results. In addition, creating opportunities for learners to engage with material online with perhaps limited help from facilitators is now even more urgent in the light of the global health crisis.

However, there is less knowledge about the learning process during research ethics training and how it could be supported. Moreover, evidence-based design principles are rarely provided to guide others to design training materials on research ethics, especially needed for tailor-made training (Clarkburn et al. 2010). In addition, a lot of decisions pertaining to teaching ethics are based on experience, opinion or belief (Löfström et al. 2015; Shephard et al. 2015). Facilitators of ethics training could greatly benefit from evidence-based knowledge about the learning process during ethics training to make pedagogically sound decisions. Our goal was to provide PhD students and researchers with supervisory experience with alternative training material to find new insights and help develop their ethics competencies. Based on the existing literature, we hypothesize that PhD students and more senior researchers would benefit from sharing the same cases and materials as it offers opportunities to form mutual understanding of expectations, which is especially important to cater for more sustainable supervisory relationships (Vehviläinen and Löfström 2014; Löfström and Pyhältö 2017) as well as bringing up the next generation of supervisors, and building the culture of integrity. To guide and monitor the development of research ethics competencies, we designed a Computer-Supported Collaborative Learning (CSCL) ethics resource that would not only focus on the individual researcher (i.e. helping the learner conduct one’s own research ethically), but would also provide competencies to guide others (following a systems approach, see Bertram-Gallant 2011).

More concretely, this paper analyses the knowledge building of collaborative group work through monitoring the learning process and outcomes of groups while using the CSCL research ethics resource, and evaluates the effectiveness of the scaffolding provided by the training. The goal was twofold: to collect insights on the learning processes and scaffolding needs of more expert researchers, and develop a training resource specifically for the needs of this target group. The current article reports results based on data collected during the second iteration of design-based research (DBR).

Theoretical background

Research ethics and integrity is gaining more prominence in HE institutions, still, there is little agreement among academics on how ethics and integrity should be taught and whose responsibility it is (Löfström et al. 2015; Hyytinen and Löfström 2017). Previous studies reflect on certain strategies for effective ethics training, namely cases and collaboration. Case-based learning can be utilised in research ethics, where facilitated discussions in groups about potential or real ethical dilemmas have been effectively used (Jordan et al. 2013). Even small-scale case-based discussions have proven to be beneficial (Clarkeburn 2002) and perceived as more enjoyable than lectures by learners (Kim et al. 2006). Also, collaborative and group work increases students’ awareness and concerns for academic misconduct and develops personal integrity and builds mutual trust (Smith et al. 2005; Cavanagh 2011).

One grounding theory for the current research is the Knowledge Building Theory (Scardamalia and Bereiter 2006), i.e. a collaborative effort of advancing mutual understanding and knowledge. Overall, the group learns by building and sharing knowledge and interacting with the learning environment. Discussion is an important element to synchronise understanding: participants in groups may ask for clarifications, and they may argue to convey their point. The learning process is not focused on obtaining factual knowledge alone: when the teacher or textbook provides additional information, this is evaluated by the group, the understanding may be elaborated and also new ideas may emerge. Groups also collaboratively create ‘epistemic artefacts/objects’, i.e. artefacts that reflect the advancement of knowledge or co-creation of knowledge.

In CSCL, knowledge is a product of collaborative work, and the role of technology is to support collaboration and knowledge building processes that would be challenging to organise without networked communication media and software tools (Stahl 2002). One of the reasons why online environments have proven to be beneficial is that they keep the learners focused on the task and on track (Lu et al. 2010). Initially, it may be necessary to decrease the task complexity but, in the later phases problematizing (Reiser 2004) can be added to guide problem-solving. Online tools provide support with goal-setting, support material, time management and planning, thus increasing group work participation and this may change teacher talking-time (Lu et al. 2010). Therefore, digital support provided in the form of visual/language help as structural scaffolding can diminish the amount of help learners require from the teacher, who is mostly needed to problematize the case in order to support reaching higher levels of understanding (Furberg 2016).

Scaffolding involves providing assistance to students when needed and fading as the learner gains competence (Wood et al 1976). Chi and colleagues (2001) consider scaffolding a critical component in facilitating learning, especially highlighting two components of scaffolding: what kind of support to provide and when. Reiser (2004) outlines two complementary mechanisms of scaffolding: structuring the task (i.e. decreasing the degree of freedom and maintaining focus) and problematizing (i.e. making the learner’s work more ‘problematic’ thus encouraging using previous experience to solve the task at hand). Scaffolding can be designed to target key challenges that learners face, namely sense-making, process management and articulation/reflection (Quintana et al. 2004). This means that, first, content knowledge should be covered as this will help with sense-making. Second, the learner should be supported explicitly on which strategy to use. Third, to help with articulation and reflection, attention should be paid to language, discourse markers and terminology.

The current research and the design of the learning material developed for this study rely on the Knowledge Building Theory (Scardamalia and Bereiter 2006), double stimulation, the zone of proximal development and scaffolding (Vygotsky 1980; Wood et al. 1976). According to the Knowledge Building Theory, groups learn by building and sharing knowledge, and also by interacting with the digital environment (Scardamalia & Bereiter 2006). Double simulation involves first stimulating the learners through task design, and then engaging them in the process of co-creating epistemic artefacts/objects to alter the problem-solving (Vygotsky 1980). The zone of proximal development indicates that people can learn in groups the things they cannot learn alone by scaffolding provided by more knowledgeable others, including peers (Vygotsky 1980).

In this study, we focus on the learning process during research ethics training utilising the CSCL approach. In order to understand how knowledge building takes place and how CSCL supports research ethics learning, we formulated the following research questions:

  1. (1)

    How is the learning process during research ethics training displayed?

  2. (2)

    What are the externally observed and self-perceived learning outcomes resulting from the interaction with the ethics resource?

  3. (3)

    How does scaffolding used during research ethics training support the learning process and achieving learning outcomes?

Methodological framework

The research and resource design was embedded into the pragmatic paradigm and adopted a design-oriented and interventive approach. Within the pragmatic paradigm, design-based research (DBR) is often used (Juuti and Lavonen 2006; Alghamdi and Li 2013). DBR is a systematic research approach focused on improving educational practices in real-life context through design, development, iterations and implementation, and leading to contextually-sensitive design principles and theory development (Bakker 2018). DBR has demonstrated its potential as an approach suitable to both research and design of technology-enhanced learning environments (Wang and Hannafin 2005). DBR emphasises ‘rigour’ and ‘trustworthiness’ over ‘validity’ and ‘reliability’. Rigour means using various sources to investigate the intervention and theories, and trials happening in iterations. The results are not so much generalizable as they are transferable. Trustworthiness is achieved by providing detailed descriptions of the process, being transparent and trackable, and providing opportunities for transferability (Bakker 2018).

This paper reports on the second iteration of a larger DBR. To report on this iteration, we follow Reinmann's model (see Fig. 1), focusing on goal setting (GS), conception (CO), development (DE), testing (TE) and analysis (AN) (Reinmann 2020). This research spanned 3 cycles focusing on development (DE), testing (TE) and analysis (AN) (a detailed description of the process below).

Fig. 1
figure 1

The DBR process applied in the research, based on Reinmann (2020)

Full size image

Description of the resource

The ethics training designed primarily for the more experienced PhD students and researchers with supervisory experience consists of cases that groups discuss collaboratively following a set of tasks. There are 5 cases in total, each pertaining to a different phase of the research process, and handling violations, asking the participants to take the role of a research team leader. The cases are followed by a set of tasks (see Appendix I) and accompanied by a ‘possible solutions’ section – as part of the learning process the participants are asked to compare their answers to the ‘expert opinion’. The ‘expert’ in the resource is an ‘imaginary’ research ethics expert who asks the participants whether they thought of certain ethical principles, stakeholders (with their rights and responsibilities) and possible courses of action (answers provided by the authors). Scaffolding is introduced in the task design including sense-making, process management, and articulation-reflection (Reiser 2004; Tammeleht et al. 2020). The focus is on guiding the team or group through understanding ethical principles and practicing ethical analysis in solving ethical cases. Ethical analysis refers to the process of identifying ethical issues and principles, identifying stakeholders, considering rules, rights and responsibilities, and identifying possible courses of action (Mustajoki and Mustajoki 2017).

Context and participants

The participants were recruited in an Estonian university in 2018–2021. The research participants were PhD students and researchers with supervisory experience. All in all, 36 individuals took part in 4 training sessions in 11 groups (see Table 1 for an overview). The groups were heterogeneous combining participants from five different nationalities and different disciplinary backgrounds.

Table 1 Overview of research participants and data collection
Full size table

As indicated in Table 1, for Cycle 1 all 11 invited participants of a research ethics training session intended for experienced researchers, accepted to join the study (distributed in 2 groups of researchers with supervisory experience and 1 group of more experienced PhD students). For Cycle 2, 10 people who had participated in research ethics training for beginner researchers the previous year, volunteered for a follow-up training. For Cycle 3, 15 PhD students accepted the invitation to take part in the training session.

Research ethics

The research followed the European Code of Conduct for Research Integrity (ALLEA 2017), the Estonian National Code of Conduct (Hea Teadustava 2017), as well as the Finnish National Board on Research Integrity guidelines (2019). As the authors are affiliated with two universities, the Finnish guidelines (National Board on Research Integrity 2019) for research in the humanities and social and behavioural sciences were applied as these outline the process for ethics review and corresponding instruction was lacking from the Estonian guidelines. No ethics review was required since the study did not involve an intervention in the physical integrity of research participants; deviate from the principle of informed consent; involve participants under the age of 15 being studied without parental consent; expose participants to exceptionally strong stimuli; cause long-term mental harm beyond the risks encountered in normal life; or signify a security risk to subjects (National Board on Research Integrity 2019). Participation was voluntary, and the participants were asked for their informed consent prior to data collection. The data were anonymized before analyses.

The DBR process and data collection

Table 1 provides an overview of the research methodology of this study. Following the holistic DBR approach (Reinmann 2020), the aim of Cycle 1 was to test the developed content of the resource and by monitoring the learning process and scaffolding of the group-work, make improvements to the material and prepare the content to be transferred to an online environment. Three groups worked on cases, one consisting of PhD students, two consisting of supervisors. All groups spent about 25 min on the case. The tasks were focused on different topics (see Appendix I), but the first two tasks contributed to achieving the outcome of task 3. One supervisor group had a facilitator, while the other did not (but they could ask questions if necessary). PhD student group had a facilitator most of the time (except for the last task). All these groups worked face-to-face and did not provide an epistemic object other than their group discussion. Based on the lessons learned from Cycle 1, several improvements were made to the resource: a group report template was developed, the material was transferred to an online environment, and visuals were provided to support group discussion. In addition, it was decided to collect learner’s self-reflections and a tool was developed for that.

During Cycle 2 the aim was to scrutinise the learning process during CSCL training. Only structural scaffolding was provided, and this took place by built-in features of the material in the online environment. Evidence of learning was collected in the form of epistemic objects: group reports, group discussions, and self-reflections. At the beginning of the session, groups were provided with the online learning materials and a shared online document with the group report template to be filled. During the session, the group discussions were audio recorded. Groups spent about 35–40 min on discussing the case. To determine the perceived level of understanding from the participants, individuals were asked to reflect on their learning in the self-reflection questionnaire highlighting the ethical principles, stakeholders, possible courses of action of the case, and asking the participants evaluate their learning experience and level of understanding. While group reports and discussions were collected per group, self-reflections were individual and collected as part of the training session. All groups worked face-to-face using one computer per team with one person mostly responsible for taking notes. After analysis the training resource was modified: some visuals were simplified and some scaffolding was included in the task design in the form of additional questions. Online environment was upgraded to improve user experience.

Cycle 3 focused on testing the improved version of the online resource by monitoring the learning process, and the achieved and self-perceived outcomes of the group-work. The epistemic object was the group report accompanied by a group presentation recording. It took groups about 40 min to fill their report and about 10 min to present their case, results and receive comments from facilitators. In addition, participants were asked to fill an online self-reflection questionnaire individually to provide the self-perceived level of understanding. During Cycle 3 the entire training session took place online. First, all participants were in the same ‘online space’ for the introduction and warm-up, then worked in smaller teams, and finally in the same ‘space’ again for group presentations. The epistemic object was compiled collaboratively online (all group members were able to write into the same document).

Data analysis

Various taxonomies have been developed to assess the learners’ level of understanding, e.g. Bloom, Revised Bloom (e.g. Bjelobaba 2020 for ethics training context), SOLO taxonomy, endorsed due to flexibility and ease of use among teachers and students (Biggs and Tang 2007; Hattie and Brown 2004; Hook 2012). For the current study, the SOLO taxonomy was chosen as it is evidence-based, hierarchical, allows evaluation of learning outcomes in HE settings, it is applicable in various fields (Biggs and Tang 2007) and has previously been applied in evaluation of ethical awareness (Löfström 2012). See Appendix II for interpretation of SOLO levels for the current research.

Instrument I: Ethical Case Assessment Grid (ECAG)

To display the learning process, the level of group understanding during different tasks was indicated on the Ethical Case Assessment Grid (ECAG) utilising the SOLO taxonomy (Tammeleht et al. 2019) (see Appendix III for examples). The tasks were focused on different stages of ethical analysis, so each stage had to be analysed separately. Since the learning activities and the group reports were carried out collaboratively, the unit of analysis was the group.

Cycle 1 group work recordings were transcribed verbatim as this was the only epistemic object available about the groups’ learning progress. The transcription was analysed using a deductive content analysis method. Themes were based on the tasks (recognising ethical principles; identifying stakeholders, their rights and responsibilities; and providing courses of action), level of understanding was deducted based on the SOLO taxonomy (see Appendix II), and results were transferred to the ECAG (see an example in Appendix III). From Cycles 2 and 3 written group reports were analysed based on the ECAG. Each report was thoroughly read and the level of understanding for different tasks was indicated on the SOLO taxonomy and then marked on the ECAG.

For group discussions (Cycle 2 and 3) thematic deductive analysis of direct audio recording was used. The reason for that was threefold: firstly, the SOLO taxonomy is simple enough to be used in the classroom for evaluation of student responses on the spot. Secondly, research shows that direct analysis method is ‘cost effective, trustworthy and possibly a superior alternative [to transcription] when used with focus group data’ (Greenwood et al. 2017 p 90) – group discussions would be in similar conditions. Thirdly, expert researchers who are familiar with the content of the training material can do it more time-effectively (Greenwood et al. 2017). Themes for analysis derived from the tasks of the training material (ethical principles—A, stakeholders – B, stakeholders’ rights and responsibilities—C, possible courses of action—D), and the level of understanding was evaluated based on the SOLO taxonomy description (see Appendix II). Recordings were analysed multiple times and notes were made on the themes, their approximate timing (for Cycle 2) and the level of understanding based on the SOLO taxonomy (for Cycles 2 and 3). The learning progress was also visualised as graphs (see Results, Fig. 5).

To analyse the self-perceived levels of understanding, the self-reflection questionnaire that builds on the SOLO taxonomy, was utilised. The observed learning outcomes were based on the groups’ highest ECAG scores. The self-perceived SOLO score was collected from participants via submitted self-reflection questionnaires that were filled individually. The self-perceived level of understanding was used to assess whether the participants evaluated their understanding accurately. In addition to collecting the self-perceived level of understanding, the self-reflection questionnaires included a possible solution to the tasks in the training – the participants were asked to compare the answers provided by their group to the ‘expert opinion’ (this being an important learning opportunity), and also reflect on their personal agreement with the group decision. Self-reflection questionnaires also asked about the perception of the learning experience. We report frequencies, i.e. how many participants provided certain kinds of opinions.

Instrument II: The Scaffolding Framework

In order to analyse the scaffolding techniques and mechanisms a scaffolding technique framework (Tammeleht et al. 2020) was used. The framework has scaffolding techniques and mechanisms outlined by Chi and colleagues (2001), Reiser (2004) and Tambaum (2016). To analyse the effectiveness of scaffolding, the structural scaffolding was compared to the learning process and outcomes. Namely, the levels of understanding (ECAG scores) were measured during different tasks and then evaluated whether the SOLO levels were at least at or above the threshold (multistructural level, 2; see Appendix II). The aim was to see whether the scaffolding offered by the ethics resource provided support to achieve a sufficient level of understanding. It was also important to identify means to improve the structural scaffolding.

Results

Results are presented for each cycle separately as the epistemic objects and aims differed (see Table 1 for an overview). Also, due to the iterative nature of DBR, it is important to take into account that the results of each cycle fed into the refinement and further development of the training resource.

Cycle 1 results (see Fig. 2) indicate that the groups advanced to at least relational level (3) of understanding for all themes. Supervisor group I and PhD students displayed an extended abstract (4) level of understanding during the last task providing courses of action (see Fig. 2) but also extending beyond the current case and considering the greater good.

Fig. 2
figure 2

Achieved levels of understanding by groups in Cycle 1

Full size image

For instance, Fig. 3 (top) illustrates how the PhD student group discussed ethical principles where they exhibit relational level (3) of understanding by exemplifying various ethical principles that may be at stake in this case (respect for autonomy; doing no harm; benefiting others). In addition, an example from Supervisor group I shows ethical sensitivity towards stakeholders and considering the implications of possible courses of action (Fig. 3 bottom).

Fig. 3
figure 3

Illustration of analysis of the PhD student group (top) and Supervisor group I (bottom)

Full size image

It should be noted that during the session both Supervisor group I and PhD student groups had facilitators who provided oral scaffolding. As a result, these groups achieved higher levels of understanding compared to Supervisor group II. Scaffolding analysis indicated that the groups mostly needed goal orientation maintenance, but also benefitted from guiding questions and pumping. It was obvious that the facilitator non-supported group would have benefitted from scaffolding as the group discussion rifted quite far from the original case and there was a lot of speculation.

Cycle 2 results indicate that compiling the group report indeed supported keeping focus and provided sufficient structural scaffolding as no outside scaffolding was provided to groups during this cycle. The initial ECAG (based only on group reports) displayed mostly understanding on the multistructural level (2) (for SOLO level explanations refer to Appendix II), occasionally the relational level (3), as depicted in Fig. 4 (top). On the other hand, the analysis of understanding of stages of ethical analysis based on group discussion (oral) indicated mostly relational level (3) for all groups (see Fig. 4, bottom).

Fig. 4
figure 4

Achieved level of understanding of groups in Cycle 2 according to the group reports (top) and the oral presentations (bottom)

Full size image

Recording analysis indicated (see Figs. 4 bottom and Fig. 5) that the discussion was actually much richer than what was seen in the written report – the groups occasionally exhibited relational and extended abstract level of understanding while it was absent from reports (e.g. in drawing parallels with similar cases they knew, giving more examples, considering various reasons for such behaviour, etc.). This means that not everything is written down during the group discussion. In addition, group discussion seemed to contribute to higher levels of understanding.

Fig. 5
figure 5

Learning process of one group based on the recording analysis

Full size image

During Cycle 2, the participants did not receive any oral scaffolding. All scaffolding was structural and included in the online environment. The groups were asked to compile a written group report and a template was provided. The structural scaffolding consisted of decomposing the task into three sections; questions were provided for the discussion to direct learners, maintain the goal, and highlight critical features.

The self-reflection questionnaire was filled by 7 participants (out of 10). Results showed that participants always found an agreement about ethical principles, stakeholders and possible courses of action in their group. Three respondents evaluated their level of understanding on the level that matched their group’s achieved level. Four people evaluated their level of understanding a bit higher than their group’s achieved level. Almost everyone saw the group as an asset; about three quarters believed it would be easier to deal with ethical issues in research in the future.

For Cycle 3 the entire group learning process, consisting of three tasks and the oral presentation, was analysed. Group report results were indicated on the ECAG, recordings were checked using the thematic deductive direct audio recording method for confirmation of achieved levels of understanding.

Groups’ learning process was indicated on the graph (see Fig. 6) where all groups displayed relational level (3) of understanding during at least one task. As illustrated in Fig. 6, Group 1 showed the highest level of understanding, especially when providing possible courses of action.

Fig. 6
figure 6

Cycle 3 achieved levels of understanding

Full size image

An example from Group I report (Fig. 7) indicates, firstly, that the group had indeed taken roles of leaders of the research group and regarded the entire research process. In addition, they explicated the purpose of the research through ethics arguments (i.e. strive for a greater good and fairness).

Fig. 7
figure 7

Illustration of analysis during Cycle 3 (Group I)

Full size image

Figure 6 also displays that Group 2 had somewhat lower level of understanding as they provided a limited written report (they did not provide much information in the written form). Still, during the group presentation, Group 2 was able to display a relational level (3) of understanding by providing additional points to illustrate their answers.

The self-reflection questionnaire was filled by 13 participants out of 15. Again, the groups had achieved mutual understanding as all respondents claimed they had agreed with the group on ethical principles, stakeholders and possible courses of action. Achieved and perceived levels were compared and they generally matched, as shown in Table 2. Only Group 2 evaluated their understanding higher than it was indicated by their responses Fig. 8.

Table 2 Achieved and perceived levels of understanding for Cycle 3
Full size table

All respondents considered working in a group as an asset, and almost everyone thought it would be easier to notice ethical issues in research after the training. The overall feedback was positive and more than half of the participants had learned something new or surprising. One participant wrote: ‘I really like the group work assignments, not only because it makes completing the tasks easier and livelier, but it also adds elements specific to working in a team—exercises of listening, considering different experiences and viewpoints, mutual respect, etc. It is also a great practice of communicating one's ideas.’

For Cycle 3, one of the authors kept records in the reflection diary on how the groups took the role of research leaders. It seemed that it was difficult for groups to take that role (i.e. guiding the team through a difficult situation and taking responsibility for the team). Four groups out of five dealt with Tasks 1 and 2 from the perspective of a PhD student or a researcher in a team (often using the word ‘they’ to denote the leader and the team). Two groups never displayed any leadership perspective. Two groups started to display their changed role by referring to themselves as ‘we’ and also taking responsibility for their team during task 3.

One group took the leadership role from the beginning and displayed high levels of understanding throughout the training (see Fig. 7). It should also be noted that the provided courses of action differed in perspective depending on whether the team had taken the role of leaders or not, also the achieved levels of understanding tended to be lower if the leader’s role was not considered.

Discussion

Looking at the learning processes, outcomes and scaffolding during CSCL research ethics training gave us insights on how the knowledge building process proceeded, and whether the training material provided sufficient scaffolding to the learners. In order to evaluate the learning process and outcomes of groups, evidence of the learning process and the achieved levels of understanding was collected.

For Cycle 1 there was no written epistemic object of the groups’ learning as we considered an option of not asking the groups to compile a group report. But as the analysis results indicated, the group discussion tended to get side-tracked and speculative, as there was not enough structural scaffolding available. In support of this finding, Lu, Lajoie and Weisman (2010) and Hakkarainen (2009) point out that the written epistemic object helps in knowledge building and keeps the focus of the group work. Thus, it may be important to provide also expert learners with an opportunity to create a written epistemic object that anchors the discussion.

It should be noted though that written epistemic objects may display more superficial and general understanding than is actually achieved, as illustrated in Cycle 2, where the achieved level of understanding was higher according to the oral epistemic object. The knowledge telling and knowledge transformation theory (Scardamalia and Bereiter 1987) explains that experts transform the knowledge before writing it down and tend to write down only one third of what they think and say, which also support the findings. All in all, both the group report and discussion should be part of the epistemic object for more expert learners, and could be used for drawing conclusions of the learning process and outcomes.

Based on the epistemic objects we tracked the learning process of the groups. The results indicate that learners usually displayed multistructural and relational levels of understanding, occasionally also extended abstract level. The design of the ethics resource supports development of research ethics competencies, rather than just obtaining declarative knowledge, by providing opportunities to learn through dealing with cases and problem-solving. Double simulation (Vygotsky 1980) seemed to be effective during the training – first the report template was provided with structural scaffolding, thus stimulating the group with a case and questions. Then the group discussion followed that provided peer scaffolding in the form of asking for clarification, argumentation and keeping focus. The resource provided tools (ethical principles and ethical analysis) and information (support material, possible answers) to help effectively make connections with other contexts, and applying the information and competencies in novel situations in the future, which is a core proposition of the Knowledge Building Theory (Scardamalia and Bereiter 2006). Moreover, the goal was to provide an online resource that would minimise the need for facilitator help so the material could also be used independently.

Based on the comparison between observed and self-perceived levels of understanding, we conclude that learners in the HE context can evaluate their learning outcomes quite accurately. The observed learning outcomes displayed by the epistemic objects coincided to a large extent with how the learners perceived their levels of understanding. The slightly higher self-evaluation may have been the result of a positive group-work experience and the description of extended abstract level in the self-reflection questionnaire. Nevertheless, the learner self-assessment could be used to triangulate the assessment of the epistemic objects, especially in those cases where facilitators cannot follow the discussions.

Scaffolding analysis revealed that there is a need for goal orientation. While experience can help recognise ethical issues more easily, it may also start interfering with the case by participants bringing in lengthy speculations and eventually losing track of the goal. This was especially evident in supervisor groups where there was no written epistemic object. Scaffolding for goal orientation appeared to be crucial. Quintana and colleagues (2004) also indicate that for more expert learners there is a need for process management, rather than sense-making. Process management was provided by pumping, redirecting the learner, maintaining goal orientation, making a fill-in-the-blank requests, asking a leading question or highlighting a critical feature – all of the above are forms of structural scaffolding and can be included in the online resource. The abovementioned scaffolding techniques would also help limit time and scope of the discussion. The need for a written group report became evident as this would help keep focus, orient towards crucial elements and urge the team to reach agreements. Groups discussed the cases and tasks and reached a common understanding, which was then written in the report. Instead of the scaffolding from the facilitator or teacher, peers can provide support as this tends to be in the zone of proximal development of the learners (Vygotsky 1980).

The analysis indicated that the scaffolding provided by the resource helped learners achieve high levels of understanding. To ensure this result, the CSCL ethics resource should be used progressively from one task to the next by filling in the group report as this ensures that ethical principles are identified before stakeholders and their rights and responsibilities are considered before providing possible courses of action. There were examples (see Fig. 5) where groups started to come up with possible courses of action already while discussing ethical principles and displayed lower levels of understanding. But devoting more time on discussing a topic and only providing solutions after considering other aspects of the ethical analysis, the group was able to display an extended abstract level of understanding. This can be related to the strategies used by research ethics experts for solving ethical dilemmas (Löfström et al. 2019) – with simpler cases the experts identify the ethical issue and then proceed to solutions, reflecting back to stakeholders and guidelines. While with more complex cases, the ethical issues are followed by iterative contemplation of stakeholders and guidelines, and only then a solution is provided. The cases in the designed ethics resource were rather complex, so the second strategy may provide higher levels of understanding, and this makes following the ethical analysis steps crucial.

Although group dynamics were not analysed per se, data indicated that not only active engagement but also how groups scaffolded their own learning is important to group achievement. Almost all of the participants considered the role of the group an asset. For example, on many occasions different group members reminded the group of the task, to keep focus and to continue in case the discussion started to become more distracted. Also, peer support was used to elaborate answers, to challenge each other and to ask for explanations.

Conclusion

The results of the study indicate that by utilising the CSCL approach it is possible to effectively support the development of research ethics competencies in HE, and the training format could be considered as an alternative element easily integrated in various courses requiring an ethics component. Based on the results, this section compiles design principles and recommendations for developing research ethics competencies.

Monitoring the knowledge building process during CSCL through only written epistemic objects may not provide enough evidence of understanding, especially with learners of higher expertise levels. Thus, in order to keep track of the learning process, it is advisable to collect evidence of learning through multiple sources, for instance:

  • Written epistemic objects provide structural scaffolding by keeping focus, advancing understanding and decreasing the need for facilitator help.

  • Oral presentations can be considered as part of the learning process and not necessarily presenting learning outcomes, as oral scaffolding (by peers or facilitators) can be provided during presentations.

  • Self-reflection helps triangulate the learning outcomes—usually learners in HE context can quite accurately evaluate their level of understanding.

We also recommend following group discussions or oral presentations. In case of not having access to the group discussion, combining epistemic objects and self-reflection, practitioners may have a more holistic view of the achieved knowledge.

Transferring the material onto an online environment decreases the need for facilitator help during group work. This has become especially important during the pandemic when all the learning and teaching had to be transferred online and facilitator help became limited. Structural scaffolding can be provided by the group report template, which presents questions and topics one by one keeping the group discussion on track. Peer scaffolding is usually provided in the zone of proximal development. This means that collaborative learning can be effectively utilised for contributing to the learners’ levels of understanding by providing appropriate support when needed. In addition, support material can be added to the online resource, which provides the learners with an opportunity to compare their answers to the ‘expert knowledge’ and this contributes to their knowledge building.

Limitations and suggestions for future research

We are aware of methodological limitations of the research. Even though the research team and participants were from various countries and disciplines, the testing and data collection was carried out in one university. In addition, qualitative research methods were more dominant while quantitative data only included descriptive statistics. Moreover, there was no alternative format of research ethics training to the one designed, so it is not possible to say whether other formats would provide different or similar results. In addition, we cannot expect an improvement in behaviour based on only one training session, even though groups displayed gradual development of understanding and achieving high SOLO levels by the end of the training.

The study indicated that group dynamics may have influenced engagement and group achievement. This observation warrants further research. Future studies could also include other universities and research institutions. In addition, data could be gathered using multimodal learning analytics and include quantitative data and analysis methods.