UvA-DARE (Digital Academic Repository) Teachers' implementation and evaluation of design principles for value-loaded critical thinking

An important aim of education is that students learn to think critically about moral values. This study examines how five secondary school philosophy teachers implement and evaluate design principles for promoting value-loaded critical thinking in whole-class dialogues. The five design principles are: address, apply and reason critically about moral values, create intercontextuality, and promote metacognitive reflection. For our evaluation, we first analyzed teachers ’ judgments of the relevance, consistency, and practicality of the design principles. Secondly, we conducted classroom observations to analyse teachers ’ implementation of the design principles. We find that teachers consider the design principles relevant, consistent and mostly practical and that they use a wider range of teaching strategies for each design principle, after participating in this study.


Introduction
Critical thinking is an important aim of education (Davies & Barnett, 2015) and a civic competence (Ten Dam & Volman, 2004).Recently, critical thinking has been conceptualized as a combination of three dimensions (Davies & Barnett, 2015;Santos Meneses, 2020): first a cognitive dimension centred on rigorous, logical reasoning skills, second a metacognitive dimension focusing on self-reflection, self-critique, and higher order thinking skills, and third an ethical dimension concerning morality, ethics and human values.Education in critical thinking should address all three dimensions, and it has been widely recognized that classroom dialogue plays a key role in doing so (Ten Dam & Volman, 2004;Kim & Wilkinson, 2019;Wilkinson et al., 2017).Arguments for a dialogic approach to critical thinking education are that such an approach actively involves students in collaborative meaning-making (Ten Dam & Volman, 2004), which promotes elaboration and reasoning (Howe, Hennessy, Mercer, Vrikki, & Wheatley, 2019;Wilkinson et al., 2017), and that dialogue makes it possible to take the perspectives of others into account (Kim & Wilkinson, 2019).
There is a growing body of research developing approaches to dialogic teaching (Howe & Abedin, 2013;Kim & Wilkinson, 2019).Even though a single synthetic framework is lacking (Howe & Abedin, 2013), most authors emphasize the importance of participants giving reasons for what they are saying, discussing multiple perspectives, listening to and building on contributions of others, and flexible power relations between teacher and students (Howe, 2017;Kim & Wilkinson, 2019).Researchers often offer pedagogical principles to dialogic teaching (that it should be collective, critical, cumulative, etc.) and classroom indicators describing the context and conditions in which these principles are realized (for instance: many students say something during the discussion and students talk more than the teacher) (Alexander, 2008;Kim & Wilkinson, 2019).However, only in a few studies the role of the teacher is analysed.As a result, very little is known about what teachers can do or say to promote critical thinking through dialogic teaching.One reason for this, is that many studies focus on small-group discussions (Howe & Abedin, 2013).The research that analyses teachers' facilitation of classroom dialogue, show that teachers can contribute to dialogue and reasoning by asking open-ended questions, probing reasoning and inviting challenges (Chinn, Anderson, & Waggoner, 2001;Nystrand, Wu, Gamoran, Zeiser, & Long, 2003;Oyler, 2015;Wilkinson et al., 2017).For the ethical dimension of critical thinking, teachers should make moral values part of the curriculum (Veugelers & Vedder, 2003), for instance via moral inquiry, discussing moral dilemmas, asking ought-questions and taking different perspectives (Bleazby, 2020;Cam, 2016;Veugelers & Vedder, 2003;Wong, 2020).However, research also shows that classroom dialogue is scarce (Howe & Abedin, 2013), and that it is hard for teachers to change the way they interact with students and adopt a more dialogic pedagogy (Chinn et al., 2001;Reznitskaya & Gregory, 2013;Sedova, Sedlacek, & Svaricek, 2016).Moreover, little is known about how teachers can facilitate classroom dialogues in which all three dimensions of critical thinking are addressed, nor how they can promote transfer of what is learned in classroom dialogues.Transfer is important, because the objective is that students can apply critical thinking in situations outside the classroom, and previous research has shown that transfer of reasoning skills taught in classroom dialogues is particularly hard to achieve (Reznitskaya et al., 2012).
In this study we evaluated design principles for teaching value loaded critical thinking, in which all three dimensions of critical thinking (cognitive, metacognitive and ethical) are addressed in a dialogic and transfer-oriented way.Value-loaded critical thinking is logically consistent and self-reflective reasoning focused on deciding what the right thing is to believe or do (Schuitema, Veugelers, Rijlaarsdam, & Dam, 2009;Frijters, ten Dam, & Rijlaarsdam, 2008).We focused on how secondary school philosophy teachers facilitate classroom dialogues that promote value-loaded critical thinking in a transfer-oriented way.The design-principles were evaluated in two ways: first, we asked the teachers' judgment on the relevance, consistency, and practicality of the design principles.Secondly, we observed lessons to gain insight into the pedagogical practices used by the teachers used to implement the principles in their lessons.In this way, we hope to gain more insight in the ways teachers can promote value-loaded critical thinking in a transfer-oriented way and into their considerations regarding the implementability of these design principles.

Value-loaded critical thinking
Consistent with Frijters et al. (2008) and Schuitema, Veugelers, Rijlaarsdam, & Dam, 2009 we define value-loaded critical thinking as 'logically consistent and self-reflective reasoning focused on deciding what the right thing is to believe or do'.In this definition the three dimensions of critical thinking can be recognized: logically consistent reasoning refers to the cognitive dimension, self-reflection to the metacognitive dimension, and deciding what the right thing is to believe or do, is the ethical dimension (Davies & Barnett, 2015;Santos Meneses, 2020).In deciding what the right thing is to believe or do, one has to consider the moral values at stake: enduring articulations about the good life and how people should live together (Schuitema, Veugelers, Rijlaarsdam, & Dam, 2009).Value-loaded critical thinking consists of reasoning steps such as identifying relevant moral values (Frijters et al., 2008;Tichy, Johnson, Johnson, & Roseth, 2010), coordinating moral and other values (Nucci, 2015), and considering the perspectives of others (Schuitema, Veugelers, Rijlaarsdam, & Dam, 2009) in order to form one's own moral judgment about the issue at hand (Tichy et al., 2010;Veugelers & Vedder, 2003).

Teaching value-loaded critical thinking in classroom dialogues
Classroom dialogue is characterized as a collaborative inquiry (Alexander, 2008;Reznitskaya & Gregory, 2013) in which the participants listen to each other, build on each other's contributions and work together in search for answers to a contestable, open-ended question (Alexander, 2008;Nystrand, 1997;Reznitskaya & Gregory, 2013).It is considered beneficial when teachers and students share responsibility for the content and process of the dialogue, so that students can express their ideas and learn about the perspectives of other participants (Schuitema, Radstake, van de Pol, & Veugelers, 2018;Wilkinson et al., 2017).
We identify three ways to promote value-loaded critical thinking in classroom dialogues: teachers should address, apply and promote critical reasoning about moral values ().Teachers can address moral values by formulating a value-loaded central question: one that is contestable, does not have a single right answer and raises questions about values and what is desirable, right or just (Reznitskaya & Wilkinson, 2017;Veugelers & Vedder, 2003).Often such questions contain 'ought', 'should' or a moral value (Cam, 2016).Another strategy to address values is to name or highlight moral values in students' contributions or to ask value-loaded follow-up questions (Oyler, 2015).Secondly, applying moral values to engaging and realistic examples is important, because people tend to agree about the importance of moral values such as justice and equality on an abstract level, but only when we consider what moral values mean in a specific situation and what the right course of action is, people tend to disagree about them (Nucci, 2015;Rescher, 2014).Analysing a realistic example in which values are at stake, can invite reflection on various perspectives, different interpretations of values and their relative importance (Nucci, 2015;Veugelers & Vedder, 2003).For this, teachers can introduce dilemma's, suggest real-life examples or thought-experiments to evaluate, or reflect on own experiences of the participants (Rescher, 2014;Rondhuis, 2005;Veugelers & Vedder, 2003).
Thirdly, teachers should promote critical reasoning about moral values, in literature the following strategies for this are described: teachers can advance reasoning about moral values and value-judgments by inviting students to articulate, elaborate, weigh and evaluate reasoning about moral values and value-judgments (Chinn et al., 2001;Oyler, 2015).When students aren't able to do this themselves, even after teacher prompts, teachers might support student's reasoning by elaborating on students' contributions (Oyler, 2015).In a dialogic approach to critical reasoning, it is also important that teachers remind students to listen to and build on the ideas F. Rombout et al. of others, by asking for reactions or reinforcement (Schuitema, Radstake, van de Pol, & Veugelers, 2018).Teachers can invite alternative perspectives into the inquiry by asking for counter-arguments, criticism or alternative points of view (Chinn et al., 2001;Oyler, 2015).When the students don't bring up additional perspectives, a teacher might do so, by explaining the ideas of a philosopher or expressing his or her own thoughts on the matter (Sprod, 2001).Tracking the inquiry is important to make sure that everyone follows the line of reasoning, and by labelling reasoning steps and summarizing contributions, teachers can make the reasoning process explicit and accessible to the participants in the inquiry dialogues (Oyler, 2015;Reznitskaya & Wilkinson, 2017).

Transfer-oriented teaching
Transfer is a person's ability to apply his or her learning acquisitions in a variety of new situations (Peters et al., 2013;Van Oers, 1998).Since this usually does not happen automatically (Bransford & Schwartz, 1999), and students find it particularly difficult to identify what they have learned in dialogic learning contexts (Reznitskaya et al., 2012), even though these are often very meaningful (Kim & Wilkinson, 2019), it is important to consciously apply transfer-oriented teaching strategies that make the transfer of value-loaded critical thinking more likely to occur (Bransford & Schwartz, 1999;Peters et al., 2013).
In order to flexibly apply their value-loaded critical thinking skills, students should also recognize contexts in which value-loaded critical thinking is relevant and transform what they have learned to apply it in a new situation (Van Oers, 1998).For this, intercontextuality is considered helpful: create an explicit connection between two or more learning contexts (Lobato, 2015;Wiig, Silseth, & Erstad, 2018).Teaching strategies to create intercontextuality are: recontextualization (provide additional contexts in which students practice what they have learned), comparison (exploration of similarities and differences between contexts) and generalizations (apply abstract concepts to various contexts, to enrich students' knowledge about the concept and raise awareness of the variety of contexts to which it might apply) (Nelissen, 2007;Rondhuis, 2005).
Another aspect of transfer-oriented teaching is metacognitive reflection: the deliberate monitoring and evaluation of the learning process and learning outcomes (Elshout-Mohr, Van Hout-Wolters, & Broekkamp, 1998).Through metacognitive reflection students become aware of how and what they are learning, which helps them to select, apply or transform learned knowledge and skills in future situations (Elshout-Mohr et al., 1998;Nelissen, 2007).For metacognitive reflection on the learning process, teachers could address strategic and social dimensions of the dialogic inquiry, by asking questions such as: 'how did we proceed in answering this question?','are we making connections to contributions of others?', and 'who is participating in the inquiry?' (Fisher, 2007;Gregory, 2008).For metacognitive reflection on the learning outcomes, teachers could ask students to explain what they learned and evaluate the progress made in the dialogic inquiry, with regard to the sub-questions, hypotheses, solutions, reasons or judgments that were raised (Fisher, 2007;Gregory, 2008).

Quality criteria for educational interventions
In the literature on educational design research, four quality criteria are distinguished for designing and evaluating educational interventions: relevance, consistency, practicality and effectiveness (Nieveen & Folmer, 2013;Plomp et al., 2013).Relevance and consistency are also referred to as validity: relevance or content validity concerns the need for an intervention, and consistency, or construct validity, whether its design is logically consistent and the elements form a coherent whole.Researchers need to be able to present convincing evidence of the validity of their design, before evaluating the practicality (Plomp et al., 2013).Practicality refers to how usable the design is for practitioners in the setting for which it has been developed: can the teachers work with it in their daily practice, in the way that was intended by the developers?Often, a distinction is made between expected practicality and actual practicality, and the latter can only be evaluated when target users have had the chance to work with the intervention.The final quality criterion is the effectiveness of an intervention: that it results in the desired learning outcomes.Measuring actual effectiveness often requires large-scale, time-consuming and costly studies, therefore Nieveen and Folmer (2013) argue that 'it is important not to carry out an effectiveness study, until the intervention is developed to such an extent that it has sufficient potential effectiveness.'For this, an intervention should at least be relevant for the educational need at hand, logically designed and practical in use, which is what we will be evaluating in this study.

This study
The main aim of the present study is to evaluate the relevance, consistency, practicality and implementation of our principles for teaching value-loaded critical thinking through classroom dialogues in a transfer-oriented way.Based on the literature, we formulated five design principles: To promote value-loaded critical thinking in classroom dialogues, teachers should: (1) …explicitly address moral values and value judgments, (2) …apply moral values and values judgments to engaging or realistic examples, (3) …promote critical reasoning about moral values.
F. Rombout et al.Fig. 1 outlines our design and its components: the design principles and the central concepts they are based on, and the teaching strategies with which they are implemented in lessons.
To investigate the relevance, consistency, practicality and implementation of our design principles, we collaborated with five secondary school philosophy teachers over the course of a school year.During this year, the teachers participated in five professional development meetings in which they learned about the design principles, and co-planned and evaluated the implementation thereof in their lessons.The teachers conducted multiple inquiry dialogues in their 10th grade philosophy classes, three of which were videorecorded in order to analyse the implementation of the design principles in the lessons (15 lesson observations in total).In addition to this, we conducted individual semi-structured interviews with the teachers to gain insight in their evaluation of the relevance, consistency and practicality of the design principles.
Our main research questions were: (A) How do teachers evaluate the relevance and consistency of the design principles?(B) How Netherlands, philosophy is an elective subject in secondary education do teachers evaluate the practicality of the design principles?(C) How do teachers implement the design principles in their lessons?

Setting and participants
In the Netherlands, philosophy is an elective subject in secondary education.This is a relevant context for value-loaded critical thinking education, because logic, critical thinking, and ethics are part of the philosophy curriculum, and philosophy teachers frequently use dialogic pedagogies (Marsman, 2010).The participants are five philosophy teachers who volunteered to participate, after a call via the association for philosophy teachers.All five teachers have a master's degree in philosophy and a master's degree in teaching: Arne (male, 5 years experience), Bas (male, 6 years experience), Caroline (female, 15 years experience), Dorus (male, 3 years experience), and Emanuel (male, 9 years experience).The teachers participated with their 10th grade philosophy classes of the pre-university track.The Ethics Committee approved our research proposal before we started recruiting.All teachers signed for informed consent and all parents were informed about the study.

Procedure and data-collection
This study consisted of five professional development meetings over the course of one school year.In each meeting we discussed design principles and prepared their implementation in classroom dialogues in the 10th grade philosophy lessons, which was evaluated in the consecutive meeting.Table 1 depicts the procedure of the study, the data-collection and the topics discussed in each meeting.In the first meeting we discussed the central concepts (classroom dialogue, value-loaded critical thinking, and transfer) and the first design principle about addressing moral values, and we also paid attention to intercontextuality, because the teachers asked many questions about this.In meetings 2 and 3 we introduced value-loaded design principles 2 and 3 and discussed and evaluated corresponding teaching strategies.In the second part of the third meeting, we started with transfer and discussed design principle 4 about intercontextuality, on which we reflected in meeting 4, before introducing design principle 5 about metacognitive reflection.In the final meeting, we reflected on the implemented teaching strategies for design principle 5 and collaboratively evaluated the implementation of all five design principles.Three types of data were collected: firstly, the transcript of the fifth meeting in which we collaboratively evaluated the relevance, consistency and practicality of our design.Secondly, transcripts of the semi-structured evaluation interviews with each teacher, in which we asked about the relevance, consistency and practicality of each design principle.These data were analysed for research questions 1 and 2 about the teachers' evaluation of the validity and practicality of our design.Thirdly, we conducted three classroom observations with each teacher, before the first, around the third, and after the final meeting.The dialogues in these 15 lessons were video-recorded, transcribed and analysed for answering research question 3 about the implementation of the design principles and corresponding teaching strategies.

Analysis
In educational design research, the target users have an important voice in the evaluation of a design (Nieveen & Folmer, 2013), which is why our first two research questions concern the teachers' evaluation of the relevance, consistency and practicality of the design principles.To analyse the data for the first research question, we coded the transcript of the final meeting and the evaluation interviews on relevance and consistency.For each design principle, we sampled all coded fragments in which the teachers said something about the importance, need and coherence of this design principle.We clustered and compared the teachers' comments to formulate an answer to the question whether the design principles were considered valid.For the second research question, about practicality, we conducted a similar analysis to select all fragments in which the teachers commented on the implementation of the design principles.In order to gain detailed insight in their experience, we asked the teachers in the interviews about how they implemented each design principle, what was helpful, what problems they encountered, whether they were planning to do this more often and how they evaluated the implementation of this design principle.
For our third research question, about the implementation of the design principles, we complement the reported teacher evaluations with an analysis of classroom observations.This research question concerns the actual implementation of the design principles in the observed lessons: which of the discussed teaching strategies did the teachers use, which didn't they use, and which additional strategies did we observe?And, how did this change over time: were the teachers using strategies in the second and third observation lessons that they haven't been using before learning about them in the collaborative design meetings?For this, we analysed the transcripts of the 15 observation lessons.To analyse the value-loaded design principles (DP 1, 2 & 3), we went through two rounds of analysis.First, we identified all teacher turns containing references to moral values or value judgments.We used the coding act for value-loaded teacher turns (Table 2) to assign a code to all teacher turns in the transcripts: value-loaded or not value-loaded.This enabled us to count the (relative) number of value-loaded teacher turns as a percentage of the total number of teacher turns in each transcript.To determine inter-rater reliability, 5 of the 15 transcripts were coded by two independent raters and with a Cohen's Kappa of 0.891 and an interrater agreement of 95,12 %, this proved very reliable (Landis & Koch, 1977).The main aim of this round of coding was to identify all relevant turns for further analysis in the second round.
Since, the amount of value-loaded teacher turns does not shed light on how the teachers are implementing the design principles, we thereafter asked ourselves for each value-loaded teacher turn: what is the teacher doing here, to which design principle does this  belong, and which teaching strategy is this?We went through several rounds of verifying our coding, critically discussing difficult cases and teaching strategies that weren't described in our theoretical framework.This round resulted in a list of observed teaching strategies for design principles 1, 2, and 3 and a revised way of looking at the implementation of design principle 2: rather than listing teaching strategies, we make a distinction between the type of examples (real-world example, thought-experiment, personal experience and shared experience) and the kind of questions teachers asked about those examples.
For our analysis of the implementation of the transfer-oriented design principles, we had to go back to the video-recordings of the lessons.The transcripts of the dialogues did not suffice, because the teaching strategies for intercontextuality and metacognitive reflection are not only applied during the dialogues, but also before or afterwards.For each lesson, we summarized the learning activities and the questions, concepts, and contexts that were discussed in each activity.To identify teaching strategies for intercontextuality (DP4), we asked ourselves: is the teacher introducing a new context, or taking up a context introduced by a student, and what question is the teacher asking about it?For this, we defined a context as a situation or example with a particular kind of action, place, participant(s) or issue.In a new context, one or more of these variables differed from the original context.For metacognitive reflection (DP5) we looked for learning activities in which the teacher initiated or facilitated reflection on the learning process or the learning content.Again, we went through several rounds of verifying our coding, critically discussing difficult cases and teaching strategies that weren't described in our theoretical framework, which resulted in a list of observed teaching strategies for design principles 4 and 5.

Relevance and consistency
This section addresses the first research question on the teachers' evaluation of the relevance and consistency of our design principles.With regard to relevance, it should be noted that the teachers weren't familiar with the concept value-loaded critical thinking at the start of this study.After learning about it, they recognized this as an important learning objective, not just for the subject philosophy but for education in general and especially for this age group.Dorus for example explained that this learning goal fits students of this age group (10th grade) because they are developing their own values and morals.Although the teachers thought their philosophy classes were contributing to students' value-loaded critical thinking, they could not explain what aspects of their teaching were helpful or effective for this.Therefore, the teachers felt a need for more explicit knowledge about promoting valueloaded critical thinking in dialogues, which the value-loaded design principles 1, 2, and 3 provided.Also, since the teachers consider value-loaded critical thinking an important learning objective that students should learn to apply flexibly in a variety of contexts, also outside of philosophy classes, they agree that this should be taught in a transfer-oriented way, as described in design principles 4 and 5.
For the teachers' evaluation of consistency, we take a closer look at the value-loaded design principles first.Design principles 1, 2 and 3 about addressing, applying and critical reasoning about moral values explain the different elements a teacher has to keep in mind when teaching value-loaded critical thinking, as was recognized by all teachers.Design principle 1 serves to make a dialogue valueloaded, to take a normative approach.However, the teachers disagreed about what this exactly meant: should this involve explicitly examining (the meaning of) relevant moral values or is it enough to ask normative questions about students' value-judgments?As Bas put it: 'moral values shouldn't be the object of the inquiry, but aspects to consider when examining a topic.' (Bas) He preferred to ask ought-and should-questions about students' judgments, whereas Dorus thought it helpful to explicitly discuss values such as welfare and justice.With regard to the design principle 2, the teachers were unanimous about its importance: Emanuel explained that: 'rich, realistic applications show the students that something is at stake, and that the philosophical inquiry is not just a theoretical endeavour, but has real life relevance.'(Emanuel) Both design principles helped to realize design principle 3, which all teachers considered the most important design principle: to promote critical reasoning about moral values.In the final meeting, the teachers concluded that critical reasoning involves producing criticism (debating arguments pro and con, or challenging the reasoning of others), as well as critique (inquiry into assumptions, meanings, implications of values and arguments).Moreover, as Arne and Emanuel said, this design principle is a reminder to promote critical reasoning of students, rather than giving a critical response themselves, which is consistent with a dialogic approach to teaching.However, according to Bas and Dorus, with an inexperienced group of students it is important that the teacher also models giving critical responses.The teachers considered the three value-loaded design principles a consistent whole, not consecutive steps to follow, but necessary elements of teaching value-loaded critical thinking in dialogues.
When we discussed design principle 4 about creating intercontextuality, the teachers compared this to Rondhuis' (2005) idea about moving back and forth from the abstract to the concrete, to which all of them were already familiar and that they considered important for teaching ways of thinking and abstract philosophical concepts.As to design principle 5 about metacognitive reflection, the teachers agreed reflection is particularly important when it concerns the skills needed to engage in dialogues, think critically and evaluate one's own opinion.However, this certainly was not something the teachers were already doing, but they thought they should be doing more often.
In summary, the teachers' evaluation of the relevance and consistency of the design principles was positive: they thought that there is a need for these design principles and that they form a coherent framework that fits their daily practice.Also, the teachers were F. Rombout et al. willing to apply this in practice, and moreover, they considered it important to do so.

Reported practicality
The teachers' judgment of the practicality of the design principles (second research question) was over all positive, they experienced few problems implementing the design principles and teaching strategies in their lessons.Here we report the themes that came up in the interviews.
About design principle 1, on addressing moral values, the teachers reported that this was the biggest shift in thinking for them, but that the design meetings and co-planning helped them to make this shift.Bas said: 'the question whether students could trust us teachers invoked a lot of discussion.This was the first time I approach epistemology in a value-loaded way and it showed me that this even works for more theoretical domains of philosophy such as epistemology and philosophy of science.' (Bas) On the other hand, when we discussed design principle 2, about applying moral values, most teachers recognized that they were already working with concrete examples quite often, before participating in this study.However, the conceptual framework proved a helpful tool for selecting different kinds of examples, and formulating questions about them (see also Table 5).Bas and Caroline, for instance, co-planned to examine students' own experiences and discussed how to ensure a safe learning environment during classroom dialogue.
With regard to design principle 3 about critical reasoning, the teachers reported that this was practical to implement, because what was addressed in the meetings closely resembled what they were already doing.However, their repertoire was expanded with additional strategies, which enabled them to vary more.The hardest part of implementing design principle 3, according to the teachers, was to balance critical reasoning with an open and inclusive dialogue.As Bas explained in the interview: 'It is our task as teachers to track the reasoning process and reveal inconsistencies, because initially the students don't do that themselves.To teach this a more directive type of facilitation is sometimes required.While, on the other hand, inviting different students to contribute to the inquiry, respond to one another and come up with their own examples, flourishes in a less structured and more open dialogue, with a less directive role for the teacher.' (Bas) In the final meeting, the teachers concluded that there is no 'one size fits all'-solution, since particularities of the group, teacher, topic, et cetera, all influence what is needed, and balancing this is part of the craft of facilitating dialogues.However, the concepts, design principles and teaching strategies were helpful tools to make the inquiry either more critical or more open and inclusive.For instance, Arne rephrases whatever he wanted to say as a question to students first: when he wanted to challenge a student, he asked the group whether someone could challenge what was just said.But he had to remind himself to keep trying, because it took a few classroom dialogues before the students started asking questions to each other and take up their responsibility.
For design principle 4, about intercontextuality, the teachers reported something similar as for design principle 2: this was very easy to implement because it resembled what they were already doing, but the conceptual framework enabled them to put this into words and make more deliberate decisions about intercontextuality.Arne and Caroline mentioned that they usually did not create intercontextuality within a single lesson, but preferred to do so over a series of lessons.However, the teachers were more divided on design principle 5 about metacognitive reflection.Arne and Caroline tried this out in their lessons, and they reported positive about this.Caroline prepared students for the reflection before engaging in the dialogue and she judged this beneficial for the students' participation in and the quality of their contributions during the dialogue and reflection.Her reflection exercise required some preparation and good time-management during the lesson, but she definitely had the intention to do this more often.Dorus experienced lack of time: he wasn't able to discuss the reflection assignment with his students.Emanuel considered this design principle not practical enough yet, he would have liked more examples of exercises for metacognitive reflection or questions to ask.
In sum, the teachers reported that the design principles were mostly practical in use, but they acknowledged that it was nonetheless challenging to facilitate inclusive dialogues while warranting high quality critical reasoning, even for experienced teachers.

Implementation in observed lessons
The third research question concerns the actual implementation of the design principles in the observed lessons.The results are discussed in two parts, concerning first the value-loaded design principles (1, 2 and 3) and second the transfer-oriented design principles (4 and 5).

Table 3
Value-loaded teacher turns in observed lessons (percentage of total, amount of value-loaded and total number of teacher turns).

Implementation of value-loaded design principles
Table 3 and Fig. 2 show the amount of value-loaded teacher turns in the three observed lessons for all teachers.A first thing to notice is that Arne, Caroline and Dorus, already made value-loaded contributions in the first observed lesson, which was conducted before the first collaborative design meeting (except for lesson D1, which was recorded just after the first collaborative design meeting).This confirms that reasoning about moral values is not completely new to these teachers.In addition to this, these results show more value-loaded teacher turns on average in the second and third lessons, than in the first lesson.This is a first sign that the teachers placed more emphasis on value-loaded critical thinking as they learned about it in the design meetings.Thirdly, we observed more value-loaded contributions in the second lesson and a slight decline in the third lesson, which was observed after meeting 4, especially in the lessons of Bas, Dorus and Emanuel.This could be explained by the fact that the value-loaded design principles were discussed in meetings 1, 2 and 3, and meetings 4 and 5 focused on transfer-oriented design principles.So, in the third lesson the teachers had additional design principles to consider.
However, the amount of value-loaded teacher turns does not shed light on which teaching strategies the teachers are using: what are they saying in those value-loaded turns?Table 4 shows the results for design principle 1 about addressing moral values.This table shows that the teachers were using additional teaching strategies after learning about this design principles in the meeting.Four of the teachers were already asking value-loaded follow-up questions to their students, but Arne and Dorus in particular used additional strategies in the later lessons.Two of these strategies were newly developed in the meetings: both Arne and Dorus explicitly told their students that value-loaded critical thinking was a learning objective for the dialogue.In addition to this, Dorus explained during the dialogue more elaborately what value-loaded critical thinking is and why it is important.Moreover, these results are consistent with the remarks of Dorus and Bas on the consistency of this design principle: Dorus frequently named and examined the meaning of moral values, whereas Bas never did this.
Table 5 depicts the results for design principle 2 about applying moral values.We show the type of examples the teachers used, and the type of question they asked their students about these examples.In the lessons before meeting 2 in which this design principle was discussed, we only found real-world examples, and in the lessons afterwards, we also observed thought-experiments and personal experiences of the teacher and students.
Additionally, some teachers combined design principles 1 and 2 (in lessons A1, A2, A3, B3, D1, and D3), and then the example functioned as the central topic of the inquiry.Oftentimes the teachers introduced this with a text or video, gave the students time to think about the example (individually or in small-groups, with or without a clear assignment), before discussing it with the wholeclass.In the other lessons, the examples were introduced during the discussion.
The results for design principle 3 about promoting critical reasoning are presented in Table 6.See Appendix A for a more elaborate version of this table, with examples of the listed teaching strategies, because these aren't described that elaborately in the literature and because the teachers specifically asked for this.These results show that some teachers applied teaching strategies in the lessons after the relevant meeting that we did not observe previously.This holds for weighing different values (A, C, D), asking for criticism (C, D, E) and asking students to track the inquiry (A, B).Also, some teachers implemented teaching strategies in observation lessons 2 and 3 that were suggested by other teachers during the meetings.Examples were: evaluate value-loaded arguments (new to D and E), question the meaning of values (new to E), remind students to build on the contributions of others (new to C and D), ask students for reinforcing arguments (new to D), explain the position of a philosopher (new to E), locate or name value-loaded arguments (new to A and E).This indicates that the teachers were able to implement new teaching strategies for design principle 3 and that they learned from each other's experiences.Additionally, the results suggest that the teachers had previous experience with promoting critical reasoning in classroom dialogues, since they already employed a variety of strategies for this in their first observed lesson.This is Fig. 2. Percentage of value-loaded teacher turns per lesson.
F. Rombout et al. consistent with the reported practicality: the teachers said that they were already promoting critical reasoning about moral values, but they had learned additional strategies in this study.

Implementation of transfer-oriented design principles
Table 7 shows the results for design principle 4 about creating intercontextuality.A first thing to notice is that Table 7 contains two teaching strategies that weren't described in our theoretical framework: both Dorus and Arne presented multiple contexts before the start of the dialogue, but selected only one of those to investigate further in the inquiry.During the dialogue the teacher and students referred occasionally to one of the other contexts.Another strategy that we did not mention before in our theoretical framework, was to

Table 4
Teaching strategies for design principle 1 -address moral values (teaching strategy, observed in lessons before relevant design meeting, observed in lessons after relevant design meeting).

Observed teaching strategies
In lessons before relevant meeting In lessons after relevant meeting  * A is teacher Arne, A1 refers to the first observation lesson for Arne, B2 is the second observation lesson for Bas, and so on.

Table 6
Teaching strategies for design principle 3 about critical reasoning about moral values (teaching strategy, example from observed lesson, observed in lessons before relevant meeting and observed in lessons after relevant meeting).make explicit connections with concepts and arguments that were discussed in previous lessons: Arne did this by asking his students to analyse the example about human enhancement using the ethical theories they learned about in the lessons before.All teachers implemented design principle 4 in their lessons.Dorus and Emanuel already did so in their first lesson, although for Dorus it is difficult to say whether this is his regular practice or something he learned, since his first observed lesson took place shortly after the first design meeting in which intercontextuality was discussed quite elaborately.Bas and Caroline, on the other hand, implemented this design principle after learning about it in the third meeting: we identified recontextualization and follow-up strategies such as comparing contexts or using generalizations in their second and third lesson.And for Arne, the results were consistent with what he said about intercontextuality in the interview: he preferred to create intercontextuality over a series of lessons rather than within a single lesson, which is why he did not introduce additional contexts to discuss (recontextualization).Table 8 shows the three examples of metacognitive reflection (design principle 5) that we observed in the observation lessons.Similar as for design principle 3, we present examples in Appendix B. We only observed three activities for metacognitive reflection: in Caroline's third lesson, after discussing this in meeting 4, and in Arne's second and Dorus' first lesson, both of which took place before metacognitive reflection was discussed.All three teachers implemented the design principle in a slightly different way: Caroline's reflection focused on the dialogue and how that went, Arne focused on the content of the discussion, the arguments for and against, and also, similar as Dorus, on the students' own ideas and how these changed after participating in the dialogue.Since Arne and Dorus were able to facilitate reflection exercises before we discussed this collaboratively, and because both Caroline and Arne reported that they were rather pleased with their reflection exercises and are motivated to do this more often, we infer that it is doable to implement design principle 5.That we did not find more examples in the lessons, can be explained by the fact that this design principle was first discussed in meeting 4, leaving only the third observation lessons (five in total) to record this.

Conclusion and discussion
In this study, we evaluated five design principles for promoting value-loaded critical thinking in dialogues in a transfer-oriented way: address, apply and promote critical reasoning about moral values, create intercontextuality and engage in metacognitive reflection.We analysed how five teachers evaluated the relevance, consistency and practicality of the design principles, and how they implemented the design principles in their lessons.Regarding the first research question, we conclude that the teachers judge the design principles relevant and consistent: there is a need for these design principles, because value-loaded critical thinking is an important educational objective and more insight is needed in how to teach this effectively.Also, the five design principles were considered logical, to complement each other, and cover the most important aspects of teaching value-loaded critical thinking in a transfer-oriented and dialogic way.As to the second research question, the teachers consider most design principles practical to implement in their lessons.The first design principle, addressing moral values or making the dialogue value-loaded, was quite a shift in thinking since this was not a central aspect of their teaching before.However, it was a manageable shift according to the participating teachers and something they were motivated to do more often.About design principle 3, promoting critical reasoning about moral values, the teachers said that it can be hard to balance this with an open, inclusive and non-authoritative way of facilitating the dialogues.The teachers were unsure about whether to prioritize rigorous reasoning or participation and why.And about design principle

Table 7
Teaching strategies for design principle 4 about creating intercontextuality (teaching strategy, observed in lessons before relevant meeting, and in lessons after relevant meeting).* A is teacher Arne, A1 refers to the first observation lesson for Arne, B2 is the second observation lesson for Bas, and so on.** Lesson D1 took place in the week after the first design meeting, in which the theoretical framework was presented and in which we elaborately discussed intercontextuality.So Dorus was already introduced to this design principle when we observed his first lesson.However, we did not discuss specific teaching strategies in great detail.

Table 8
Teaching strategies and examples for design principle 5 about metacognitive reflection (teaching strategy and lesson in which this was observed).

Teaching strategy Lesson
Reflect on the process: on social and cognitive dimensions of the dialogue C3* Reflect on learning outcomes: on the results of the dialogue A2 Reflect on self: students' opinions, judgments and how these changed D1 * C3 is the third observation lesson for Caroline, A2 refers to the second observation lesson for Arne and D1 tot Dorus' first observation lesson.
F. Rombout et al. 5, on metacognitive reflection, two teachers reported that this was not yet practical enough, because examples of teaching strategies and reflection activities were lacking.Below, we will discuss these three design principles in more detail.For the other two design principles no practicality issues were reported.
Regarding the third research question, about the actual implementation of the design principles in the observed lessons, we found that the teachers were using a wider range of teaching strategies for each design principle, after they participated in the collaborative meetings.Moreover, the teachers were not only using teaching strategies from the literature that were suggested in the meetings, but had also developed their own ways to translate the design principles into practice, such as explaining the importance of value-loaded critical thinking and making connections to ethical considerations from other lessons.

How to address values?
With regard to design principle 1 about addressing values, we identified two different approaches: asking for students' valuejudgments with normative questions containing ought or should (not), and, on the other hand, naming and specifying the meaning of relevant values, such as justice or wellbeing.Some teachers employed both methods, but others were hesitant to spent too much time and energy to discussing the meaning of values.Cam (2016) distinction between conceptual tools and reasoning tools sheds light on this: students should learn to articulate their own values, clarify its meaning, analyse and compare values of their own of others (Cam, 2016;Veugelers, 2011).For this, a developed and nuanced understanding of 'the language of morals' is required (Cam, 2016), this is what Cam refers to as conceptual tools needed for ethical inquiry: clarification, distinction making, defining, classification, etc. Strategies such as naming and examining the meaning of relevant values, address this conceptual dimension.On the other hand, students should develop moral reasoning tools (Bleazby, 2020;Cam, 2016;Veugelers, 2011): to make inferences, weigh options and evaluate moral arguments.Asking students about their value-judgments and probing reasoning about their positions, can address this dimension.A balanced approach to addressing moral values is aimed at developing both reasoning and conceptual tools.

Balancing participation and rigorous reasoning
The difficulty described by the teachers with principle 3, to balance rigorous reasoning with maintaining equal participation, relates to a central topic in literature on dialogic teaching (Schuitema, Radstake, van de Pol, & Veugelers, 2018;Howe & Abedin, 2013): it is considered beneficial for learning when students elaborate their ideas and comment on those of others (Alexander, 2008), and more student-control over the dialogue can increase student engagement with the discussion (Schuitema, Radstake, van de Pol, & Veugelers, 2018;Chinn et al., 2001).However, stronger teacher guidance is thought to be important for the content quality of the dialogue (Schuitema, Radstake, van de Pol, & Veugelers, 2018;Wilkinson et al., 2017), which is consistent with what the participating teachers taught.According to Reznitskaya and Wilkinson (2017), teachers should gradually release responsibility for dialogue to guide students toward independence: with unexperienced students the teachers strongly guide the dialogue, through modelling and giving instruction, and as the students gain skill, the teachers shift to guided practice in which (s)he is 'procedurally strong but substantively self-effacing' (Splitter & Sharp, 1996).The teacher encourages norms of reasoned discourse, asks questions and prompts students' reasoning and models when needed.Eventually the group might be able to conduct dialogic inquiry without much teacher support (Reznitskaya & Wilkinson, 2017).An underlying principle to this approach is that students are potential sources of knowledge, but that the teachers are experts in value-loaded critical thinking and that they should use this to develop students' expertise (Reznitskaya & Gregory, 2013).The teachers' remarks and classroom observations in this study are consistent with this model, as the teachers are trying out different strategies to release responsibility to students and take students' contributions seriously.An advice to teachers, in order to make informed decisions about which aspect deserves more attention, is to evaluate participation patterns and reasoning quality in their classroom dialogues, preferably in collaboration with students.

Metacognitive reflection
The fifth principle about metacognitive reflection is the only one that was not directly implementable according to two teachers.Insights from research on professional development of teachers can be used to make reflection more manageable for teachers: Sedova et al. (2016) emphasize the importance of phasing when teaching something as complex as dialogic pedagogy (Sedova et al., 2016).Phasing is the gradual introduction of elements, in this case of value-loaded critical thinking.For example, a teacher can start by explaining the importance of listening to the contributions of others and reflect on this with the students after the dialogue.In a next lesson another element is added and the group reflects on both elements, and so on: adding elements each time, and dropping aspects that go well.In this way, teachers can introduce elements of the three categories of metacognitive reflection (self, learning content and learning process) step by step, in a way that is more manageable and for them and for students.
A more methodological consideration is that our definition of metacognitive reflection might be too narrow to capture all relevant examples.Particularly, teacher feedback on students' contributions during dialogue, can also make students aware of their valueloaded critical thinking competencies (Howe & Abedin, 2013;Sedova et al., 2016).

Limitations
A limitation to this study is that we only worked with philosophy teachers, whom we expect to be more experienced in dialogic teaching and promoting critical thinking about moral values than most other teachers.Therefore, the insights on the implementability F. Rombout et al. of these design principles might be hard to generalize to a broader group of teachers.Previous research shows that it is difficult for teachers to implement a dialogic pedagogy and change the way they interact with their students, especially if their regular practice doesn't resemble this (Chinn et al., 2001;Reznitskaya & Gregory, 2013).However, we expect that the list of teaching strategies that was developed during this study, can be a helpful tool to bridge the gap between theoretical principles and classroom practice (Reznitskaya & Gregory, 2013).
Another limitation is that we did not analyse the perspective of students.Research shows that students' attitude to classroom dialogue and the extent to which they consider this an opportunity for learning, is an important influence on their participation and on teachers' facilitation of classroom dialogues (Chinn et al., 2001;Howe & Abedin, 2013).Therefore, the students' perspective could have provided additional insight into the teachers' implementation of the principles and more insight into why some strategies were used instead of others.

Future research and implications for educational practice
All in all, we have shown that the participating teachers were able to implement the five design principles for promoting valueloaded critical thinking in a transfer-oriented way.Moreover, we gained insight into the teachers' considerations about the relevance, consistency and practicality of the design principles, which can be helpful for developing and implementing similar interventions in the future, for instance with other groups of teachers.As such, this study provided convincing evidence for the potential effectiveness of these design principles, so an effect measure could be a next step.Also, these results can have direct implications for teachers interested in conducting classroom dialogues about value-loaded issues: the described teaching strategies provide practical examples of how to facilitate dialogues and how to promote transfer.On a more conceptual level, the design principles and the considerations of participating teachers on relevance and practicality can help teachers to think carefully about the learning objectives of classroom dialogues and how to realize those.

Fig. 1 .
Fig. 1.Schematic representation of our design, with central concepts, design principles and implementation in lessons.

F
.Rombout et al.

Table 1
Procedure of the study and data-collection.Each teacher conducted classroom dialogues with the 10th grade weekly, in which he or she tried out teaching strategies for the discussed design principles.Three dialogues were observed by the first author and video-taped, to analyze the implementation of the design principles and how this changed over the course of the design project.A1 refers to the first observation lesson for teacher Arne, B2 is the second observation lesson for Bas, and so on….**Lesson observation D1 took place in the week after the first meeting, so Dorus was already introduced to our theoretical framework when we observed his first lesson.*** The interviews took place after the final meeting and third observation lesson. *

Table 2
Coding act for value-loaded teacher turns.
Code DescriptionValue-loaded teacher turn The teacher … … refers to or mentions moral values … asks a value-loaded (follow-up) question (ought, should) … responds to or builds on a previous value-loaded contribution … expresses a value-judgment Not value-loaded None of the above F. Rombout et al.
A is teacher Arne, A1 refers to the first observation lesson for Arne, B2 is the second observation lesson for Bas, and so on. *

Table 5
Examples and questions for design principle 3 about applying moral values (types of examples/questions, observed in lessons before relevant meeting and observed in lessons after relevant meeting).