Triangulation in history education research , and its limitations : A view from the UK

This paper explains how different forms of triangulation have been used in recent history education research in the UK, and attempts to assess the strengths and weaknesses of different approaches to triangulation. It also draws attention to the limitations of triangulation as a means of making claims about the validity of research outcomes. In spite of the pronouncements of policymakers in the UK that education reforms will be ‘evidence based’, there are many examples of distortion and misrepresentation in the field of history education research. The paper gives some examples of the ways in which triangulation and mixed methods have been used in research in history education in the UK, and argues that without an underpinning commitment to veracity and respect for evidence, sample size, research approach and range of triangulation methods cannot ensure that reasonable claims are made for the outcomes of research. The concluding section of the paper suggests ways of complementing triangulation as a means of moderating judgements and claims in history education research, and argues that it is important that history teachers have an intelligent and well-informed understanding of the potential usefulness and the limitations of research studies in history education. Although the examples of research cited are from the UK, the question of how to optimize the use of mixed methods in history education research is an important issue for researchers and academics in history education worldwide.


Introduction
In spite of the long history of mixed-methods research (Gage, 1989;Gobo, 2016), and the substantial number of papers published that focus on the application of mixedmethods approaches to research into subject disciplines (Riazi and Candlin, 2014), there are still substantial variations in definitions of mixed-methods approaches to research in the social sciences.Johnson et al. (2007) suggest 19 differing definitions of mixed-methods research and outline some of the connections between mixedmethods approaches, and the concept of 'triangulation' in research, a phenomenon defined by Denzin (1978: 291) as 'the combination of methodologies in the study of the same phenomenon'.Whereas mixed methods is often construed as indicating that a combination of quantitative and qualitative research approaches has been employed (see, for example, Johnson et al., 2007;Kelle, 2006), triangulation of approach does not necessarily indicate that both qualitative and quantitative approaches to collecting data have been used.Triangulation can be achieved by the employment of more than one qualitative or quantitative form of data collection; for example, the use of questionnaires and follow-up interviews.Denzin (1978) describes research that is limited to one paradigm -either quantitative or qualitative -as 'within-methods' triangulation, considering such approaches as of limited value as they would be unable to transcend the respective limitations of the two main paradigms.Thus, in Denzin's terms, questionnaires, numerically/statistically analysed, and follow-up interviews with open-ended questions, would be defined as 'between-methods' triangulation, and interviews with open-ended questions and classroom observation would be an example of 'within-methods' triangulation.Denzin (1978) outlined four forms of triangulation: 1) data triangulation (the use of a variety of sources in a study) 2) investigator triangulation (the use of several different researchers) 3) theory triangulation (the use of multiple perspectives and theories to interpret the results of a study) 4) methodological triangulation (the use of multiple methods to study a research problem.
This paper draws on the broader interpretations of mixed methods and triangulation, that is to say, including studies that do not necessarily encompass both quantitative and qualitative paradigms, and those which rely on within-methods triangulation, as well as between-methods triangulation.The paper will also argue that in history education research, there is a wider range of approaches to triangulation than the four identified by Denzin.
One indicator of the quality of educational research is the extent to which it presents an accurate picture (or 'tells the truth') about the research questions that are being addressed.The philosopher Bernard Williams (2002) makes the point that there are two basic components in the quest for truth: accuracy and sincerity, and that both elements are important.This paper argues that as well as the technical issues involved in research design, and the implications of mixed-methods approaches, consideration needs to be given to the integrity of the researcher/s, and the extent to which they are doing their utmost to 'get at the truth'.(For further development of this point, see Haydn, 2017b.)

Context
Debates about the status of mixed-methods approaches to educational research, and the desirability of incorporating some element of quantitative research into all educational research, have been given extra relevance in England by the decision of the Department for Education to give priority to funding random controlled trials for educational research, as it was claimed that this would provide 'a more rigorous approach towards evidence' (DfE, 2013: n.p.).
Research in education in England has traditionally been predominantly qualitative in nature (Bennett, 2013;Goldacre, 2014), but in recent years the Department for Education has embarked on a high-profile campaign to extol the need for 'evidencebased practice' in education (Gibb, 2017;Gove, 2014).As part of this commitment, the department commissioned Ben Goldacre to undertake a review of educational research, and to report with recommendations for future practice.Goldacre is most famous in the UK for his book and journalism on 'Bad Science', focusing primarily on science, medicine and the pharmaceuticals industry (Goldacre, 2009).The report, Building Evidence into Education, criticized the 'woefully infrequent' use of randomized trials in UK education, and recommended that the department should invest more in studies that had at least some quantitative element (Goldacre, 2013a).While acknowledging that 'qualitative research has a great deal to offer', he argued that, 'it is not good at measuring which interventions work best' (Goldacre, 2013b: n.p.).
The current support for mixed-methods research is not confined to England.Over past decades, the use of mixed-methods approaches in educational research has become orthodox and established, rather than being seen as a novelty or a departure from the norm (Archibald, 2016;Bryman, 2006;Creswell, 2009;Morgan, 2014;Kelle, 2006).Flick (2017: 1) goes so far as to suggest that current thinking about what constitutes the most robust and rigorous research approaches is such that there are 'expectations and requirements on the side of funding agencies that research in many fields should include a combination of quantitative and qualitative methods and a readiness to preferring such projects and proposals'.
Among the claims made for the use of mixed methods is that it offers (in various ways) a degree of triangulation that will add to both the validity and reliability of the information obtained in relation to any research question.In a robust defence of mixed-methods approaches, and the virtues of triangulation, Olsen (2004: 4) argued that 'triangulation plays an important role in good social research', and suggests that between-methods triangulation offers advantages over purely quantitative approaches, both in terms of enhancing the validity of claims, and in terms of offering more comprehensive insights into areas researched.This unconditional approbation for the superiority of mixed-methods approaches is echoed by Webb et al. (1966: 3), who claim that, 'once a proposition has been confirmed by two or more independent measurement processes, the uncertainty of its interpretation is greatly reduced.The most persuasive evidence comes through a triangulation of measurement processes.'The idea underpinning the claim that a range of methods or research approaches will enhance the validity of research findings is also articulated by Arksey and Knight (1999: 23), when they argue that 'cumulatively, the weaknesses of one research method are offset by the strengths of others'.
Although more circumspect in his appraisal of mixed-methods approaches, Kelle (2006: 293) argues that the advantages of mixed-methods approaches 'are twofold', and that 'it can serve for the mutual validation of data and findings as well as for the production of a more coherent and complete picture of the investigated domain than mono-method research can yield'.However, this still raises the question of whether this is a priori the case, or whether the deployment of mixed-methods approaches merely increases the possibility of such advantages.
Other commentators have limited their claims for the advantages of mixedmethods research to the argument that it adds to the richness of data, and can lead to a more nuanced and comprehensive understanding of the issue being researchedwithout necessarily strengthening claims to validity.In the words of Mason (2002: 190), triangulation has the potential 'to encourage the researcher to approach their research questions from different angles, and to explore their intellectual puzzles in a rounded and multi-faceted way'.Flick et al. (2004: 182) also offer more qualified support for triangulation, arguing that triangulation 'is particularly useful for theory-development, when it can elucidate divergent perspectives'.However, in an earlier paper, Flick (2002: 227) also points out some limitations of triangulation, arguing that 'Triangulation ... the combination of different methods ... is less a strategy for validating results ... than an alternative to validation which increases scope, depth and consistency'.There are significant disagreements about the precise or particular advantages claimed for mixed-methods approaches, and triangulation is not without its critics.Sandelowski (2003: 328) argues that its fashionability has led to a position where it is overused to the point where it has become 'a near talismanic method for democratising inquiry'.Richardson and St Pierre (2008: 963) make the point that the advocates of triangulation make an assumption that somewhere, 'there is a "fixed point" or "object" that can be triangulated', in a way that does not correspond to everyday realities.Bazely (2013: 406) also questions some of the claims made for triangulation, noting its often labyrinthine complexity, and suggesting that 'rarely does it provide a clear path to a singular view of what is the case'.Denzin and Lincoln (2013) warn of the dangers of oversimplification in interpreting the influence or impact of mixed methods, and of making excessive claims for the truth claims of such approaches.(For further points about the limitations of triangulation, see also Archibald, 2016;Flick, 2014.)These diverging views raise two important questions about the use of mixed methods and triangulation in research in history education.Does the deployment of a range of approaches to data collection necessarily enhance the validity and reliability of the research findings, and is triangulation primarily about trying to increase insight into a research problem or is it about strengthening claims for the 'warrant' or validity of the research?(And can it be both?) These questions are not designed to advance an argument against the use of mixed methods in history education research.A review of the literature relating to the use of mixed methods in the social sciences suggests that many commentators and researchers see the deployment of mixed methods as offering advantages over a single-stranded approach to research.The idea that there should be a clear distinction between quantitative and qualitative approaches (and, in particular, that quantitative approaches offer a more scientific and superior research paradigm) is often viewed as outmoded and unhelpful (see, among many others, Archibald, 2016;Denzin, 2012;Fielding, 2012;Gobo, 2016;Goldacre, 2014;Onwuegbuzie, 2012).However, it is important to be clear about exactly in what ways, and in which contexts, mixed methods might add to the quality and utility of educational research.The next section examines these two questions in the context of recent history education research in the UK.Goldacre (2013a) is correct in his assertion that educational research in the UK has not frequently gone down the road of randomized controlled trials, and this extends to research in history education.A search for UK history education research projects employing randomized controlled trials over the past three decades did not unearth any results.(Some speculations about the reasons for this vacuum will be given in the conclusions to the paper.)

Triangulation in history education research in the UK
However, there are some examples of research in history education that employ various forms of triangulation (and, in some cases, the use of between-methods triangulation), and that raise interesting points about both the affordances and limitations of triangulation.There are several studies that do not meet all the criteria for randomized controlled trials (for example, in terms of sampling protocols), but that do use a combination of quantitative and qualitative methods (see, for instance, Harris andBurn, 2016, 2018;Haydn and Harris, 2010;Lee et al., 1997).A further point that might be made is the comparative dearth of funded research projects in history education (as in some other educational jurisdictions) as a result of governmental preoccupation with STEM subjects in education (science, technology, English and maths).
The CHATA Project (Concepts of History and Teaching Approaches, 7 to 14), directed by Lee, Ashby and Dickinson in the 1990s was, in terms of citations and international reach, one of the most influential history education research projects in the UK.Its aim was to develop greater insight into students' ideas about history, and the project used more than one method of obtaining data.In addition to focus group triads of students discussing historical scenarios, there was a range of written tasks, asking students from the age of 7 upwards to explain their thinking in relation to various historical episodes.In addition to this between-methods triangulation, the outcomes were analysed and coded using statistics to formulate tentative progression models relating to students' ideas about historical issues such as causation, and the problems posed by differing accounts of the past (see, for example, Lee et al., 1997).This is a useful example of what Kuckartz (2014) describes as 'transfer design', where qualitative data are aggregated into 'types of answer' and then compared to each other in statistical form to see which types of answer are most or least prevalent.The statistical analysis, and the graphical presentation arising from this analysis, enabled the researchers to at least tentatively estimate the proportion of students in different age ranges who were operating at more or less sophisticated levels in terms of their understanding of a range of second-order concepts.Thus, in response to a question about why the Emperor Claudius decided to invade England in AD 43: general policies like extending the empire appeared in 8% of both year 6 and year 7 responses, and 10% of those at year 9; 1.3% of year 6, 4% of year 7 and 7.8% of year 9 responses mentioned mid-range policies like spreading the Roman way of life.(Lee et al., 1997: 240) The inclusion of some quantitative interpretation of the data enabled comparisons to be made across the nine schools involved in the project, to explore the extent to which 'teaching' factors might have an influence on the sophistication of students' thinking, rather than simply student maturation.
The use of student testimony in the published papers provided insights into students' ideas about the past in a very different way.In response to two different 'stories' about the end of the Roman Empire, students were asked to decide which of two possible dates for the end of the empire was most appropriate.'Christopher', a Year 7 student (aged 11-12) stated, 'No one was there when the empire ended so nobody really knows', whereas 'Natasha' (Year 9, 13-14 years old) responded: Because it all depends on your opinion and whether you are thinking about the empire physically ending or mentally ending.If people still thought about the empire and talked about it in 1453, then it had not ended until then, mentally.I think different bits of it ended at different times, i.e. it ended physically when it was all no longer governed by the same person, mentally when it was no longer thought about and spiritually, when there was no longer a shadow of it in people's lives.We still use Latin derivatives now, so the empire's influences haven't ended now.(Lee, 2004: 153) These extracts from student testimony enable the researchers to make a particular point (that there are wide variations in the sophistication of students' understanding of why accounts might differ) in a powerful way, and in a way that would be difficult to achieve using quantitative data.The quantitative elements of the study, which provide insight into variation in the 'levels' achieved by students in different schools, enable the researchers to make at least a tentative claim that differences in the sophistication of students' understanding of some second-order concepts arise from the ways in which students in those schools are taught, rather than simply student maturation.The study shows that some younger students are operating in more advanced ways than their older peers, in terms of their understanding of second-order concepts such as causation and change.Lee (2004) and his colleagues do not at any point claim that the use of more than one research approach adds to the validity of the claims made.The instruments measure different things, and are an attempt to gain insight into different aspects of students' ideas about the way the past works.
Another example of between-methods triangulation can be found in a study on students' ideas about the usefulness of history as a school subject (Haydn and Harris, 2010).Drawing on data from 1,740 student questionnaires and 160 students in focus group interviews, the study aimed to gain insight into students' ideas about why they study history at school.The closed questions in the questionnaire were subjected to statistical analysis, to ascertain how important students felt it was to do well in history, compared to other school subjects, and how useful they thought history was as a school subject.The open questions in the questionnaire and the focus group interviews were analysed to try to find out why students thought that history was or was not useful.Statistical analysis of the closed questions, using SPSS software, showed that students regarded history as less important than maths, science and English (with a mean of 3.26 on a six-point Likert scale, with 5 = Important and 0 = Not important), but more important than religious education and art.Statistical analysis also showed significant variations in the proportion of students who regarded history as useful across the 12 schools, from 80 per cent in one school, down to 62.2 per cent in another.Coding of the open questions also allowed for some analysis of variations between schools in terms of why they felt that history was or was not useful.This suggested that 'school' or 'departmental' effect was a significant factor in students' ideas about the utility of history.
However, the non-statistical outcomes of the research, in terms of examples of student testimony to illustrate the sort of thinking that influenced students' ideas about the utility of school history, provided a different form of insight into their thinking.Many students had naively vocational ideas about the subject, for example: 'I think history can't be used in everyday life, and only specialists -need to know it -authors, historians'; 'I don't know, it might be useful if you become a history teacher' (Haydn and Harris, 2010: 248).Others were politely negative: 'I'm not being rude, but it doesn't actually help you in your daily life'; others more vituperatively so: 'To punish people'; 'To bore us to death'; 'Handwriting and patience'; 'Because pupils need sleep so they made history up'.Others simply said that they did not know why they had to do history: 'I don't really know but some people really enjoy it'; 'I don't have a clue'; 'They don't let you know'.Some students gave idiosyncratic or 'trivial pursuit' type answers: 'to get a bit of ancient stuff into your brain'; 'so that can know a bit more knowledge that you don't use much'; 'it helps you on quiz shows and pub quizzes'.Interestingly, one student felt that it was 'to influence children's brains with patriotism and not to let them have independent views ' (ibid.: 249-53).
These examples are provided to show that student testimony can provide a different sort of insight into students' ideas about the usefulness of school history, compared to the statistical aggregation of data.As with the Project CHATA research Triangulation in history education research, and its limitations 41 cited earlier in this paper, the range of instruments and research approaches was not aimed at enhancing the validity of claims made for the outcomes of the research; rather, it aimed at exploring different ways of trying to get at students' thinking, and at addressing different research questions.The Likert-scale questions enabled comparisons with other subjects, and made possible some conclusions about the scale and prevalence of student indifference to school history.The open questions and focus group interviews provided more insight into why students regarded school history as useful or not useful.
There was also some within-instrument triangulation, a strategy widely used in psychometric testing.In addition to asking students to rate the usefulness of school history on a Likert scale (Question 2), a subsequent question asked students to 'declare a position' on school history, asking them to declare it either 'Useful' or 'Not very useful'.This provided two different ways of trying to get at students' ideas about the utility of school history -and to see whether the outcomes were broadly comparable in terms of face validity, rather than contradictory.The second of the questions also provided a further form of triangulation, in enabling some comparison with two earlier major surveys focusing on students' ideas about the usefulness of history, one conducted in 1967 and one in 1984 (see Aldrich, 1987 for details).
With the usual caveats about the weaknesses and limitations of longitudinal research (for example, the impossibility of isolating all other variables over the period in question), this made it possible to make comparisons about student perceptions of the uses of school history at three points over approximately fifty years.Although all three surveys suggested that there has always been a sizeable proportion of students who think that school history is 'useless', the comparisons over time showed interesting variations.The triangulation over time provides some evidence to suggest that 'new history', with its emphasis on active student involvement in learning and its concern to 'make the subject meaningful for all children and young people by relating history to their lives in the 21st century' (Schools History Project, 2015: n.p.), was considered by students to be more useful than traditional modes of history teaching.(See Slater, 1989 andSylvester, 1994 for a developed explanation of 'new' and 'traditional' history.)Table 1 shows the differing results for the three surveys.Source: Aldrich (1987), Haydn and Harris (2010: 247) Another example of within-instrument triangulation (and longitudinal triangulation) can be seen in Box 1, which shows part of a simple pencil-and-paper test that attempts to assess (among other things) the extent to which 12-year-old students in the UK understand the concept of 'century'.Of course, such an approach to testing the different dimensions and complexities of concepts is more traditionally described in terms of strengthening construct validity, rather than 'classical' triangulation defined in Denzin's (1978) terms, but it is nevertheless a useful reminder to researchers that it can be helpful to ask a question in more than one way in order to obtain an accurate picture of (in this case) the security of student understanding of a commonly used historical term.There is also, of course, the possibility that the responses of some students are due in part to deficiencies in numerical understanding, but there remains the strong probability that the students' understanding of the concept of 'century' is limited, and that this is exposed by posing questions about centuries in different forms.

Box 1: An example of within-instrument triangulation
Holt (1964) argued that that when attempting to assess the extent of learner understanding of particular terms and concepts, it is helpful to ask questions in different forms to check that the learning is not just regurgitation without understanding.A simple pencil-and-paper test of students' understanding of some basic time concepts and terms attempted to assess the extent to which 12-year-old students understood the concept of 'centuries' and were able to use the concept accurately and consistently.This is an instance of triangulation that does attempt to increase the validity of a claim that a student understands (and can operate) the concept of 'centuries': 1) Which century are we living in?
2) If the Battle of Hastings was fought in 1066, in which century did the battle take place?3) Which century were the following years in?
The outcomes of the test showed that only a small proportion of students were able to answer all the questions correctly, and there were frequent inconsistencies in student strategies for calibrating the correct response (Haydn, 2017a).The test was first used in the 1990s, and has recently been repeated to see if there are any significant changes in students' understanding of basic time concepts and vocabulary.The outcomes suggest that in spite of the high profile given to time and chronology in curriculum documentation compared to the 1990s, substantial deficits in student understanding remain.
In addition to longitudinal triangulation, a broader interpretation of triangulation than that suggested by Denzin (1978) can also embrace triangulation by educational jurisdiction or 'geographical' triangulation, which can point to divergences in how the past is approached in schools in terms of national education systems.Returning to the arguments of Richardson and St Pierre (2008), triangulation does not always lead to a narrowing down to a fixed point or object.It can elucidate differences and divergences.As Haward (2017: 56) points out, 'It is this interplay between convergence and divergence that holds promise of a useful paradigm for thinking about data'.Two examples can be used to illustrate this point.Carretero (2017) explores how Columbus's landing in the New World is represented in textbooks, and interpreted by students, across four different countries (using both quantitative and qualitative approaches).The study found a strong correlation between students' ideas and those presented in textbooks in Mexico, Chile and Argentina, but not in the case of Spain.This approach also suggested that factors outside of schools -such as public debate and national media -influence students' thinking.In a similar vein, the EHISTO Project explored the treatment of responsibility for the outbreak of the First World War in popular history magazines in Poland, Sweden, Spain, Germany and England, finding wide variations in how the issue was addressed (Popp et al., 2015).
Another variant on triangulation is that of triangulation by theory, or theoretical framework/s (Van Boxtel, 2017;Van Drie and Dekker, 2013).An example of this approach in UK history education research can be found in Phillips' research on the formulation of the first national curriculum for history in the UK, introduced in 1991.Drawing on Ball's (1990) theoretical framework for researching education policy, which drew on a range of analytical perspectives in order to cultivate 'an applied sociology' to explain policy formation, Phillips (1998) wrote a book to explain why the first national curriculum for history took the form that it did.The book considered the creation of the new history curriculum from three differing perspectives: 'the context of influence (who was in power at the time; how were the key players in drawing up the national curriculum for history selected); 'the context of text production' (who produced the curriculum document; what were the steps and tensions that influenced the final version of the text); and 'the context of practice' (how was the document translated into practice by teachers).Phillips (1998) uses Ball's (1990) notion of 'discourse', to highlight the gap between policy intent and actual implementation.This is another example of what Haward (2017) terms 'divergent' triangulation, where rather than the differing perspectives focusing on an explanatory 'truth', the research provides a more comprehensive and nuanced explanation of why the intentions for the curriculum varied from the outcomes, and the complexity of the factors influencing the process of policy formation in education.The study helps to explain why the original national curriculum for history took the form it did, and helps to explain why it did not turn out as those dominating 'the context of influence' intended.It demonstrates the interaction between triangulation and complexity theory (Bates, 2015).
Other studies that used elements of triangulation were the attempts by the Department for Education to estimate the extent to which computers were being used in schools.These large-scale biennial surveys took place for over a decade, and in addition to trying to find out how much computers were being used in the teaching of school subjects, there was an attempt to evaluate the extent to which the use of computers was improving learning outcomes (see, for example, DES, 1991;DfES, 2002).The surveys asked head teachers and heads of department how much use they made of the computers that had been provided by funding from the department.Responses for history, from both heads of department and head teachers were consistently positive across the decade.The 1995 report (DfE, 1995) suggested that over 20 per cent of history teachers were using ICT 'regularly' (defined as twice a week or more).By 1998, it was estimated that history teachers were using computers in almost one lesson in ten (DfEE, 1998).However, in spite of the element of triangulation in seeking responses from both head teachers and heads of department, there was no attempt to triangulate by asking students how much they used computers in their history lessons.Other surveys that did include 'student perspectives' suggested a much less extensive use of new technology.This disparity illustrates the potential corruption processes that can be involved in research, and the temptation to limit triangulation to strategies that might encourage positive outcomes, given that both head teachers and heads of history departments might both be understandably keen to demonstrate that they were making good use of the computers that the department had provided (Haydn, 2004).
It also makes the important point that a large sample size is no guarantee of accuracy or validity in history education research.
The use of meta-analyses or 'systematic review' offers another form of triangulation in history education research.The high profile of Hattie's meta-analyses of interventions likely to lead to progress in student learning (Hattie, 2009), and the seductive idea that a study of all academic studies in a particular field is likely to provide strong corroboration of the warrant for particular educational approaches, have led to a call in the UK for a 'what works' approach (DfE, 2013;Wrigley, 2016).However, here too there are corruption processes at work.MacLure (2005) and Wrigley (2016) point to the dangers of both researchers and policymakers 'cherry-picking' studies, distorting equivalence and resorting to selective citation in using systematic review.
As an example of this, former Secretary of State for Education Michael Gove (2013) claimed that, 'survey after survey has revealed disturbing historical ignorance' among young people, blaming progressive teaching methods and attempts by Marxist history teachers to undermine democracy for these knowledge deficits.After considerable obfuscation, and under pressure from freedom of information requests, the Department for Education conceded that the Secretary of State had cited polls originating from Premier Inn and UKTV Gold press releases, rather than from peerreviewed academic studies (see Ball, 2013 for a more developed account).
In similar vein, Michael Gove and Minister for Schools, Nick Gibb, praised and popularized an untriangulated and non-peer-reviewed pamphlet by Derek Matthews (2009) criticizing progressive teaching methods in history (Gibb, 2012;Gove, 2011).The highly polemical nature of the pamphlet, the fact that it was not subject to a testretest trial with a group who were not Matthews's students, and the fact that it was not subjected to peer review by a community of professional practice raises the possibility that even if Matthews were to triangulate the research by using individual interviews, focus groups, alternative question choices, participant observation and field notes, visits to schools, interviews with history examiners and teachers, the other instruments might be designed with a view to confirming the researcher's view that 'progressive' teaching approaches in school history are pernicious and ineffective.
This section of the paper has attempted to illustrate some examples of the ways in which triangulation has been used in research in history education in the UK, in order to provide some insight into both the advantages and limitations of triangulation.It also notes the tendency or temptation for policymakers to use research to support the policies and ideas that they wish to promote, irrespective of the appropriate knowledge warrant for the research cited.

Conclusions
Reflection on the use of triangulation in history education research in the UK suggests that there are many examples where the use of more than one research approach has made an important contribution to the validity of research findings, and (to a much more extensive degree) has added to 'breadth and depth of understanding' of a research problem (Johnson et al., 2007: 123).Given the nature of history as an academic discipline, with its concern to discover the truth about the past, to give as accurate a representation of it as possible, and to explore issues and problems as thoroughly and open-mindedly as possible (Evans, 1997(Evans, , 2003)), this is not surprising.For the respectable academic historian, and for history education researchers with integrity, it is almost axiomatic that there should be as thorough, broad, and openminded an approach to considering as wide a range of relevant evidence as possible Triangulation in history education research, and its limitations 45 in response to the research question at hand.A narrow, and in some ways artificial, approach that makes an arbitrary distinction between different forms of evidence, or that disdains to even consider some forms of evidence, seems inimical to rigorous historical enquiry.There is also the point that, as noted in some of the examples given above, a quantitative approach may be more helpful for answering some questions, while a qualitative approach may be more appropriate for others.Some of the studies described earlier in the paper support the claim that mixedmethods approaches can lead to 'richer, thicker data … the synthesis or integration of theories … it can uncover contradictions' (Johnson et al., 2007: 115).Triangulation can in the right hands contribute to the researcher's reflexivity and 'to the realisation that researchers and their methods are entangled with the politics of the social world they study' (Holliday, 2012: 138), or, in the words of Johnson et al. (2007: 129), the researcher is 'cognizant, appreciative and inclusive of local and broader socio-political realities, resources and needs'.
However, triangulation does not guarantee any of these advantages.In discussing the potential of mixed-methods research, Kelle (2006: 309) consistently uses the word 'can' rather than 'does' when explaining the possible advantages of combining qualitative and quantitative research.The use of mixed methods is usually less important to the quality and validity of a piece of research than the integrity and intelligence of the researcher.For all the pious claims about the concern of the Department for Education for the rigour and quality in educational research, there is evidence to suggest that politicians (of all parties) are likely to 'misuse evidence, and distort it to shameful degrees' (Goldacre, 2011: 48).This makes the important point that triangulation is not just a matter of technique, but of integrity, open-mindedness and willingness to explore the negative hypothesis (Popper, 1935).As I have argued elsewhere, although politicians may be the most egregious species in terms of willingness to distort research for their own ends, there are also journalists, scientists, researchers and historians whose desire to prove their hypotheses or lend support to their preconceived opinions and beliefs outweighs their concern for truth (Haydn, 2017b).In an exception to the discourse of many UK politicians on education, former Secretary of State for Education Keith Joseph argued for the importance of veracity as a desired disposition, which the teaching of history might cultivate: The complicated interplay of evidence which is itself not certain and subject to interpretation gives history a particularly valuable part in the development of an adult understanding.It helps pupils to understand that there is a range of questions -be they political, economic, social or cultural -on which there is no single right answer, where opinions have to be tolerated but need to be subjected to the test of evidence and argument.As the pupil progresses in this encounter with history, he should be helped to acquire a sense of the necessity for personal judgements in the light of facts -recognising that the facts often be far from easy to establish and far from conclusive.And it should equally awaken a recognition of the possible legitimacy of other points of view.In other words, it seems to me that the teaching of history has to take place in a spirit which takes seriously the need to pursue truth on the basis of evidence, and at the same time accepts the need for give and take in that pursuit and that teaching in that spirit should encourage pupils to take a similar approach.(Joseph, 1984: 11) The same plea about the need to pursue truth on the basis of evidence might also be usefully applied to researchers in education, and policymakers drawing on that research.Goldacre's (2013a) lament on the dearth of random controlled trials in educational research in the UK fails to take fully into account some of the differences between research that takes place across schools, and that in medicine, science and pharmacology, where it is often easier to isolate particular variables.That is not to say that it is not possible to conduct such a study in education, but that there may be significant differences to the manageability and reliability of such studies.As MacDonald (1971: 12) points out, 'No two schools are sufficiently alike in their circumstances that prescriptions of curricular action can adequately supplant the judgement of people in them.Historical/evolutionary differences alone make the innovation gap a variable which has significance for decision making.' The appropriateness of quantitative approaches is also influenced by sample size.Small-scale exploratory studies, for example an enquiry into how UK schools approach the teaching of the British Empire (Haydn, 2019), which was based on a survey of just 15 high schools, do not lend themselves to measures of statistical significance, whereas the recent national surveys of history teachers in England conducted by the Historical Association, involving several hundred respondents, do make statistical analysis of data worthwhile (see, for example, Harris and Graham, 2019).This last study provides a good example of the advantages of analysis that incorporates statistical elements with qualitative data, which provides insight into history teachers' feelings and beliefs influencing their curriculum choices and decisions.
In spite of advocating more use of randomized controlled trials in education, Goldacre is not dismissive of qualitative research, and in a chapter on the use of evidence-based policy in education, he argues for more training for teachers, researchers and policymakers in understanding the strengths and weaknesses of various research approaches: Learning the basics of how research works is important, not because every teacher should be a researcher, but because it allows teachers to be critical consumers of the new research findings that will come out during the many decades of their career.It also means that some of the barriers to research that arise from myths and misunderstandings can be overcome.(Goldacre, 2014: 215) Among the consequences of such an approach would be a need to educate all those involved in education policy about not just the pros and cons of quantitative, qualitative and mixed-methods approaches to research, but also to develop understanding of some of the overarching ethical and practical issues arising out of research.This would include issues such as understanding the importance of confirmation bias, Hawthorn effect, selective citation, focusing effect, situational understanding (Elliott, 1991), the part played by communities of practice and peer review in history education research, and the use of 'case record' in verifying case studies (Skilbeck, 1983;Stenhouse, 1981).An awareness of Campbell's law, on the part of both researchers and policymakers would also help to improve the use of research in history education.In the words of MacDonald (2000: 31), 'the more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor'.
The use and potential of triangulation in research in history education is an interesting and important issue.However, a better understanding of the issues discussed above, and more attention and concern for the moral and ethical dispositions (and 'position') of researchers is also needed.This would provide teachers, researchers and policymakers with a broader and more helpful understanding of the problems surrounding the warrant for research studies in history education than is provided by debates that are limited to technical and theoretical consideration of research approaches.Although the examples of research cited in the paper are from the UK, the question of how to optimize the use of mixed methods in history education research is an important issue for researchers and academics in history education worldwide.

Notes on the contributor
Terry Haydn is Professor of Education at the University of East Anglia.His research interests are in the purposes of school history, the role of values and dispositions in history education, and issues relating to student behaviour in schools.He is lead author of Learning to Teach History in the Secondary School, a standard text for history student teachers.
So, in what ways do researchers in history education in the UK use mixed methods and triangulation in their research?This section of the paper attempts to provide insight into some of the ways in which researchers in history education have used triangulation in their practice.It also raises the question of what ways, and to what extent, the use of mixed methods and triangulation impact on the quality of this research, in relation to the claims made for such approaches in the preceding section.To what extent does it or can it enhance the warrant or authority of the research?

Table 1 :
Students' views on the usefulness of school history