Student essays expressing historical thinking : A quantitative and dually qualitative analysis of 1 , 100 papers for the History Contest of the German President

History educationalists internationally agree on describing historical thinking and learning on the basis of narrative theoretical models. However, it is striking that there is often a significant difference between rigorous, demanding theoretical and normative expectations, and disappointing empirical results. In the context of the student dissertation project presented here, a substantially wide range of essays has been chosen. More than 1,100 essays from history students between the ages of 7 and 21, with an average length of 33 pages, provide the foundation of this research project. Considering the amount of relevant data (the essays, and submissions using other genres) involved, we developed as a methodological evaluation tool a mixed-methods approach with an explanatory three-phase design (QUANT – QUANT/QUAL – QUAL). This paper presents theoretical and methodological approaches drawing upon examples plus exemplary results of the quantitative and qualitative evaluation of student essays. Thus, we present for discussion the potential of the research field for mixed-methods research.


Student essays expressing forms of historical thinking
'My brother Joseph and I were born in Halle/Saale.On our way to kindergarten we would always pass by the house Burgstraße 43, where Hans Litten was born' (Contribution to the history essay contest 20150188: 4).This quotation provides the reason given by two student joint authors to explain the choice of topic for the essay they submitted to the Geschichtswettbewerb des Bundespräsidenten (History Contest of the German President), (Körber-Stiftung, 2015), a very prestigious competition.The topic of the contest in 2014/15 was, 'Being different: Outsiders in history'.There were 5,109 participants and 1,500 contributions.Most of the contributions to the competition are classifiable as students' extended historical narratives, that is, dissertations or long essays.These narratives are mainly based on either the individual historical research interests of the students, or the desire to achieve a high ranking in the competition.
The competition essays are suitable for an empirical research project regarding the historical thinking of young people because they relate to the competences of historical narratives that correspond to several other similar research studies (Wineburg, 1991;Van Drie and Van Boxtel, 2008;Waldis et al., 2015;Henríquez and Ruiz, 2014;Carretero, 2014).It is striking that in many such empirical studies, there is a significant disparity between rigorous, demanding, theoretical and normative expectations and requirements of student dissertation writing and disappointing empirical outcomes.Hence the question arises whether these expectations and requirements are too ambitious (see Schönemann et al., 2011).What the studies referred to above have in common is that they measure historical thinking in artificial test situations detached from the students' school and college history educational context.These test situations involve the students having to comment on fixed/given historical questions and material in either oral or written form, often only having had a short amount of time to answer the questions.
The results are primarily intended for academic purposes.Therefore, they have little importance for the students who were tested, that is, they are low-stakes activities (see Waldis et al., 2015;Henríquez and Ruiz, 2014).However, in some other instances, the test students take examinations that have significant impact on their futures, that is, they are high-stakes activities.Such immense pressure can have a major impact upon performance, and this has to be kept in mind when making comparisons with their performance on non-critical academic tests (see Schönemann et al., 2011).The history contest essays differ in two respects from such high-stakes testing: the students can individually choose their favourite topics, and they have a period of six months to intensively research their topics and write their essays.
In the essay quoted above, Hans Litten was the central point of attention.(Contribution to the history essay contest 20150188: 4).He was a Jewish communist and lawyer who was an advocate (as a barrister) for people prosecuted by the National Socialist regime .The two joint authors emphasized, 'We can take his altruistic and honest manner as an example.His moral values and beliefs are still relevant for us today' (ibid.: 5).From this perspective, such essays gain an individual orientation, which helps differentiate them from other essays.
On the basis of these extended essays -that is, small-scale dissertationssystematically analysed by Christopher Wosnitza's dissertation research project, we discuss the question as to how a mixed-methods research design can assess the quality of students' historical thinking involved in such discursive writing.

Theoretical and empirical approaches towards historical thinking
History educationalists internationally agree on describing historical thinking, learning and consciousness on the basis of models in relation to theories of constructivism and narrative.However, these models differ widely in their analytical frameworks.
In the model of Sam Wineburg in Reading like a Historian, the focus is on dealing with historical sources analytically.Wineburg focuses specifically upon four central aspects using the terms 'Sourcing', 'Contextualisation', 'Close reading', and 'Corroborating' (Wineburg et al., 2013).A central aspect, which is referred to by later models, is the idea of supporting students' insights into aspects such as perspective and the intention of an author, as well as contributing to the development of a critical consideration of differing interpretations in multiple sources.
Within the educational implementation of the theory of 'second-order historical thinking concepts' from the 1990s, as pioneered and exemplified by Rosalyn Ashby and Peter Lee, the focus is on conceptual categorization of insights into the logic of historical thinking, rather than the procedural, syntactic knowledge and related 90 Wosnitza and Meyer-Hamme processes and skills that Wineburg cites (Lee and Ashby, 2000).Second-order concepts aim at historical training that focuses upon discrete disciplinary concepts -insights -that allow a student to understand terms used in the discipline of history and the related causal connection of events at a high level.
Conceptually, Lee and Ashby's (2000) model does not intend to address propositional, substantive knowledge, historical interests (topics) or their related firstorder substantive concepts.Instead, their model categorizes 'second-order concepts' and the underlying structures of historical thinking by focusing on four such syntactic, disciplinary concepts: 'evidence', 'explanation', 'change' and 'accounts'.It should be noted that Lee and Ashby's (2000) approach was developed from two major research projects into history education in schools, particularly CHATA -Concepts in History and Teaching Approaches.
Based on the thinking of Lee and Ashby (2000), Peter Seixas and Tom Morton (2013) differentiate between six other inter-related meta-concepts of historical thinking: 'historical significance', 'primary source evidence', 'continuity and change', 'cause and consequence', 'historical perspectives' and 'the ethical dimension of historical interpretations'.This model principally differentiates transferable insights into history lessons.Its primary goal is to provide teachers with categories and structures for history teaching.It does not intend, however, to describe historical thinking of students per se.On the basis of this model, a new curriculum has been introduced, one in which students are confronted with the key question: 'What is history?'.The main idea is that they should deal with the significance and value of meta-concepts that involve working with primary source evidence and using it to construct arguments.
Categorizing insights, as with the second-order concepts and the procedural models of historical thinking, is represented in other iterations.Hence, Van Drie and Van Boxtel's (2008: 90) model of historical reasoning also explains the operation of historical thinking, that is, 'asking historical questions, use of meta-concepts, use of sources, use of substantive concepts, contextualization and argumentation', while not addressing the question of the role of historical orientation.This is especially relevant regarding historical thinking that aims at the personal historical construction of meaning (Meyer-Hamme, 2009).
For the German-speaking debate on historical thinking, the concept of 'competence' is particularly relevant.This term has become especially important since 2000, when the OECD's (Organisation for Economic Co-operation and Development) large-scale PISA (Programme for International Student Assessment) report confronted the German political establishment with below average results compared to other countries with highly developed modern economies.As a result, there was a shift within the German educational system, from focusing on taught content ('input orientation') to the promotion of cognitive abilities and skills.This includes the added focus on motivational and volitional readiness to find solutions to problems in a wide range of different cases.Ideally, students should acquire these competences by the end of the education process in schools ('outcome orientation') (Weinert, 2001: 21).Following PISA, several competence models of historical thinking have been developed (Barricelli et al., 2012).These incorporate the main elements of the history models presented above.However, these models frame and connect competences in a slightly different manner.
The model of historical thinking competences currently receiving the most attention is the one of the FUER group (Schreiber et al., 2006;Körber, 2014).The FUER-Geschichtsbewusstsein project (Förderung und Entwicklung eines reflektierten Geschichtsbewusstseins; or Promotion and Development of a Reflected Historical Student essays expressing historical thinking 91 Consciousness) was an EU project from which a competence model of historical thought emerged.This FUER model is based on Jörn Rüsen's model of the process of historical thinking that differentiates between four competence categories to which various sub-competences refer.Its main focus is on identifying domain-specific abilities, skills and readiness 'that a member of society must develop to be able to participate in social debates about history -by being independent, responsible and demonstrating one's ability to form independent judgements' (Körber, 2014: 74).The model differentiates between more or less reflective forms of historical thinking.It consciously extends beyond the school context, reflecting the underlying assumption that historical thinking is not limited to this sector.This makes it particularly useful for analysing and describing the historical learning processes in all contexts.These include formal historical learning in schools, as well as non-formal and extracurricular forms of learning outside of school, which are typical of the processes that led to the essays discussed in this article.
The so called 'FUER model' differentiated between three categories of procedural competence: • The historical enquiry competence includes the ability to ask historical questions that arise from the temporal need for orientation and to evaluate their practicality.Furthermore, it includes the procedural ability to deal with other historical questions.
• The historical methodical competence includes the ability to independently develop explanatory historical accounts (reconstruction competence), as well as the ability to analyse the historical accounts of others ('know that' knowledge) with regard to the decisions made concerning their construction and explanations of how they were constructed ('know how' knowledge; deconstruction competence).• The competence of historical orientation includes the affective ability to incorporate new findings into one's own historical consciousness, as well as the constructions of identity and 'otherness'.It further includes the ability to evaluate behavioural dispositions.
These three competences are connected elements in historical subject competence, a recognized model of historical thinking.One of the characteristics of this model is that it includes the procedural aspects of historical thinking mentioned above, as well as conceptual categorical aspects.Hence, it can be declared an inclusive model.Moreover, it includes not only the reconstructive synthesizing processes of historical thinking, but also the deconstructive analytical processes that are differentiated into separate levels (Körber, 2014).What the diverse approaches presented above have in common is that they model historical thinking on the basis of a narrative-constructivist concept of history (Danto, 1965;Rüsen, 1983Rüsen, -9, 2013)).Most of them are based on the 'concept of historical consciousness' that was introduced by Jörn Rüsen in the anglophone community, and focus on reflective historical learning, but their relevance for historical thinking and learning is most clearly demonstrated in the FUER model.
Based on the research data sets relating to the student dissertations, it was no surprise that the results were rather disappointing in relation to the statement that models of historical thinking have an impact on how students learn and write history.This is perhaps because, depending on the perspective, they give different significance to their different learning topics (Meyer-Hamme, 2009).Hence, it is advantageous to focus on the historical narratives of teenagers who frame their own historical questions reflecting an underlying interest in historical imperative.To identify the potential need for the historical orientation of students, it is beneficial to ask about their own historical identity and to assess their existing historical knowledge.For these reasons, we examined the student essays entered for the History Contest of the German President 2014/15 to gain knowledge and understanding of the candidates' historical thinking competences and their historical perspectives.

Methodology, material and research questions
The 5,109 participants in the 2014/15 History Contest of the German President make this one of the chief international history competitions.Students between the ages of 7 and 21 are permitted to participate, regardless of whether they are still in school.The participants have the chance to win several prizes/awards, differentiated between 'advancement/promotion awards' (Förderpreise), 'federal state awards' (Landespreise) and 'national awards' (Bundespreise).Students are probably highly motivated as participation is voluntary.Additionally, most of the students receive support from tutors.The essays examined here therefore differ in many ways from those previously referred to.Despite this, they still allow us to gain insights regarding the ability of students to think historically.
The research project was based on 1,133 students' historical accounts.These accounts were long essays or dissertations with an average length of 33 pages.The essays were digitized, together with data collection sheets of the participants and all work reports.Considering the scope of the material, the first task was to develop an appropriate reduction method with respect to the research focus.
On the basis of the extensive material, a mixed-methods approach in the form of an explanatory three-phase design (quant -QUAL/QUANT -QUAL) was developed as a methodological evaluation instrument (see Cresswell and Plano Clark, 2011;Kuckartz, 2014) (see Table 1: Methodological approach).The first step included the quantitative analysis of the papers (N = 1,133) with the consideration of an inductively developed encoding system.The scope of the account (essay/dissertation), the gender, stage of education, awards and the type of school were recorded for the purpose of obtaining a comprehensive background set of data for a representative sample drawn from a full range of contrasting submissions.With this background, 120 papers were extracted.The second step was qualitative and quantitative.More specifically, the para-texts (that is, introduction, conclusion, work report) were analysed with the help of an encoding system.This system was developed deductively on the basis of history didactic theories and existing encoding approaches, as well as inductively on the basis of the insights gained from the essays.The aim of this second quantitative step was to identify qualitative and quantitative differences regarding subject references, motives, material and interpretation schema, and to make a selection for the second qualitative part of the analysis.Within this, the research project analysed the texts of 30 essays with regard to the students' handling of historical sources, literature and argumentation.Therefore, the encoding system was deductively derived from history didactical theories and inductively supplemented by the material used.
The research project was based on the hypothesis that a mixed-methods research approach is especially suitable, as its combination of quantitative and qualitative evaluation enabled the handling of the extensive research data.This meant there was breadth and depth in analysing the students' historical thinking and narrative patterns, unlike in previous research.Additionally, systematic results from qualitative and quantitative analysis helped develop new perspectives and methods of empirical research about narrative competence.

Quantitative analysis
Questions, aims and approach First, it is important to note that the submissions to the competition take various forms and types, including written essays, films/videos and other diverse, creative genres.Thus, the first step was to extract the essays from the other submissions to ensure comparability.Despite this first selection, the data still consisted of 44,000 pages of text (about 1,200 essays), as well as socio-demographic data and related information.
The next research step involved quantitative analysis of the students' forms containing social background information based on a set of categories; for example, the scope of the essay, gender, stage of education, type of school attended when the essay was finished and which federal state the participants were from, as well as information on the historical era with which the projects dealt, along with additional data.The main goal was to obtain an overview of the material for selection of a maximum range of contrasting essays for qualitative-quantitative analysis.

Selected results of the quantitative analysis
The robust representative sample (N = 120) was significant, allowing deeper examination and analysis of the quantitative findings.There was a striking difference between the number of essays from females and males: 661 dissertations/essays 94 Wosnitza and Meyer-Hamme were submitted by females, and 321 by males.It should also be noted that the result for primary school students is unreliable due to the low number of participants (see Figure 1).A fuller analysis of the students from Classes 11-13 (see Figure 1, under the category 'Upper school'), is again striking in that about 70 per cent of the submissions were from females.This presents an interesting point for further discussion, namely, whether the gender discrepancy in the number of participants can be linked to age differences, developmental differences, or even sociopolitical factors.However, this article does not focus primarily on gender aspects, but rather on subject references that participants made throughout their essays.Also, there were no significant differences when comparing the number of submissions in written form to the total numbers of submissions.Written submissions were not clearly preferred by female as compared to male participants.
In regard to who won which awards, 30 to 40 per cent of the award-winning submissions were from students who attended a Gymnasium or gymnasiale Oberstufe (see Figure 2, under the category 'Secondary School/Gymnasium and Upper school').These numbers change considerably for submissions developed in the context of different types of schools or from a context outside of school.This is especially true for the participants from elementary schools, as well as for those participants no longer in school.However, as already stated, the numbers for elementary schools are unreliable, as there were only three submissions in total.Nevertheless, it is remarkable that two of the three were award-winners.Also of note, contestants who were no longer in school (see Figure 2, under the category 'No school') often worked without a tutor or support of any kind, so this may have contributed to their below-average performance.This becomes even more apparent when looking at the tutoring of students.The majority of tutors for the essays were history teachers from school.Also, analysis of the number of prizes that are awarded to the different types of schools reveal a marked discrepancy.Whereas 7.6 per cent of the participants from Years 5-10 of the comprehensive schools (see Figure 2, under the category 'Comprehensive school') were awarded national awards, only 3.5 per cent of the participants from the same grade levels at a Gymnasium, and 2.7 per cent of the participants from Years 11-13 received national awards (see Figure 2, which shows who won what kind of prizes (X-axis) and how many of them (Y-axis)).The graph shows the relation between achieved prizes and delivered contributions.This might also be due to the different assessments by jury members judging the essays, who receive not only the final submissions but also information about the socio-demographic background of the contestants.The first step for carrying out a qualitative and quantitative analysis to detect qualitative differences of essays was deductively to develop a set of codes based on theoretical approaches about historical thinking.For the second step, the set of codes was complemented by other codes, prime examples and subcategories generated inductively from the analysis of the material.The results of the quantitative analysis were then used to select the research sample of essays.This was done in a two-step procedure: first, compilation of the most divergent examples with regard to factors such as gender, type of school, stage of education and award won.Second, a subsequent randomization principle was used to complete the sample.In total we analysed 120 essays, 10 per cent of the total number.
Based on the set of codings previously mentioned, the research programme classified through content analysis the para-texts of the participants' essays (that is, introduction, conclusion, list of references, work reports) into categories (Kuckartz, 2016).To illustrate this approach, we present the 'subject references' and the corresponding sub-codes in Table 2.In addition to the codes used, Table 2 lists definitions as well as explicit prime examples for the respective codes.The term 'subject reference' refers to any kind of reference made by the author related to himself or herself.This 'subject reference' was further subdivided into the elements shown in Table 2.

Results of the first qualitative-quantitative analysis and potential implications
The research programme presented and analysed qualitatively some paragraphs taken from one essay to exemplify both the potential of such material for research and our methodological approach.This involved using the categories to designate and classify the 'subject references'.We share and discuss some of the results of the first analysis of the 'subject references' in the 120 research essays.The exemplar essay analysed here is from the 48-page dissertation about Hans Litten.The two authors of the essay used 7 scientific monographs, 2 scientific essays, an encyclopaedia, 20 archive sources, a documentary, 12 photographs taken by themselves, 15 photographs taken by other people, 27 sources from the internet and a newspaper article.They visited two historic sites, three memorials, three archives besides their work at school.Additionally, they included several subjective, personal references.
My brother Joseph and I were born in Halle/Saale.On our way to kindergarten we would always pass by the house Burgstraße 43, where Hans Litten was born.… Another relation to Hans Litten was established by our grandma's work in the Berliner Luisenstädtischen Bürgerverein … We have been living in Jena for several years and since it is not far from Weimar, it is a school tradition to go on an excursion to Buchenwald in the ninth grade and to deal with National Socialism intensively.'(Contribution to the history contest 20150188: 4-5) The three explicit 'subject references' in the extract, the starting points for the authors engagement in their study of Hans Litten, were coded as 'local reference', 'family reference' and 'commemorative culture' (see Table 2).This demonstrates how this methodological approach can be used to classify and analyse qualitatively while making statements about the quantity of 'subject references'.Of the 120 essays analysed in the first qualitative analysis of the research project, 102 contained local 'subject references'.The reason for the high number of 'local references' is that these types of references were clearly required and explicitly mentioned in the contest guidelines (see Körber-Stiftung, 2016).However, 10 essays do not contain any 'local references', and 8 do not refer to a local place, but do refer to a family context.It is notable that the submissions made by primary school students and young adults no longer in school all contain 'local references'.
Figure 3 shows findings of subject-references in relation to different class levels.The statistics outline shows how many subject references were made in the submissions written within a school context.Essays could contain several subject references, which the researchers then coded individually.This shows a considerable difference to the previous research projects on historical thinking, where students could not, or did not have to, develop their own research questions.However, this is an essential, relevant aspect of historical thinking that arises from a student's identification of a need for orientation (Rüsen, 1983-9).While analysing the students' essays, it is necessary to ask what types of questions they asked.At this point, the students refer to characteristics such as perspectivity and the reliance of history on the historian's point of view (historical subject competence).Moreover, they also explicitly establish a link between history and the present, that is, historical time and present time.Thus, the quotation above was coded as a 'narrative constructivist concept of history'.Furthermore, the authors stated their reasons and motivation for focusing on Hans Litten: 'We can take his selfless and honest attitude as an example.His moral values and attitudes are still up-to-date even nowadays' (ibid.: 5).In this way, these statements are also an expression of their competence of historical orientation.In the last quotation they use an exemplary construction of meaning.Hans Litten is referred to as a personal example and role model, based on their opinion that values such as selflessness and honesty are timeless, as well as stating their motivation.Based on their implicit assumption that their construction of meaning is timeless, they try to persuade the reader to agree to their construction of meaning by using an inclusive 'we'.These statements were, therefore, coded as 'mentioning brave behaviour', as well as 'learning from history for the present time, or taking history as an example'.These codes were further complemented by the codes 'giving a voice to victims', because the two authors mentioned in the next line that they want to keep the memory of Hans Litten alive.Additionally, these excerpts underline the question of historical orientation in the FUER model and the ethic dimension by Seixas and Morton (2013).

Discussion
In what way does the perception of students' competences of historical thinking change due to the evaluation of these student essays?The material used for this research project provides unique insight, differing from other research projects considered in at least four ways.Because of this, the results have to be interpreted accordingly.First, almost all of the materials analysed contain at least one, and often several, personal references.Second, this indicates an interest in genuine historic insights and a subconscious demand for orientation.Third, many papers were part of a six-monthlong essay project, giving students the opportunity to try working like an historian (for example, by going to archives or historic places).Fourth, the participants' motivation was further fostered by the desire to win an award.Despite all of these advantages, the material also presents some new challenges.For instance, the different contexts in which these papers were written make comparisons extremely difficult.The development of a mixed-methods research approach can be seen as a response to this challenge as it helps logically to reduce the amount of data, while still appreciating the different contexts.Thus, the mixedmethods approach helps in the analysis of large amounts of empirical data, as well as enabling deeper insights into historical thinking.Moreover, this approach is useful in further investigating the narrative patterns commonly used by young adults in historical compositions.
Thus, it seems likely that the disappointing results regarding the historical method competence made by the studies analysed might be due to the context in which the studies were carried out.In contrast to these findings, the results from the analysis of this data suggest that young adults do possess elaborated forms of historical thinking, but that it very much depends on whether participants ascribe a personal meaning to the topic.When interpreting studies that measure historical thinking of students, one needs to check if and how participants are offered the possibility to bring their

Figure 1 :
Figure 1: Educational context of participants in the competition

Figure 2 :
Figure 2: Who won which types of award in the competition

Figure 3 :
Figure 3: Subject references in relation to class levels

Table 2 :
Category: subject-references plus related subcategories 'Before we decided on a topic we were gathering ideas.Since I am Catholic I decided to take on this topic.'(Cothc 20150324, S.1)