Does a theory of action approach help teachers engage in evidence-informed self-improvement ?

This article reports on how using Theories of Action (TofA) can help teachers scale up evidence-informed teaching practices by aiding their understanding of why such interventions have been effective and which aspects are key to driving change. This paper reports on a specific approach: a partnership between an academic and three schools. The findings, based on interviews with 15 teachers and school leaders (the whole of the federation’s teaching staff), and preand post-intervention surveys (undertaken with 13 staff members) suggests that the scale-up of evidence-informed practice, when aided by TofAs, can lead to substantial impact on teacher and pupil outcomes. The paper concludes that the effective scale-up of evidence-informed interventions is grounded in teachers’ understanding of why interventions have been successful and how that success might be realized in a new context. Correspondingly, when teachers are shown how to use TofAs to tailor interventions, this helps them ascertain how such interventions can be realized most effectively in their own settings.


Introduction Achieving research-informed teaching practice
Research-informed teaching practice (RITP) is described by Walker (2017) as the process of teachers accessing, evaluating and applying the findings of academic research in order to improve teaching and learning in their schools.RITP forms a key part of many recent policy initiatives by governments worldwide that seek to foster school improvement from the bottom up or in ways that are self-improving (Greany, 2014;Brown et al., 2017).In keeping with this policy push, in 2014 the Education Endowment Foundation (EEF) launched a funding call for projects with a focus on increasing research-informed teaching practice in schools.The approach described in this paper -the research learning communities (RLC) model -was originally developed in response to this call.Other projects also funded by the EEF at this time include two so called 'literacy octopus' trials (named for their multi-armed design) in which 13,323 English primary schools were involved in testing commonly used ways of disseminating evidence (for example, online research summaries, magazines, webinars and conferences).Although RLCs are a relatively new concept, their impact has been evaluated through the use of both a randomized control trial and case study research.Evidence from these evaluation studies indicates that RLCs are successful in: (1) increasing teachers' confidence to use research; (2) building the capacity of school leaders to lead research-informed change and develop school conditions that foster the use of research; (3) enhancing teaching practice; and (4) improving student outcomes (Brown, 2017;Rose et al., 2017).

The study
The Chestnut Learning Federation represents a family of three small church infant schools based in Hampshire who work closely together under the leadership of the Federation Executive Head Teacher and governing body (as per the ethical requirements of the project, the name of the federation has been changed to preserve anonymity).One of the federation's improvement plan objectives is for it to become evidence-informed, so that all three schools collaborate to rigorously evaluate the quality of the education they offer, understand what they need to do to improve, take appropriate research-informed action and evaluate the impact of their actions, enabling them to achieve together.To meet this objective, the Executive Head Teacher of the federation adopted the research learning community (RLC) model of professional learning for the 2016/17 academic year (see Box,p. 349).To facilitate this approach, the Executive Head Teacher took the bold step of dedicating four of the statutory staff professional development (inset) days allocated to schools in England entirely to research-informed professional development.The subject of the RLC was effective teacher-student feedback, chosen by the Federation Executive Head Teacher as a key area for improvement.The subject of teacher-student feedback also has a substantive, detailed and secure research base with which to engage teachers -for example, see the Education Endowment Foundation's 'toolkit' (Education Endowment Foundation, n.d.).Using a cycle of enquiry approach, the aim of the Chestnut RLC has been to enable all teachers in the federation to engage collaboratively with research on feedback (which was provided in the form of literature reviews produced by the second author), to identify new practices, to trial these practices, to measure their impact and then roll out the most successful within and across schools in the federation.
Research learning communities (RLC) are groups of teachers who come together to engage with research in order to enhance both their practice and also the practice of their colleagues.Each RLC typically comprises 6 to 12 teachers from three to six schools, although this number can vary depending on circumstances.A key principle of the RLC approach is to ensure that the 'right people are in the room': those who can make change happen in schools.Over the course of a school year, the RLC process involves four whole-day workshops that lead participating teachers through a research-informed cycle of enquiry.Here participants engage with research and are facilitated to: (1) relate this research to their own practical knowledge and knowledge of their context; (2) develop an intervention, grounded in the research findings and their own practical knowledge, which is designed to improve specific aspects of teaching and learning; (3) trial and refine their intervention to maximize its effectiveness; and (4) ascertain impact and roll out impactful interventions within and across schools.In between workshops, participants are expected to work with school colleagues to share research knowledge and to enable colleagues to assist with the development, trial, roll-out and impact assessment of interventions.
Importantly, the RLC approach developed for the Chestnut Federation was revised to incorporate the concept of theories of action (TofA) (Brown and Graydon, 2017;Hubers, 2016) to help teachers consider how to develop effective research-informed teaching practices.Theories of action are perhaps best thought of as a journey guide for impact -TofAs provide the route map that steers educators towards their intended long-term outcomes, or the difference an innovation is designed to make for a given group or set of stakeholders.Correspondingly, to help educators reach this long-term vision, TofAs provide the steps that need to occur along the way.
The TofA used by Chestnut Federation comes from Brown and Graydon (2017).Synthesizing seminal impact measurement literature (for example, Earl and Timperley, 2015;Wenger et al., 2011), Brown and Graydon (2017) suggest that interventions can be conceived as being informed by, and affecting change across, a number of 'domains'.These domains are identified as: 1. the context in which the school or setting is situated 2. the problem or driver for the intervention 3. detail on the intervention and how it was intended to result in change 4. activities and interactions related to the introduction and roll-out of the intervention 5. the learning that results from teachers engaging in these activities/results from these interactions 6. changes in teachers' behaviour, and the extent to which something is being used 7. the difference behavioural changes have made to pupil outcomes.
In the Chestnut RLC, Brown and Graydon's (2017) model was used to help teachers think about how and why their research-informed interventions were likely to lead to improved outcomes for teachers and pupils.Specifically, teachers used the Brown and Graydon TofA as a basis for considering why their interventions might lead to changed teacher learning and knowledge, how they might change teaching activity and, as a result, how this revised teaching activity might impact on pupil outcomes.
This article provides an evaluation of the approach in relation to three key areas: (1) did the RLC approach help teachers become engaged with research; (2) did the RLC approach help teachers use the research to develop approaches to feedback with clear pathways to impact; and (3) do teachers in the federation perceive that these new approaches have achieved impact?

Approach
The report is based on interviews with 15 teachers and school leaders (representing the whole of the federation's teaching staff), and pre-and post-intervention surveys relating to teachers' use of research (completed by 13 respondents).Interviews took place after the final workshop in June 2017.As part of the interview process, participants were asked to bring with them impact data (such as end-of-year attainment data) relating to their interventions in order to facilitate a way to triangulate their responses and provide a level of objectivity to their accounts.External observation is provided by Ofsted, England's accountability body, since an Ofsted inspector also visited one of the three schools involved towards the end of the project.Data were analysed thematically using the impact domains (4) to (7) from those listed above as a means to interpret respondent's findings and ascertain the meaning of their impact data.

Did the activities undertaken help participants engage with the research?
Respondents suggested that the activities helped them engage effectively with the research and relate it to their context/setting and area of practice that required improvement in the following ways: 1.By providing access to research where previously this had been difficult: '[previously] that's the bit that I've found hardest with the inquiry, is accessing that kind of material … knowing more where to go and accessing [research].So having access to that and time to read through things was really helpful' (Respondent 3). 2. The quotation from Respondent 3 above also highlights the value placed on having time to engage with research.Other similar comments about the model providing the time needed to do research included: 'having those inset days made all the difference this year.You know, when we were trying to fit it in, sometimes it didn't happen, and we'd grab half an hour and it didn't have the momentum it had this year' (Respondent 3).(Respondents 5, 8, 9, 10, 13 and 14 also made similar points.)3. The approach to research engagement was seen to have two key components: participants enjoyed the collaborative discursive nature of the activities: 'I'm not one to sit and read through reams of research, but actually when we did, everyone read a little bit and then fed back and discussed it.I found that a much easier way to engage with the research … to go through and talk about, or to analyse together' (Respondent 2); 'the communication and working as part of a team is important, if you can sit down with [research] and unpick [its meaning] together.I think that's better than trying to work in isolation' (Respondent 7).(Similar points also made by respondents 10, 11, 12, 13 and 14.) Furthermore, the structured and facilitated approach to research engagement meant that participants felt they were able to engage more meaningfully with the literature (respondents 2, 5, 9, 13 and 14). 4. Reflecting nascent literature (for example, Galdin O'Shea, 2015), respondents also appreciated that they were being encouraged to experiment and take risks: 'I think for me, it was the knowledge that it was okay to get it wrong.That didn't matter, because it's not necessarily finding the answer' (Respondent 6).Likewise, Respondent 9 noted of the federation leader that: 'she is always reassuring us that "if you trialled it and it didn't work, that's fine"'.
Effective engagement with research also requires that teachers can understand strengths and limitations of different research methods, can contextualize research findings (that is, see how research findings can be applied to one's own setting and practice) and can engage in learning conversations using research as part of collaborative approach to designing new teaching strategies.These three requirements are reflected in survey questions 1, 2 and 3 in Table 1.While not based on an experimental approach (that is, there was no counterfactual data for individuals not engaging in the project), the data from the surveys does provide promising indications that for all three areas respondents typically believed that their knowledge and skills had improved over the course of the project.Specifically, average scores appear to have moved from below the midpoint score of 3 ('average') at the start of the project to closer to 4 ('above average') by its end.Did the activities help participants develop interventions with clear pathways to impact?
From analysing the interview data, it could be seen that all respondents could espouse a theory of action (TofAs) (pathways to impact) for their developed intervention.In other words, respondents were able to state what their intervention was, the logic underpinning its design, how it was intended that the intervention be realized and the changes it was intended should result.An example of one such TofA is set out in Table 2.As can be seen in the table, Respondent 4 sets out in detail how they were able to deconstruct the nature of their intervention and its intended and actual changes in knowledge and practice, as well as evidence the impact on pupils that resulted.Respondent 3 suggested that the TofA approach had made her realize the importance of being systematic and rigorous in how interventions are developed, as well as how baselines are established and how impact is assessed.Furthermore, the TofA approach meant that if interventions were not delivering the desired impact, then tweaking and refinement could be undertaken by re-examining the logic of the approach and whether its constituent parts were being implemented or supported effectively.This was also reflected by Respondent 5, who noted that the TofA approach meant that they were able to systematically explore 'What is the problem?What am I doing about it?What's changed?'.In addition, it was also recognized that the TofA approach could be used generally to explore and tackle issues of practice: 'if you've got your theory of action, I find that you can then drop in a variety of questions, can't you?And, it's a similar process.I mean, once you've got the process of the research and that systematic approach and looking at it, then I feel that you can drop any question in [and] explore how to address it' (Respondent 12).Alternatively, the TofA approach can help refine or fix interventions that appear to be unsuccessful: it also helps you address 'Well, actually, it didn't work, so where do I go now?' Or, to somebody else, they come back and say, 'Well, it did work for me, but it didn't work for B.' 'It did work for you, why?Why? Was it your approach?Was it the cohort?' So, then it opens up another question on where you're looking at (Respondent 12).
Interview data also suggest that the TofAs developed by respondents were fully grounded in the research they engaged with in the first workshop.In particular, three respondents could specifically identify the research underpinning their intervention: for example, see Table 2 for Respondent 4's responses.Others could not recall the name of the research(er) but could describe what the research was about and its implications for practice.Furthermore, survey data too suggest that participants felt, by the end of the project, that they had developed the skills to interpret and then apply academic research to the design of new teaching and learning strategies.Survey questions 4 and 5 in Table 1, for instance, indicate that over the course of the project respondents typically believed that they had substantively more confidence than before in interpreting research findings.They also reported a higher ability to employ research effectively when developing new pedagogies.These responses reinforce the suggestion that the TofA developed for interventions had a basis in the research concerned.

Problem or driver for intervention
As a school we have been tasked with supporting more children to exceed expectations in writing.For our early years children, we felt that this wasn't going to be reached through more handwriting practice or more time sat at tables.Our previous observations and experience led us to believe that something else must happen before children would exceed in their writing.

The intervention
We had noticed over several years that many children were fearful of failure, getting things wrong or not being able to achieve something, and that this was inhibiting them in taking risks in their learning.They would keep doing what they could easily do rather than taking a risk with something new or tricky that might possibly go wrong.We felt that this may well be what was preventing our children from exceeding.Our intervention was informed by Carol Dweck and her work around growth mindsets.From this work, we hypothesized that if we were able to change children's feelings and attitudes towards failure/ struggle and getting things wrong, then they would be more likely to take risks in their learning.

Activities and interactions
We have introduced the idea of being a 'brave learner'.This has not just been applied to writing and maths, but to all aspects of learning and being.We have created two brave learner characters and have identified the characteristics of being a brave learner.Children have been awarded a certificate when they have been a brave learner, and their picture is added to our brave learner display board in school.

Learning
The teachers involved better understand the need to show to children that getting it 'wrong' is part of the learning process and that only by having another go, changing strategies or practising will they get better: failure and getting things wrong are part of the learning process.They now also have an understanding of the need to give children a language to articulate their feelings while learning.

Changes in behaviour
When a child has been awarded a certificate, we now talk about how the child felt about the struggle they have had to be a brave learner.We now praise their effort and resilience, and their endurance, not whether they were successful in their quest.

Difference
Over the last six months we have seen a huge change in the attitudes of our children.They talk about being a brave learner and when we, the adults, talk about needing to be a brave learner they know what they have to do.They also talk about how they and others have been, or need to be, brave learners.We feel our brave learner programme has impacted positively on all children's attainment in writing, especially for those for which writing has been a struggle.
The children have begun to understand that struggle is part of learning, not an indication they will never get there.
Do teachers in the federation perceive that these new approaches have achieved impact?
In all cases, respondents could easily attribute changes from their interventions to learning, behaviours and outcomes for children.An exemplar response in its entirety is set out in Table 3.For other respondents, we have sought to provide example vignettes that capture changes in practice and children's outcomes in order to provide an illustration of what had been achieved as respondents journeyed along their TofAs.For example, Respondent 2's research question was 'if they're better risk takers, and Research for All 2 (2) 2018 they're more willing to try things, are their reading levels coming up?' Respondent 2's approach was to create: a small focus group [and work with the group using] books and empathy of characters [to help them understand that] you can't learn without being uncomfortable, and all those sorts of things.So, break down the barriers, and make them risk takers, and that linked with the empathy, because we're all in the pit at different times.Bar one, the whole focus group did get to [working above expectations], so, it seemed to have been successful … but I've been doing it with all of them.I think it's been, outside of that group, it's been effective, as well.

Learning
The aim was to improve teachers' understanding of the effective characteristics of learning, and whether this approach impacts on writing outcomes for summer-born children.Specific learning included: 'the approach has changed our perspective on the importance of some core skills [and has led to an] improved understanding of why certain provision is important to specific groups and individuals.From our staff questionnaire, it is clear that teachers and teaching assistants all have a greater knowledge of the learning characteristics.'

Changes in behaviour
Changes in teacher practice noted by Respondent 11 included: • 'changes to teachers' planning activity -using characteristics of effective learning to move away from curriculum-specific foci' • that 'learning values are now driving teaching practice [rather than end-of-year goals]' • that teachers were 'more actively looking for effective learning behaviours and planning activities to develop these behaviours' • that across the school there was a more general focus on 'getting children to use the language of learning, so reflecting on their own learning'.
It was also noted that, depending on the cohort/class, 'we have had to change the focus from role-play writing opportunities to individual interests … we have also had to do much more fine/gross motor work'.In other words, teachers were also taking a differentiated, learning-centred, approach, employing their understanding of the effective characteristics of learning.

Difference
Leuven scale data show greater engagement in learning by children; interview data with children suggest greater confidence and understanding.
Parent questionnaires indicate that parents can see the differences in children's writing.For example, one parent noted that: 'the forming of [Jill's] letters and her interest in writing have both improved significantly'.(As per the ethical requirements of the project, the names of children and staff have been changed to preserve anonymity.)Furthermore, the school's writing data for 2015 highlighted that only 60 per cent of summer-born children met their end-of-Year-R early learning goals for writing.This compares to 83 per cent of autumn-born children.Respondent 11 argued that the changes in practice noted earlier worked extremely well, ultimately, in her view leading to a rise in the number of children meeting their writing early learning goals to 86 per cent in 2016, and to 82 per cent in 2017.In other words, sustained improvements of over 20 percentage points per year.
Respondent 5 noted that with their project: there were six boys who I was trying to get to age-related expectations for writing, and at the beginning of the year they were predicted that they might not make it.Out of that, four have made it, two haven't, so I guess the data is saying that it's more successful than not [in fact the data provided showed that the four pupils in question had exceeded expectations].The Talk for Writing [an approach developed by Pie Corbett, which research says is successful] works in particular for stamina of writing.When [the pupils] came in September, their stamina and confidence to write at length was zero.The Talk for Writing just gives them the toolkit to do that.They can regurgitate, shall we say, the story and it helps them think about actually the mechanics of the writing rather than, 'I have to think what to write and then how to write it.'It's that stepping stone and it's been a good scaffold for them.It has helped them grow in confidence and ability.
Respondents 6 and 8 were working collaboratively on a feedback project.Here it was noted that: using the Leuven capture sheet, it was clear that our focus children were slow to settle to a given task.Having checklist prompt cards and strategy cards [derived from research by Gibbs and Simpson, 2004] have certainly made things quicker, and the children are all now engaged positively with their writing.The quality of writing has improved and outcomes in reading and writing [according to the end-of-year learning expectations] are now significantly above average (Respondent 8).
Furthermore, data provided by these two respondents shows that the gap between highest and lowest achieving pupils in terms of meeting or exceeding age-related expectations has closed during the course of the project from 10 per cent to 6 per cent.Finally, Respondent 12's project was to explore children's understanding of mastery with the aim of helping them exceed age-related expectations in writing and maths, and drew on research by Yarker (2016).Two focus groups of children were selected, and learning conversations were held about the notions of mastery.Subsequently, a language of learning was introduced across Year 1 to help children see that mistakes are part of the learning process rather that setbacks, and that these mistakes could help children master their learning.Modelling of mastery language and skills was undertaken by the teachers and teaching assistants.End-of-year data show that the number of children in Year 1 meeting their age-related expectations at the end of the year has risen in writing from 76 per cent to 83 per cent, and in maths from 83 per cent to 92 per cent.In conclusion, then, it seems clear that that Chestnut Federation's new approach to professional development has enabled teachers to successfully engage with research evidence on effective pedagogic practices, but that through the use of TofA, it has achieved this in such a way that both teachers and children are benefitting as a result.

Conclusion
In this paper, we have suggested that what is required to achieve research-informed teaching practice is an approach that can help teachers engage effectively with research evidence in order to adapt existing interventions, such that they achieve desired impact.Our approach for this has been to use notions of TofA: presenting research to make theories of action visible and explicit, and helping teachers consider how to tailor theories of action in order to ensure interventions operate most effectively in their own settings.From the analysis above, we suggest that this approach has enabled Chestnut Federation's teachers to successfully engage with research evidence on effective pedagogic practices.Perhaps more important, however, is that the paper presents evidence to suggest that the effective scale-up of research-informed interventions is less to do with the instrumental replication of existing strategies, and more to do with understanding why interventions have been successful and how that success might be realized in a new setting and context.
The world of education is replete with examples of scale up failure (Dede, 2016): Bradford and Braaten (2017), for example, undertaking research into the centralized roll-out of an initiative referred to as 'great teaching', note that by enforcing the equivalent replication of a given toolkit, teachers involved in the initiative felt both unable to employ their professional judgement and were prevented from prioritizing what they valued and regarded as great teaching and learning.Ultimately, this enforced use of the intervention served to demoralize teachers, but it also meant that the reform was only engaged with in a cursory way: 'great teaching' thus never became fully integrated into existing pedagogy.At the same time, many academics continue to pursue strict notions of fidelity (for example, Fixsen, 2017), insisting that once research has demonstrated that an intervention is successful that the intervention should then be rolled out elsewhere as an exact copy.The analysis in this paper, however, starts to address how to resolve the apparent contradiction between 'treatment fidelity' and the need for adaption that comes with the conceptual engagement that typifies teachers' use of research (Klieme, 2017).
In doing so, we suggest that it is time to reconsider the importance of fidelity to the scale-up of research-informed interventions.Or perhaps, to be more precise, to reconsider what fidelity really means and why it is important in relation to teachers' engagement with research.To us, our data show that it is the theory of action that is key: teachers need to understand why an approach works and which aspects of an approach drive change.Of course, examples of how it has been implemented (that is, toolkits) provide useful illustrations for teachers.But if an approach has been developed in a given setting, there is no guarantee that in the specificities of a different school it is either possible or desirable to roll-out the exact same approach.Instead, what is needed is to find ways of achieving similar success by helping teachers tap into the same social drivers as the original intervention (assuming they hold in a new setting), but to do so by using approaches that are suitable to the resources available, the children being taught, the skills of the teachers in place and so on.Fidelity, then, should be regarded primarily as fidelity to a TofA, not necessarily to the specific way that TofA has been operationalized.
In keeping with the growing understanding among implementation scientists that 'rolling interventions out as an exact copy' rarely works in a real-world setting (for example, Bryk, 2016;Dede, 2016;LeMahieu, 2011;Moss, 2013), our data in fact shows that customized replications can be impactful.In all cases, teachers were engaged with research that had examples of specific interventions that could have been implemented through direct copy and paste (for example, Assessment for Learning feedback or Talk for Writing).In all but one situation, teachers used the TofA to engage with the research in a conceptual way in order to develop a bespoke intervention that worked best for them.In all situations, teachers reported impact in terms of their knowledge, their practice and outcomes for their children.In some cases, this impact appears to be substantive (for example, see Table 3).Although we argue that this impact was due to the use of TofAs within the RLC approach (especially given previous impact evidence associated with RLCs), we accept that with the current research design it is not possible to definitively attribute impact in this way.Nonetheless, we still believe that this evidence supports our assertion that there should be more nuanced understanding of what it means to scale-up a research-informed approach.
Notes on the contributors Jane Flood has been an infant teacher for more than 20 years, and has worked in a variety of schools in various roles, from supply teacher to deputy head, teaching in small infant to large primary schools.Jane is currently Head of Learning in a federation in Hampshire.She has just started a part-time PhD, at the University of Portsmouth, focusing on ways to manage the competing priorities of teacher-researchers.
Chris Brown is Professor of Education at the University of Portsmouth.With a longstanding interest in how evidence can aid education policy and practice, Chris continues to publish extensively and present internationally in this field.In 2015, Chris was awarded the American Educational Research Association 'Emerging Scholar' award (Education Change SIG), the 2016 AERA Excellence in Research to Practice award and the UCEA Jeffrey V. Bennett Outstanding International Research award.

Table 1 :
This report thus lends further weight to the notion that the approach and activities used have been successful in helping teachers engage in research evidence and collaboratively develop research-informed teaching practices to tackle areas requiring improvement.
specific aspects of teaching and learning.In particular, the report notes that: 'leaders have embedded a research-based culture where strategies to improve teaching are investigated and evaluated in terms of pupil outcomes.As a result, the whole school community is deeply dedicated to continuous improvement and sharing expertise to raise standards further.'

Table 2 :
An example of one respondent's TofA

Table 3 :
An example of one respondent's impact statementImpact domain Impact text and data(Respondent 11)