Studies in Educational Evaluation

This special issue explores the complexity and interconnectedness of the many components of data-based decision making. The selection of papers represents many countries (i.e., Belgium, The Netherlands, New Zealand, Norway, and the United States), theories, methods, and foci. All the papers seek to explicate how data are used at the di ﬀ erent level of the system, ranging from students, teachers, schools, and districts. Together these papers o ﬀ er a view of the current data-based decision making landscape, including in-and pre-service professional development, district and school organizational capacity, the data use process (from goal setting to collaborative instructional decision making), and e ﬀ ects on student achievement. The intent of the special issue is to stimulate future work in terms of impact on research, theory, policy, and practice.


Background
This special issue is grounded in recent sessions on data-based decision making held at the annual conferences of the American Educational Research Association (AERA) and the International Congress for School Improvement and Effectiveness (ICSEI) as well as critical publications. The two organizations, AERA and ICSEI, have an established record of promoting the enhancement of work around data use through AERA's special interest group (SIG) on Data-Driven Decision Making in Education and ICSEI's Data Use network 1 .
In recent years the field of data-based decision making evolved from a focus on one source of data (standardized assessment data) and one outcome measure (student achievement) to the use of a variety of data sources (e.g., classroom observations, student voice data, parent surveys) and a broad range of outcome measures (e.g., student achievement, student learning, wellbeing). However, still many misconceptions and (sometimes valid) criticism exist in the field and have also become apparent in the sessions organized by ICSEI and AERA. Common misconceptions include that data can only be used for accountability purposes; data use does not lead to increased student learning and achievement; data equals test results; and data literacy equals assessment literacy. These are some of the misconceptions that formed the starting point for this special issue, and which will be addressed in the various papers.
In this special issue, we have provided a unique and broad perspective of the international landscape around data-based decision making (DBDM), and we address some of the existing misconceptions and criticism. This special issue demonstrates how complex the field of DBDM is, and why it is so difficult to achieve definitive outcomes about the impact of data use on classroom practice and student performance. DBDM is a complex system of interacting components that combine to facilitate or impede effective and responsible data use. Directly addressing the misconceptions and complexity issue not only extends but also helps to clarify and explain work presented in prior special issues on DBDM.
With this special issue we have provided diverse settings to depict similarities and differences across international settings. We hope that the special issue will stimulate creative strategies to address some of the pressing issues and questions surrounding DBDM. A parallel intention for the special issue is to engage young scholars and those who heretofore have not worked in the data field.

The articles
An overview of all the resulting articles in this paper can be found in Fig. 1, although it has to be noted that several articles focus on more than one topic. Firstly, it is important to acknowledge that data use requires capacity development. Moreover, the process of data use is https://doi.org/10.1016/j.stueduc.2020.100906 Received 3 June 2020; Received in revised form 6 July 2020; Accepted 7 July 2020 influenced by the district and school organization, which can enable or hinder the use of data. It is important that the process of data-based decision making (in collaboration) use does not start with data, but with clear, measurable and collectively agreed upon goals. Although we put "in collaboration" in brackets, because not all the papers focus on the collaboration aspect, we do want to stress the importance of collaboration here. Data use should not happen in isolation. Sense making is a crucial component in the data use process and making sense of data requires a collective discussion. Finally, if done well, the use of data can lead to achieving the goals set at the beginning of the process, the desired effects. The papers in this special issue include diverse methods and originate from different countries as can be seen in Table 1.

Data-based decision making (in collaboration)
The lead article by Mandinach and Schildkamp (2020) is a direct result of the 2018 AERA symposium where goals, misconceptions, and concerns about (the process of) data-based decision makingwere discussed. The article provides a landscape view of the literature to support, refute, or challenge several pervasive topics and criticisms. The article concludes with a set of recommendations to the field about needed work to address and ameliorate the misconceptions.
The following articles focus on aspects of the process of data-based decision making, sometimes with a focus on collaboration. Datnow, Lockton and Weddle (2020) examine how teachers use data and evidence of student thinking to inform instructional planning and instruction. They study how, through collecting information on students' thinking, teachers identify and address student misconceptions in their thinking and learning. The authors also touch on the importance of teacher capacity; that is, teachers' abilities to use data and translate those sources of evidence into their classroom practices.
The article by Fjørtoft and Lai (2020) explores data use in terms of social semiotics, the study of signs and symbols. It poses two cases studies and examines how the affordances of teacher collaboration around data impact practice. The authors discuss systems of representations and their affordances. They also identify and discuss five challenges to social semiotics in data use. This paper uses a theoretical perspective to examine systemic components to data use as well as teacher collaboration.
The Jimerson, Garry, Poortman, & Schildkamp (2020) article focusses on the implementation of a Netherlands-based data team intervention in the context of education in the United States. The authors take a systemic perspective by identifying enablers and challenges to the enculturation of data use within the U. S. educational system. This article focuses on many aspects of DBDM, but primarily examines the interconnections and systemic nature of the components that facilitate or impede data use. Powell et al. (2020) examine how data-based individualization affects student mathematics performance for those who exhibit learning difficulties. Project STAIR (Supporting Teaching of Algebra with Individual Readiness) includes intensive professional development, ongoing coaching, and frequent progress monitoring of students on teachers' instructional practices and students' algebra readiness. Effects of these intervention were found on teachers' understanding of data-based individualization, the importance of evidence-based instructional practices, and some, but not all of the mathematics outcomes studied.
Van Gasse, Goffing, Vanhoof and Van Petegem (2020) focus on the importance of collaboration in the data use process. It examines the interactions among teachers, focusing on popularity, proximity, and bonds. The authors use social network analysis to examine the interactions and address how teachers collaborate while using data.
Vanlommel, Van Gasse, Vanhoof and Van Petegem (2020) explore  the data use process, focusing on the use of intuition as well as data in teacher decision making. They studied teachers' decision process regarding the transition of 30 pupils during their last year of primary education. They conclude that both data-driven and intuitive processes are important for contextualized and unbiased decisions.

Data capacity development
The next set of papers focus on data capacity development (in-and pre-service professional development)in the use of data. Abrams, Varier and Mehdi (2020) discuss capacity building at the school level as well as the teacher level. Abrams and colleagues use mixed methods approach to study an intervention to improve teacher data literacy, teacher collaboration through data teaming, and data use. The authors seek to explore the organizational and structural factors that support the enculturation of data (use). This paper also intersects diverse aspects of data use impacting instructional practice, teacher capacity, data teams, and data cultures.
Beck and Nunnaley (2020) examine issues around data literacy. They touch on the conflation of data literacy with assessment literacy. Moreover, based on a literature review they developed a continuum of data literacy skills and knowledge from novice to expert and the intermediate points for individual development. This continuum can serve as a roadmap for instrument development and the integration of data literacy into educator preparation courses.
The aforementioned paper by Jimerson and colleagues focuses on the process of data use in an in-service professional development intervention. The authors focus on capacity building through a data team approach and focuses on the enablers and barriers for implementing such an approach.
Visscher (2020) compares six data use intervention studies in Dutch schools by reviewing the literature and examining empirical evidence of the impact of data use. Similar to the previous article, Visscher looks at factors that may enable data use. These include goal-setting, feedback, school context, and teacher professionalization. Visscher concludes by looking at two future topics that show promise: professionalization and rapid feedback.

District and school organizations
Two articles focus on district and school organizationalissues and development with regard to data use. The Lasater, Bengtson and Albiladi (2020) paper focuses on the organizational aspects of data use that promote the use of a deficit model. Deficit models focus on the weakness and failures of students rather than capitalizing on student strengths, weaknesses, and interests. Deficit models are strongly associated with the accountability movement which has often given DBDM a bad reputation. The authors contrast accountability versus instruction and focus on how teachers view students. The authors also discuss the need for a safe environment in which teachers can explore data. This article intersects systemic components of data use, equity models, instructional actions, and teacher perspectives.
The article by Wang (2020) is an examination of how artificial intelligence (AI) can be used in DBDM. Wang examines the opportunities and challenges to implementing AI in instructional settings and beyond. The author discusses how AI can potentially enhance data use (e.g., take care of some data literacy skills needed in schools, such as collecting and analyzing data), while cautioning about the potential misuses of the technology.

Effects of data use
Two of the articles in this special issue (Powell et al., 2020;Visscher, 2020) also focus on the effects of data use on teachers and on student outcomes. All the interventions studied in these papers had positive effects on teacher professional development and student achievement, albeit not on all outcome measures.
The issue concludes with a synthesis tying together the diverse papers, noting their contributions, and explaining how they can inform future work and practice in DBDM.

Concluding thought
Circling back to the original impetus for the misconceptions article, we sincerely hope that these papers will allay or address some of the concerns and criticisms that persist in DBDM. After all, if applied well, the use of different kinds of data does not only lead to improvement in the lives of our children in terms of achievement, but also in terms of well-being.