Recent studies have reported negative relationships between inquiry-based instruction and science achievement in international large-scale assessments (ILSAs; Areepattamannil et al., 2011; Cairns, 2019; Cairns & Areepattamannil, 2019; Lavonen & Laaksonen, 2009; OECD, 2016a; Oliver et al., 2019). The inquiry-based instruction constructs employed in these assessments are conceptualised in broad terms. For example, the Program for International Student Assessment (PISA) 2015 survey included an inquiry-based instruction scale that encompassed items describing inquiry activities that had been previously categorised at varying levels of teacher guidance (Jiang & McComas, 2015) and four conceptual domains (OECD, 2014). The experimental inquiry-based learning literature reports positive effects on student science performance (e.g. Espinoza, 2015; Furtak et al., 2012; Minner et al., 2010). The largest postive effects were reported for inquiry approaches that focused on (a) specific conceptual domains (epistemic; Furtak et al., 2012; Minner et al., 2010), (b) provided structured inquiry learning experiences (Furtak et al., 2012; Hmelo-Silver, 2011; Hmelo-Silver et al., 2007; Salamon et al., 1991), and (c) where students had sufficient prior subject (van Riesen et al., 2018) and process skills knowledge (Bredderman, 1985; Klahr & Nigam, 2004).

There is limited range of correlational research using ILSA data investigating teacher-directed approaches and their associations with science achievement (Cairns, 2019; Denoël et al., 2017). This study intends to address this by exploring the effects of teacher-directed practices (measured as individual items in the student questionnaire) included in the PISA 2015 scaled indexes, and the mediating effects of teacher explanations on any significant relationships found. All analyses were carried out using a representative sample of 15-year-old students in Australian schools from the PISA 2015 dataset (OECD, 2017a). The PISA project is an OECD initiative involving a standardised assessment of the capacity of students to apply reading, mathematics, and science knowledge to authentic, workplace-related, contexts. The PISA 2015 survey focused on scientific literacy. The use of the term scientific literacy to reflect student performance in science (science achievement) illustrates that the application of scientific knowledge to authentic contexts is central to the OECD’s beliefs relating the intended outcomes of science education (OECD, 2017b). The construct of scientific literacy requires students to be able to explain phenomena scientifically, evaluate and design scientific inquiry, and interpret data and evidence. These competencies consist of three types of knowledge: (1) content, (2) procedural, and (3) epistemic. Accordingly, the terms scientific literacy and science achievement will be used interchangeably in this study.

Teacher-Directed Instruction

Teacher-directed instruction (TDI) is a construct included in the PISA 2015 student background questionnaire that measures student responses to items relating to teacher-led instruction. The scale measures students’ exposure to teaching practices whereby the teacher is wholly responsible for initiating and structuring student learning (OECD, 2014). An early form of TDI, coined Direct Instruction (DI), was originally developed by Engelmann in the 1960s (Bereiter & Engelmann, 1966) and focuses on conceptual modelling, conceptual reinforcement, and informative feedback targeting progression towards mastery (Joyce et al., 2000). Direct Instructional theory is underpinned by two key concepts: (a) learners can only access new materials and tasks when they have mastered the prerequisite knowledge and skills and (b) the instruction must be explicit and unambiguous. DI theory is not predicated on constructivism and the uniqueness of the learner but rather on the mastery of learning through carefully sequenced and highly structured examples (Stockard et al., 2018). Upon mastery of essential knowledge and skills, DI theory states that this existing “foundation” facilitates the next stage of learning. Thus, learners are required to have achieved mastery before progressing and the curriculum itself must be designed to provide clear, progressive learning pathways (Stockard et al., 2018). For example, Gersten et al. (2001) reported a conceptualisation of DI that enabled a focus on the learning of discrete strategies and skills, educational concepts, and higher-order thinking skills involving the teaching of skills, initially in isolation (e.g. conclusion writing in a range of contexts) followed by the application of these skills (in combination with other skills, as appropriate), in ever more complex scenarios. Due to the selective processes in the DI model, learning must also be rewarding for the learners. This is achieved through continuous positive reinforcement (of learning behaviours) and frequent celebrations of student successes (such as progression in the successful acquisition of knowledge and skills; Stockard et al., 2018).

As such, DI encompasses both curriculum content and instructional approaches. Similar teacher-directed methods that are concerned only with teaching approaches (how to teach) include “direct instruction”, “explicit teaching” (e.g. Stanovich, 1980), and “systematic instruction” (Katz, 2007). These approaches conceptually overlap with DI teaching practices. For example, Rosenshine (2012) described essential direct instructional practices for successful teachers from a cognitive science, classroom practice, and cognitive support perspective. The 17 principles outlined included familiar practices such as giving clear and detailed instructions and explanations, providing a high level of active practice for all students, providing guidance for students as they begin to practice, providing model solutions to problems and modelling processes and strategies, providing regular feedback, re-teaching and re-explaining as necessary, ensuring students are well-prepared for independent practice, and supporting them when they start working independently (Rosenshine, 2012).

There is also a growing consensus in the instructional quality literature that the four basic elements of quality teaching (instructional clarity, cognitive activation, discourse features, and a supportive climate) include multiple elements of TDI (Klette et al., 2017). For example, cognitive clarity is considered an essential condition to positively influence students’ achievement. Cognitive clarity promoting approaches include explicit teaching, not only of domain specific concepts, but also how and when to use specific strategies (Afflerbach et al., 2013) supported by the modelling of how strategies can be applied in authentic contexts (Afflerbach et al., 2013; Gersten et al., 2009). Another component of effective instruction is cognitive activation through high-quality, challenging learning tasks (Klette et al., 2017) where connections between concrete and abstract concepts, and new and old content, skills and learning strategies are made clear through explicit instruction (Lipowsky et al., 2009).

Teacher-Directed Instruction in Practice

Implementation Studies

In terms of TDI in science lessons, a study designed to compare TDI, guided instruction, and minimal instruction when children are learning to design investigations (Hushman & Marley, 2015) reported that TDI was more effective in terms of learning process skills, recalling conceptual knowledge, application of process skills to different contexts, and effective evaluation of experimental designs.

A meta-analysis of 50 years of DI effectiveness literature showed that DI approaches were consistently positive across all subject areas (Stockard et al., 2018). More years of intervention and increased frequencies of exposure to DI methods further strengthened the observed effects (Stockard et al., 2018). Also, DI and other TDI approaches are reported as effective in research supporting children with learning difficulties (e.g. Datchuk, 2017; Head et al., 2018). Although there is a body of evidence to suggest that explicit instructional approaches are effective when working with academically at-risk students, there is no inherent reason why these students should learn differently to higher performing students (Martin, 2015). The key for an effective teacher is knowing when (and how often) to use appropriate instructional approaches (Martin, 2013).

In a review of TDI and academic achievement studies, Liem & Martin, 2013 concluded that teacher-led approaches (where the teacher’s role is an activator of learning) were more effective than open-ended, student-led approaches (where teachers served as facilitators of learning) due to the lowering of cognitive demands in terms of the working memory and executive functioning.

Correlational Studies

An analysis by Denoël et al. (2017) using the PISA 2015 dataset found that students who experienced a blend of TDI and inquiry-based instruction had the best science outcomes in European countries. They suggested that content mastery is a pre-requisite for successful inquiry-based learning evidenced by a positive correlation between science literacy scores and students experiencing teacher-directed approaches in most lessons and inquiry-based approaches in some lessons. Another study demonstrated, through the disaggregation of the inquiry-based instruction scale into the component items, that the majority of teaching strategies included in the scale had a negative correlation with science attainment (Cairns, 2019). However, in a similar vein to the McKinsey report (Denoël et al., 2017) an “optimum” level of inquiry approaches was posited. For example, the frequency of students spending time in the laboratory doing experiments associated with the highest science scores was in “some” lessons.

Although the recent addition of the TDI scale to the PISA 2015 background questionnaire enables researchers to test for associations between teacher-led instructional practices and science achievement, the items in the TDI scale do not appear to relate closely to the practices described above (OECD, 2016b) or to other detailed descriptions of teacher-directed instructional approaches (e.g. Hattie, 2012).

In the previously mentioned study of the effects of items from the inquiry-based learning scale, Cairns (2019) reported that two of the items that involved teacher explanations ‘the teacher explains how a science idea can be applied’ and ‘the teacher clearly explains relevance of science concepts to our lives’ exhibited a significant, positive relationship with science achievement. The explanation given for this relationship suggested that this positive association was due to the importance of students’ understandings of how scientific knowledge is applied and how it relates to their everyday lives. However, these statements consist of two parts: (1) the teacher-led explanation of a concept and (2) the concepts themselves (relevance and applications of science). It is possible that the mode of instruction (teacher-led explanations) could also be a factor contributing to the apparent success of these two approaches.

The Present Study

Considering the limited body of research regarding TDI in the correlational literature relating to ILSAs, this study investigated the relationships of teacher-led approaches on science achievement using the PISA 2015 dataset. We investigated these relationships by disaggregating the inquiry-based instruction and TDI scaled indexes, and identifying and analysing the teacher-directed approaches included in these constructs (while controlling for the effects of student-led approaches). The teacher-directed strategies identified from the 13 items (see Appendix for a full list of the items) are “the teacher explains how a science idea can be applied”, “the teacher clearly explains the relevance of science concepts to our lives”, “the teacher explains scientific ideas”, “the teacher discusses our questions” and “the teacher demonstrates an idea”. This identification is derived directly from the wording of the items and informed by prior research (Hattie, 2012; Lau & Lam, 2017). Lau and Lam (2017) identified the first three items listed above as an “interactive application” construct through exploratory factor analysis of the Hong Kong data. The item “the teacher discusses our questions” was included as a component of TDI construct conceptualised by Hattie (2012) and “the teacher demonstrates an idea” item was included due to the unambiguous wording of the item.

For the next step, we investigated the mediating effect of the teacher explanation item (“the teacher explains scientific ideas”) on the significant, teacher-directed approaches from the inquiry-based instruction scale to explain any associations found between these variables. These approaches were applied to answer the following research questions:

  1. 1.

    What is the relationship between the reported frequency of experiencing teacher-directed instructional approaches in science lessons and science achievement?

  2. 2.

    Is the relationship between the reported frequencies of teacher-led instructional approaches in science lessons and science achievement mediated (or explained) by the frequency of students experiencing teacher-led instructional explanations?

Analysis

Data

The data for this study were drawn from the questionnaires in the PISA 2015 database (http://www.oecd.org/pisa/data/2015database) for Australia only. The sample consisted of 14,530 15-year-old students (male = 7367, 51%; female = 7163, 49%) from 758 schools. The schools were selected to represent the national target population (OECD, 2017a) and the sampling frame included any school that could currently be educating 15-year-old students. Schools were allocated to sampling strata appropriate to the Australian school system such that each type of school was proportionately represented. The stratification variables used in the Australian school sample included: state/territory (NSW, QLD and so on); sector (Catholic, independent or government schools); and modal grade (year 10; OECD, 2017a). For example, there were 23 independent schools from Queensland with year 10 students enrolled included in the PISA 2015 sample.

Measures

The outcome measure for this research was the students’ 10 plausible value scores in the scientific literacy domain. These plausible values, which are a range of multiply imputed values derived from the student’s performance using IRT scaling, were combined with a regression model based on the student’s answers to the background questionnaire (OECD, 2009). As a result, we ran 10 separate analyses and pooled the results to yield the parameter estimates for a given model.

For the purposes of this study, individual items from the inquiry-based instruction and TDI constructs (raw scores) were used as predictor measures, as the effects of specific practices (described by individual items on the scales) were our area of interest.

Statistical Analysis

To answer research question 1, a two-level model was fitted that included the inquiry-based instruction items and the TDI items as predictors and the student-, school-level variables as covariates. To address research question 2, the mediating effect of the TDI scale item ‘the teacher explains scientific ideas’ on the teacher-led inquiry items (‘the teacher explains how a science idea can be applied’ and ‘the teacher clearly explains relevance of science concepts to our lives’) and their associations with scientific literacy was estimated by running two-level models with each explanatory variable regressed on the mediator variable in order to estimate the direct (the relationship between the dependent and independent variable), indirect (the product of the relationship between the independent, mediator and dependent variable), and total effect (the sum of the direct and indirect effects) on the outcome variable.

Further details regarding the individual items and their item codes, the control variables employed, and the specifics of analysis methods used are included in the Appendix.

Results

The means, mean differences (between Australia and the OECD average for each item), standard deviations, and correlations between the predictor variables used in the models are shown in Table 1. Items belonging to the same scales are positively correlated and items from different scales were negatively correlated with each other. The difference between responses from students in Australia compared with the mean responses across all regions included in PISA 2015 did not show any interesting deviations. Students in the Australian PISA sample reported experiences in their classrooms that are largely similar to the mean reported experiences across all sampled OECD regions. The correlations between the independent variables were sufficiently low (< 0.7), indicating an acceptable level of multicollinearity, which led to the inclusion of all 13 items in the model.

Table 1 Descriptive statistics for inquiry and teacher-directed instruction items
Full size table

The relationship between individual inquiry and TDI items with science literacy was estimated using a two-level MLM approach. The multi-level intercept-only model estimated a grand mean science achievement score of 504.92 (γ00 = 504.92), and statistically significant variation in the random effects for science literacy score values between students (level 1: σ2 = 8294.40) and across schools (level 2: τ00 = 2656.66). The intra-class correlation coefficients (ICCs) indicated that 78% of the science achievement score variance occurred between students and 22% of the variance occurred across schools justifying the use of a multi-level analytical approach.

Research Question 1

The relationships between the 13 items from the PISA 2015 inquiry-based and TDI scales as estimated by the full MLM model are shown Table 2.

Table 2 Instructional approaches by item predicting science achievement
Full size table

The approaches of giving students opportunities to explain their ideas, asking students to draw conclusions from an experiment they have conducted, allowing students to design their own experiments, having class debate about investigations, the teacher discussing students questions, and the teacher demonstrating an idea had no significant, linear associations with science achievement, when accounting for other teaching approaches and student- and school-level demographics. A range of teaching practices (as described by the items) were found to have a significant, negative relationship with science achievement. Where a science lesson involves having a whole class discussion with the teacher a one-unit increase in the frequency of experiencing this approach (for example, from “some” lessons to “most” lessons) resulted in a nearly 10-point decrease in the association with science literacy (B = − 9.82, p < 0.001). Similar negative relationships were observed for the items “students spend time in the laboratory doing practical experiments” (B = −7.95, p < 0.01), “students are required to argue about science questions” (B = −8.71, p < 0.001), and “students to do an investigation to test ideas” (B = −7.14, p < 0.001).

Classroom practices that were associated with a significant increase in science achievement were teacher-directed approaches that included the teacher explaining how a science idea can be applied (B = 8.56, p < 0.001), the teacher clearly explaining the relevance of science concepts to students’ lives (B = 6.02, p < 0.01) and, the teacher explaining scientific ideas (B = 29.61, p < 0.001). The positive relationship for the teacher explaining scientific ideas was by far the largest of all the teaching approaches in the model. A one-point increase in the frequency of experiencing teacher explanations about scientific ideas, on average, corresponds to a 29.61 point increase a student’s science score. In other words, the average student experiencing teacher explanations in all science lessons is predicted to score nearly 90 points higher in scientific literacy than a student that never experiences a teacher delivering scientific explanations. This is approximately equivalent to 2.25 years of learning progression (Jerrim et al., 2019). In a previous study, Cairns (2019) demonstrated that the explicit, teacher-led approaches included in the inquiry-based instruction scale were highly, positively related to science achievement (albeit for 69 countries). The results described above suggest that this relationship is weaker when items from the TDI scale are included in the model–specifically the approach of the teacher explaining scientific ideas. This observation suggests the presence of a mediation effect, which is addressed by research question 2.

Research Question 2

The mediation analysis showed a statistically, significant indirect effect of “the teacher explains how a science idea can be applied” on science achievement through the mediator “the teacher explains scientific ideas”, estimate = 6.82, 99% CI [5.20, 8.45]. There was also a statistically, significant direct effect on science achievement, estimate = 8.62, 99% CI [3.48, 13.78]. The total effect was 15.45, 99% CI [10.28, 20.62]. The direct effect in the mediation model remained significant (see Fig. 1), suggesting the presence of a partial mediation effect. The total effect of the teacher explaining how a science idea can be applied is 15.45 points in science achievement. 6.82 of these points are related to the effect of the teacher explaining how a science idea can be applied through the process of explaining scientific ideas. 8.63 points of the total effect on students’ science scores are related directly to the teacher explaining how a science idea can be applied (without influence from the teacher explaining scientific ideas).

Fig. 1
figure 1

The regression coefficients for the association between the reported frequency of a teacher explaining how a science idea can be applied with science achievement and a teacher explaining the relevance of science concepts to the lives of students with science achievement, mediated by the reported frequency of a teacher explaining scientific ideas. All teaching practices, student- and school-level demographics, and socio-economic factors were included in the analyses; but for clarity of presentation purposes, only the main measures of interest are shown. **p < 0.001

Full size image

There was also a significant indirect effect of “the teacher clearly explains the relevance of science concepts to our lives” on science achievement through the mediator “the teacher explains scientific ideas”, estimate = 7.02, 99% CI [5.26, 8.79] (see Fig. 1). There was also a significant direct effect, estimate = 6.19, 99% CI [1.36, 11.02], and total effect, estimate = 13.21, 99% CI [8.40, 18.03]. As such, the total effect of the teacher clearly explaining the relevance of science concepts to students’ lives is 13.21 points of science achievement. 7.02 of these points are related to the effect of the teacher clearly explaining the relevance of science concepts through the process of explaining scientific ideas. 6.19 points of the total effect on science scores are related directly to the teacher clearly explaining the relevance of science concepts to students’ lives.

Discussion

The approaches found to be associated with increased levels of scientific literacy all involve teacher-led practices. These include, “the teacher explains scientific ideas”, “the teacher explains how a science idea can be applied”, and “the teacher clearly explains the relevance of science concepts to our lives”. However, some likely teacher-led teaching strategies (namely, “the teacher discusses our questions” and “the teacher demonstrates an idea”) did not have a statistically significant relationship.

These results are noteworthy as teacher demonstrations are common practice in science lessons and are used to build conceptual knowledge through methods such as conceptual conflict and anchor-bridge models (Thijs & Bosch, 1995). The pedagogical rationale for a teacher using demonstrations usually revolves around the students’ lack of mastery regarding the manipulative and process skills involved (van den Berg & Giddings, 1992). However, other (non-cognitive) motivations may include time pressures, limited resources, and health and safety concerns. The lack of a relationship between the frequency of teacher demonstrations and science achievement appears counter-intuitive for such a well-used teaching strategy but there are some studies that question the efficacy of science teacher demonstrations (Roadruck, 1993; Roth et al., 1997). One important aspect of successful demonstrations is the alignment of the content with the cognitive level of the students (Roadruck, 1993). For a student to accurately decode the meaning of a demonstration, they must have sufficient understanding of the underlying principles of the phenomena they are observing (Roth et al., 1997). As the PISA background questionnaires do not provide information regarding the quality of teacher demonstrations, one possible explanation is that the effectiveness of teacher demonstrations varies considerably between schools and teachers. A similar issue may exist with regards to the item “the teacher discusses our questions” where low quality classroom discussions (also see the item “a whole class discussion takes place with the teacher”, below) may be taking place. There is evidence that classroom discussions, especially discussions involving students talking about their learning that lead to instructional adaptation, have a positive impact on achievement (Hattie, 2012).

Teaching strategies related to a decreased level of science achievement included the following: “students spend time in the laboratory doing practical experiments”, “students are required to argue about science questions”, “students are asked to do an investigation to test ideas”, and “a whole class discussion takes place with the teacher”. Although students spending time in the laboratory doing experiments results in a negative relationship with science achievement, previous studies have reported a curvilinear relationship whereby there is an optimum frequency (in some lessons), beyond which the relationship is negative (Cairns, 2019; Oliver et al., 2019) possibly due to instructional time constraints. Cairns (2019) suggested that, for laboratory-based work to lead to improved phenomenological understanding, sufficient time should be allocated for reflective opportunities; this would be difficult to provide if experimental work took place in all lessons.

Statistically significant direct, indirect, and total effects were estimated for the mediating effect of “the teacher explains scientific ideas” on the relationship between “the teacher explains how a science idea can be applied” and “the teacher clearly explains the relevance of science concepts to our lives” with science achievement. A previous analysis of the individual items on the inquiry-based instruction scale (Cairns, 2019) suggested that these positive associations were related to the pedagogical effectiveness of applying scientific concepts and relating classroom-based science learning to every-day contexts. However, these findings suggest that teacher explanations themselves play a significant role in the associations of these two items (“the teacher explains how a science idea can be applied” and “the teacher clearly explains the relevance of science concepts to our lives”).

Intuitively, explanations must play a major role in science teaching, and they are certainly ubiquitous in educational environments. Explanations can be delivered in many formats and contexts, and learners generally know when they have received a sufficiently good explanation (Geelan, 2013). Although there is a paucity of research regarding explanations and quality in educational settings (instructional explanations), some researchers have developed conceptual frameworks for understanding the effectiveness of explanations (Wittwer & Renkl, 2008). Examples of instructional explanations in science include explicit reference to the underlying principles behind natural phenomena (such as motion) allowing learners to generalise to other, more complex, contexts (Chi et al., 1989). Effective instructional explanations can be characterised as being aligned with the learner’s current level of prerequisite knowledge, centred on underpinning concepts and principles, and (perhaps most importantly; Webb & Farivar, 1999; Webb et al., 1995) associated with the current learning in such a way that allows for the practical application of the knowledge shared in the explanation (Wittwer & Renkl, 2008). Although the PISA questionnaires only provide information regarding the perceived frequency of experiencing various approaches, it is reasonable to assume that a learner knows when they have received a high-quality explanation as it will have fulfilled their cognitive needs (Keil et al., 2000). It follows then, that when a student reports “the teacher explains scientific ideas” in, for example, most lessons it can be assumed that the teacher explanation was successful in order for the learner to have recognised the existence of the explanation.

Implications for Teacher Development

In terms of classroom practice, instructional explanations should be very carefully constructed to ensure they are effective, as clear, instructional explanations are related to considerably higher science literacy scores. The effectiveness of instructional explanations is dependent on the teacher’s awareness of the learners’ current levels of understanding (namely, the existence of prior and sometimes faulty knowledge). Indeed, types of instructional explanation that are inappropriate for advanced learners can be superfluous (or even detrimental; Wittwer & Renkl, 2008). As such, effective teacher explanations are inextricably linked to high-quality assessment practices.

Once the current level of knowledge and understanding of learners has been ascertained by the science teacher, appropriate scientific instructional explanations should focus on: clarifying the underlying principles of scientific phenomena; clear guidance on when to apply scientific strategies (for example, when to draw on particle theory to solve a scientific problem); modelling of how to apply scientific strategies (for instance, specifically how to draw a conclusion from collected data); demonstrating how to make links in related content (such as abstract and concrete concepts or prior learning with new learning); and explicitly establishing how to reflect on the science learning process.

Limitations

There are three major limitations to the study. First, this study employed a correlational research design. Although correlational research designs, like observational research designs, tend to demonstrate relatively high ecological validity, causation cannot be assumed from correlation. Further research employing longitudinal or experimental research designs is warranted to examine causal relations among the variables of interest in the study. Second, correlational studies of this nature often use indirect measures of instructional practices. Instead of observing teachers’ actual instructional practices in classroom, such studies frequently rely on proxy measures such as students’ perceptions of their teachers’ instructional practices. Future research based on observational studies may better capture teachers’ actual instructional practices in classroom. Third, data for the present study were drawn from only one of the 72 PISA 2015 participating countries and economies, thereby limiting the generalisability of the findings. More cross-national and cross-cultural research is needed to further validate the findings and conclusions of this study.

Conclusions

Despite these limitations, this study provides empirical support for the importance of providing clear, instructional explanations for students, in all science lessons. Although these explanations are directly related to science achievement, they also underpin other teaching approaches (such as applying conceptual knowledge to practice and relating science concepts to the everyday lives of students). Previous studies have also suggested that teacher explanations (as determined from the TDI scale, a scale that has been shown by this study to be dominated by the positive effect of teacher explanations) are likely a pre-requisite for effective inquiry-based instruction (Denoël et al., 2017). Further work in this area should focus on the analysis of the effectiveness of science teaching methodologies that include explicit, teacher-led instruction (for a given concept) prior to students experiencing open, inquiry-based approaches.