Highlighting considerations in experimental design: the case of multimeters

The instructional lab setting has been found to be dominated by prescribed tasks and pre-prepared lab kits. This was explained by teachers’ need to guide students to simultaneously progress through a lab curriculum, which prompts them to standardize the lab experience. Nevertheless, prominent professional associations have persistently called to better represent experimental research practices in the lab, and to grant students more agency in the experimental process by orienting them towards more open-ended lab experiences. This paper reports a lab activity designed to advance students’ agency in the practice of experimental design, in a setting governed by a high-stakes national matriculation exam. Three hundred teachers of advanced level high-school physics experienced the lab activity in a national network of professional learning communities (PLCs). The activity was anchored in an experiment to determine the relationship between the current through a battery and its terminal voltage. It was designed to problematize students’ considerations underlying the choices of the location of the voltmeter and the measuring scale of the ammeter, and the possible implications for the validity of the experimental results; e.g. control of the variables, as well as the range and the accuracy of measurements. Teachers first performed the lab activity as learners, discussed it in the PLC meeting, and finally reflected on their experience. Individual responses to the lab worksheets and the reflections were analyzed. Initially, teachers’ considerations did not portray key aspects related to the validity of the experimental results, such as how design choices related to the location of the voltmeter and the ammeter measuring scale impacted the accuracy and range of the measurements and the control of variables. The teachers were highly engaged in the peer discussion in the PLC and found the lab activity valuable in raising students’ awareness of important considerations in experimental design.


Introduction
Current position papers (National Research Council 2013, Kozminski et al 2014 have called to better represent experimental research practices in science instruction, and to grant students more agency in experimental work. These experimental research practices include the stages in an investigation such as experimental design, involvement in the planning of the experimental setup to investigate a phenomenon in the real world, selecting and applying the measurement tools, analyzing and representing the data, constructing empirical and theoretical models, and communicating scientific arguments. They also include self-regulation and the teamwork required for an extended, explorative process (Etkina et al 2010).
Setting experimental research practices as key learning goals differs significantly from the widespread approach that sees experimental work as a way to reinforce classroom instructionan approach that was not found to be effective (Holmes and Wieman 2018). However, teachers' views and instructional practices are shaped by their past experiences (Schoenfeld 1998, McDermott et al 2000. In the context of the lab, they shape their instructional practices in the prevailing traditional labs. These traditional labs are often characterized by the use of pre-prepared lab kits designed to guarantee that the apparatus is 'operator proof', thus allowing instructors to minimize the time to achieve the goal of reinforcing classroom instruction. However, such kits limit students' agency over experimental design (Wieman 2015). This paper reports a lab activity designed to advance students' agency in the practice of experimental design. It requires students to investigate the relationship between the current through a battery and its terminal voltage. This activity elicits students' considerations related to the choice of the location and the measuring scale of multimeters, and the possible implications for the validity of the experimental results; e.g. control of the variables, as well as the range and the accuracy of measurements.
The lab activity was part of a long-term professional development (PD) process aimed to promote experimental design considerations in the instructional lab (Levy et al 2021). It was carried out in a national network of professional learning communities (PLCs) for high-school physics teachers (Eylon et al 2020). In the PLC activity, the teachers performed the lab activity as learners, and discussed the target learning outcomes and the expected instructional challenges with respect to the implementation of this activity in their classrooms.
To allow for large-scale implementation, and to address the constraints in which most teachers operate, the lab activity involved a modest restructuring of a lab manual that relates to a widely used experimental kit and highlights a few selected experimental research practices. The design of the activity took two factors discussed in the educational literature into account: (a) the design of activities for the instructional lab to support the development of experimental research practices; (b) students' difficulties in understanding concepts in electric circuits related to the lab activity.

The design of activities in the instructional lab to support the development of experimental research practices
International comparisons have shown that the instructional lab setting focuses predominately on developing students' theory-based conceptual frameworks rather than experimental practices. This can be accounted for by structural features; namely, the need to manage lab work given the time constraints, and the curriculum where all students progress simultaneously and teachers tend to standardize the lab experience (Tiberghien et al 2001, Abrahams and Millar 2008, Wieman 2015. Reformed instruction in the lab can implement several approaches. One approach, a prototype of which is the real-time physics inquirybased sequence (Sokoloff et al 2007), focuses on the construction of physical models for observed physical phenomena, and the modification of common conceptions to refine the understanding of general principles of physics. The interactions with lab apparatus follow a learning cycle consisting of prediction, observation, comparison, analysis and quantitative experimentation, and are less structured than in traditional labs, to provide opportunities for discussions of students' ideas. The findings indicate that in this approach students do indeed develop a better conceptual understanding. Other more recent reforms in lab instruction integrate the real-time physics approach (Doucette et al 2019). While this approach was found to promote a meaningful discussion about physics concepts, students were less engaged in decision making about how to collect, process, or present their data.
Another approach, utilized today in several introductory level labs across the US, is the investigative science learning environment (ISLE; Etkina et al 2010), which centers students' work on the experimental design of observational and testing experiments. Students generate scientific evidence and explanations while planning and carrying out experimental investigations. While ISLE supports students' learning of the concepts and laws of physics, the added value of this environment as compared to the real-time physics approach is its emphasis on the experimental research process: observing a series of carefully selected experiments, using the available tools to synthesize patterns in the data, devising possible physical explanations for these patterns, and testing them.
Yet another approach is made up of activities operationalizing some specific sub-goals from the set of experimental research practices recommended in position papers, rather than the whole investigative process (Holmes and Smith 2019); e.g. control of the variables (Boudreaux et al 2008) or which paradigms students use with regard to measurement uncertainty when evaluating their experiments (Pollard et al 2020).

Students' difficulties understanding concepts in electric circuits related to lab activities
Linking experimental observations and measurements to physics concepts and principles constitutes a significant cognitive challenge. In the context of the lab activity presented here on electric circuits, studies have shown that students find it difficult to apply formal concepts; e.g. current, voltage or resistance and to reason qualitatively about the behavior of the circuit. Students tend to focus on the current as the primary concept in solving problems and confuse terms, often assigning the properties of current to voltage and/or resistance. They frequently rely on current while ignoring or greatly underestimating the importance of the resistance of circuit components. Many students solve problems using the V = IR equation even when it is not applicable, and do not apply a mental model of voltage. Students commonly perceive the battery as a source of constant current, and the concepts of emf and internal resistance are not well understood. In general, students have problems dealing with a simultaneous change of several variables in the circuit (Cohen et al 1983, Millar and Beh 1993, Liégeois et al 2003, Engelhardt and Beichner 2004. The use and interpretation of formal representations of an electric circuit are known to be difficult. In particular, students fail to treat meters as circuit elements, and do not regard the ammeter and voltmeter as instruments that should be connected such that they do not affect the current or voltage to be measured. These issues are not successfully addressed by standard textbooks or traditional lab manuals (McDermott and Shaffer 1992). Standard textbook representations may even reinforce the belief that the experimental circuit in the lab will behave exactly like the model circuit presented in the textbook. Indeed, developing experimental research practices and agency in the lab while enhancing students' conceptual understanding can create a significant cognitive load (Chandler and Sweller 1991). This is because experimental research involves the modeling of 'messy' real-world phenomena, whereas instructional strategies designed to promote a conceptual understanding commonly use simplified scenarios that focus students' attention on specific difficulties.

S Levy et al
The lab activity discussed here was aimed to problematize aspects of experimental design in the context of a simple electric circuit. Below we describe the PLC lab activity, and a study of teachers' considerations with regard to experimental design. In particular, we discuss their choices of the location and the measuring scale of the multimeters. Finally, we present the possible implications for lab instruction.

The PLC lab activity
We first describe the PLC setting, the lab activity, and finally the PLC activity.

The PLCs
The national network of PLCs for advanced-level high-school physics teachers served as the setting for the PD activity. The design of the PLCs adheres to the cognitive apprenticeship paradigm (Collins et al 1991), which situates the PD in the teachers' practice and provides opportunities to collaboratively reflect on their classroom experiences. The PLCs network operates in a 'fan model' (figure 1): 24 Teacher-Leaders (TLs), all practicing high-school physics teachers, participate in a PLC operated by the PER group at the Weizmann Institute of Science (TL-PLC). Simultaneously, they lead 11 nationwide regional PLCs of physics teachers (N ≈ 275, teaching ∼15 000 high-school physics students). The TL-PLC and the regional PLCs meet alternately, twice a month, during the school year (for a total of 60 h). The TLs first experience each new instructional activity as learners in the TL-PLC and reflect on its rationale and the expected learning outcomes; then they implement the new activity in their own classes, and in the next TL meeting they reflect collaboratively on their classroom experiences and suggest improvements to the activity. Later on, they implement the new activity in the regional PLC in a similar process (Levy et al 2018). The vast majority of the PLC teachers participate in the program for several years.

The lab activity
The PLC teachers conduct their lab lessons in the high-stakes setting of a national lab matriculation,  an oral examination where students demonstrate their ability to carry out an experiment and defend their lab reports portfolio collected over 3 years. The lab lessons usually cater to 20-25 students, working in pairs with a pre-prepared lab kit and tightly prescribed lab manuals.
The lab activity here involved an experiment to determine the relationship between the current through a battery and its terminal voltage, and measure the internal resistance of the battery (figure 2). The lab kit was composed of the following components 1 : a 1.5 V battery, two digital multimeters (one serving as a voltmeter, and one 1 The values of the components described here are based on the typical school lab kit used in the TL-PLC lab activity. School labs are equipped with a variety of similar lab kits with different components. Hence, some of the values may be different, but the considerations remain the same. as an ammeter), a rheostat, and the connecting wires (assumed to have negligible resistance). As discussed below, the ammeter, which is usually treated as 'ideal', emerged in this setup to be 'nonideal', since its internal resistance (R ⩾ 0.1 Ω) was shown to be non-negligible in relation to that of the rheostat (0 < R < 15 Ω) and the battery (≈0.2 Ω < r < ≈1 Ω).
The accompanying lab worksheet was designed to problematize the considerations underlying the experimental design. We focus here on two worksheet questions that targeted design considerations related to the location of the voltmeter (Q1), and the measuring scale of the ammeter (Q2).

Q1:
Could a different location of the voltmeter influence the measurements? If so, how?
Exemplary answer 1: The location of the voltmeter may affect the validity of the measurement, since the voltmeter reads a different voltage depending on whether it was connected across the battery (figure 3(a)) or across the rheostat ( figure 3(b)). This difference is due to the fact that the lab ammeter is not 'ideal', namely, its internal resistance cannot be neglected in relation to that of the rheostat (R max ≈ 15 Ω). Since the terminal voltage of the battery is the sum of the voltage across the rheostat and the voltage across the ammeter, when the voltmeter is connected across the rheostat it does not read the correct value of the terminal voltage of the battery. Thus, to measure the terminal voltage, it is better to connect the voltmeter across the terminals of the battery ( figure 3(a)).  a typical lab kit the battery emf is ∼1.5 v and the resistance of the rheostat is in the range of 0 < R < 15 Ω, the expected currents are 100 mA ⩽ I ⩽ 3 A (this is an estimation, based on the given components). Thus, to encompass the whole range of measurements, the appropriate ammeter scale is 10 A, which enables the constant internal resistance of the ammeter and therefore control of the variables. However, to increase the accuracy of the measurements, the appropriate ammeter scale is 200 mA, which can only be used for part of the measurements in the expected range of values. In this case, the ammeter scale needs to be increased during the experiment to measure higher currents. Thus, there are conflicting considerations: shifting the ammeter scale during the experiment increases the accuracy of the measurements, but it causes a change in the internal resistance of the ammeter during the experiment and thus undermines the control of variables. In the given experimental setup, the decision to start with the 200 mA scale and shift to 10 A scale as needed is better, since many of the measurements can be taken in the 200 mA scale and the change in the ammeter's internal resistance is negligible.

The PLC activity
The activity in the PLCs adhered to the Knowledge Integration design guidelines (Linn and Eylon 2011) by providing opportunities for teachers to (a) explicate existing ideas; (b) add peers and research-based ideas: (c) sort out ideas and (d) reflect. In particular the teachers (see figure 5): (a) individually examined a lab activity intended for their students and filled in the accompanying lab worksheet; (b) worked in small groups and discussed their answers; (c) sorted out the considerations underlying the experimental design in a PLC discussion, conducted by the TLs; (d) wrote a reflection paper at the end of the PLC meeting. They were encouraged to implement the activity in their classrooms.

Methodology
The teachers' considerations with regard to experimental design were examined at the beginning and end of the PLC activity: Which experimental design considerations were expressed by the teachers? How did these considerations relate to the validity of the experimental design?

Data collection
At the beginning of the PLC meeting the teachers were asked to fill in the lab worksheet individually, including questions 1 and 2 (see section 2.2) related to the location and the measuring scale of the multimeters. At the end of the meeting the teachers wrote a reflection paper, articulating their insights following the PLC activity.

Participants
Approximately 100 teachers (members of 11 regional PLCs) completed the individual lab worksheet in its online version, of whom ∼67 teachers completed the online version of the individual reflections at the end of the activity as well. The participants had a wide range of teaching experience (1-49 years, Md = 7) and experience in preparing students for the national matriculation lab exam (0-30 years, Md = 5). They represent diverse school profiles-a broad range of socioeconomic levels. More than half are secondcareer teachers, who transferred to physics teaching after an engineering career. The gender distribution was ∼1/3 females.

The location of the voltmeter 4.1.1. Beginning of the PLC activity.
Most of the teachers (∼¾) responded that the measurements could be affected if the voltmeter was in a different location (figure 6). A quarter of the teachers referred specifically to the implications when connecting the voltmeter across the rheostat instead of directly across the battery (see figures 3(a) and (b)), but did not explain why, while another quarter explained that the results are affected by the internal resistance of the ammeter, which cannot be neglected in this experiment. A quarter of the teachers responded that a different location of the voltmeter would not influence the measurements.

End of the PLC activity.
Most of the teachers (79%) referred to the contribution of the activity to their awareness of the ammeter's internal resistance when considering the appropriate location of the voltmeter (figure 7). As mentioned above, roughly 25% referred to the role of the ammeter's internal resistance in the lab worksheet at the beginning of the PLC meeting. A little more than half (55%) of the teachers noted that the discussion during the PLC meeting helped them refine their considerations: 36% referred to the ammeter's internal resistance in the lab worksheet and mentioned it again as meaningful for them in their reflection papers, and 19% referred to this only in their reflections. Crucially, 21% of the teachers did not refer at all to the considerations underlying the voltmeter's location.  pre-estimating the values that are expected to be measured in the experiment).

End of the PLC activity.
In their reflection papers at the end of the PLC activity the teachers highlighted the numerous features that had been meaningful to them (figure 9). The teachers mainly noted the implications of the ammeter's internal resistance and considerations related to choosing its scale. Additionally, the teachers acknowledged the activity's contribution to other considerations in experimental design, in particular the need to strengthen students' grasp of the theory behind the experiment and increasing students' agency in lab work.

Classroom implementation
After the PD activity, most of the PLC teachers acknowledged the importance of involving their students in experimental research practices and granting their students more agency in experimental design. Almost 50% of the teachers who taught the relevant grade implemented the described lab activity in their classrooms, in spite of the strict curriculum and the time constraints. A further 25% used the activity with some changes and adaptations. The preliminary results indicate that the teachers were satisfied with the outcomes and intend to use the activity next year as well.

Discussion and implications
The work described here is part of a large scale, long-term PD project in the national network of PLCs of high school physics teachers. The project aims to address the calls to better represent experimental design practices in the instructional lab (National Research Council 2013, Kozminski et al 2014 by means of lab activities tailored to teachers from a variety of schools. To address the constraints in which most of these teachers operate, a PLC activity was designed, where teachers experienced a modestly restructured widespread lab as students and reflected on their experiences from their perspective as teachers. The lab activity problematized specific aspects of experimental design (e.g. control of variables, the range and the accuracy of measurements), by probing considerations underlying the location of the voltmeter and the measuring scale of the ammeter.
The ammeter's internal resistance cannot be neglected in this experimental setup. However, at the beginning of the PLC activity, most of the participants did not consider its possible implications for the validity of the measurements. The ammeter's internal resistance is indeed relatively small and in most familiar experiments is considered negligible. Accordingly, the teachers were surprised to discover that the voltmeter reads different values when it is connected across the rheostat than when it is connected across the battery, since these two locations are considered the same in textbook problems. Thus, the difficulty of recognizing meters as circuit elements surfaced, pointing to the challenges in the use and interpretation of formal representations of electric circuits (McDermott and Shaffer 1992), and highlighting the differences between the experimental context of the lab and the formal model of electric circuits in the textbook. Teachers' considerations in determining the optimal scale of the ammeter revealed that many teachers preferred the maximal ammeter scale for maintenance reasons, rather than validity related considerations; e.g. control of the variables, the range and the accuracy of measurements. These key design considerations were problematized by the lab worksheet and provoked a rich discussion in the PLC explicating the considerations one should take into account when designing an experimental setup.
Although we focused here on a sample of teachers, it can be assumed that these are widespread lab norms in the national education system in which the PLC teachers operate. The literature on students' views on the nature of experimental investigation and their role with respect to a range of experimental practices (Wilcox and Lewandowski 2018) reveal that students have a variety of beliefs about the nature and importance of experimental physics. Their study showed that students often enter and leave lab courses with ideas about experimental physics that are inconsistent with those of practicing physicists. Transformed instructional approaches, which aim to develop students' practical lab skills rather than reinforce physics content, and include open-ended activities that provide opportunities for students to take some agency in what and how physical phenomena are investigated, encourage students to adopt beliefs about the nature and importance of experimental physics that are consistent with those of practicing physicists.
Most of the teachers perceived the PLC activity as contributing to their awareness of considerations related to the validity of the experimental setup, as well as other experimental design considerations. Teachers' experiences in the PLC meetings motivated them to implement the activity in their lab lessons and to grant their students more agency in experimental design, in spite of the strict curriculum and the time constraints. The preliminary results indicate that the teachers were satisfied with the outcomes and intend to use the activity next year as well. These findings emphasize the importance of problematizing and discussing experimental design considerations and their realization in a specific experimental setup, both in teachers' PDs and in the classroom.

We hereby declare that
The Weizmann Institutional Review Board (IRB-Education) reviewed the research project prior to data collection. The research was conducted in accordance with the principles embodied in the Declaration of Helsinki and in accordance with local statutory requirements. All the participants volunteered to participate in the research and signed a written Informed Consent Form. The participants anonymity was protected.

Data availability statement
The data that support the findings of this study are available upon reasonable request from the authors.