Investigating and addressing student difficulties with a good basis for finding perturbative corrections in the context of degenerate perturbation theory

Degenerate perturbation theory (DPT) is a powerful approximation method for finding the energies and the energy eigenstates for a system for which the time-independent Schrödinger equation is not exactly solvable and there is degeneracy in the unperturbed energy spectrum. However, many students struggle with DPT because they have difficulty identifying whether a given basis is a good basis for finding perturbative corrections and determining a good basis for a given system. Here, we first discuss an investigation of student difficulties with determining a good basis and finding the corrections to the energies and energy eigenstates in the context of DPT carried out in advanced quantum mechanics courses by administering free-response and multiple choice questions and conducting individual interviews with students. We find that students share many common difficulties related to this topic. We then describe how the research on student difficulties was used as a guide to develop and validate a quantum interactive learning tutorial (QuILT), which strives to help students develop a functional understanding of a good basis for finding the perturbative corrections in the context of DPT. We discuss the development and validation of the QuILT on DPT and its evaluation in undergraduate and graduate courses.

There has been relatively little research conducted into student understanding of advanced topics in quantum mechanics, e.g. degenerate perturbation theory (DPT) [37,38]. Here, we discuss an investigation of student difficulties with DPT and the development and evaluation of a research-based QuILT that makes use of student difficulties as resources to help them develop a solid grasp of DPT. We first summarise the basics of DPT that students should learn. Then, we describe the methodology for investigating student difficulties and the common student difficulties found. We describe how the difficulties were used as a guide to develop the QuILT and its in-class evaluation in undergraduate and graduate QM courses 1 .

Basics for DPT
Perturbation theory is a powerful approximation method for finding the energies and the energy eigenstates for a system for which the time-independent Schrödinger equation (TISE) is not exactly solvable. The HamiltonianĤ for the system can be expressed as the sum of two terms, the unperturbed HamiltonianĤ When the eigenvalue spectrum ofĤ 0 has degeneracy (i.e. two or more eigenstates ofĤ 0 have the same energy and two or more diagonal elements ofĤ 0 are equal), equations (1) and (2) from non-DPT theory are still valid provided one uses a good basis. For a givenĤ 0 and ¢ H , we define a good basis as consisting of a complete set of eigenstates ofĤ 0 that 1 The findings reported in this article were initially published in the conference proceedings article [37]. Here we expand upon our methodology for investigating student difficulties as well as the methodology for the development of the QuILT to help address these common difficulties.
diagonalizes ¢ H in each degenerate subspace ofĤ 0 . Therefore, the terms y y ¢ ⟨ |ˆ| ⟩ H m n 0 0 in equation (2) for the wavefunction are zero when ¹ m n so that the expression for the corrections to the wavefunction in equation (2) does not have terms that diverge when = E E m n 0 0 . Only if a good basis is chosen, equation (1) is valid for finding the first order corrections to the energies (which are the diagonal elements of the ¢ H matrix as given by equation (1)). SinceĤ 0 is the dominant term and ¢ H provides only small corrections to the energies, we must ensure that the basis states used to determine the perturbative corrections to the energies in equation (1)

Methodology for investigating student difficulties
As can be seen from the brief review in the previous section, there are many concepts that students must consider when applying DPT correctly. It is not surprising that students struggle to develop a consistent and coherent knowledge structure and a functional understanding of DPT. Student difficulties with finding the corrections to the energies and energy eigenstates using DPT were first investigated using five years of data involving responses to open-ended and multiple choice questions administered after traditional instruction in relevant concepts to 64 upper-level undergraduates in a second-semester junior/senior level QM course and 42 first-year physics graduate students in the second-semester of the graduate core QM course. Additional insight was gained concerning these difficulties via responses of 13 students (graduate and undergraduate students) during a total of 45 h of individual interviews. A 'think aloud' protocol was used during the interviews in which students were asked to think aloud as they answered the questions posed without being disturbed [39]. Once the students had answered each question to the best of their ability, we asked them to clarify their reasoning and probed deeper into certain difficulties. The interviews were generally conducted in one sitting, but there were two interviews that took place over the course of 2 d. In all the questions discussed here, students worked through examples involving DPT that are restricted to a three-dimensional Hilbert space (with a two-fold degeneracy inĤ 0 ).
The purpose for restricting the problem solving to three dimensions was to ensure that students focus on the fundamental concepts instead of working through cumbersome calculations that may detract from the focus on why it is important to determine if the initial basis is a good basis to find perturbative corrections. In all the questions discussed, the Hamiltonian operator was given in matrix form and we will refer to the basis used to generate these initial matrix representations of the Hamiltonian operator as the initially chosen basis.
To probe student understanding of a good basis for finding perturbative corrections to the energies and energy eigenstates, we posed questions regarding the following four systems (given by the Hamiltonians H1-H4) in which the Hilbert space is three-dimensional and  is a small parameter (  1). For each system, the normalised basis states are y | ⟩   The basis given in equation (3) is a not good basis for the Hamiltonians H1 and H2 as each ¢ H matrix is not diagonal in the degenerate subspace of the correspondingĤ 0 . The basis given in equation (3) is a good basis for the Hamiltonians H3 and H4 since each ¢ H matrix is diagonal in the degenerate subspace of the correspondingĤ 0 .

Student difficulties
Throughout our analysis of student responses to the multiple choice and open-ended questions, we found that many students struggled to determine a good basis and the corrections to the energies and energy eigenstates. It was often the case that students had difficulty even starting some of the open-ended problems after traditional lecture-based instruction in relevant concepts. We conducted individual think aloud interviews to gain a better understanding of student difficulties. Below, we discuss some of the common student difficulties with DPT found via interviews in the context of a three-dimensional Hilbert space with a two-fold degeneracy inĤ 0 . It was not possible to discern the underlying cognitive mechanism and reasoning for student responses via the students' written responses to multiple choice or openended questions. It was during the interviews that we probed further into the students' reasoning and were able to uncover reasoning for some of the common student difficulties with DPT. When possible, in the discussion below, we will give the percentage of the interviewed students who displayed a given difficulty. We note that certain student responses generated further probing and so those probing questions may not have been asked to all of the interviewed students. Therefore, we will only report the percentage of difficulties for interviewed students for questions that were common to all the interviewed students. In the results section, we present in-class student performance data that suggest that students gained a better understanding of the concepts related to DPT after working through the QuILT.
Interviews suggest that many of the following difficulties may partly be a result of the students' overloaded working memory [40][41][42][43] and the fact that they did not have a strong background in linear algebra or they struggled to apply linear algebra concepts correctly in the context of DPT. In DPT, students must integrate a number of different concepts to solve a single problem and some students struggled to incorporate these concepts coherently to solve problems involving degeneracy. For example, one cannot simply focus on the unperturbed HamiltonianĤ 0 or the perturbation ¢ H , but one must consider bothĤ 0 and ¢ H when determining a good basis and the first order corrections to the energies and energy eigenstates. The unperturbed Hamiltonian dictates whether one should use DPT, and the perturbating Hamiltonian ¢ H in the degenerate subspace ofĤ 0 determines whether the initial basis is a good basis. It is often difficult for students who are still developing expertise in the context of DPT to apply all these concepts correctly. Additionally, DPT problems require the application of linear algebra concepts in the context of QM. It is not enough to simply diagonalize a matrix, which is a familiar task for many students from their mathematics courses. In DPT, one must be able to identify whether a basis is a good basis, whether a matrix must be diagonalized, what needs to be diagonalized ( ¢ H in each degenerate subspace ofĤ 0 ), and also understand that the degeneracy in the energy spectrum ofĤ 0 is what allows us to diagonalized ¢ H in the degenerate subspace ofĤ 0 while keepingĤ 0 diagonal everywhere. The difficulties found are consistent with many prior studies focusing on student difficulties in connecting the mathematics and physics concepts and how constraints on working memory can negatively impact student performance in areas in which their expertise is still evolving [40][41][42][43].

Difficulty realising that a good basis is required for corrections to the energies
Most of the interviewed students (85% of the interviewed students) realised that the first order corrections to the energy eigenstates y | ⟩ n 1 are not valid unless we choose a good basis. When examining equation (2), they identified that there will be terms in which the denominator is zero due to the degeneracy in the energy spectrum. However, many of these same students (38%) thought that equation (1) is still valid to find the first order corrections to the energies since no divergent terms appear in equation (1). They claimed that any basis which consists of eigenstates ofĤ 0 is a good basis for finding the first order corrections to the energies, but that this same basis may not be a good basis for finding the first order corrections to the energy eigenstates. These students did not realise that if a basis is not a good basis for finding the corrections to the energy eigenstates, then that same basis cannot be a good basis for finding the corrections to the energies. When calculating the first order corrections to the energies, students with this difficulty used the diagonal matrix elements of ¢ H as the first order corrections to the energies whether the initially chosen basis was a good basis or not (whether ¢ H in that basis was a diagonal matrix in the degenerate subspace ofĤ 0 or not). For example, when given the system with Hamiltonian H1 in equation (4), students with this difficulty incorrectly claimed that the initially chosen basis was a good basis for finding the first order corrections to the energies. They incorrectly stated that the first order corrections to the energies are all zero. However, when diagonalizing ¢ H in the degenerate subspace ofĤ 0 in equation (4), one finds that the first order corrections to the energies are òV 0 , −òV 0 , and 0, respectively.

Difficulty identifying
Many students had difficulty identifying the ¢ H matrix in the degenerate subspace ofĤ 0 when the HamiltonianĤ for the system was provided in a matrix form. In particular, students had difficulty with the fact that, in order to determine ¢ H in the degenerate subspace ofĤ 0 , they should start by identifying whether there is any degeneracy in the energy spectrum ofĤ 0 . In fact, we found that some students (31% of the interviewed students) incorrectly focused on the diagonal elements of the perturbation ¢ H to determine whether there was 'degeneracy' in ¢ H and whether they should use DPT. For example, students were given the Hamiltonian in H2 in equation (5) and were asked in a multiple choice format to identify ¢ H in the degenerate subspace ofĤ 0 . Some interviewed students incorrectly identified ¢ H in the degenerate sub- because  2 appears twice as a diagonal matrix element of ¢ H .
However, the same diagonal matrix elements of ¢ H has nothing to do with whether one should use DPT.
Additionally, many students (38% of the interviewed students) were unable to identify ¢ H in the degenerate subspace ofĤ 0 given the HamiltonianĤ in the matrix form if the degenerate basis states were not in adjacent rows/columns. For example, in the system given by the Hamiltonian H3 in equation (6), students were asked to identify ¢ H in the degenerate subspace ofĤ 0 . In this system, the degenerate states are y | ⟩ , which is diagonal. The initially chosen basis is a good basis. Several students (38% of the interviewed students) who correctly identified the matrix elements ofĤ 0 corresponding to the degenerate unperturbed energies were unable to correctly identify ¢ H in that degenerate subspace ofĤ 0 because the degenerate states are not in adjacent rows/columns. Some of the interviewed students (31% of the interviewed students) with this difficulty would then look for 'degeneracy' in the diagonal elements of ¢ H and determine if the initially chosen basis was a good basis based upon whether ¢ H had same diagonal elements (in other words, they looked for the 'degenerate' subspace of ¢ H as opposed to the degenerate subspace ofĤ 0 ).

Difficulty determining whether the initially chosen basis is a good basis
A good basis is one that keeps the unperturbed HamiltonianĤ 0 diagonal while diagonalizing the perturbation ¢ H in the degenerate subspace ofĤ 0 . However, many students had difficulty determining whether the basis in which the Hamiltonian was given in matrix form was a good basis. For example, students were given the system with the Hamiltonian H2 in equation (5) and were asked if the initially chosen basis is a good basis. In this case, ¢ H in the degenerate , which is not diagonal. Therefore, the initially chosen basis is not a good basis. However, some students (15% of the interviewed students) incorrectly stated that the initially chosen basis is a good basis because it consists of a complete set of eigenstates ofĤ 0 (Ĥ 0 is diagonal in the initial basis) without considering whetherĤ 0 had any degeneracy and the implications of the degeneracy inĤ 0 for finding a good basis. These students did not consider the ¢ H matrix before determining whether the initial basis was a good basis for finding the perturbative corrections.
Other students only examined the basis in a general manner and did not focus on either H 0 or ¢ H . For example, one student incorrectly stated that the basis is a good basis if 'it forms a complete Hilbert space.' Another student incorrectly claimed that the only condition to have a good basis is that 'the basis vectors are orthogonal', regardless of the fact that the unperturbed HamiltonianĤ 0 had degeneracy in the situation provided.
Another common difficulty students had with identifying a good basis was considering onlyĤ 0 or ¢ H when determining whether a basis was a good basis. For example, students were asked to consider the system with the Hamiltonian H4 in equation (7) and asked if the basis in which the Hamiltonian is written in the matrix form is a good basis. Since ¢ H in the , which is diagonal, the initially chosen basis is a good basis. However, many students (46% of the interviewed students) had a tendency to focus on eitherĤ 0 or ¢ H , but not both, as is necessary to correctly answer the question. For example, during the interview, one student said, ' ¢ H must be diagonal (everywhere) in the good basis'. Equivalently, another student incorrectly claimed that the basis was not a good basis 'since ¢ H has off-diagonal terms in this basis'. These types of incorrect responses suggest that students have difficulty with the fact that a good basis is one in which ¢ H need only be diagonal in the degenerate subspace ofĤ 0 . Students with these types of responses often focused on diagonalizing the entire ¢ H matrix (rather than diagonalizing ¢ H in the degenerate subspace of H 0 ). They did not realise that ifĤ 0 and ¢ H do not commute,Ĥ 0 will become non-diagonal in a basis that diagonalizes the entire ¢ H matrix, which is inappropriate since we are finding small corrections in perturbation theory.
Moreover, some students (31% of the interviewed students) had difficulty with the fact that even when the initially chosen basis is not a good basis, it may include some states that are good states that can be used to find the first order corrections to the energies using equation (1). For example, when asked to consider the system with the Hamiltonian H2 in equation (5), many students claimed that none of the three basis states in equation (3) (5). Roughly one-third of the students were unable to correctly identify whether each state in the initially chosen basis is a good basis state or not. For example, during the interview, one student said, 'We cannot trust non-degenerate basis states for finding corrections to the energy. We must adjust all the basis states since we cannot guarantee any will be the same.' This student and others with this type of response assumed that if the unperturbed Hamiltonian has degeneracy, none of the initially chosen basis states are good states. However, any state belonging to the non-degenerate subspace ofĤ 0 is a good state.
Other students struggled with the fact that if ¢ H is already diagonal in a degenerate subspace ofĤ 0 , the initially chosen basis is a good basis and equation (1) can be used to determine the perturbative corrections without additional work to diagonalize ¢ H in the subspace. For example, students were given the system with H4 in equation (7) and were asked to find the first order corrections to the energies. Some students (15% of the interviewed students) with this difficulty attempted to diagonalize the ¢ H matrix in the degenerate sub- , these students attempted to diagonalize a matrix that was already diagonal. They appeared to have memorised a procedure for finding the first order corrections and often made mistakes when diagonalizing ¢ H in the degenerate subspace ofĤ 0 . Interviews corroborated the fact that students with this type of response did not have a functional understanding of DPT partly because of difficulties with linear algebra and also not thinking globally about the problem. Many students (45% of the students after traditional lecture-based instruction) did not realise that when the initially chosen basis is not a good basis and the unperturbed HamiltonianĤ 0 and the perturbing Hamiltonian ¢ H do not commute, they must diagonalize the ¢ H matrix only in the degenerate subspace ofĤ 0 . For example, students were given the system with Hamiltonian H4 in equation (7) on a written test and asked to determine the first order corrections to the energies. In the Hamiltonian H4,Ĥ 0 and ¢ H do not commute. In this situation, 45% of the students diagonalized the entire ¢ H matrix instead of diagonalizing the ¢ H matrix only in the degenerate subspace ofĤ 0 . When presented with a similar system and asked to determine the first order corrections to the energies, one interviewed student who attempted to diagonalize the entire ¢ H matrix justified his reasoning by incorrectly stating, 'We must find the simultaneous eigenstates ofĤ 0 and ¢ H .' This student, and others with similar difficulties, did not realise that whenĤ 0 and ¢ H do not commute, we cannot simultaneously diagonalizeĤ 0 and ¢ H since they do not share a complete set of eigenstates. Students struggled with the fact that ifĤ 0 and ¢ H do not commute, diagonalizing ¢ H produces a basis in whichĤ 0 is not diagonal. SinceĤ 0 is the dominant term and ¢ H provides only small corrections, we must ensure that the basis states used to determine the perturbative corrections in equations (1) and (2)  Some students (23% of the interviewed students) did not realise that ¢ H can be diagonalized in the degenerate subspace ofĤ 0 while keepingĤ 0 diagonal even whenĤ 0 and ¢ H do not commute. For example,when considering the Hamiltonian H4 in equation (7) in whichĤ 0 and ¢ H do not commute, one student in the interview stated, 'We cannot diagonalize a part of ¢ H , we must diagonalize the whole thing.' In general, students had great difficulty with the fact that the degeneracy in the eigenvalue spectrum ofĤ 0 provides flexibility in the choice of basis in the degenerate subspace ofĤ 0 so that ¢ H can be diagonalized in that subspace (even ifĤ 0 and ¢ H do not commute) while keepingĤ 0 diagonal. For example, if we consider the case in whichĤ 0 has a two-fold degeneracy, then y y y y y y ,a n d 0 , must remain an eigenstate ofĤ 0 with the same energy E 0 . Many students (31% of the interviewed students) did not realise that since any linear superposition of the initial basis states that correspond to the degenerate subspace ofĤ 0 remains an eigenstate ofĤ 0 , one can choose a special linear superposition of the initial basis states which diagonalizes ¢ H in the degenerate subspace ofĤ 0 .

Development and validation of the QuILT
The difficulties described show that many students struggle in determining a good basis for finding corrections to the energies in the context of DPT. Therefore, we developed a QuILT that takes into account these difficulties. The development of the DPT QuILT started by investigating of student difficulties via open-ended and multiple choice questions administered after traditional instruction to advanced undergraduate and graduate students and conducting a cognitive task analysis of the requisite knowledge from an expert perspective [44]. The QuILT strives to help students build on their prior knowledge and addresses common difficulties found via research, some of which were discussed in the previous section. The QuILT is inspired by Piaget's 'optimal mismatch' framework [45] as well as the preparation for future learning framework of Bransford and Schwartz [46]. In Piaget's 'optimal mismatch' framework, students are intentionally placed in a situation in which their current knowledge structure of relevant concepts is inadequate and they are then given the opportunity and support to reorganise their existing knowledge structures or develop new structures to reconcile this conflict. Bransford and Schwartz's preparation for future learning framework emphasises that learning occurs when elements of innovation and efficiency are both present. Although there are many interpretations of the framework, in one interpretation, innovation and efficiency describe two orthogonal components of instructional design. Innovation describes aspects that are new to students, such as new concepts or new problem solving skills. Efficiency is a measure of the structure and organisation of the instructional design and learning tools, as well as how proficient the student is with the instructional design and learning tools. Instructional design that incorporates only one of these elements leads to students becoming disengaged. If instruction is too innovative, students cannot connect what they are learning with their prior knowledge and may become frustrated. When the instruction is too efficient, students may become disengaged with the repetitious material that is too easy and that does not provide intellectual stimulation.
In the QuILT, innovation is incorporated by presenting students with novel tasks. Whether by examples, hypothetical conversations, or quantitative reasoning, the QuILT strives to help students develop a deeper understanding by actively working through the guided enquiry-based sequences. Student difficulties are incorporated in these questions to create a cognitive conflict after which the students are provided scaffolding support designed to resolve these issues and develop a robust knowledge structure. Efficiency is addressed in the QuILT in several ways. First, the QuILT follows a guided enquiry-based learning sequence laid out in the cognitive task analysis. It is organised to build on the students' prior knowledge and each guided enquiry-based sequence in the QuILT builds upon the previous guided enquiry-based sequences. This organisation strives to help students build their own knowledge structures in a coherent manner. Second, students are provided scaffolding support to help address common difficulties, thus resolving the cognitive conflicts. Third, the QuILT progressively reduces the scaffolding support so that students develop self-reliance and are able to solve the problems without any assistance. Finally, as the students work through the different tasks, they develop proficiency in applying the concepts in diverse contexts.
The development of the QuILT went through a cyclic, iterative process. The preliminary version was developed based upon the task analysis and knowledge of common student difficulties. Next, the QuILT underwent many iterations among the three physics education researchers and then was iterated several times with three physics faculty members to ensure that they agreed with the content and wording. It was also administered to graduate and advanced undergraduate students in individual think aloud interviews to ensure that the guided approach was effective, the questions were unambiguously interpreted, and to better understand the rationale for student responses. The next step involved evaluating student responses during the interviews and their corresponding posttest responses to determine the impact of the QuILT on student learning and whether difficulties remained. Finally, modifications and improvements were made based upon the student and faculty feedback before it was administered to students in various courses.

Structure of the QuILT
The QuILT uses a guided enquiry-based approach to learning and actively engages students in the learning process. It includes a pretest to be administered in class after traditional instruction in DPT. Next, students engage with the tutorial in small groups in class (or alone when using it as a self-paced learning tool in homework), and then a posttest is administered in class. As students work through the tutorial, they are asked to predict what should happen in a given situation. Then, the tutorial strives to provide scaffolding and feedback as needed to bridge the gap between their initial knowledge and the level of understanding that is desired. Students are also provided checkpoints to reflect upon what they have learned and to make explicit the connections between what they are learning and their prior knowledge. They are given opportunities to reconcile differences between their predictions and the guidance provided in the checkpoints before proceeding further.
The DPT QuILT uses a blend of qualitative and quantitative reasoning to improve students' understanding. For example, the QuILT requires qualitative understanding while students respond to the hypothetical conversations and quantitative reasoning to determine the first order corrections to the energies and energy eigenstates. In addition, students are asked to verify predictions about the validity of the statements in hypothetical conversations via quantitative reasoning by working through problems. The QuILT strives to help students with linear algebra difficulties relevant for DPT by incorporating a combination of quantitative and qualitative questions in the guided enquiry-based sequences. Students are asked to reflect upon their answers and reasoning and then provided checkpoints to reconcile their initial reasoning with the correct reasoning.

Addressing student difficulties
In the QuILT, students actively engage with examples involving DPT that are restricted to a three-dimensional Hilbert space (with two-fold degeneracy inĤ 0 ). In this manner, students focus on the concept of a good basis in DPT without working through complex calculations. In particular, for a givenĤ 0 and ¢ H , when there is degeneracy in the eigenvalue spectrum of H 0 , students learn about why some bases are not good even though they may consist of a complete set of eigenstates ofĤ 0 . The QuILT strives to help students develop a functional understanding of whether the basis is a good basis and how to change the basis to one which is good (if the initial basis is not good for a givenĤ 0 and ¢ H ) so that equations (1) and (2) can be used to find the first order corrections. Below, we discuss how the QuILT addresses student difficulties and strives to help students learn about a good basis for finding perturbative corrections.
Helping students realise that a good basis is required even for finding first order corrections to the energies: by engaging with the QuILT, students learn to reason about why a basis that is not a good basis for equation (2) cannot be a good basis for equation (1). There are several questions in which students must identify that ¢ H is not diagonal in the degenerate subspace ofĤ 0 and therefore is not a good basis. For example, students consider the following system and are asked to determine whether the basis is a good basis: 0 are not zero so that equation (2) contains divergent terms since E 1 =E 2 . Thus, it is not a good basis for finding perturbative corrections to the energies and energy eigenstates. The following is an excerpt from a hypothetical student conversation in which the students must consider each hypothetical student's statement and explain why they agree or disagree with each statement. The conversation strives to help students reflect upon the fact that the same basis cannot be a good basis for equation (1) while at the same time NOT be a good basis for equation (2).  After the students work through the question and consider the validity of each statement in the hypothetical conversation, they are provided further scaffolding. They are then asked to summarise when equations (1) and (2) In question Q1(A), students must identify that sinceĤ 0 is diagonal in the initially chosen basis, the basis consists of a complete set of eigenstates ofĤ 0 . Thus, the initially chosen basis satisfies one of the conditions for a good basis. The next question Q1(B) asks students to identify whether there is degeneracy in the energy spectrum ofĤ 0 so that DPT must be used. Students must identify V 2 0 as the two-fold degenerate unperturbed energy in order to correctly identify the degenerate subspace ofĤ 0 .
The subsequent question in the guided enquiry-based sequence asks students to identifŷ H 0 and ¢ H in the degenerate subspace ofĤ 0 after identifying the degeneracy inĤ 0 in question Q1(B) as follows: is the correct answer to Q1(C). Students must correctly identify the degenerate subspace ofĤ 0 and identify ¢ H in the degenerate subspace ofĤ 0 for question Q1(C). Option (i) is given as a distractor because students often incorrectly focused on the matrix elements of ¢ H when determining the degenerate subspace ofĤ 0 . In option (i), the matrix representation of ¢ H in the degenerate subspace ofĤ 0 is incorrectly given as the matrix representation of ¢ H in the 'degenerate' subspace of ¢ H . The final part to this enquiry-based sequence asks the following: (probed in Q1(C)). However, students who had difficulty identifying whether the initially chosen basis is a good basis for finding the perturbative corrections often selected option (i) in Q1(C) and they determined that the initially chosen basis is NOT a good basis as ¢ H in option (i) is not diagonal in the given subspace. Scaffolding is provided after this question in the form of student conversations and checkpoints to help students reconcile the differences between their initial responses and correct ideas.
After students work through several examples to determine whether the initially chosen basis is a good basis when the degenerate states are in adjacent rows/columns ofĤ 0 , students are also given an example to help them identify ¢ H in the degenerate subspace ofĤ 0 when the degenerate states are not in adjacent rows/columns. Helping students identify that if ¢ H is not diagonal in the degenerate subspace ofĤ 0 , it is not a good basis for finding the perturbative corrections: in other examples in the QuILT, students learn that if ¢ H is not diagonal in the degenerate subspace ofĤ 0 , it is not a good basis for finding the perturbative corrections. For example, the following is an excerpt from a guided enquiry-based sequence in the QuILT:   In parts (a) and (b) of question Q4, students verify that the linear combination of eigenstates ofĤ 0 from the same degenerate subspace ofĤ 0 is an eigenstate ofĤ 0 . Q4(C) strives to help students learn that we can find a particular linear combination that diagonalizes ¢ H in the degenerate subspace ofĤ 0 while keepingĤ 0 diagonal to find a good basis for DPT.
Students are given the opportunity to check their answer in Q4 via quantitative reasoning.

Evaluation of the QuILT
Once the researchers determined that the QuILT was successful in one-on-one implementation using a think aloud protocol, it was administered in graduate and upper-level undergraduate QM classes. Both undergraduate and graduate students were given a pretest after traditional instruction in relevant concepts in DPT but before working through the tutorial. The pretests were never returned to the students. After working through and submitting the completed tutorial, both groups were given the posttest in class. Students were given enough time in class to work through the pretest and posttest. The posttest was similar to the pretest with minor changes to the degenerate subspaces. The pretest, tutorial, and posttest each counted as components of the students' course grades. The pretest was scored for completeness for both groups. The posttest was scored for correctness for the undergraduates in all three years. However, the posttest was scored differently for the graduate students in the two different years. In Year 1, the graduate students' posttest was scored for completeness while in Year 2 it was scored for correctness. For the undergraduate students, the QuILT (including pretest, tutorial, and posttest) contributed to roughly 2.5% of their course grade in Years 1 and 2. In Year 3, for the undergraduate students, it contributed to roughly 7% of their course grade. For the graduate students, roughly 1% of the course grade was associated with the QuILT. Regardless of how the pretest and posttest counted towards the students' course grade, each was scored for correctness to investigate the effectiveness of the QuILT for research purposes. These scores are the scores that are reported here. The entire pretest and posttest each consist of 10 questions related to DPT. We will present the results from the three questions that focused on finding a good basis and first order corrections to the energy for a system restricted to a three-dimensional Hilbert space. To probe student understanding of DPT in the context of three-dimensional Hilbert space, the following questions QI-QIII were administered on the QuILT pretest and/or the QuILT posttest. All the questions were asked on both the pretest and posttest in Years 1 and 3. In   (1)). . Also, the average score of the undergraduates is given for two problems that were given on the final exam six weeks later.

Undergraduate students Graduate students
Question The open-ended questions were graded using rubrics which were developed by the researchers together. A subset of questions was graded separately by them. After comparing the grading, they discussed any disagreements and resolved them with a final inter-rater reliability of better than 95%. Table 1 shows the performance of undergraduate and graduate students on the pretest and posttest. Table 1 also includes the average gain, G, and normalised gain [47], g. The normalised gain is defined as the (posttest percent-pretest percent)/ (100 − pretest percent). The undergraduate students had the same instructor (Instructor 1) in Years 1 and 2. The instructor (Instructor 3) for the graduate level course was the same in Years 1 and 2 (it was a different instructor than the undergraduate course). Performance on questions QII and QIII on pretest were comparable in Years 1 and 2 and were combined into a single percentage in table 1. Similarly, the posttest scores for the undergraduate and graduate students on QII and QIII in Year 1 and Year 2 were comparable and were combined. Both the undergraduate and graduate instructors in Years 1 and 2 used a traditional lecture-based approach. Instructor 2 for the undergraduate students in Year 3 used active engagement teaching involving in-class clicker questions with peer discussion. The performance of the undergraduates on the pretest in Year 3 is significantly better than that of the performance of the undergraduate students on the pretest in Years 1 and 2. However, after engaging with the QuILT, there is no statistically significant difference in the performance of the undergraduate students on the posttest based upon instructor and all classes performed well regardless of the instructor. These results are encouraging and suggest that the QuILT is effective at reducing the gap between courses taught with traditional lecture-based instruction and those that incorporate active engagement activities while also achieving a high normalised gain for the students regardless of their performance on the pretest. The posttest scores are significantly better than the pretest scores on all of these questions for both undergraduate and graduate students with the exception of Q1(B) in Year 3 (in which the active learning instructor's students performed well on both the pretest and the posttest).
To investigate retention of learning, the undergraduates in Year 1 were given questions QI(A) and QI(B) again as part of their final exam. The final exam was six weeks after students engaged with the tutorial. The average score on QI(A) was 97.8% and on QI(B) was 91.0%. In QI(A), all 11 students provided an ¢ H matrix that was diagonal in the degenerate subspace ofĤ 0 . In QI(B), 10 out of 11 students provided an ¢ H matrix that was not diagonal in the degenerate subspace ofĤ 0 . Table 1 shows that the performance of the undergraduate students on all the questions in the posttest was exceptional. However, as can be seen from the pretest scores in table 1, traditional lecture-based instruction was not particularly effective at developing a functional understanding of these topics. We also note that this second-semester, upper-level undergraduate QM course is an elective honours physics course that majority of the students take in preparation for graduate school to pursue a PhD. They are highly motivated to learn the material if appropriate guidance and support is provided (which the QuILT, that uses research on student difficulties as a guide, strived to do). This may help to explain why the undergraduate students did so well on the posttest after engaging with research-validated guided enquiry-based learning tutorial. The majority of these honours students are high achieving undergraduate students and a large fraction go on to graduate school at top universities. In addition, we note that while many students were able to answer questions QI-QIII correctly, it is encouraging that most students provided correct reasoning along with their work on the posttest questions. Students' written reasoning indicated that they had developed a good understanding of how to determine a good basis and the first order corrections to the energy rather than simply memorising an algorithm. Figure 1  identified the first order corrections to the energies as 0 and   V 0 . Many students provided similar solutions that clearly justified their reasoning and demonstrated a correct problem solving approach to questions QI-QIII.
We also note that this investigation was part of a larger study of student understanding of DPT. The QuILT focusing on DPT in a three-dimensional Hilbert space was one of a series of QuILTs developed to help improve student understanding of DPT. The QuILT discussed here was developed to help students gain a functional understanding of fundamental concepts in DPT in the context of a three-dimensional Hilbert space which are necessary for understanding more complex applications of DPT. For example, we have developed a QuILT that builds on the QuILT discussed here and strives to help students find a good basis and the first order corrections to the energy spectrum of the hydrogen atom placed in an external magnetic field. In this situation, students must determine a good basis and find the first order corrections to the energies for principal quantum number = n 2 in an eight-dimensional subspace. We note that in these more complex situations involving the hydrogen atom, the students do not perform as well on the posttest as they do on the posttest described in this paper that focused on DPT in a three-dimensional Hilbert space. However, they still show a dramatic improvement over their pretest scores after traditional lecture-based instruction only. We plan to discuss these investigations in future work.
As can be seen in table 1, the graduate students generally performed better than the undergraduates on the pretest. However, the undergraduates outperformed the graduate students on the posttest on most questions (see table 1). One possible explanation for the undergraduates outperforming the graduate students on the posttest could be the grade incentive associated with the QuILT. As discussed earlier, the QuILT accounted for a larger percent of the undergraduates overall course grade and the components of the QuILT were accounted for differently for the course grade for the two groups of students. In particular, the posttest for the undergraduate students was graded for correctness in all three years while the posttest for the graduate students was graded for completeness in Year 1 and for correctness in Year 2. Additionally, the undergraduate students knew that the material from the QuILT could appear on their examinations while the graduate students were told but the graduate instructor that this material was a review of the undergraduate QM and that no material from the QuILT would appear on their examinations, instead more complex problems on the DPT would appear on the exams. The fact that the graduate students were given very small grade incentive to learn the material in the QuILT may have decreased their motivation to engage as deeply with the QuILT as the undergraduates and may explain why the graduate students did not perform as well as the undergraduate students on the posttest. We also note that prior studies in the context of introductory physics suggest that more time on task does not improve student understanding and students need to engage with research-based approaches in a meaningful way for them to develop a good grasp of concepts [48].

Summary
We developed and evaluated a research-based QuILT which focuses on helping students reason about and find perturbative corrections to the energies using DPT. We found that the advanced physics students who are still developing expertise in QM had difficulty after traditional lecture-based instruction in reasoning about the DPT concepts while solving problems. This difficulty is in part due to the fact that students' working memory can get overloaded by the demands of the DPT problems (partly due to the fact that the paradigm of QM is novel and partly due to the difficulty with mathematical sense making in a physics context involving degeneracy). One major cause of the difficulties is the fact that DPT relies heavily on applying linear algebra in the context of QM and many students struggled to apply these mathematical concepts correctly in the context of DPT. In particular, a majority of students were not able to integrate all the different concepts coherently to solve a given problem after traditional lecture-based instruction. We used the common difficulties of advanced students with DPT found via research as resources in order to develop and validate the QuILT. The research-validated QuILT strives to provide appropriate scaffolding and feedback using a guided enquiry-based approach to help students develop a functional understanding of DPT. The preliminary evaluation shows that the QuILT is effective in improving undergraduate and graduate students' understanding of a good basis in the context of DPT. Future investigations will focus on evaluating the effectiveness of the QuILT at other universities where the student in this type of undergraduate QM course are not so selective.