Investigating and improving student understanding of quantum mechanical observables and their corresponding operators in Dirac notation

In quantum mechanics, for every physical observable, there is a corresponding Hermitian operator. According to the most common interpretation of quantum mechanics, measurement of an observable collapses the quantum state into one of the possible eigenstates of the operator and the corresponding eigenvalue is measured. Since Dirac notation is an elegant notation that is commonly used in upper-level quantum mechanics, it is important that students learn to express quantum operators corresponding to observables in Dirac notation in order to apply the quantum formalism effectively in diverse situations. Here we focus on an investigation that suggests that, even though Dirac notation is used extensively, many advanced undergraduate and PhD students in physics have difficulty expressing the identity operator and other Hermitian operators corresponding to physical observables in Dirac notation. We first describe the difficulties students have with expressing the identity operator and a generic Hermitian operator corresponding to an observable in Dirac notation. We then discuss how the difficulties found via written surveys and individual interviews were used as a guide in the development of a quantum interactive learning tutorial (QuILT) to help students develop a good grasp of these concepts. The QuILT strives to help students become proficient in expressing the identity operator and a generic Hermitian operator corresponding to an observable in Dirac notation. We also discuss the effectiveness of the QuILT based on in-class evaluations.


Introduction
Learning quantum mechanics is challenging for students partly because it is abstract and unintuitive [1][2][3][4][5][6][7][8][9][10][11][12]. Investigations have focused on helping students visualise quantum phenomena [13,14], learn about the formalism of quantum mechanics and Dirac notation [15][16][17][18][19], and develop a good grasp of quantum measurement [20][21][22], probability distributions for measuring physical observables [23], expectation values and their time dependence [24,25], and addition of angular momentum [26]. Student difficulties with quantum experiments [27] have also been investigated. Researchers have also investigated the difficulties of PhD physics students with quantum mechanics concepts [28]. Other investigations in quantum mechanics have focused on students' cognitive issues and reasoning difficulties, transfer of learning, expertise involving categorisation of problems, and grading approaches [29][30][31][32][33][34][35][36][37]. Investigations of student difficulties in quantum mechanics are important for developing learning tools that help students develop a solid grasp of quantum mechanics [38][39][40][41][42][43][44][45][46][47][48][49][50][51][52][53][54], such as quantum interactive learning tutorials (QuILTs) on measurement [39,40], addition of angular momentum [41], quantum key distribution [42], degenerate perturbation theory [43], Stern-Gerlach experiment [44], double-slit experiment [45][46][47], Larmor precession of spin [48], Mach-Zehnder Interferometer [49][50][51][52], and Dirac notation [53,54]. The effectiveness of pedagogies in quantum mechanics courses have been investigated [55][56][57][58][59][60][61][62][63][64], such as peer instruction [55,56] and just-in-time-teaching [57]. However, few prior research investigations have focused on student difficulties with quantum operators in the context of Dirac notation [18], a compact and convenient notation commonly used in advanced quantum mechanics. The completeness relation (i.e. the spectral decomposition of the identity operator) when the identity operator is written in Dirac notation in a given basis can be useful, e.g. for decomposing a generic state into its components along each chosen basis vector. It can also be helpful in determining the probability for measuring a particular value of an observable and the expectation value of an observable in a given quantum state. Similarly, a generic quantum operator written in Dirac notation in terms of its eigenvalues and eigenstates can be helpful, e.g. in determining the expectation value of the observable in a given quantum state. Writing a generic quantum operator in terms of its eigenvalues and eigenstates in Dirac notation can also be beneficial for reasoning about the fact that the matrix representation of an operator corresponding to an observable is diagonal in its own basis with the eigenvalues of the operator along the diagonal (whereas the operator may not be diagonal in other representations).
We have been investigating student difficulties and developing learning tools in the context of Dirac notation in quantum mechanics (see, e.g. 53,54,62). Our previous published research has focused on investigating and improving student difficulties with probability distributions for measuring physical observables [53] as well as expectation values of observables [54]. Probability distributions and expectation values of observables are closely connected to the concept of observables and their corresponding operators in Dirac notation. Thus, here we focus on student difficulties with expressing the identity operator and any Hermitian operator corresponding to a physical observable in Dirac notation after traditional, lecture-based instruction in quantum mechanics courses. Then, we describe how student difficulties were used to develop a QuILT to improve students' understanding of these concepts. The QuILT involves a guided inquiry-based approach to learning and was developed using an iterative approach.
Below, we begin with a brief background on the identity operator and a generic operator corresponding to an observable in Dirac notation (which summarises the concepts and procedures students should learn). We then describe the methodology for investigating student difficulties and categorising the difficulties found. Next, we discuss the development and assessment of the QuILT suggesting that the QuILT was effective in improving students' understanding of quantum mechanical observables and their corresponding operators in Dirac notation.

Summary of Dirac notation formalism
Depending upon the convenience in using a basis in a given situation, a complete set of orthonormal eigenstates of a Hermitian operator Q corresponding to an observable Q with discrete or continuous eigenvalues is usually selected as the basis. Such a basis can be used to write the completeness relation which can be useful, e.g. for decomposing a generic state vector Y | ⟩ for a quantum system into its components along each of the basis states or for expressing an operator Q in terms of its eigenvalues and outer products of its eigenstates. In an N -dimensional Hilbert space, the completeness relation (or the spectral decomposition of the identity operator, Iˆ) in terms of the orthonormal eigenstates { q n N , 1, 2, 3 } of an operator Q with a discrete eigenvalue spectrum Q q q q n n n = (ˆ| ⟩ | ⟩) can be expressed as I q q . | ⟩⟨ | shows that in an N -dimensional Hilbert space, the identity operator can be represented as an N Ń diagonal matrix with ones along the diagonal regardless of the basis chosen. Similarly, the completeness relation or the spectral decomposition of Iˆin terms of the complete set of the eigenstates of an operator Q with a continuous eigenvalue spectrum is I q q q d , where q {| ⟩} forms an 'orthonormal' (in the Dirac sense) basis for an infinite-dimensional Hilbert space (e.g. formed with a complete set of eigenstates q | ⟩ with eigenvalues q of an operator Q corresponding to an observable Q, i.e. Q q q q . =| ⟩ | ⟩) The identity operator Iˆdoes not change the quantum state it acts on. Therefore, the identity operator can be a very useful tool for introducing a basis or changing the basis used to express a quantum state in a given situation. Moreover

Methodology for the investigation of student difficulties
Both before and during the development of the QuILT, we investigated the difficulties students have with expressing the identity operator and a Hermitian operator corresponding to a physical observable in Dirac notation. We note that the methodology for the present investigation was the same as that reported in our previous work on investigating and improving student difficulties with probability distributions and expectation values of physical observables [53,54], as well as the PhD dissertation by Marshman [62]. Student difficulties were investigated by administering open-ended and multiple-choice questions after traditional lecture-based instruction in relevant concepts to upper-level undergraduate and PhD level students and observing difficulties on in-class quizzes. The undergraduate students were enroled in an upper-level quantum mechanics course and the PhD students were enroled in a first-year core graduate quantum mechanics course. Table 1 shows a list of the questions Table 1. Questions related to the identity operator Iˆand an operator Q corresponding to an observable and the number of students (N) who answered each question. The correct answer is bolded for the multiple-choice questions. Students were asked to assume that Q was a Hermitian operator corresponding to an observable Q and that all of the summations were over all possible values of n and m.
} forms a complete set of orthonormal eigenstates of an operator Q corresponding to a physical observable with non-degenerate eigenvalues q .
n Iˆis the identity operator. Choose all of the following statements that are correct. that were administered to students. These questions were developed and refined over a period of several years and were revised based upon feedback from several faculty members and individual interviews with students. Questions Q1-Q3 involve the identity operator Iˆand questions Q4-Q6 involve a generic Hermitian operator Q corresponding to an observable Q. Some of the questions used different notations for the eigenvalues and eigenstates of a generic operator. For example, questions Q1, Q4 and Q6 involve an operator Q with eigenvalues q n n N 1, 2, , = ¼ ( ) whereas Q3 and Q5 involve an operator Q with eigenstates e j | ⟩ and eigenvalues j l j N 1, 2, . = ¼ ( ) The questions involve different notations for the eigenstates and eigenvalues because different instructors were more comfortable with particular notations. Moreover, the different notations helped us investigate whether students' difficulties are similar across different notations. Furthermore, we note that student responses are sensitive to the wording of a question, particularly for multiple-choice questions which include an explicit mention of a particular difficulty [3]. In particular, even if only 5%-10% of the students show a certain type of difficulty in a particular context (e.g. open-ended format), it is likely that a higher percentage will display the same difficulty in a different context (e.g. a multiple-choice format) [3].

UG
The multiple-choice questions Q1 and Q4 were administered to 184 undergraduate students after at least one semester of traditional, lecture-based instruction in upper-level undergraduate quantum mechanics at four US universities (see table 1, questions Q1 and Q4). The open-ended quiz questions Q2, Q3, Q5, and Q6 were administered to students after traditional lecture-based instruction in quantum mechanics at the University of Pittsburgh over approximately ten years (these students were different from the 184 undergraduate students that were given questions Q1 and Q4). The numbers of students answering the openended questions Q2, Q3, Q5, and Q6 differ in table 1 because, in some of the years, students were not given certain questions.
The open-ended questions were graded using rubrics which were developed by the two investigators together. A subset of the open-ended questions was graded and categorised for student difficulties separately by the two investigators. After comparing the grading and categorisation of the difficulties, the investigators discussed any disagreements and resolved them with a final inter-rater reliability of better than 95% on all questions in all cases.
Student difficulties were also investigated by conducting individual interviews with 23 student volunteers enroled in quantum mechanics courses. These students were a subset of students from the University of Pittsburgh (these students are not included in table 1). The individual interviews employed a think-aloud protocol to better understand the rationale for student written responses on questions Q1-Q6. During the semi-structured interviews, students were asked to 'think aloud' while answering the questions. Students first read the questions individually and answered them without interruptions except that they were prompted to think aloud if they were quiet for a long time. After students had finished answering a particular question to the best of their ability, we asked them to further elaborate on issues they had not clearly addressed earlier. Table 2 shows that, in response to question Q1, only 45% of the students selected the correct answer. 83% of the students correctly identified that the identity operator shown in option 1 (i.e. I q q n n n å = | ⟩⟨ |) is correct by selecting option A, D or E. Since statement (1) is in three out of the five answer choices, there was a 3/5=60% chance of guessing correctly and student performance on identifying the identity operator is somewhat higher than the level of random guessing. On question Q2, students had great difficulty when they were asked to write the spectral decomposition of the identity operator. Table 2 shows that, on question Q2, only 9% of the undergraduate students and 54% of PhD students were able to correctly write the spectral decomposition of the identity operator using a complete set of eigenstates q | ⟩ of the operator Q. Furthermore,  table 2 shows that on question Q4, only 35% of the students were able to identify the correct expression for the spectral decomposition of a Hermitian operator corresponding to an observable in terms of its eigenvalues and eigenstates, i.e. Q q q q . n n n n å = | ⟩⟨ | Interviews suggest that even those students who recognised that the identity operator in terms of the eigenstates of Q is I q q n n n å = | ⟩⟨ | on question Q1 were unable to use this knowledge to determine the spectral decomposition of the generic operator Q q q q n n n n å = | ⟩⟨ | in question Q4. Interviews also suggest that most students did not remember the expression for the spectral decomposition of the operator Q q q q n n n n å = | ⟩⟨ | nor could they reason about it conceptually simply by looking at the expressions provided in question Q4. Therefore, they needed to derive it. Interviews also suggest that many students did not understand conceptually that when the eigenstates of an operator are selected as the basis states, an operator can be represented as a square matrix with its eigenvalues along the diagonal. In fact, in response to question Q4, 29% of the UG students selected option (C) that involves off-diagonal matrix elements (since there were two sums involved).

Student difficulties
Below, we summarise students' common conceptual and procedural difficulties with the identity operator and a generic Hermitian operator Q that were observed in the written responses and interviews. In particular, we find that many difficulties stem from the fact that students have difficulty differentiating between related concepts of identity operator and projection operator, struggle to distinguish between operators, numbers, and states, and have procedural difficulties in generating expressions for the identity operator Iˆand a generic Hermitian operator Q.

Difficulty differentiating between the identity operator and a projection operator
Students struggled to make distinctions between related concepts, e.g. discerning the difference between a projection operator and the identity operator. On question Q2, table 3 shows that 7% of the PhD students claimed that the identity operator was I q q = | ⟩⟨ | and on Table 2. Percentages of undergaduate (UG) and PhD (G) students who correctly answered questions in the open-ended format and the distribution of responses for the multiple-choice questions (correct responses are bolded).
Question Percentage of students 12% UG, 52% G question Q3, 7% of the undergraduate students claimed that the identity operator was I e e , j j = | ⟩⟨ | even though these are actually projection operators. Interviews suggest that many students with this type of response often had difficulty differentiating between the related concepts of the identity operator and a projection operator. They lacked a conceptual understanding of these operators and the impact of acting with each of these operators on a state . Y | ⟩ In particular, these students were often unable to articulate, e.g. that the projection operator q q | ⟩⟨ | acting on a state Y | ⟩ gives q q , Y | ⟩⟨ | ⟩ which is along state q | ⟩ and is multiplied by the component of Y | ⟩ along the state q . | ⟩ Moreover, they could not explain why Iˆis the sum over all projection operators formed with a complete set of orthonormal basis states.
This difficulty sometimes led students to derive an incorrect expression for Q in terms of its spectral decomposition. In particular, on questions Q5 and Q6, students sometimes confused the identity operator with a projection operator. For example, one student stated: Other students also attempted to show that the identity operator e e i j j i , å | ⟩⟨ | is equal to the number 1. Interviews suggest that many students did not conceptually understand that the identity operator is not a number but an operator (and can be represented by an N Ń matrix with ones along the diagonal and zeroes everywhere else regardless of the orthonormal basis chosen).
⟨ | ⟩ Interviews suggest that the common difficulties of this type are often not only due to a lack of procedural facility with Dirac notation but also the fact that many students could not make sense of the meaning of the outer product (which is an operator) and inner product of quantum states (which is a number). Table 2 shows that in response to question Q4, 13% of the students incorrectly selected option (A), i.e. Q q q q . n n n n å = ⟨ | ⟩ Interviews suggest that this type of response often originated from the inability to differentiate between an outer product and an inner product. Students sometimes had difficulty with the fact that an operator is represented by an outer product in Dirac notation as opposed to an inner product, which is a number.

Incorrectly claiming that b
Q ¼ ∑λ j . Another difficulty involved students incorrectly claiming that Q was equal to the sum of its eigenvalues, i.e. Q .
On question Q5, table 3 shows that 6% of the students claimed that Q .
Interviews suggest that this difficulty can stem from several reasons, e.g. the fact that students confused the identity operator with a projection operator, i.e. I e e . Interviews also suggest that on question Q4, some students who incorrectly selected options (A) or (D) had difficulty in differentiating the inner and outer products and believed that they are equivalent and Q j l = å because e e 1. j j = ⟨ | ⟩ These types of difficulties indicate that many students have difficulty reasoning about operators in Dirac notation and they do not understand that an operator in an N -dimensional Hilbert space can be represented by an outer product (or an N Ń matrix) and it is not a number. Therefore, it does not make sense to claim that an operator is a sum of its eigenvalues, i.e. Q .
Interviews suggest that asking students why an operator can be written as the sum of its eigenvalues was not useful for helping them realise that there may be some mistake in their expression and that they needed to check their work. Some students were unclear about the difference between an operator and a scalar and thought that the fact that the instructor has stated that the operator was diagonal in its own basis with the eigenvalues along the diagonal can be used to infer that the operator was equal to the sum of the eigenvalues Q . j l = å 4.2.4. Incorrectly claiming that b Q ¼ λ j . One difficulty found in some students' written responses and interviews was that they thought that Q was equal to one of its eigenvalues, e.g. Q . Responses of this type indicate that students are confused about the difference between an operator and an eigenvalue of the operator, e.g. . j l In some cases, this difficulty was also coupled to the fact that students did not realise that Q e e j j j | ⟩ which happens to be an eigenstate of Q). In some situations, instead of invoking mathematical arguments, students used incorrect reasoning based upon their understanding of quantum mechanics to conclude that Q . j l = These students often knew that for each physical observable, there is a corresponding operator in quantum mechanics and measurement of the observable gives one of the eigenvalues of the corresponding operator. They used these correct postulates of quantum theory to conclude that Q .
While a robust grasp of linear algebra may have prompted students with these types of arguments to realise that an operator cannot be equal to a scalar, their reasoning was mainly based upon their conceptual reasoning about quantum postulates. Q ¼ ∑q n |q n 〉. Question Q6 differs from question Q5 in that question Q6 does not state that the operator Q can be written in terms of its spectral decomposition as Q q q q .  Table 2 shows that question Q6 was more difficult for students than question Q5; only 12% of undergraduate students were able to answer question Q6 correctly as opposed to 39% of the undergraduate students who answered question Q5 correctly. On question Q6, table 3 shows that 8% of the undergraduate students and 10% of the PhD students claimed that Q q q .   | ⟩ [18,54] has been found when students are asked to generate an expression for the expectation value. This type of difficulty demonstrates that students have some correct knowledge, i.e. they know that one can write Y | ⟩ as a superposition of the eigenstates of a generic operator Q and use this linear superposition to find an expression for Q in terms of its spectral decomposition. However, some lack a conceptual understanding of what this expansion means and do not realise that expansion coefficients along each eigenstate are needed in the expansion instead of simply writing q .
n n å Y = | ⟩ | ⟩ Also, some students have difficulty reasoning about operators in Dirac notation and they do not understand that an operator in an N -dimensional Hilbert space can be represented by an outer product (or an N Ń matrix) and is not represented by a ket vector, i.e. Q q q .
At this point, the student assumed that he had finished the problem since his goal was to show that I 1 = (which was incorrect). Thus, this student's response included several difficulties-assuming that I 1, = difficulties with indices, and using a specific state e j | ⟩ instead of a generic state Y | ⟩ in the given situation. Such difficulties were common in many student responses after traditional instruction in relevant concepts.  table 3 shows that 13% of the undergraduate students and 16% of the PhD students incorrectly used a summation as opposed to an integral, e.g. I q q n n n Interviews suggest that some advanced students have difficulty differentiating between the cases when the eigenvalue spectrum is discrete versus continuous. Despite explicitly being asked to write the spectral decomposition of the identity operator using a complete set of eigenstates of an operator Q with a continuous eigenvalue spectrum, they replaced the integral with a sum.

Development and validation of the QuILT
As described in the preceding section, students have many common difficulties expressing the identity operator and an operator corresponding to an observable in a given basis in Dirac notation after traditional instruction in relevant concepts. Thus, we developed a QuILT that takes into account the common difficulties found and strives to help students develop a good grasp of the concepts. The QuILT uses a guided inquiry-based approach in which various concepts build on each other and can be used in upper-level undergraduate and PhD level quantum mechanics courses after students have had instruction in relevant topics.
The development of the QuILT went through a cyclic, iterative process which included the following stages before the in-class implementation (see [53,54] for more details): (1) Development of a preliminary version of the QuILT which was a. Inspired by Vygotsky's notion of the 'zone of proximal development' (ZPD), which refers to the zone defined by the difference between what a student can do on his/her own and what a student can do with the help of an instructor who is familiar with his/ her prior knowledge and skills [65]. Providing appropriate feedback and scaffolding support is at the heart of this ZPD model and can be used to stretch students' learning beyond their current knowledge using carefully crafted learning tools.
b. Based on research on student difficulties with relevant concepts and a cognitive task analysis of the underlying knowledge from an expert's perspective [66]. The cognitive task analysis involves a careful analysis of the underlying concepts in the order in which they should be invoked and applied in each situation to accomplish a task (i.e. answer the quantum physics questions in our case). (2) Individual administration to students via think-aloud interviews with 23 student volunteers. After each individual interview with a particular version of the QuILT (along with the administration of the pretest and posttest), modifications were made based upon the feedback from the students. (3) Several iterations with three faculty members who are experts in these topics and two graduate students who conduct physics education research to ensure that the content and wording of the questions were appropriate. Modifications were made based upon their feedback.
When we found that the QuILT was working well in individual administration, it was administered in quantum mechanics courses for upper-level undergraduate and PhD students.

Structure of the QuILT
The structure of the QuILT is as follows (see also [53,54]): (1) A pretest is administered right after traditional instruction on the relevant concepts but before students engage with the QuILT. The pretest is not returned to students. The questions on the pretest are in free-response format.
(2) Students work through a QuILT 'warm-up' that builds on their prior knowledge about a vector in a physical three-dimensional vector space they are familiar with from introductory physics. (3) Students then learn about the basics of Dirac notation including scalar products, projection operators, the identity operator, a generic operator corresponding to an observable, and the expansion of a quantum state using a complete set of eigenstates of an operator corresponding to an observable. (4) Students are given one week to work through the entire QuILT as a homework and then a posttest is administered in class. The posttest questions are in free-response format, and the posttest is returned to students after grading.
The QuILT can be used in class to give students an opportunity to work together in small groups and discuss their thoughts with peers, which provides peer learning support. The QuILT can also be used as a self-paced learning tool so long as the pretest and posttest are administered in class. Below, we give some typical examples of how some of the common difficulties found via research are incorporated as resources and how student learning is scaffolded via the QuILT.

5.3.
Addressing student difficulties via the QuILT 5.3.1. Addressing the difficulties in identifying and generating the identity operator for a given orthonormal basis in Dirac notation. Students had difficulty identifying and generating expressions for the identity operator in a given orthonormal basis in Dirac notation. Students sometimes claimed that I 1 = and had difficulties with indices when explicitly asked to show that I e e j N j j 1 å = = | ⟩⟨ | in terms of a complete orthonormal basis set. Also, as noted, students sometimes did not differentiate between an outer product and an inner product when generating expressions for the identity operator. In the QuILT warm-up, students learn about the identity operator in the context of a familiar three-dimensional vector space. For example, the following questions are part of a learning sequence in the QuILT to help students generate the identity operator in a three-dimensional vector space for a given orthonormal basis in Dirac notation: representation, the normalised basis vectors are chosen as i Compute the outer products i i , | ⟩⟨ | j j | ⟩⟨ |, and k k | ⟩⟨ | in matrix form. Add the matrices to find the operator I i i = | ⟩⟨ |+ j j k k + | ⟩⟨ | | ⟩⟨ | in this basis.
• Use matrix multiplication to compute Iˆis called the identity operator. Describe the effect of operator Iˆon F | ⟩ in a sentence.
The students also compute the identity operator in a different, three-dimensional orthonormal vector space to verify that the identity operator is always a square matrix with ones along the diagonal and zeroes elsewhere. Additional guided learning sequences help students extend what they learned about the identity operator in three dimensions to an N-dimensional Hilbert space. Further scaffolding support is provided to ensure that students are in the ZPD. They are asked to check whether their responses are consistent with follow up questions, reconcile possible differences between their initial responses and the correct concepts, and build a coherent understanding of the concepts.

5.3.2.
Addressing the difficulty involving confusion between the identity operator and a projection operator. The following question is a part of a learning sequence in the QuILT to help students reason about the difference between a projection operator and the identity operator. Students were asked the following question given that q n , 1, 2, 3 n = ¼¥ {| ⟩ } forms a complete set of orthonormal eigenstates of an operator Q corresponding to a physical observable with non-degenerate eigenvalues q : n • Consider the following conversation between Student A and Student B.
• Student A: I thought that q q n n | ⟩⟨ | was equal to the identity operator.
Was not that what we had learned earlier in this tutorial? How is it that the same expression is the identity operator and a projection operator at the same time?
• Student B: The expression that was equal to the identity operator was q q , n N n n 1 where there is a sum over a complete set of basis vectors. Applying that on a state Y | ⟩ would give the same state back. An example of a projection operator is q q n n | ⟩⟨ |. Acting with q q n n | ⟩⟨ | on a state Y | ⟩ gives the projection of that state along the direction of q n | ⟩ as follows: Do you agree with Student B's explanation? Explain why or why not.
Following these questions and peer discussion, an inquiry-based approach builds on student difficulties and strives to help students develop a coherent understanding of the identity operator. 5.3.3. Addressing student difficulties with identifying and generating an expression for a generic operator b Q corresponding to an observable. After traditional instruction in relevant concepts, students had great difficulty identifying and generating an expression for a generic Hermitian operator Q in terms of its eigenvalues and eigenstates. For example, on questions Q5 and Q6, some students claimed that Q q q , n N n n 1 The following questions from a learning sequence in the QuILT require students to identify an expression for an operator Q based upon their understanding up to that point and provide guidance and support to help them reconcile possible differences between their initial responses and the correct concepts: • Which one of the following relations is correct about an operator Q with eigenstates (c) Using your answers to the preceding parts, determine the expression for a Hermitian operator Q with eigenstates q | ⟩ with continuous eigenvalues q in terms of the eigenstates q | ⟩ and eigenvalues q.
Additional questions in the QuILT strive to scaffold student learning by providing opportunities for students to check their responses and reconcile possible differences between their responses and the correct understanding.
After working on the QuILT, students are expected to be able to identify and generate expressions in Dirac notation for the identity operator Iˆand a generic Hermitian operator Q corresponding to an observable Q in terms of the eigenstates and eigenvalues of the operator Q. The QuILT strives to help students develop a coherent understanding of the identity operator in the familiar context of a three-dimensional vector space before learning about the identity operator in an N -dimensional Hilbert space. Students are also given opportunities to identify expressions for a generic Hermitian operator Q corresponding to an observable in terms of its eigenstates and eigenvalues and check their work via the guided, inquiry-based approach in the QuILT.

Evaluation of the QuILT
We note that the evaluation of the QuILT in the present investigation followed the same process as reported in our previous work on investigating and improving student difficulties with quantum mechanics concepts in Dirac notation [53]: 'After the QuILT appeared to be effective in individual administration to students during interviews, it was administered to upper-level undergraduate and PhD students. Undergraduate students (N 87 = ) in four upperlevel undergraduate quantum mechanics courses first had traditional instruction in relevant concepts. Then, students were given a pretest on these topics in class. All students had sufficient time to work through the pretest. Students then worked through the QuILT in class and were given one week to work through the rest of the QuILT as homework. The pretest and QuILT counted as a small portion of their homework grade for the course. The pretest was not returned to students. Undergraduate students were then given a posttest in class (all students had sufficient time to take the posttest). The posttests were graded for correctness as a quiz for the quantum mechanics course. In addition, the upper-level undergraduate students were aware that topics discussed in the tutorial could also appear in future exams since the tutorial was part of the course material. ' As noted in [53], 'the QuILT was also administered to PhD students (N 97 = ) who were simultaneously enroled in the first semester of a graduate-level core quantum mechanics course and a course for training teaching assistants in four consecutive years. In the teaching assistant training class, the PhD students learned about instructional strategies for teaching introductory physics courses (e.g. tutorial-based approaches to learning). They first worked on the pretest (all students had sufficient time to take the pretest). The PhD students worked through the QuILT in the teaching assistant training class to learn about the effectiveness of the tutorial approach to teaching and learning. They were given one week to work through the rest of the QuILT as homework. Then, a posttest was administered to the PhD students in class (all students had sufficient time to take the posttest). The PhD students were given credit for completing the pretest, QuILT, and posttest, but they were not given credit for correctness. The PhD students' scores on the posttest did not contribute to the final grade for the teaching assistant training class (which was a Pass/Fail course)'.
To evaluate the effectiveness of the QuILT in helping students develop a coherent understanding of the identity operator Iˆand a generic operator Q corresponding to an observable Q, we compared the scores on questions Q1 and Q4 (shown in table 1) of students who worked on the QuILT versus students who did not work on the QuILT. Table 4 shows Table 4. Distribution of student responses on questions Q1 and Q4 for students who did not work through the QuILT (non-QuILT group) and students who worked through the QuILT (QuILT group). Some of the percentages do not add up to 100% due to rounding of the percentages.

Non-QuILT group (N=184)
QuILT group (N=124) the distribution of students' responses on questions Q1 and Q4 for the students who did not work through the QuILT (non-QuILT group) and for those who did (QuILT group). Questions Q1 and Q4 were administered to 184 undergraduate students at four universities in the US who did not work through the QuILT but had at least one semester of upper-level undergraduate quantum mechanics (these are the same 184 students listed in table 1). Questions Q1 and Q4 were also administered to a subset of the undergraduate and PhD students who worked through the QuILT (at least one month after the students had worked through the QuILT) and can be considered to test how much the students retained what they had learned after working through the QuILT at least one month later (these students were not listed in table 1, but they are listed in table 4). The performance of the undergraduate and PhD students was not significantly different on questions Q1 and Q4 so we do not differentiate between the two groups in table 4. We discuss students' performance on questions Q1 and Q4 below. Question Q1: Identifying an expression for the identity operator. In response to question Q1, only 45% of the students who had not worked through the QuILT selected the correct answer. On the other hand, 85% of the students who had worked through the QuILT answered question Q1 correctly. Furthermore, 96% of the students who worked through the QuILT correctly identified that the identity operator shown in option 1 (i.e. I q q n n n å = | ⟩⟨ | ) is correct by selecting option A, D or E. Students who worked through the QuILT performed better than random guessing when identifying the correct expression for the identity operator on question Q1.
Question Q4: Identifying an expression for an operator Q corresponding to an observable Q. In response to question Q4, only 35% of the students who had not worked through the QuILT were able to identify the correct expression for the spectral decomposition of an operator in terms of its eigenvalues and eigenstates, i.e. Q q q q . n n n n å = | ⟩⟨ | On the other hand, students who had worked through the QuILT performed better on this question-67% of them were able to identify the correct expression for the spectral decomposition of a generic operator Q.
Questions Q2 and Q6: Generating an expression for the identity operator Iˆand an operator Q corresponding to an observable Q. We also evaluated the effectiveness of the QuILT in improving students' understanding of the identity operator and an operator corresponding to an observable by giving the open-ended questions Q2 and Q6 (shown in table 1) on the QuILT pretest and posttest. These students are the same students who answered questions Q2 and Q6 in table 2. Students were given full credit on question Q2 if they wrote q q q d ò -¥ ¥ | ⟩⟨ | and half credit if they wrote an identity operator in terms of the eigenstates of an operator Q with a discrete eigenvalue spectrum, i.e. q q . n n n å | ⟩⟨ | On question Q6, students were given full credit if they wrote Q q q q n n n n å = | ⟩⟨ | and zero otherwise. Below, we discuss pretest (after traditional lecture-based instruction in relevant concepts) and posttest (after working on the QuILT) results for students who worked on the QuILT. Table 5 shows the percentages of students who correctly answered questions Q2 and Q6 on the pretest and posttest, the average normalised gain 〈g〉 [67], and effect size in the form of Cohen's d [68]. The number of students on the posttest does not match the pretest because students' scores on the posttest were not counted if they did not work through the entire tutorial. We note that 48% of the undergraduate students answered question Q6 correctly on the posttest, which is much lower than the percentage of PhD students who answered question Q6 correctly on the posttest. We have revised this component of the QuILT and provided more scaffolding to help students develop a better understanding of how to write a generic operator Q corresponding to an observable in Dirac notation. We plan to evaluate the effectiveness of the QuILT on this topic in future years.

Summary
We find that advanced students in quantum mechanics courses have common difficulties in expressing the identity operator and a generic Hermitian operator corresponding to a physical observable in Dirac notation in terms of the outer product of a complete set of orthonormal basis vectors. We found that these difficulties were common across different question contexts (e.g. questions with similar underlying concepts in which different types of notations were used), and they were similar to those found in other contexts. For example, some students struggled to differentiate between the concepts of the identity operator and a projection operator. In another context, students sometimes struggled to differentiate between the concepts of a stationary state (eigenstate of the Hamiltonian) and eigenstates of other Hermitian operators corresponding to physical observables (such as position eigenstates or momentum eigenstates) [3]. Furthermore, in this investigation, many students had difficulty differentiating between operators and numbers, which is consistent with a prior investigation in which students incorrectly claimed that the Hamiltonian operator Ĥ is equal to the energy of the system [3]. Students also had procedural difficulties, e.g. with indices when finding the expectation values of observables [3].
We developed a QuILT that strives to help students learn concepts related to writing the identity operator and an operator corresponding to a physical observable in a given basis. Although these concepts are very challenging, the evaluation of the QuILT is encouraging. Moreover, in an end of semester survey in one of the undergraduate quantum mechanics courses in which this QuILT was incorporated, many students reported that they felt the QuILT was very helpful in helping them learn these concepts. Since students struggled with writing operators corresponding to physical observables in Dirac notation, in the future, we plan to report our findings on student difficulties with writing quantum operators in position or momentum representations using Dirac notation and learning tools that help students learn these concepts better. We also plan to report on our findings regarding student difficulties with Dirac notation in the context of a three-dimensional vector space and learning tools that build on students' prior knowledge of three-dimensional vectors in a familiar context of introductory mechanics to help them learn about Dirac notation in quantum mechanics. Table 5. Percentages of undergraduate (UG) and PhD students (G) who correctly answered the pretest and posttest questions Q2 and Q6 that involved generating expressions for the identity operator Iˆand a generic operator Q corresponding to an observable Q, respectively, along with normalised gains 〈g〉 and effect sizes. Students who did not work through the tutorial in its entirety were removed from posttest data (N is the number of students).
Percentage of students who were correct on pretest Percentage of students who were correct on posttest