Towards testing the unitarity of the 3X3 lepton flavor mixing matrix in a precision reactor antineutrino oscillation experiment

The 3X3 Maki-Nakagawa-Sakata-Pontecorvo (MNSP) lepton flavor mixing matrix may be slightly non-unitary if the three active neutrinos are coupled with sterile neutrinos. We show that it is in principle possible to test whether the relation |V_{e1}|^2 + |V_{e2}|^2 + |V_{e3}|^2 = 1 holds or not in a precision reactor antineutrino oscillation experiment, such as the recently proposed Daya Bay II experiment. We explore three categories of non-unitary effects on the 3X3 MNSP matrix: 1) the indirect effect in the (3+ 3) flavor mixing scenario where the three heavy sterile neutrinos do not take part in neutrino oscillations; 2) the direct effect in the (3+\mathbbm{1}) scenario where the light sterile neutrino can oscillate into the active ones; and 3) the interplay of both of them in the (3+\mathbbm{1}+ 2) scenario. We find that both the zero-distance effect and flavor mixing factors of different oscillation modes can be used to determine or constrain the sum of |V_{e1}|^2, |V_{e2}|^2 and |V_{e3}|^2 and its possible deviation from one, and the active neutrino mixing angles \theta_{12} and \theta_{13} can be cleanly extracted even in the presence of light or heavy sterile neutrinos. Some useful analytical results are obtained for each of the three scenarios.


Introduction
The ν e , ν µ and ν τ neutrinos are active in the sense that they take part in the standard weak interactions. They are significantly different from their corresponding mass eigenstates ν 1 , ν 2 and ν 3 as a result of nondegenerate neutrino masses and large lepton flavor mixing [1]. Whether there exist the sterile neutrinos, which do not directly take part in the standard weak interactions, has been an open question in particle physics and cosmology. One is motivated to consider such "exotic" particles for several reasons. On the theoretical side, the canonical (type-I) seesaw mechanism [2] provides an elegant interpretation of the small masses of ν i (for i = 1, 2, 3) with the help of two or three heavy sterile neutrinos, and the latter can even help account for the observed matter-antimatter asymmetry of the Universe via the leptogenesis mechanism [3]. On the experimental side, the LSND [4], MiniBooNE [5] and reactor [6] antineutrino anomalies can all be explained as the active-sterile antineutrino oscillations in the assumption of one or two species of sterile antineutrinos whose masses are below 1 eV [7]. Furthermore, a careful analysis of the existing data on the Big Bang nucleosynthesis [8] or the cosmic microwave background anisotropy, galaxy clustering and supernovae Ia [9] seems to favor at least one species of sterile neutrinos at the sub-eV mass scale. On the other hand, sufficiently long-lived sterile neutrinos in the keV mass range might serve for a good candidate for warm dark matter if they were present in the early Universe [10]. That is why the study of sterile neutrinos becomes a popular direction in today's neutrino physics [11].
In the presence of small active-sterile neutrino mixing, the conventional 3×3 Maki-Nakagawa-Sakata-Pontecorvo (MNSP) lepton flavor mixing matrix [12] is just the submatrix of a (3 + n) × (3 + n) unitary matrix V which describes the overall flavor mixing of 3 active neutrinos and n sterile neutrinos in the basis where the flavor eigenstates of the charged leptons are identified with their mass eigenstates:  Hence the 3×3 MNSP matrix itself must be non-unitary. From the point of view of neutrino oscillations, one may classify its possible non-unitary effects into three categories: • the indirect non-unitary effect arising from the heavy sterile neutrinos which are kinematically forbidden to take part in neutrino oscillations; • the direct non-unitary effect caused by the light sterile neutrinos which are able to participate in neutrino oscillations; • the interplay of the direct and indirect non-unitary effects in a flavor mixing scenario including both light and heavy sterile neutrinos.
In each of the three cases, no matter how small or how large the mass scale of sterile neutrinos could be, the experimental information on the matrix elements of V is essentially different from that in the standard case (i.e., the case in which V is exactly a 3 × 3 unitary matrix). Hence testing the unitarity of the 3 × 3 MNSP matrix is experimentally important to constrain the flavor mixing parameters of possible new physics and can theoretically shed light on the underlying dynamics responsible for the neutrino mass generation and lepton flavor mixing (e.g., the 3× 3 MNSP matrix is exactly unitary in the type-II [13] seesaw mechanism but non-unitary in the type-I [2] and type-III [14] seesaw mechanisms).
Following a similar strategy for the precision test of the unitarity of the 3 × 3 Cabibbo-Kobayashi-Maskawa (CKM) quark flavor mixing matrix [1], here we concentrate on a possible experimental test of the normalization relation |V e1 | 2 + |V e2 | 2 + |V e3 | 2 = 1 of the 3 × 3 MNSP matrix in a precision reactor experiment which is expected to be able to distinguish between the oscillation modes induced by ∆m 2 31 and ∆m 2 32 . The recently proposed Daya Bay II reactor antineutrino oscillation experiment [15] is just of this type, so is the proposed RENO-50 reactor experiment [16]. At present a very preliminary constraint on the sum of |V e1 | 2 , |V e2 | 2 and |V e3 | 2 is [17] |V e1 | 2 + |V e2 | 2 + |V e3 | 2 = 0.994 ± 0.005 (2) at the 90% confidence level, implying that the 3 × 3 MNSP matrix is allowed to be non-unitary at the 1% level. In this note we shall discuss how to determine |V e1 | 2 , |V e2 | 2 and |V e3 | 2 via a precision measurement of the ν e → ν e oscillation and examine whether their sum deviates from one or not. To be more specific, we are going to consider three typical scenarios of active-sterile neutrino mixing to illustrate possible non-unitary effects on the 3 × 3 MNSP matrix as listed above: • The (3+3) flavor mixing scenario with three heavy sterile neutrinos which indirectly violate the unitarity of the 3 × 3 MNSP matrix; • The (3+½) flavor mixing scenario with a single light sterile neutrino which directly violates the unitarity of the 3 × 3 MNSP matrix; • The (3+½+2) flavor mixing scenario in which the light and heavy sterile neutrinos violate the unitarity of the 3 × 3 MNSP matrix directly and indirectly, respectively.
In each case the sum |V e1 | 2 + |V e2 | 2 + |V e3 | 2 can be expressed in terms of the active-sterile neutrino mixing angles in a given parametrization of the overall (3 + n) × (3 + n) flavor mixing matrix. Taking the parametrization proposed in Ref. [18] for example, we shall show that it is possible to determine the active neutrino mixing angles θ 12 and θ 13 without any contamination coming from the sterile neutrinos. We hope that the points to be addressed in the remaining part of this note will be helpful for the next-generation reactor antineutrino oscillation experiments, either to test the standard 3 × 3 MNSP flavor mixing picture or to probe new physics via its possible non-unitary effects.

Non-unitary effects
In the presence of n species of sterile neutrinos, no matter whether they are very light or very heavy, the amplitude of the active ν α → ν β oscillation (for α, β = e, µ, τ ) in vacuum can be expressed as ββ describe the production of ν α via the weak charged-current interaction, the propagation of free ν i and the detection of ν β via the weak charged-current interaction, respectively [17]- [20] (a schematic diagram is shown in Fig. 1 for illustration). Here m i is the mass of the light (active or sterile) neutrino ν i , E denotes the neutrino beam energy and L stands for the distance between the source and the detector. It is then straightforward to calculate the probability P (ν α → ν β ) = |A(ν α → ν β )| 2 . We obtain 1 Here we only pay interest to the reactor antineutrino oscillations in vacuum, Note that the energy-independent term on the right-hand side of Eq. (5) is not equal to one if there are heavy sterile antineutrinos which do not take part in the oscillation. This point will be made clear in the subsequent discussions, in which a few typical active-sterile neutrino mixing scenarios will be taken into account. Given the fact that the typical value of E is a few MeV and that of L is usually less than 100 km for a realistic experiment, it is completely unnecessary for us to consider the negligibly small terrestrial matter effects on P(ν e → ν e ).

The (3 + 3) flavor mixing scenario
Let us first consider the (3+3) flavor mixing scenario with three heavy sterile neutrinos which indirectly violate the unitarity of the 3 × 3 MNSP matrix. In this case a full parametrization of the 6 × 6 flavor mixing matrix has been given in Ref. [18], and the elements in its first row read where c ij ≡ cos θ ij andŝ ij ≡ sin θ ij e iδ ij with θ ij being the flavor mixing angles and δ ij being the CPviolating phases. In view of the fact that the active-sterile neutrino mixing angles θ 14 , θ 15 and θ 16 are at most of O(0.1) in magnitude [17], the actual values of the three active neutrino mixing angles θ 12 , θ 13 and θ 23 should be very close to those extracted from current neutrino oscillation data by assuming the 3 × 3 MNSP matrix to be exactly unitary. Eq. (6) leads us to 1 Note that (V V † ) αα and (V V † ) ββ in front of P (ν α → ν β ) are the normalization factors and can essentially be canceled by the same factors coming from the production of ν α at the source and the detection of ν β at the detector [17], which are both governed by the weak charged-current interactions. Hence we focus on P(ν α → ν β ) in this work.
implying that the sum of |V e1 | 2 , |V e2 | 2 and |V e3 | 2 is possible to deviate from one either at the percent level or at a much lower level. Is such a small effect really detectable in a precision reactor ν e → ν e oscillation experiment?
In the present scenario the three hypothetical heavy sterile neutrinos are kinematically forbidden to participate in neutrino oscillations. So Eq. (5) can be simplified to where ∆m 2 32 = ∆m 2 31 − ∆m 2 21 has been used, and Note that the third oscillation term in Eq. (8) is sensitive to the unknown sign of ∆m 2 31 (i.e., ∆m 2 31 > 0 and ∆m 2 31 < 0 correspond to the normal and inverted neutrino mass hierarchies, respectively), and it might be measurable in a precision reactor antineutrino oscillation experiment in the foreseeable future. Some comments on the implications of Eqs. (8) and (9) are in order.
• The non-unitarity of the 3 × 3 MNSP matrix, characterized by the small deviation of I 0 from one, can be tested via the energy-independent zero-distance effect [19] in such a disappearance antineutrino oscillation experiment: This effect is in principle measurable with the help of a near detector, although it might in practice be indistinguishable from the background which at least includes the uncertainties associated with the antineutrino flux. If the energy spectrum of the ν e → ν e oscillation can be fully established, however, it should be possible to determine or constrain the size of I 0 .
• The flavor mixing factors A, B and C are simple functions of the matrix elements |V e1 | 2 , |V e2 | 2 and |V e3 | 2 . It is therefore straightforward to obtain Provided A, B and C are all measured to a good degree of accuracy, one may also use Eq. (11) to calculate the sum |V e1 | 2 + |V e2 | 2 + |V e3 | 2 = I 0 and examine whether it departs from one or not. On the other hand, the value of I 0 to be determined in this way can be compared with the one to be measured from the zero-distance effect as shown in Eq. (10).
• Given the concise parametrization in Eq. (6), the active neutrino mixing angles θ 12 and θ 13 can be determined from the ratios A/I 0 , B/I 0 and C/I 0 without any contamination coming from the heavy sterile neutrinos. This point is clearly seen as follows: A I 0 = 1 4 sin 2 2θ 12 cos 4 θ 13 + sin 2 θ 12 sin 2 2θ 13 , Taking θ 12 ≃ 34 • and θ 13 ≃ 9 • for example [21], we immediately have A/I 0 ≃ 0.212, B/I 0 ≃ 0.024 and C/I 0 ≃ 0.0075. The smallness of C makes it rather difficult to be measured (in other words, a determination of the sign of ∆m 2 31 or the neutrino mass ordering is a big challenge to the reactor antineutrino oscillation experiments [22]- [26]).
To observe the non-unitary effects in the (3+3) flavor mixing scenario under consideration, one may also study the appearance neutrino oscillation (e.g., the ν µ → ν τ oscillation) experiments with reasonably long baselines. In such a precision accelerator neutrino oscillation experiment, even new CP-violating effects are likely to show up at the O(10 −3 ) or O(10 −2 ) level provided the relevant active-sterile flavor mixing angles are not strongly suppressed [27].
At this point it is worth mentioning that the type-I seesaw mechanism around the TeV energy scale can simply lead to the (3+3) flavor mixing scenario with appreciable non-unitary effects [28]. Similar effects may result from some interesting extensions of the type-I seesaw mechanism, such as the inverse seesaw scenario [29], the linear seesaw scenario [30] and the multiple seesaw scenarios [31]. In such model-building exercises, presumably significant lepton-flavor-violating effects and non-standard neutrino interactions are also expected to show up and their interplay with the non-unitary effects offers an important and complementary window for probing the true mechanism of neutrino mass generation and lepton flavor mixing [32]. On the experimental side, the precision reactor antineutrino oscillation experiments will be complementary to the precision accelerator neutrino experiments for our physics goal and thus deserve particular attention.
• Given a light sterile antineutrino whose absolute mass scale is unspecified or ∆m 2 41 is unknown, it is in principle possible to determine or constrain the active-sterile flavor mixing angle θ 14 via a precision measurement of A ′ , B ′ and C ′ . The deviation of these three parameters from their values in the standard case is characterized by nonzero sin 2 θ 14 . Taking θ 12 ≃ 34 • , θ 13 ≃ 9 • and θ 14 ≃ 10 • as a typical example, we obtain A ′ ≃ 0.208, B ′ ≃ 0.023 and C ′ ≃ 0.0071, which can be compared with A ′ ≃ 0.212, B ′ ≃ 0.024 and C ′ ≃ 0.0075 in the θ 14 ≃ 0 • case.
• Of course, a direct measurement of the oscillation term driven by ∆m 2 41 will provide the direct evidence for the existence of a light sterile antineutrino in the ν e → ν e oscillation. The most optimistic case, which seems to be very unlikely, is that X ′ , Y ′ and Z ′ could all be determined. Considering θ 12 ≃ 34 • , θ 13 ≃ 9 • and θ 14 ≃ 10 • , we have X ′ ≃ 0.029, Y ′ ≃ 0.0089 and Z ′ ≃ 0.00072. One can see that Z ′ is too small to be measured, but it does not matter because the sign of ∆m 2 41 2 Note that the other solution of |V ei | 2 (for i = 2, 3, 4) is expected to be inconsistent with the observed neutrino mixing pattern, and thus it is ignored here.
is essentially determinable from a precision measurement of the interference term associated with 2Y ′ in Eq. (13). In particular, it should be noted that Y ′ /Z ′ = sin 2 θ 12 cot 2 θ 13 ≃ 12 is a result independent of the input value of θ 14 .
• Provided ∆m 2 4i ≫ ∆m 2 31 ≃ 2.5 × 10 −3 eV 2 holds (for i = 1, 2, 3) and the experimental baseline length L favors the oscillation terms driven by ∆m 2 31 and ∆m 2 32 , the corresponding active-sterile antineutrino oscillation terms sin 2 [∆m 2 4i L/(4E)] will practically be averaged to 1/2 because they oscillate too quickly. In this case the overall effect induced by the sterile antineutrino becomes the zero-distance effect: A near detector is certainly difficult to measure such a small effect because of the uncertainties associated with the antineutrino flux. Nevertheless, we remark that the similar effect could be cross-checked from a precision determination of the flavor mixing factors A ′ , B ′ and C ′ .
In the same way one may discuss an analogous flavor mixing scenario with two or three light sterile antineutrinos which take part in the ν e → ν e oscillation.

The (3 + ½ + 2) flavor mixing scenario
Given three sterile neutrinos, one of them can be assumed to be light enough so as to either interpret the existing sub-eV antineutrino anomalies or account for the keV warm dark matter [33]. In such a (3+½+2) flavor mixing scenario the two heavy sterile neutrinos may play the role in realizing the seesaw and leptogenesis mechanisms [34]. The deviation of the sum |V e1 | 2 + |V e2 | 2 + |V e3 | 2 from one is also described by Eq. (6), but the ν e → ν e oscillation probability turns out to be in which We see that Eq. (18) consists of both direct and indirect non-unitary effects on the 3 × 3 MNSP matrix. A straightforward calculation allows us to express |V ei | 2 (for i = 1, 2, 3, 4) in terms of the flavor mixing factors ( A, B, C) and ( X, Y , Z) as follows: Of course, |V e4 | 2 = Y Z/ C holds too 3 . Using the neutrino mixing angles given in Eq. (6) Some discussions and remarks are in order.
• Switching off the light sterile neutrino (i.e., setting |V e4 | 2 = 0 or equivalently X = Y = Z = 0), one can easily reproduce Eq. (11) from Eq. (20). If the two heavy sterile neutrinos are switched off, then it is straightforward to reproduce Eq. (14) from Eqs. (19) and (21). The interplay between the direct non-unitary effect measured by nonzero θ 14 and the indirect non-unitary effect characterized by nonzero θ 15 and θ 16 is therefore transparent. In particular, the relationship holds, and I 0 can in principle be determined from the zero-distance effect.
• Provided A/ I 0 , B/ I 0 and C/ I 0 are all measured in a precision reactor antineutrino experiment, it should be possible to determine the flavor mixing angles θ 12 , θ 13 and θ 14 in a way independent of the flavor mixing angles θ 15 and θ 16 . The same observation is true for the flavor mixing factors X/ I 0 , Y / I 0 and Z/ I 0 , although they must be much more difficult to be measured. Of course, the situation might be simplified to some extent if the existence of a light sterile antineutrino could be established somewhere else (e.g., with the help of a suitable accelerator neutrino (or antineutrino) oscillation experiment). But the important point is that the reactor experiment can always provide some independent and complementary information on the same physics, no matter whether it is new or just conventional.
• In the present flavor mixing scenario, the effective neutrino mass terms of the tritium beta decay with m e = m 2 1 c 2 12 c 2 13 + m 2 2 s 2 12 c 2 13 + m 2 3 s 2 13 being the standard contribution from the three active neutrinos and with m ee = m 1 (c 12 c 13 ) 2 +m 2 (ŝ * 12 c 13 ) 2 +m 3 (ŝ * 13 ) 2 being the standard contribution from the active neutrinos, respectively. Here we have assumed that a possible contribution from the heavy sterile neutrinos is negligible [35]. We see that m ′ e ≥ m e always holds, but it is difficult to judge the relative magnitudes of | m ee | and m ′ ee because the relevant CP-violating phases may lead to more or less (even complete) cancelations of different terms in them [36].

Concluding remarks
In the era of precision neutrino physics one of the important jobs is to test the unitarity of the 3 × 3 MNSP flavor mixing matrix and probe possible new physics which might give rise to some observable non-unitary effects on it. Such effects serve as a special example of possible non-standard interactions associated with massive neutrinos and their oscillations [37]. Starting from this point of view, we have classified three typical categories of non-unitary effects on the MNSP matrix and illustrated them in a precision reactor antineutrino oscillation experiment: 1) the indirect effect in the (3+3) flavor mixing scenario where the three heavy sterile neutrinos do not participate in neutrino oscillations; 2) the direct effect in the (3+½) scenario where the light sterile neutrino can oscillate into the active ones; and 3) the interplay of both of them in the (3+½+2) scenario. We have shown that both the zero-distance effect and flavor mixing factors of different oscillation modes can be used to determine or constrain the sum of |V e1 | 2 , |V e2 | 2 and |V e3 | 2 and its possible deviation from one. In addition, the active neutrino mixing angles θ 12 and θ 13 can be cleanly extracted even in the presence of light or heavy sterile neutrinos. Some useful analytical results, which can be applied to a numerical analysis of the future experimental data, have been presented for each of the three scenarios under consideration.
We expect that some points of view in this work will be helpful for a realistic reactor antineutrino oscillation experiment, such as the proposed Daya Bay II experiment which aims to measure the neutrino mass ordering and test the unitarity of the 3 × 3 MNSP matrix. We admit the big challenges that one has to face in practice, either in measuring the zero-distance effect or in detecting different oscillation modes, or in both of them. Nevertheless, we strongly hope that the Daya Bay II experiment and other precision reactor antineutrino oscillation experiments may play an important role in probing possible new physics in the lepton sector, or can at least be complementary to the precision accelerator neutrino oscillation experiments in this respect in the foreseeable future.
An obvious drawback of this work is the lack of a numerical analysis of the future experimental sensitivities to the potential non-unitary effects on the MNSP matrix. This is simply because the main characteristics of the currently proposed Daya Bay II and RENO-50 experiments remain too preliminary and incomplete, and the existing constraints on the non-unitary effects are also preliminary and more or less dependent on the hypothetical active-sterile flavor mixing scheme. In this case we have to focus on the analytical discussions about the salient features of direct and indirect non-unitary effects (and their interplay) in the present work, leaving a quantitative study of the same topic in a future work.
The author is indebted to Y.F. Li and S. Luo for intensive and valuable discussions, and in particular to S. Luo for her partial involvement at the early stage of this work. He is also grateful to C. Giunti and S. Zhou for helpful comments on the interplay of direct and indirect non-unitary effects. This work is supported in part by the National Natural Science Foundation of China under Grant No. 11135009. + _ Figure 1: The schematic diagram for the production of the ν α neutrino via the weak charged-current interaction, the propagation of free ν i neutrinos in vacuum and the detection of the ν β neutrino via the weak charged-current interaction in the ν α → ν β oscillation process (for α, β = e, µ, τ ), in which there might be possible non-unitary effects on the 3 × 3 MNSP flavor mixing matrix due to the presence of light and (or) heavy sterile neutrinos.