Hidden flavor symmetries of SO(10) GUT

The Yukawa interactions of the SO(10) GUT with fermions in 16-plets (as well as with singlets) have certain intrinsic ("built-in") symmetries which do not depend on the model parameters. Thus, the symmetric Yukawa interactions of the 10 and 126 dimensional Higgses have intrinsic discrete $Z_2\times Z_2$ symmetries, while the antisymmetric Yukawa interactions of the 120 dimensional Higgs have a continuous SU(2) symmetry. The couplings of SO(10) singlet fermions with fermionic 16-plets have $U(1)^3$ symmetry. We consider a possibility that some elements of these intrinsic symmetries are the residual symmetries, which originate from the (spontaneous) breaking of a larger symmetry group $G_f$. Such an embedding leads to the determination of certain elements of the relative mixing matrix $U$ between the matrices of Yukawa couplings $Y_{10}$, $Y_{126}$, $Y_{120}$, and consequently, to restrictions of masses and mixings of quarks and leptons. We explore the consequences of such embedding using the symmetry group conditions. We show how unitarity emerges from group properties and obtain the conditions it imposes on the parameters of embedding. We find that in some cases the predicted values of elements of $U$ are compatible with the existing data fits. In the supersymmetric version of SO(10) such results are renormalization group invariant.


Introduction
In spite of various open questions, Grand Unification [1,2] is still one of the most appealing and motivated scenarios of physics beyond the standard model. The models based on SO(10) gauge symmetry [3,4,5] are of special interest since they embed all known fermions of a given generation and the right handed neutrinos in a single multiplet 3 . One of the open questions is to understand the flavor structures -observed fermion masses and mixings, which SO(10) unification alone can not fully address 4 . Moreover, embedding of all the fermions in a single multiplet looks at odds with different mass hierarchies and mixings, and in particular with the strong difference of mixing patterns of quarks and leptons.
The Yukawa sector of the renormalizable 5 version of SO(10) GUT [7] with three generations of matter fields in 16 F is given by where the 3 × 3 matrices of Yukawa couplings, Y 10 , Y 126 and Y 120 correspond to Higgses in 10 H , 126 H and 120 H . The masses and mixings of the Standard Model (SM) fermions are 3 We consider here theories with no extra vector-like matter which could mix with SM fermions. This is the case of the majority of available models, but may not be the case if SO(10) is coming from E 6 . 4 Partially it sometimes can: for example, b − τ unification can be related to the large atmospheric mixing angle in models with dominant type II seesaw [6]. 5 To realize eventually our scenario these Yukawa couplings should be VEVs of fields which transform non-trivially under some flavor group G f -so they will be non-renormalizable, or we should ascribe charges to the Higgs multiplets. determined by these Yukawa couplings Y a , the Clebsch-Gordan coefficients and the VEV's of the light Higgs(es). So, to make predictions for the masses and mixing one needs, in turn, to determine the matrices Y a (a = 10, 126, 120).
There are various attempts to impose a flavor symmetry on the Yukawa interaction (1.1) to restrict the mass and mixing parameters, see for example [8] for continuous symmetries, [9,10,11] for discrete symmetries, and [12] for reviews. In most cases flavor symmetries appear as horizontal symmetries -which are independent of the vertical gauge symmetry SO (10).
Two interesting ideas have been discussed recently which employ an interplay between the GUT symmetry and flavor symmetries and may lead to deep relation between them.
1. Existence of "natural" ("built-in") or intrinsic flavor symmetries [13]. Examples are known from the past that some approximate flavor symmetries can arise from the "vertical" gauge symmetries. One of these is the antisymmetry of the Yukawa couplings of the lepton doublets with charged scalar singlet. The neutrino mass matrix generated at 1-loop (Zee model [14]) has specific flavor structure with zero diagonal terms.
It is well known that SO (10) have such flavor symmetries. The three terms in (1.1) have symmetries dictated by "vertical" SO (10): symmetricity of the Yukawa coupling matrices of the 10-plet and 126-plet of Higgses and antisymmetricity of the Yukawa coupling matrix of the 120-dimensional Higgs multiplets: The first equality (symmetricity) implies a Z 2 × Z 2 symmetry [13]. For the antisymmetric matrix (second equality) the symmetry (Z 2 ) has been taken in [13] (or (Z 2 ) 2 if negative determinants are allowed).
2. Identification of the natural symmetries with residuals of the flavor symmetry [13]. This idea is taken from the residual symmetry approach developed to explain the lepton mixing. It states that some or all elements of the natural symmetries of SO (10) are actually the residual symmetries which originate from the breaking of a bigger flavor symmetry group G f [15,16,17,18,19]. In [13] it was proposed to embed the residual (Z 2 ) n , which are reflection symmetries, into the minimal group with a three-dimensional representation. This leads to the Coxeter group and finite Coxeter groups of rank 3 and 4 have been considered. The embedding of natural symmetries into the flavor (Coxeter) group imposes restrictions on the structure of Y a and consequently on the mass matrices, which reduces the number of free parameters.
In this paper we further elaborate on realizations of these ideas, although from a different point of view. While the intrinsic symmetries of Y 10 and Y 126 are Z 2 × Z 2 , as in [13], we find that Y 120 has a bigger symmetry -SU (2). Furthermore, we consider the situation when SO(10) singlet fermions are present. From the embedding of intrinsic symmetries and with the use of symmetry group relations [20,21] we obtain predictions for the elements of the relative mixing matrix U a−b (a, b = 10, 126, 120) between the Yukawa couplings Y a and Y b (U a−b connects the bases in which matrices Y a and Y b are diagonal). These unitary matrices U a−b are basis independent, in contrast to the matrices Y a and Y b themselves. We re-derive these relations and elaborate on the unitarity condition, showing that it follows from group properties. We confront the predictions with the results of some available data fits.
The paper is organized as follows. In sect. 2 we explore the intrinsic symmetries of the SO(10) Yukawa couplings. In sect. 3 we identify (part of) the intrinsic symmetries with the residual symmetries and consider their embedding into a bigger flavor group. Using the symmetry group relations we obtain predictions for different elements of the relative matrix U . We elaborate on the unitarity condition which gives additional bounds on the parameters of embedding. We consider separately the embeddings of the 120 H couplings. This case has not been covered in [13] and we develop various methods to deal with it. In sect. 4 we confront our predictions for the mixing matrix elements with the results obtained from existing fits of data. In sect. 5 we consider symmetries in the presence of the SO(10) fermionic singlets. In sect. 6 we summarize the concept of intrinsic symmetry and the relative mixing matrix. Summary of our results and conclusion are presented in sect. 7. We compare our approach with that in [13] in Appendix A, suggesting an equivalence.

Relative mixing matrices
The matrices of Yukawa couplings are basis dependent. It is their eigenvalues and the relative mixings which have physical meaning. The relative mixing matrices, which are the main object of this paper, are defined in the following way. The symmetric matrices Y 10 and Y 126 can be diagonalized with the unitary transformation matrices U 10 and U 126 as and Mixing is generated if the matrices Y a can not be diagonalized simultaneously. The relative mixing matrix U 10−126 is given by This matrix, in contrast to matrices of Yukawa couplings, does not depend on basis and has immediate physical meaning. In a sense, it is the analogy of the PMNS (or CKM) matrix which connects bases of mass states of neutrinos and charged leptons. Similarly we can introduce the relative mixing matrices for other Yukawa coupling matrices as e.g., U 10−120 , U 120−126 , etc.
The symmetry formalism we present below (symmetry group relations) will determine elements of the relative matrices immediately without consideration of the symmetric matrices Y a and their diagonalization.

Intrinsic symmetries
All the terms of the Lagrangian (1.1) have the same fermionic structure, being the Majorana type bilinears of 16 F . This by itself implies certain symmetry. For definiteness let us consider the basis of three 16 F plets in which the Yukawa coupling of the 10-plet is diagonal:  8) and the diagonal transformations equal (We use generators with Det[S i ] = +1, so that they can form a subgroup of SU (3)).
The transformations (2.9) can be written as and a, b = 1, 2, 3. All these transformations (reflections) obey The matrix Y 126 is invariant under another, G 126 = Z 2 × Z 2 group consisting of U − transformed elements where U 126 is defined in (2.2). This intrinsic symmetry is always present independently of parameters of the model due to the symmetric Yukawa matrices Y 10 and Y 126 [13] which follow from SO(10) symmetry.
In the case of antisymmetric Yukawa interactions of 120 H the situation is different. The antisymmetric matrix Y 120 can be put in the canonical form by the unitary transformation U 120 as The matrix (2.14) is invariant with respect to SU(2)×U(1) transformations Again we will bound ourselves to group elements with Det(g) = 1, keeping in mind possible embedding into SU (3). Then there is no U(1), and therefore The SU (2) transformation element g can be written as and τ being the Pauli matrices.
Although the symmetry of the Yukawa matrix connected to the 120-plet is continuous, we should use only its discrete subgroup to be a part of G f , since G f itself has been assumed to be discrete. This means that the angle φ should take discrete values such that for some integer p. The angle can be parametrized as φ = 2π n pφ , n = 1, . . . , p − 1, (2.20) whereφ ≡ φ/| φ| (so thatφ 2 = 1). In this paper we will consider a Z p subgroup of the Abelian U (1) ⊂ SU (2). So, the elements g φ , g 2 φ , . . . , g p−1 φ can be written as More on intrinsic symmetries and the mixing matrices can be found in Appendix 6. Intrinsic symmetries for the SO(10) singlets are discussed in sect. 5.
We assume throughout this paper that the Higgs multiplets are uncharged with respect to G f . Introduction of Higgs charges can lead to suppression of some Yukawa couplings but does not produce the flavor structure of individual interactions.

Embedding intrinsic symmetries
Following [13] we assume that the intrinsic symmetries formulated in the previous section are actually residual which result from the breaking of a larger (flavor) symmetry group G f . In other words, some of the symmetries G 10 and G 126 or/and G 120 are embedded into G f . In the following we will derive various constraints on the relative mixing matrix U between two Yukawa matrices.

Embedding of two transformations
We recall the symmetry group relation formalism [20,21] adopted to our SO(10) case. The formalism allows to determine (basis independent) elements of the relative mixing matrix immediately without explicit construction of Yukawa matrices. Let us first consider the Yukawa couplings of 10 H and 126 H . Suppose the covering group G f contain S d j ∈ G 10 and S i ∈ G 126 . Since S i , S d j ∈ G f , the product S i S d j should also belong to G f : S i S d j ∈ G f . Then the condition of finiteness of G f requires that a positive integer p ji exists such that This is the symmetry group relation [20,21] which we will use in our further study. Inserting Furthermore, we will impose the condition keeping in mind a possible embedding into SU (3). We will comment on the case of negative determinant later.
The simplest possibility is the residual symmetries Z , that is Z 2 for Y 10 and another Z 2 for Y 126 . In this case the flavor symmetry group G f is always a finite von Dyck group (2, 2, p), since 1 2 for any positive integer p. Let us elaborate on the constraint (3.1) further, providing derivation of the relations slightly different to that in [20,21]. According to the Schur decomposition we can present W ij in the form (3.7) The diagonal elements of W upper ij are the (in general complex) eigenvalues of W ij which we denote by λ α . Therefore, where Inserting (3.6) into condition (3.2) and using unitarity of V we obtain the off-diagonal elements in the LH side should be zero to match with the RH side). Consequently, the eigenvalues of W equal the p ji -roots of unity: Finally, Eq. (3.8) gives where a p ji is defined in (3.9). The p ji −roots of unity can be parametrized as For p ≥ 3 the number of p-roots is larger than 3, and therefore there is an ambiguity in selecting the three values to compose a p ji . However, not all combinations can be used, and certain restrictions will be discussed in the following. Restriction on a p ji arises from the following consideration. The eigenvalues λ α satisfy the characteristic polynomial equation W ij : where a p ji is defined in (3.9) 7 . Consider the conjugate of Eq. (3.14). Using the expression for (3.3) and taking into account that S d i 2 = I we obtain This in turn gives for the LHS of the conjugate equation (3.17) Therefore the set of eigenvalues {λ α } coincides with the set {λ * α } [22]. Then it is easy to check that this is possible only if one of λ α equals unit, e.g. λ 1 = 1, and two others are conjugate of each other: λ 3 = λ * 2 ≡ λ. Thus, a p ji = a * p ji = 1 + λ + λ * = 1 + 2Reλ, (3.18) or explicitly, a p ji = 1 + 2 cos (2πk ji /p ji ) = −1 + 4 cos 2 (πk ji /p ji ). (3.19) On the other hand, from definitions of S j (2.10) and (3.3), we find explicitly or using (3.12) (see also [23]) Notice that the trace (3.20) is real, and therefore a p ji = a * p ji , leading to the form (3.18). Finally, inserting a p ji from (3.19) we obtain This can be obtained noticing that Similar expression has been obtained before in [24] in the Dihedral group model for the Cabibbo angle (V us ). The expression appears also in [25]. Thus, we obtain thus a relation for a single element of the matrix U , as the consequence of the Z residual symmetry. The element |U ji | is determined by two discrete parameters -arbitrary integers p ji and k ji = 0, . . . , p ji − 1. The expression does not depend on the selected S i . The elements S i and S j just fix the ij− element of the matrix U , but not its value, the value is determined by p ji and k ji .

Embedding of bigger residual symmetries and Unitarity
Following the derivations in [22,26] we summarize here the embedding of bigger residual symmetries, when we take Z 2 × Z 2 from one of the interactions (10 H or 126 H ) and one Z 2 from the other interaction. Now there are three generating elements: for Z Consequently, we have two symmetry group conditions: which determine two elements of the matrix U from the same column i: |U ji | and |U ki |.
Repeating the same procedure of the previous section we obtain The second possibility is Z with one generating element for Y 10 and two for Y 126 . This gives also two symmetry group conditions but for two elements in the same row of U . This is enough to determine the whole row (or column in the first case) from unitarity. Possible values of matrix elements for this case have been classified in whole generality [22,26].
Using the complete symmetry Z one can fix 4 elements of U , and consequently, due to unitarity, the whole matrix U . This matrix is necessarily of the type classified in [22,26]. 8 The eigenvalues λ 1,2,3 of W ij satisfy (|λ α | 2 = 1) If a * pji = a pji , one eigenvalue is equal to Det(W ij ).
Notice that values of the elements of the relative matrix U have been obtained using different group elements S j (for fixed S i ) essentially independently. They were determined by the independent parameters p j , k j . However, there are relations between the group elements S j which, as we will see, lead to relations between parameters p j , k j , which are equivalent to relations required by unitarity of the matrix U .
According to (3.22) |U ji | ≤ 1 for any pair of values of k and p. For two elements in the same line or column unitarity requires and it is not fulfilled automatically. (In this section we omit the second index of k and p, which is the same for both. Keeping in mind that both are from the same line or the same column.) Furthermore, the inequality (3.28) can not be satisfied for arbitrary k i and p i , and therefore gives certain bounds on these parameters. This, in turn, affects the embedding (covering group). In what follows we will consider such restrictions on parameters k and p that follow from relations between the group elements.
The elements of Z 2 × Z 2 group in 3 dimensional representation (2.9) or (2.10) satisfy the following equalities and Let us find the corresponding relations between the parameters p i and k i . Summation over the index i of the traces Tr The last expression in this formula together with equalities (3.29) and (3.30) gives −T r[S d j ] = 1. Therefore i Tr[W ij ] = 1 and according to (3.12) we find a p 1 + a p 2 + a p 3 = 1. (3.32) Finally, insertion of expressions for a p i in eq. (3.19) leads to This coincides with the unitarity condition: Eq. (3.33) is nothing but i |U ij | 2 = 1, where the elements are expressed via cosines (3.22). So, the unitarity condition is encoded in the relation (3.29) which is equivalent to the unitarity. Thus, the unitarity condition which imposes relations between p j and k j can be obtained automatically from properties of the group elements. The condition is highly non-trivial since it should be satisfied for integer values of p i and k i . It can be fulfilled for specific choices of (k 1 /p 1 , k 2 /p 2 , k 3 /p 3 ). There are just few cases which can satisfy (3.33). Some of these constraints have been found in [27,28] from specific assumptions on G f . In general, it has been shown [22,26] (see also [25]) that the only possibilities are The values in (3.34) correspond to Another solution, is the golden ratio. In this case They correspond to For instance for k 0 /p 0 = 1/2 we obtain c i = (0, 1, 0), for k 0 /p 0 = 1/3:

The case with 120 H coupling
The symmetry transformation of Y 120 is given by the elements g n φ of a discrete subgroup Z p of U (1) ⊂ SU (2) (2.21). Since g 2 φ = I for p > 2, the embedding symmetry group is not a Coxeter group, and so this analysis goes beyond the assumptions of [13]. If we assume that the element g φ from the Z p intrinsic symmetry of Y 120 and the element S d j from the Z 2 intrinsic symmetry of Y 10 (or Y 126 ) are residual symmetries, from the definition of a group this is true also for all g n φ , n = 1 . . . , p − 1. Therefore, the symmetry group relations now contain the products of U g n φ U †9 -any of the symmetry elements of Y 120 , and S d j which belongs to the symmetry of Y d 10 (or Y d 126 ): Eq. (3.43) can be rewritten as Tr W n jφ = a pn (k n , l n ), (3.44) where we will assume again that Det[W ] = 1, so that the sum of the eigenvalues equals a pn (k n , l n ) = e 2πi(kn/pn) + e 2πi(ln/pn) + e −2πi(kn/pn+ln/pn) .
Since now Tr [W j ] is complex, Eq. (3.44) provides two relations on the mixing parameters for each n. For the real part we get Re (a pn (k n , l n )) = −1 + 2 |U j1 | 2 ((1 − cos (2πn/p)) , (3.46) which depends on the absolute value |U j1 | with the column index 1 and the latter is related to the form of g( φ) (2.18) in which g 11 is isolated (decouples from the rest). Changing place of this element to 22 or 33 will fix another column. Also interchanging g and S j we can fix a row rather than a column. The imaginary part equals Im (a pn (k n , l n )) = 2 sin (2πn/p) 1 − |U j1 | 2 êφ (3.47) with the unit vectorsê andφ defined aŝ If |U j1 | = 1 the r.h.s. of (3.47) vanishes. There are thus 2 × (p − 1) equations (3.46) and (3.47) to solve, i.e. for all possible values of n = 1, . . . , p − 1. This is possible only if p n , k n , l n depend on n. Essentially |U j1 | can be found from Eq. (3.46), while Eq. (3.47) provides a constraint on the angleêφ. We will say more about possible solutions in section 4.2.
Notice that now the constraint on possible matrices U found in [22,26] is not valid, since in the case with 120 H , the matrix element |U | 2 is not related to k and p only, as in (3.22) or (3.25), but must satisfy more complicated equations (3.46)-(3.47).
Let us now give three examples involving the system with 120.
As a first example consider the case of p = 4. We thus have to find (p − 1) = 3 triples (n = 1, 2, 3) T n ≡ (k n , l n , −(k n + l n ) mod p n ) /p n (3.50) which satisfy the (p − 1) = 3 equations (3.46) and p − 1 = 3 equations (3.47), allowing a solution for |U j1 | andêφ. An example of possible solution is given by In fact it is easy to see explicitly that the ratios Re(a pn (k n , l n )) + 1 are, for triples (3.51), either undefined (0/0) or independent on n, giving |U j1 | = 0 and eφ = 1. Other solutions of (3.44) will be given in section 4.2.
In the second example consider three Yukawa couplings We assume that G f contains the following p + 1 symmetry elements from these Yukawas: As for g n φ (n = 1, . . . , p − 1) we should select a finite Abelian subgroup of the SU(2), Z p , to embed into discrete G f : Then the embedding of S 10 , S 126 and S n 120 into G f implies the symmetry group relations (3.63) on the product of the two: U 126−120 = U † 10−126 U 10−120 . More precisely, from the real part of (3.62) we obtain |(U 10−120 ) j1 | 2 , while the real part of (3.63) gives (3.64) Imaginary parts give constraints onφê 10−120 andφê 126−120 according to (3.47) with the definition (3.48).
In the third example we consider a system with two Yukawas, e.g. Y 10 and Y 120 . We can, similarly as in section 3.2, see how unitarity restricts possible solutions when two (and thus due to group relations all three) among S d j in (3.43) are residual symmetries. We thus have T r W 1 2π n p = a p n (k n , l n ), (3.65) T r W 2 2π n p = a p n (k n , l n ), (3.66) T r W 3 2π n p = a p n (k n , l n ). (3.67) What we have to do is (restricting the solutions to p, p , p , p ≤ 5) to find in Table 3 three solutions for the same p with the sum 3 j=1 |(U 10−120 ) j1 | 2 = 1. (3.68) Up to permutations of elements we get We can ask if just unitarity is enough to get these solutions, repeating the arguments of section 3.2. Summing the three equations (3.65)-(3.67), we find the relation a p n (k n , l n ) + a p n (k n , l n ) + a p n (k n , l n ) = −1 − 2 cos (2πn/p). (3.70) Although solving this equation (either by explicit numerical guess or using the techniques of [29]) is not problematic, one needs to combine n = 1, . . . , p − 1 such solutions. In other words, satisfying the equation for the sum (3.70) is necessary but, in general, not sufficient condition for solving the whole system (3.65)-(3.67).

Confronting relations with data
The possible values of |U ji | found in sect. 3 The non-zero neutrino mass comes from both type I and II contributions: where the left-handed, M ν L , and right-handed, M ν R , Majorana mass matrices are generated by non-vanishing (in the Pati-Salam decomposition) SU(2) R triplet VEV v R and SU(2) L triplet VEV v L : Relations (4.1) -(4.3) and the experimental values of the SM fermion masses and mixing allow to reconstruct (with some additional assumptions) the values of the Yukawa matrices Y 10 and Y 126 . Then diagonalizing these matrices as in (6.1) we can get the relative matrices, e.g. U 10−126 = U † 10 U 126 . The procedure of reconstruction of Y a from the data is by far not unique and a number of assumptions and further restrictions are needed to get Y a . Here we will describe few cases from the literature, where the unitary matrices U a are explicitly given. For other fits see for example [30]. 10 Here we assume supersymmetry; in the non-supersymmetric case, the Higgs 10-plet and 120-plet are in principle real. In this case v d 10,120 = v u 10,120 * , w d 120 = (w u 120 ) * .

The case of Y 10 + Y 126
Consider first check whether equality (3.22) is satisfied for one or more elements of the reconstructed relative matrix U 10−126 . A fit of the Yukawas has been done, for example, in [31], where the SUSY scale was assumed to be low. Let us start with the Yukawas displayed in eq. (18) of [31]. The corresponding matrix U (only absolute values of its elements are important) can be found easily: One should also take into account possible uncertainties in the determination of elements of (4.4), which we estimate as 10 − 20%. The element |(U 10−126 ) 22 | is numerically close to |cos (π/5)|. Furthermore, |(U 10−126 ) 23 | = 0.46 ≈ 0.5 = cos (π/3). The third element in the same row is |(U 10−126 ) 21 | = 0.36 ≈ 0.31 = cos (2π/5). This is one of the cases in which a full row of the relative matrix is determined by a residual symmetry, namely by the solution in (3.37). One can interpret this as an experimental evidence for the existence of G f . The second example comes from the Yukawa couplings shown in eq. (22) of [31]. They lead to the relative mixing matrix (4.5) The matrix element |(U 10−126 ) 23 | is numerically close to |cos (2π/5)|, however the other elements in the same row or column are not close to any value determined by symmetry.
With large probability this can be just accidental coincidence.
We reconstruct U 10−120 from the Table 2 p. 39 of [32]: (4.7) Confronting the first column in this matrix with predictions of the Table 3  (4.8) Again, coincidences with predictions of the Table 3 can be found.

RG invariance of the residual symmetry
Since we consider here the symmetry at the SO(10) level, the relative mixing matrix U b−a , determined by the residual symmetries, should be considered at GUT or even higher mass scales. One would expect that renormalization group equation running change the value of this unitary matrix. This, indeed, happens in most of the cases, for example when residual symmetries are applied to quarks or leptons in the standard model: the CKM or PMNS matrices run, so that the validity of the residual symmetry approach is bounded to an a-priori unknown scale.
In any supersymmetric SO(10) a residual symmetry inposed at the GUT scale will remain such also at any scale above it. Indeed, due to supersymmetry the renormalization is coming only through wave-functions. This means that up to wave-function renormalization of the 10 H and 126 H the Yukawa matrices Y 10 and Y 126 above the GUT scale renormalize in the same way: The different renormalization of (different) Higgses H a gives just an overall factors, and as such appears as a common multiplication the corresponding Yukawa matrices Y a , without change of the relative mixing matrix U b−a . This is different from other cases, where a residual symmetry is valid at a single scale only. Here if the symmetry exists at the SO(10) GUT scale, it is present also at any scale above it, thanks to the combined effect of supersymmetry and SO(10).

SO(10) model with hidden sector
Another class of SO(10) models includes the SO(10) fermionic singlets S which mix with the usual neutrinos via the Yukawa couplings with 16 H (see [33] and references therein). This avoids the introduction of high dimensional Higgs representations 126 H and 120 H to generate fermion masses. Neutrino masses are generated via the double seesaw [34] and this allows to disentangle generation of the quark mixing and lepton mixing, and therefore naturally explain their different patterns. The Lagrangian of the Yukawa sector is given by The terms in (5.1) have different intrinsic symmetries: 1. The first one has the Klein group symmetry G 10 = Z 2 × Z 2 , as the terms in (1.1). 2. The last term is also symmetric and has G 1 = Z 2 × Z 2 symmetry. 3. The second, "portal" term obeys a much wider intrinsic symmetry: U (1) × U (1) × U (1). In the diagonal basis it is related to independent continuous rotation of the three diagonalized states. This term can be considered as the Dirac term of charged leptons in previous studies of residual symmetries. To further proceed with the discrete symmetry approach we can select the discrete subgroup of the continuous symmetry, e.g. G 16 = Z m × Z n × Z l , or (to match with previous considerations in literature) even single subgroup G 16 = Z n , under which different components have different charges k = 0, 1, ...n − 1. So, the symmetry transformation, T , in the diagonal basis 16 F = T 16 F , S = T † S becomes: with k 1 + k 2 + k 3 = 0 mod n to keep Det(T ) = 1.
There are many possible embeddings of the residual symmetries G 10 , G 16 , G 1 which will lead to restriction on the relative mixing matrices between Y 10 , Y 16 , Y 1 . These matrices will determine eventually the lepton mixing (and more precisely its difference from the quark mixing). Recall that the difference may have special form like TBM or BM-type.
According to the double seesaw [34] the light neutrino mass matrix equals In terms of the diagonal matrices and relative rotations it can be rewritten as Then the embedding of G 10 and G 16 (or their subgroups) into a unique flavor group G f determines (restricts) the relative matrix U 10−16 . Embedding of G 16 and G 1 into G f determines U 16−1 . Further embedding of all residual symmetries will restrict both U 10−16 and U 16−1 . Let us mention one possibility. Selecting the parameters of embedding one can, e.g. obtain U 10−16 = I and U 16−1 = U T BM . Then imposing Y d 10 Y d−1

16
= I (which would require some additional symmetries [35] [33]) one finds that is, the TBM mixing of neutrinos. Detailed study of these possibilities is beyond the scope of this paper.

Intrinsic symmetries and relative mixing matrix
Let us further clarify the conceptual issues related to the intrinsic and residual symmetries. Intrinsic symmetries are the symmetries left after breaking of a bigger flavor symmetry. These symmetries exist before and after G f breaking. By itself these symmetries do not carry any new information about the flavor apart from that of symmetricity of antisymmetricity of the Yukawa matrices. So, by itself the intrinsic symmetries do not restrict the flavor structure.
These symmetries do not depend on the model parameters or on symmetry breaking. Recall that depending on the basis the form of symmetry transformation is different. So, changing the basis leads to the change of the form.
In a given basis symmetry transformations for different Y a can have different form, and it is this form of the transformation that encodes the flavor information. In other words, not the symmetry elements (generators) themselves, but their form in a given (and the same for all couplings) basis that encodes (restricts) the flavor structure. Changing basis for all couplings simultaneously does not change physics.
Breaking of the flavor symmetry fixes the form of the intrinsic symmetry transformations. In other words, G f breaking can not break the intrinsic symmetries but determine the form of symmetry transformations in a fixed (for all the couplings) basis.
In a sense, the intrinsic symmetries can be considered as a tool to introduce the flavor symmetries and study their consequences. Indeed, in the usual consideration symmetry determines the form of the Yukawa matrices in a certain basis. Changing the basis leads to a change of the form of Y a , but it does not change the relative mixing matrix between different Y a , which has a physical meaning. On the other hand the form of Y a determines the form of symmetry transformations. Therefore studying the form of transformations we obtain consequences of symmetry.
Let us show that the matrix which diagonalizes Y a determines the form of symmetry transformation. For definiteness we consider two symmetric matrices Y a and Y b , and take the basis where Y b is diagonal. The diagonalization of Y a in this basis is given by rotation U : (Recall that here U is the relative mixing matrix U b−a and we omit subscript for brevity). Let us show that U determines the form of the intrinsic symmetry transformation as where S d is the intrinsic symmetry transformation in the basis where Y a is diagonal: Using (6.2) and (6.1) we have where in the second equality we used the invariance (6.3). According to (6.4) S defined in (6.2) is indeed the symmetry transformation of Y a .
Let us comment on intrinsic and residual symmetries. Not all intrinsic symmetries can be taken as residual symmetry which originate from a given flavor symmetries. On the other hand, residual symmetries can be bigger than just intrinsic symmetries, i.e. include elements which are not intrinsic. The variety of residual transformations does not coincide with the variety of intrinsic symmetry transformations.
We can consider another class of symmetries under which also the Higgs bosons are charged. The symmetries are broken by these Higgs VEVs. In the case of a single Higgs multiplet of a given dimension, this does not produce flavor structure.
Let us comment on possible realization and implications of the residual symmetry approach. We can assume that three 16 F form a triplet of the covering group G f (A 4 can be taken as an example). If we assume that Higgs multiplets H a , a = 10, 126, 120, 16, are singlets of G f , then the product 16 T F Y a 16 F should originate from G f symmetric interactions. Apart from trivial case of Y a ∝ I (implied that 16 T F 16 F is invariant under G f ), Y a should be the effective coupling that appears after spontaneous symmetry breaking, so it is the function of the flavon fields φ, ξ, which transform non-trivially under G f : Y a = Y a (φ, ξ). In the A 4 example we may have, e.g., that Y 10 = h 10 y( φ), Y 126 = h 126 y(ξ), (6.5) where φ = (φ, φ , φ ) are flavons transforming as 1, 1 , 1 representations of A 4 and ξ transforms as a triplet of A 4 . The effective Yukawa couplings are generated when the flavons get VEV's. Then Y 10 will be diagonal, whereas Y 126 -off-diagonal.
To associate 10 H with certain flavons we need to introduce another symmetry in such a way that only φ10 H and ξ126 H are invariant. For instance, we can introduce a Z 4 symmetry under which φ, 10 H , ξ, 126 H transform with −1, − 1, i, − i, respectively.

Summary and Conclusion
We have explored an interplay of the vertical (gauge) symmetry and flavor symmetries in obtaining the fermion masses and mixing. In SO(10) the GUT Yukawa couplings have intrinsic flavor symmetries related to the SO(10) gauge structure. These symmetries are always present independently of the specific parameters of the model (couplings or masses). Different terms of the Yukawa Lagrangian have different intrinsic symmetries. Due to SO(10) the matrices of Yukawa couplings of 16 F with the 10 H and 126 H are symmetric and therefore have "built-in" G 10 = Z 2 × Z 2 and G 126 = Z 2 × Z 2 symmetries. We find that the matrix of Yukawa couplings of 120 H , being antisymmetric, has G 120 = SU (2) symmetry and some elements of the discrete subgroup of SU (2) can be used for further constructions. If also SO(10) fermionic singlets S exist, their self couplings are symmetric and therefore G 1 = Z 2 ×Z 2 . The couplings of S with 16 F have symmetries of the Dirac type G 16 = U (1) 3 , and the interesting subgroup is G 16 = Z n .
We assume that (part of) the intrinsic (built-in) symmetries are residual symmetries which are left out from the breaking of a bigger flavor symmetry group G f [13]. So G f is the covering group of the selected residual symmetry groups. This is an extension of the residual symmetry approach used in the past to explain lepton mixing. The main difference is that in the latter case the mass terms with different residual symmetries involve different fermionic fields: neutrino and charged leptons. Here the Yukawa interactions with different symmetries involve the same 16 F (but different Higgs representations). Higgses are uncharged with respect to the residual symmetries but should encode somehow information about the Yukawa couplings. In the presence of the fermionic singlets, also the fermionic operators can encode this information.
We show that the embedding of the residual symmetries leads to determination of the elements of the relative mixing matrix U a−b which connects the diagonal bases of the Yukawa matrices Y a and Y b . In our analysis we use the symmetry group condition which allows to determine the elements of the relative matrix immediately without the explicit construction of the Yukawa matrices and their diagonalization. We show the equivalence of our approach and the one in [13] in few explicit examples.
In the case of the minimal SO(10) with one 10 H and one 126 H the total intrinsic symmetry is G 10 × G 126 = (Z 2 × Z 2 ) 10 × (Z 2 × Z 2 ) 126 . In this case the covering group is the Coxeter group. If one Z 2 element of G 10 and one element of G 126 are taken, so that the residual symmetry is Z 2 × Z 2 , only one element of the relative mixing matrix U 10−126 is determined. The value of the element is given by the integers p, k of the embedding and therefore has a discrete ambiguity.
If one Z 2 element of G 10 (or G 126 ) and both elements of G 126 (or G 10 ) are taken as the residual symmetries, then two elements in a row (column) are determined. Furthermore, as a consequence of unitarity, the whole row (column) is determined. We show that the unitarity condition emerges from the group properties. Unitarity is not automatic and it imposes additional conditions on the parameters of the embedding, and therefore on possible values of the matrix elements.
If all Z 2 elements of G 10 and G 126 are taken as the residual symmetries, then 4 elements of U , and consequently, the whole matrix U is determined.
Using elements of G 120 opens up different possibilities. Taking the Abelian Z p subgroup of SU (2) the covering group is not a Coxeter group anymore for p > 2, and so not covered by [13]. Even if we start with one single element of g ∈ Z p (g p = I) being a residual symmetry, so must be g 2 , . . . , g p−1 . This follows simply from the definition of a group, it is not our choice or assumption. So each of the elements g n , n = 1, . . . , p − 1, must satisfy a group condition if also a Z 2 element of G 10 is a residual symmetry. In the case of residual symmetry with one Z 2 element of G 10 and the p − 1 elements of Z p a total of 2 × (p − 1) real equations for one element |(U 10−120 ) j1 | and one angleêφ (plus various integers) must be satisfied. Solutions can exist only because each complex equation can have a different choice of p n , k n , l n .
Using unitarity 3 × 2 × (p − 1) relations on elements of U 10−120 (plus some integers) appear if the whole G 10 and p−1 elements from G 120 = Z p are taken as residual symmetries.
If one Z 2 from G 10 , another Z 2 from G 126 and p − 1 elements Z p from G 120 are identified as the residual symmetries, we obtain relations between the elements of both U 10−126 and U 10−120 .
We confronted the obtained values of elements of the relative mixing matrices with available results of data fits. We find that in the case of G 10 and G 126 embedding the predictions for one and two elements are compatible with some fits. Also for G 10 and G 120 embeddings some predictions for elements of U 10−120 exactly or approximately coincide with data. These values as well as residual symmetries in general are renormalization group independent in supersymmetric SO (10).
The fits to data are not unique and typically several local minima with low enough χ 2 exist. This is one of the reasons why we cannot conclude yet that SO(10) data point toward residual symmetries, and more work should be done. The other reason is the unavoidable possibility that a coincidence between data and the theoretical expectation could be simply accidental.
A Comparison with the approach in [13] The invariance of the symmetric Yukawa matrix Y is expressed as The intrinsic symmetry can be easily realized in the basis where the Yukawa matrix Y is diagonal. A diagonal matrix Y d = diag(y 1 , y 2 , y 3 ) with arbitrary (non-degenerate) elements y i is invariant with respect to transformations For a symmetric matrix the invariance is defined as The elements, being reflections, satisfy (S d i ) 2 = I. There are 2 3 = 8 different transformations in (A.2), including the identity matrix. So, the maximal intrinsic symmetry group is Z 3 2 , since transformations with s j = −s i having opposite signs of determinants, do not produce additional restrictions on m. If we take elements with Det(S i ) = 1, only 4 elements are left which correspond to the Z 2 × Z 2 group.
In general, different Yukawa matrices can not be diagonalized simultaneously. Therefore, in a given basis, their symmetry elements can be obtained performing the unitary transformation: where U i connects a given basis with the diagonal basis for S i . Using (S i ) 2 = I it is easy to show that for two different elements (S i S j ) n = (S j S i ) n with n ≥ 2. The group formed by the reflection elements S i is called the Coxeter group 12 .
In [13] it is suggested that different terms of the SO(10) Yukawa Lagrangian, and consequently different mass matrices generated by these terms, are invariant under different elements S i . Furthermore S i are identified with the residual symmetry left over from the breaking of the Coxeter group. Invariance of the Yukawa matrices leads to restriction of their elements.
Let us show that the approach in this paper is equivalent to that in [13]. Consider two Yukawa matrices (or "fundamental" mass matrices as in [13]) Y a and Y b invariant with respect to S a and S b . Then the elements S a and S b being residual symmetry elements satisfy the relation (S a S b ) p = I. Expressing S a and S b in terms of diagonal elements (A.4) we This coincides with the symmetry group condition (3.2). U a−b connects two basis in which Y a , Y b are diagonal, that is, the relative mixing matrix. This matrix does not depend on the basis and has a physical meaning.
In our approach we use immediately the symmetry group condition to get bounds on U a−b , whereas in [13] the symmetries S a and S b were used to obtain bounds on the corresponding mass matrices. Diagonalization of these restricted matrices and then finding the relative mixing should lead to the same result.
Let us illustrate this using two examples. We will consider the Coxeter group A 3 . It has three generators and the group structure is (S 1 S 3 ) 2 = I, (S 1 S 2 ) 3 = I, (S 3 S 2 ) 3 = I. (A.5) In the first example we take Y 10 to be invariant with respect to S 10 = S 1 and Y 126 with respect to S 126 = S 3 . From the first group relation in (A.5) it follows that S 1 and S 3 commute. Therefore the basis can be found in which both S 1 and S 3 are diagonal simultaneously. We can take S 10 = S d 1 and S 126 = S d 3 , where S d 1 and S d 3 are given in (2.9). Let us underline that in this example it is the commutation of S 1 and S 3 (which is a consequence of the group structure relation) that encodes the information about embedding.
As the consequence of symmetries, the matrices should have the following vanishing elements This result can be obtained immediately from our consideration (3.22). Indeed, in this case the generators S 1 and S 3 are involved, so we fix the element U 13 . In this example p = 2 and k = 1 that lead according to (3.22) to U 13 = cos (π/2) = 0.
In the second example we take again S 10 = S 1 as the symmetry of Y 10 but S 126 = S 2 as the symmetry of Y 126 . Now p = 3 (A.5) and the generators do not commute, so they can not be diagonalized simultaneously. In the basis S 10 = S d 1 according to [13] the third element equals This element can be represented as where S d 2 = diag(−1, 1, −1) and, as can be obtained explicitly from (A.10) and (A.11), in U only the second column is determined: |U j2 | = (1/2, 1/ √ 2, 1/2) T . The matrix U is nothing but the relative matrix which connects two diagonal bases for S i . In particular, we have |U 12 | = 1/2. Again this result can be obtained immediately from our consideration. Since the generators involved are S 1 and S 2 , the 1 -2 element is fixed. For p = 3 and k = 1 (or k = 2) we have from (3.22) |U 12 | = cos (π/3) = 1/2.
Notice that in the matrix U only one column is determined, and so there is an ambiguity related with certain rotations. Also in the first example we could write the symmetry group condition as (S d 1 S d 3 ) 2 = I, that is, U = I which is consistent with U 13 = 0. Again here we have an ambiguity related to rotations (A.8).