Lepton masses and mixing in a scotogenic model

We consider an extension of the standard model with three Higgs doublet model and S3 × Z2 discrete symmetries. Two of the scalar doublets are inert due to the Z2 symmetry. We have calculated all the mass spectra in the scalar and lepton sectors and accommodated the leptonic mixing matrix as well. We also show that the model has scalar and pseudoscalar candidates to dark matter. Constraints on the parameters of the model coming from the decay μ → eγ were considered and we found signals between the current and the upcoming experimental limits, and from that decay we can predict the one-loop μ → eeē channel.

components, and also co-annihilate in two dark particles [6]. In fact, this might be the only possibility to accommodate several astrophysical observations. For instance, the electron/positron excesses observed by many experiments and recently confirmed by AMS-02 [7] and the excess of gamma rays peaking at energies of several GeV from the region surrounding the Galactic Center [8,9] must need at least two component DM. Moreover, the latter case can avoid some constraints on the one component DM from the AMS-02 data [10]. Notwithstanding there are alternative interpretations for these gamma rays excess, see [11,12]. It is interesting that the one-doublet inert model has at least two DM components. However see [13].
Here we will work out a similar mechanism but in the context of the model with two inert scalar doublets proposed in Ref. [14]. Moreover, the inert character is due to the Z 2 symmetry, and the S 3 symmetry makes the scalar potential more predictive and easier to be analyzed. Although three right-handed neutrinos are introduced, the active neutrino masses do not arise through the type-I seesaw mechanism but via the scotogenic mechanism, at the 1-loop level. We need to add two real singlet scalar fields in order to accommodate the charged lepton masses and the Pontecorvo-Maki-Nakagawa-Sakata (PMNS) mixing matrix. In the context of inert doublet models, the latter issue is to the best of our knowledge done for the first time.
The outline of this paper is as follows. In the next section we discuss the model while in Sec. III analyse the scalar sector. Lepton mass matrices and the leptonic mixing matrix is shown in Sec. IV. In Sec. V we show that the model provides a multi-component DM spectra, although we do not consider the most general case. In Sec. VI we consider the decays µ → eγ, Subsec. VI A, and µ → eēe in Subsec. VI B. Our conclusions appear in Sec. VII.

II. THE MODEL
In Ref. [14] it was proposed an extension of the electroweak SM with three Higgs scalars H 1,2,3 transforming as doublets under SU (2) and having Y = +1. One H 1 ≡ S transforms as singlet of S 3 , and the others as doublets, D = (H 2 , H 3 ) ≡ (D 1 , D 2 ). Here we will extend the model of Ref. [14] by adding three right-handed sterile neutrinos, N 1R transforming as singlet, and N d = (N 2R , N 3R ) transforming as doublets of S 3 , and two real scalar singlets of SU (2) (Y = 0) but doublets of S 3 , ζ d = (ζ 1 , ζ 2 ). See the other quantum numbers in Table I. The vacuum alignment is given by S = v SM / √ 2, and D 1 , D 2 = 0, ζ 1,2 = v ζ . In the charged lepton and quark sectors, all usual fields of SM transform as singlet under S 3 .
With these fields the Yukawa interactions in the lepton sector, invariant under the gauge, S 3 and Z 2 symmetries (see Table I) are given by where i, j = e, µ, τ (we omit summation symbols), L i (l iR ) and N s,d denote the usual left-handed lepton doublets (righthanded singlets) and the right-handed neutrinos, respectively; according to the S 3 multiplication rules, andD 1,2 = iτ 2 D * 1,2 . Notice that the doublets D and ζ d couple only with neutrinos. We assume that ζ 1,2 Λ, where Λ is an energy scale much larger than the electroweak one. It is also interesting to note that the right-handed neutrinos in the S 3 doublet are mass degenerated, with mass M d , which is different from the mass of the right-handed neutrino in singlet of S 3 , which has a mass denoted by M s . Notice that at three level active neutrinos are still massless.

III. THE SCALAR SECTOR
The scalar sector of the model is presented as follows: plus the singlets The scalar potential invariant under the gauge and S 3 ⊗ Z 2 symmetries is with µ 2 d > 0 that is guaranteed by the Z 2 symmetry. We can write Eq. (3) explicitly as where where we have used [ζ d ζ d ] 2 ′ = (ζ 1 ζ 2 + ζ 2 ζ 1 , ζ 1 ζ 1 − ζ 2 ζ 2 ). Notice that the term µ 2 12 breaks softly the S 3 symmetry but not the Z 2 . Notice also that the Z 2 symmetry forbids trilinear terms in the scalar potential [D † ⊗ D] 1 ζ i and ζ 3 i . From derivation of Eq. (5), we obtain the constraint equations: where we have defined c = c 1 + c 2 . Notice from (6) that neither v 1 = 0 nor v 2 = 0 are allowed if µ 2 12 = 0, hence we have that v 1 = 0, v 2 = 0, or v 1 = v 2 ≡ v ζ . We have chosen the latter case, so the constraint equations become The scalar potential has to be bounded from below to ensure its stability. In the SM it is easy to ensure the stability of this potential, we just have to ensure that λ > 0. In theories in which the number of scalars is increased, it is more difficult to ensure that the potential is bounded from below, in all directions. A scalar potential has a quadratic form in the quadratic couplings, i.e. A ab φ 2 a φ 2 b , where φ 2 a and φ 2 b represents the scalar fields, S, D and ζ d . If the matrix A ab is copositive it is possible to ensure that the potential has a global minimum. Assuming a quadratic form (e.g. considering only the quartic terms of the potential) is valid, even if there exist trilinear terms, because in the case where the fields assume large values, the terms of order 2 and 3 are negligible compared to the terms of order 4. For more detail see Refs. [23,24]. We consider all quartic couplings positive, i.e. all A ij are positive. Below we will denote B = a 1 − 2a 2 − a 3 and C = a 5 − 2a 6 + a 7 . And finally the following limits guarantee that the scalar potential is bounded the from below: Next, we consider the scalar mass spectra. In the CP -even sector the mass matrix becomes in block diagonal form with one 3 × 3, M 1R sub-matrix and one 2 × 2 matrix, M 2R . The first one in the basis (ReS 0 , ζ 1 , ζ 2 ) is given by The respective eigenvalues are From m 2 1 we see that µ 2 12 < 0, hence m 2 1 may have large mass. The SM-like Higgs boson may be identified with the scalar with mass m 3 in Eq. (10). In order to see this just make v 2 ζ = 0. The second mass matrix M 2R in the basis (ReD 0 1 , ReD 0 2 ) reads with the eigenvalues The mass matrix in the CP odd sector has the form in the basis (ImD 0 1 , ImD 0 2 ) (the would-be Goldstone bosons has been already decoupled) with eigenvalues Above we have defined a ′ = a 5 + a 7 + 2a 6 and a ′′ = a 5 + a 7 − 2a 6 . The model allows four neutral scalars with different masses that could contribute to the DM relic density in different proportion: two CP even and two CP odd. Notice also that a 6 is the term in the scalar potential that transfer the L violation to the active neutrino sector.
In the charged scalars sector, besides the charged would-be Goldstone boson, we have two charged scalar fields (we have already omitted the charged would-be Goldstone boson) with the non-zero eigenvalues given by Notice that Notice that µ 2 d > 0 does not disappear in the mass difference above because, in oder to reproduce the Klein-Gordon equation for each component, a real scalar has a 1/2 factor in the mass term related to the mass of a complex scalar.

IV. LEPTON MASSES AND THE PMNS MATRIX
In Sec. II we have seen that at tree level the neutrinos are massless, the a 6 term in Eq. (5) induce diagrams like those in Fig. 1 and it is possible to implement the mechanism of Ref. [4] for radiative generation of neutrinos mass. In fact, the diagram in Fig. 1 are exactly calculable from the exchange of ReD 0 1,2 and ImD 0 where a = 1, 2; k = s, d; m Ra and m Ia are the masses of ReD 0 1,2 and ImD 0 1,2 , respectively. In Eq. (18) Y ik Y jk corresponds to G ν is G ν js when the coupling is with the N s ; and Y ik Y jk corresponds to G ν id G ν jd when the coupling is with the N d , and finally M s is the mass of the right-handed neutrino N s , and M d is the common mass of the righthanded neutrino in the doublet of where M s is the mass of the right-handed neutrino N s and M d is the common mass of the neutrinos N d . Under the condition in which the scalars are mass degenerated i.e., b 3 = 0 in (17) we obtain just a factor 2 in Eq. (19). Below, for simplicity, we will consider the case b 3 = 0. In order to obtain the active neutrinos masses we assume a normal hierarchy and, without loss of generality, that M s ∼ M d and will be represented from now on by 2 23 ). Taken the central values in PDG we havê M ν ≈ (0, x, y).
In the charged lepton sector we assume their masses at the central values in PDGM l = (0.510, 105.658, 1776.86) GeV. It is important to note from these considerations, that there exist a multitude of other possibilities which satisfy also the masses squared differences and the astrophysical limits in the active neutrino sector. Each one corresponds to different parameterization of the unitary matrices V l L,R , V ν L . We will obtain the neutrinos masses from Eq. (19). We have as free parameters a 6 , M R and the Yukawas [m +1 = m +2 ≡ m + are also still free but they will enter only in the leptonic decays considered in Sec. VI]. In the Fig. 2 we show the dependence of a 6 with respect to the main Yukawas G ν τ d in (a) and G ν es in (b) for fixed M R values. Notice that G ν τ d are essentially of the same order of magnitude, while the rest G ν ed,µd,µs,τ s are suppressed by four orders of magnitude when comparing with any specific value of those G ν τ d,es . L V ν L , it means that this matrix appears in the charged currents coupled to W + µ . We have tested the robustness of our fitting of the lepton masses and the leptonic mixing matrix by using several parametrization corresponding to the values of the Yukawa couplings given in Table II. We omit the respective matrices V l L and V ν L but in all cases we have obtained: which is in agreement within the experimental error data at 3σ given by [15] |V and we see that it is possible to accommodate all lepton masses and the PMNS matrix. Here we do not consider CP violation.

V. DARK MATTER
As we said before, the present model may have a multi-component DM spectrum, which means that many particles may contribute to the relic density of DM, but we will consider the simplest example where one of the CP even scalar, say R 1 , and one of the CP odd scalar, say I 1 , as the dark matter candidates, each case is considered separately for simplicity. A two inert doublet model without right-handed neutrinos and scalar singlet was considered in Ref. [16].
As usual, in order to determine the relic density, we solve the Boltzmann equation. Firstly, considering R as the candidate, we have where σ|v| is the annihilation cross section already thermally averaged and H is the Hubble constant. In the thermal equilibrium, the number density of DM [17] is where g = 1 for a scalar DM. When solving the Boltzmann equation we obtain the equation for the relic density: where M P l = 1.22 × 10 19 GeV is the Planck mass, x F = m R /T F , where m R is the mass of the neutral scalar and T F is the temperature at freeze-out, the terms a and b result from the partial wave expansion of σ|v| = a + bv 2 . The number of relativistic degrees of freedom g * = 118.375 is a result of the SM particles plus three right-handed neutrinos, five neutral scalars, two pseudo-scalars and two charged scalars. The evaluation of x F leads to where the unitary parameter c ≈ 5/4. Here we will consider the solution for the relic density which, at the same time, solves the charged lepton masses and neutrino masses given in Eq. (18) for the sets of parameters showed in Tables III and IV, in order to obtain the PMNS matrix. We call them scenario 1 and 2, when R 1 and I 1 is the DM candidate, respectively. In both scenarios the Yukawas values adjust the squared masses differences for the neutrinos and the PMNS.
With the numbers in Tables III and IV we obtain the mixing matrix, that is: As previously defined the mixing matrix for the leptonic sector V P MN S = V l † L V ν L , From Eqs. (26) and (27) we obtain again Eq. (20).
To perform DM calculation we have used MicrOmegas package [18]. For instance, let us consider scenario 1, where R 1 is the DM candidate. In the range of parameters used by us, DM annihilates mainly in W + W − . Once again we emphasize that other solutions in other annihilation channels do exist. We have chosen the following parameters for the couplings, and vacuum expected value: 2 , for values of other parameters see Table III. With this parameters choice m R1 = 85.15 GeV. The parameter dependence of m R1 is presented in the last sections. So, the dominant contributions for Ω are 99% in R 1 R 1 → W + W − . In this case, x F ∼ 23.8. The annihilation cross section is σv = 1.0 × 10 −26 cm 3 /s and the DM-nucleus cross section for spin-independent elastic scattering is numerically given by σ p SI = 5.84 × 10 −46 cm 2 and σ n SI = 6.70 × 10 −46 cm 2 .
In scenario 2 we consider I 1 as the DM candidate. In this case, as a result of the parameter choice (G τ d = 3.75×10 −7 , G τ s = 1 × 10 −11 , v ζ Λ = 1000 GeV, λ ′ = 1, λ ′′ = 0.12, µ 2 d = 2830.24 TeV 2 ), I 1 annihilates 97% in I 1 I 1 → hh and 2% in I 1 I 1 → bb. The value of other parameters can be seen in Table III. The annihilation cross section is σv = 4.55 × 10 −28 cm 3 /s and the DM-nucleus cross section for spin-independent elastic scattering is numerically given by σ p SI = 6.40 × 10 −45 cm 2 and σ n SI = 7.35 × 10 −45 cm 2 . In these two scenarios, we had set R 1 and I 1 as DM candidates, making them lighter than the others possible neutral scalars. We emphasize that other choices for DM are possible so that other annihilation channels may also give interesting signatures. There is also the possibility that two, three or even four of the neutral scalars contribute partially to the DM density, but this case is beyond the scope of this paper.
In Fig. 3   In this section we study the impact of the new particles, the charged scalars D + 1,2 and the right-handed neutrinos N s,2,3 in the lepton flavor violating processes l i → l j γ. Here we will consider these rare decays in two cases: one in which we do not care with DM solutions and one in which we use the parameters for having a DM candidates that also give the correct lepton masses and the PMNS.
In terms of the leptons mass eigenstates, the interactions with charged scalars from Eq. (1) are written as i, k = e, µ, τ , the values of the entries of the matrices G ν i,d and G ν i,s are given in Table III. In the model, the allowed lepton flavor violation (LFV) decays l i → l j γ and l i → l j l klk arise only at the 1-loop level. These diagrams are generated by the known SM contribution W & ν l and by the new content D + 1 &N 2 , D + 2 &N 3 , D + 1 &N s and D + 2 &N s . For l i → l j γ, it is known that the SM contribution is extraordinarily suppressed with respect to the experimental capabilities of detection, see Table V. As we will show below, the new particle content in the model predicts signals close to the experimental upper limits for the space of our considered allowed parameters. Regarding the three body decay l i → l j l klk , it arises when in l i → l j γ we attach to the photon the γll coupling. In the following we are interested in presenting the µ → eeē channel, because it provides interesting results near the experimental upper limit, while all the other channels are out of the experimental interest region because the devices are unable of reaching such suppressed signals.
In our study we have solved the amplitudes and the loop integrals with the help of Mathematica, FeynCalc [19,20], and Package-X [21].
A. Predictions of µ → eγ and µ → eeē in the scotogenic model without dark matter.
We start our numerical analysis of the decays in the scotogenic model without DM content. Accordingly to the Yukawa values derived in the Sec. IV (see Fig. 2), the obtained values of the Yukawas are G ν ad,as ∈ [10 −11 , 10 −1 ], they satisfy the neutrino masses. Notice that G ν τ d ≃ G ν es ≫ G ν ed,µd,µs,τ s . We recall that all the previous analyse were done in the case of b 3 = 0 in which the charged scalars are mass degenerated. As starting point we consider the mass of the charged scalar in the range m + ∈ [80,750] GeV, N 2,3,s degenerated as well with values m N ∈ [250, 4000] GeV. We have tested the five different parameterizations of the V l L matrices derived in the Sec. IV (see Table II). They can be separated into two sets which will have two different behaviours in the processes, the set A is conformed by the parameterizations P1 and P5, and the B by P2, P3 and P4.
For the channel µ → eγ such situation occurs when G ν τ d = G ν es ∼ 10 −1 and G ν ed,µd,µs,τ s ∼ 10 −5 . In the Fig. 5 (a)-(b) it is shown the Br(µ → eγ) as function of the sterile neutrino mass m N ∈ [250, 4000] GeV with given values for the charged scalar m D + = 80, 250, 500, 750 GeV. The current experimental upper limit Br(µ → eγ) Exp < 4.2 × 10 −13 , indicated with the red line in the plots, will constrain the right-handed neutrino mass m N for given values of m + in order to respect such limit, those constraints are listed in the Table VI, where it is evident that the V l L parameterization A in Fig. 5 (a) allows a lighter mass for the right-handed neutrino than the parameterization B in Fig. 5 (b).   Regarding to the subcase µ → eeē, we are able to predict the branching ratio from the neutrino right-handed mass constraints obtained for the µ → eγ channel. These predictions are organized in the Table VII, and such values are indicated with the green line in the Fig. 5 (c) and (d), being of the same order of magnitude ∼ 10 −15 for both scenarios. About the analogous tau decays, for the same space of parameter values than in the µ → eγ case, and respecting the obtained mass bounds, we have found that Br(τ → eγ) ≤ 10 −12 and Br(τ → eγ) ≤ 10 −14 , which are beyond the current and upcoming experimental capabilities of detection, see Table V for comparison.
B. Predictions of li → ljγ in the scotogenic model with dark matter As commented in the Sec. V, the model can be extended to include DM. In order to estimate the consequences on the transition µ → eγ, we consider the Yukawa values given in Table III. The resulting prediction with G ν τ d = G ν es = 10 −7 is which is beyond the scope of detection.

VII. CONCLUSIONS
Here we have considered an extension of the SM with three scalar doublets of SU (2) with S 3 and Z 2 symmetries. We had analysed all the mass spectra in the scalar sectors and used the scotogenic mechanism for generating neutrino masses. Moreover, we had obtained the PMNS matrix once the unitary matrices which diagonalize the lepton masses are obtained. Although the model can have many DM candidates, we have shown two cases in which the DM candidate is a CP even scalar (scenario-1) and other one in which the DM is composite of CP odd scalar (scenario-2). But we emphasize that other possible choices for DM candidates are possible, considering for example, smaller masses, since besides the SM-like scalar, we have eight additional neutral scalars in the model. The study of other candidates and other channels of annihilation will be done soon. We had exemplified in some range of parameters space, two DM candidates for the model. For the scenarios 1 and 2 presented, DM annihilates mainly in W + W − and hh respectively. We have also presented some fluxes for this model. Of course, there may be other possible scenarios which could explain the Galactic gamma ray excess, as well as the the PAMELA and AMS-02 results. These processes could tightly constrain the parameter space of this sort of scotogenic models. The considered scotogenic model without DM provides optimistic predictions for possible detection of the LFV decay µ → eγ due to our solution space of the Yukawa values, which adjusts the squared masses differences for the neutrinos and the PMNS matrix. Our estimations predict a mass for the right-handed neutrino starting from m N > 295 GeV, and from µ → eγ we predict Br(µ → eeē) 10 −15 . On the other hand, considering DM content in the model we found that Br(µ → eγ) ∼ 10 −34 , which is out of detection range.