Rare Higgs three body decay induced by top-Higgs FCNC coupling in the littlest Higgs Model with T-parity

Motivated by the search for flavor-changing neutral current (FCNC) top quark decays at the LHC, we calculate rare Higgs three body decay $H\rightarrow Wbc$ induced by top-Higgs FCNC coupling in the littlest Higgs model with T-parity(LHT). We find that the branching ratios of $H\rightarrow Wbc$ in the LHT model can reach $\mathcal O(10^{-7})$ in the allowed parameter space.


INTRODUCTION
The discovery of a Higgs-like resonance near 125 GeV [1] at the LHC is a great triumph for theoretical and experimental particle physics. So far, most measurements of this new particle are consistent with the Standard Model (SM) prediction, but the experimental investigation of this new particle has only just begun. It is not impossible that more in-depth studies will reveal its non-SM properties.
Compared with the normal decay modes, the flavour-changing neutral current (FCNC) decays are highly suppressed in the SM due to the Glashow-Iliopoulos-Maiani (GIM) mechanism [2]. So, any large enhancements in these branching ratios will be smoking-gun signals beyond the SM.
As the heaviest known elementary particle, the top quark is widely speculated to be sensitive to the electroweak symmetry breaking (EWSB) mechanism and new physics at TeV-scale. An interesting possibility is the presence of FCNC interactions between the Higgs boson and the top quark. This interaction not only participate in the top quark FCNC decays [3], but also participate in the Higgs FCNC decays [4].
Except for the dominant decay mode H → bb, the so called below-threshold decay modes induced by the HV V (V = W ; Z) couplings are also very important, where the decay H → V V with one (or two) V's being off-shell and decaying to fermions. In some new physics, the decay mode of Higgs bosons is much richer and 3-body decays may be even more important. Now, almost all Higgs boson decay modes have been measured at the LHC, but they are plagued by large SM backgrounds. So, the rare Higgs 3-body decays may bring us more surprises. In some new physics models, the GIM suppression can be relaxed and/or new particles can contribute to the loops, so that the top-Higgs FCNC couplings tqH, especially the tcH coupling, can be enhanced by orders of magnitude larger than those of the SM [5].
In this paper, we study the rare Higgs 3-body decay H → W bc induced by top-Higgs FCNC coupling in the littlest Higgs Model with T-parity(LHT). This decay includes the FCNC vertex tcH, which receives the contribution from the new T-odd gauge bosons and T-odd fermions. The results of this process will help to test the SM and probe the LHT model.
The paper is organized as follows. In Sec.II we give a brief review of the LHT model related to our work. In Secs.III we calculate the rare Higgs 3-body decay H → W bc induced by top-Higgs FCNC coupling in unitary gauge under current constraints. Finally, we draw our conclusions in Sec.IV.

II. A BRIEF REVIEW OF THE LHT MODEL
The LHT model is based on an SU(5)/SO (5) non-linear σ model [6]. At the scale f ∼ O (TeV), the SU (5) global symmetry is broken down to SO(5) by the vacuum expectation value (VEV) of the σ field, Σ 0 , given by After the global symmetry is broken, there arise 14 Goldstone bosons(GB) which are described by the "pion" matrix Π. Then the kinetic term for the GB matrix can be expressed in the standard non-linear sigma model formalism as The σ field kinetic Lagrangian is given by with the [SU(2) ⊗ U(1)] 2 covariant derivative defined by where W µ j = 3 a=1 W µ a j Q a j and B µ j = B µ j Y j are the heavy SU(2) and U(1) gauge bosons, with Q a j and Y j the gauge generators, g j and g ′ j are the respective gauge couplings. The VEV Σ 0 also breaks the gauged subgroup [SU (2) where g and g ′ denote the SM SU(2) and U(1) gauge couplings, respectively. v represents the VEV of the Higgs doublet, which is related to the SM Higgs VEV v SM = 246GeV through the following formula: In the quark sector, the T-odd mirror partners for each SM quark are added to preserve the T-parity. The up and down-type mirror quarks can be denoted by u i H and d i H , where i(= 1, 2, 3) is the generation index. One can write down a Yukawa interaction to give masses to the mirror quarks After the EWSB, their masses up to O(v 2 /f 2 ) are given by where κ i are the eigenvalues of the mass matrix κ.
Under T -parity, in order to cancel the large radiative correction to Higgs mass parameter induced by top quark, an additional T-even heavy quark T + and its T-odd mirror partner T − are introduced. Their masses are given by where x L is the mixing parameter between the top-quark and heavy quark T + . This mixing parameter can also be expressed by a ratio R = λ 1 /λ 2 with where λ 1 and λ 2 are two dimensionless top quark Yukawa couplings.
When the mass matrix √ 2κ ij f is diagonalized by two U(3) matrices, a new flavor structure can come from the mirror fermions. In the mirror quark sector, the existence of two CKM-like unitary mixing matrices V Hu and V Hd is one of the important ingredients.
It's worth noting that V Hu and V Hd are related through the SM CKM matrix: Follow Ref. [7], the matrix V Hd can be parameterized with three angles θ d 12 , θ d 23 , θ d 13 and three phases δ d 12 , δ d 23 , δ d For the down-type quarks and charged leptons, there are two possible ways to construct the Yukawa interaction, which are denoted as Case A and Case B [8]. At order O (v 4 SM /f 4 ), the corresponding corrections to the Higgs couplings are given by (d ≡ d, s, b, l The Feynman diagrams of the tree level H → W + bc and the rare decay H → W + bc are shown respectively in Fig.1 and Fig.2, which includes the W + and W − modes. The rare Higgs decay H → W bc is mediated by the same Yukawa coupling that leads to the t → cH decay [9], so we show the Feynman diagrams of the LHT one-loop correction to vertex V tcH in unitary gauge in Fig.3, where the Goldstone bosons do not appear. We can see that the flavor changing interactions between SM quarks and mirror quarks are mediated by the heavy gauge bosons W ± H , Z H , A H . We find that dominant contribution to the branching ratio of the decay H → W bc is from the interference between the Fig.1 and Fig.2. Each loop diagram is composed of some scalar loop functions [10], which are calculated by using LOOPTOOLS [11].
In our numerical calculations, we take the SM parameters as follows [12] G F = 1.16637 × 10 −5 GeV −2 , sin 2 θ W = 0.231, α e = 1/128, m H = 125GeV, The LHT parameters related to our calculations are the scale f , the mixing parameter For the Yukawa couplings, the search for the mono-jet events at the LHC Run-1 [14] give the constraint κ i ≥ 0.6. Considering the constraints in Ref. [13], we scan over the free parameters f , κ 12 and κ 3 within the following region For the parameters in the matrices V Hu , V Hd , we follow Ref. [15] to consider the two scenarios as follows • Scenario I: V Hd = I,V Hu = V † CKM ; • Scenario II: Furthermore, we will consider the constraint from the global fit of the current Higgs data and the electroweak precision observables (EWPOs) [16]. In Fig.4, we present the  where the scale f is small and the Yukawa coupling κ 3 is either too small or too large.
According to the Ref. [15], the branching ratio of t → cH is enhanced by the mass splitting between the three generation mirror quarks, the same thing will happen to the branching ratios of H → W bc. In order to see this dependence, we show the branching ratios of H → W bc in the | M 3 − M 12 |∼ f plane for two scenarios in Fig.6, respectively.
We can see that the small branching ratios correspond to the region that has small mass splitting | M 3 − M 12 | values. The largest branching ratios lie in the upper-left corners of  For the observability, the SM decay H → W W * → W bc is an important irreducible background that will generate the same final state. Due to the off-shell top in the signal decay H → t * c → W bc, we can use the invariant masses cut |M W b − m t | > 20 GeV to isolate the signal. Besides, the c-jet in our signal comes from the Higgs decay, which usually is harder than that in the SM background H → W W * → W bc. Thus, we can use the high transverse momentum p c T cut to suppress the background. Due to the same Yukawa couplings that lead to the t → cH decays, the decays H →

IV. CONCLUSIONS
In this paper, we calculated rare Higgs three body decay H → W bc induced by top-Higgs FCNC coupling in the LHT model. According to the parameters in the mixing matrices, we considered two scenarios and found that the branching ratio for H → W bc can reach O(10 −7 ) in the allowed parameter space.