Searches for the FCNC couplings from top-Higgs associated production signal with $h\to \gamma\gamma$ at the LHC

In this paper, we study the observability of the top-Higgs flavor changing neutral current (FCNC) $tqh$ coupling through the process $pp\to qg\to t(\to \ell^{+}b\nu)h(\to \gamma\gamma)$ at the Large Hadron Collider~(LHC), where $\ell=e, \mu$. Our numerical results show that, in some parameter regions, the LHC may observe the above signals at the $5\sigma$ level. Otherwise, the branching ratios $Br(t\to uh)$ and $Br(t\to ch)$ can be respectively probed to $0.036\%$ and $0.13\%$ at $3\sigma$ level at 14 TeV LHC with the high integrated luminosity of 3000 fb$^{-1}$. On the other hand, studying the charge ratio for the lepton in top quark decay can be not only used to discriminate between signal and backgrounds, but also used to discriminate between $tuh$ and $tch$ couplings, for which anomalous single top production comes from the up initiated channel and charm initiated channel.


I. INTRODUCTION
The discovery of a Higgs boson with a mass about 125 GeV is the undisputed highlight of Run-I of the CERN's Large Hadron Collider (LHC) [1,2]. So far the measured couplings of the Higgs boson with fermions and gauge bosons are found to be in agreement with the predictions of the Standard Model (SM) [3,4]. A major target of the future LHC programme is to study the intrinsic properties of the discovered Higgs boson. In view of the large top quark mass and the large number of top quarks produced at the LHC, it is attractive to investigate the anomalous flavor-changing neutral current (FCNC) top-Higgs couplings [5][6][7][8][9][10][11][12][13].
In the SM, the FCNC couplings in the top sector are strongly suppressed due to the Glashow-Iliopoulos-Maiani (GIM) mechanism [14], which can only occur at loop-level with the expected branching ratios of order about 10 −15 − 10 −12 [15,16]. However, in many new physics (NP) models beyond the SM, such as the Minimal Supersymmetric Standard Model (MSSM) [17][18][19][20][21], Two-Higgs-Doublet Models (THDM) [22][23][24][25][26], Extra Dimensions (ED) [27], Little Higgs Models (LHT) [28], and the other miscellaneous models [29][30][31][32][33], some FCNC processes involving the top quark can be greatly enhanced by extending the flavor structures, which makes them potentially accessible at current and future high-energy colliders. Thus, any signal for top quark FCNC process at a measurable rate would be a robust evidence for NP. Since we do not know which type of NP models will be responsible for the possible deviation, it is better to study these processes with a model-independent method. So far, there are already many studies on the probe of the anomalous FCNC couplings in the top quark sector within model-independent method [34][35][36][37][38][39][40].
Recently, the ATLAS and CMS collaborations have set the upper limits of Br(t → qH) < 0.79% [41] and Br(t → cH) < 0.56% at 95% confidence level (C.L.) [42]. The production of the top pair (tt) and associated top-Higgs (th) via FCNC couplings has been emphasized in the recent studies [43][44][45][46][47][48][49][50][51][52][53][54]. Especially, the author of [43] studied the anomalous production of th via the FCNC interaction of tqh at the LHC through the h → bb channel including complete QCD next-to-leading order (NLO) corrections. The anomalous production of th at the LHC originating from FCNC interactions in tqg and tqh vertices has also been studied via the h → bb decay channel [45]. Although the branching ratio of Higgs diphoton decay channel is small, it has the advantages of good resolution on the Higgs mass and small QCD backgrounds. Thus in this paper, we mainly investigate the top-Higgs FCNC interactions through pp → th with the sequent decays t → W + b → ℓ + νb and h → γγ at the LHC. This paper is organized as follows. In Sec. II, we give a brief introduction to the anomalous FCNC tqh couplings and our selected production channel. In Sec. III, we discuss the observability of the top-Higgs FCNC couplings through the process pp → t(→ W + b → ℓ + νb)h(→ γγ) at 14 TeV LHC. In Sec. IV, we discuss the leptonic charge ratio of the signal and backgrounds. Finally, we summarize our conclusions in Section. V.

A. Top-Higgs FCNC couplings
In general, the effective Lagrangian describing the FCNC Yukawa interactions of a light up-type quark with the top quark and a Higgs boson can be written as [55] L = λ tuht Hu + λ tcht Hc + h.c., where the real coefficient λ tqh (q = u, c) denotes the strength of the top-Higgs FCNC coupling. At the leading order (LO) and the NLO, the decay widths of the dominant top quark decay mode t → W b could be found in Ref. [56]. After neglecting all the light quark masses and assuming the dominant top decay width t → bW , the branching ratio of t → qh can be approximately given by [44]: with the Fermi constant G F , the top quark mass m t , the W boson mass m W , the Higgs mass m h and x i = m i /m t (i = W, h). Here the factor κ QCD is the NLO QCD correction to Br(t → qh) and equals about 1.1 [57].
Currently, the stringent constraints on the anomalous FCNC couplings are set exploiting the experimental data of the ATLAS and CMS Collaborations [41,42]. On the other hand, the low energy observables, such as D 0 −D 0 mixing [58] and Z → cc [59] can also be used to constrain the top quark flavor violation in the tqH vertex. With the 125 GeV Z → cc decay [50]. The author of [60] also derived model-independent constraints on the tcH and tuH couplings that arised from the bounds on hadronic electric dipole moments.

B. The production processes
At the LHC, the parton level signal process at the tree-level via the FCNC htq couplings can be expressed as where q is u or c quark. The Feynman diagrams are shown in Fig. 1. Obviously, the conjugate processt + h production can also occur at the tree level. We first implement the tqH FCNC interactions by using the FeynRules package [61].
The LO cross section are computed using MadGraph5-aMC@NLO [62] with CTEQ6L parton distribution function (PDF) [63], setting the renormalization and factorization scales to be µ R = µ F = µ 0 /2 = (m t + m h )/2. In this work, we assume λ tqh ≤ 0.1 to satisfy the direct constraints from the ATLAS and CMS results [41,42]. The SM input parameters are taken as follows [64]: In Fig. 2, we show the dependence of the cross sections σ th and σt h on the top-Higgs FCNC couplings λ tqh at 14 TeV LHC for different processes. From Fig. 2, we can see that the cross section of ug → th is larger than that for other processes. To be specific, when √ s = 14 TeV and λ tqh = 0.1, the production cross section σ is about 20 pb, which is roughly an order of magnitude larger than that for the conjugate processūg →th due to the difference between the u-quark andū-quark PDF of the proton. Thus, for a given center-of-mass (c.m.) energy and luminosity, more leptons will be observed than anti- On the other hand, since the c-quark andc-quark have the similar small PDF, the production rates of top and anti-top quarks from the processes of gc(c) → ht(t) are almost the same and smaller than that for the process ug → th. This implies that the sensitivity to the coupling λ tuh will be better than λ tch .

III. SIGNAL AND DISCOVERY POTENTIALITY
In this section, we perform the Monte Carlo simulation and explore the sensitivity of 14 TeV LHC to the top-Higgs FCNC couplings through the channel, where ℓ = e, µ. The Feynman diagram of production and decay chain is presented in Obviously, the signal is taken as the single top plus a Higgs boson followed by the leptonic top quark decay and the Higgs boson decay into two photons, which is characterized by two photons appearing as a narrow resonance centered around the Higgs boson mass.
The main SM backgrounds which yield the identical final states to the signal are W hj, W jγγ and tjγγ, where j denotes non-bottom-quark jets. Besides, with fake photons due to misidentified jets or electrons, the reducible backgrounds such as W jjγ, tjjγ, tjγ, ttγ, ttγγ can be important as well. On the other hand, the SM thj, ZW h, W + W − h and tth events can also be the sources of backgrounds for our signal. We have not included these backgrounds in the analysis due to very small cross sections. After including the branching ratios and applying the cuts, a negligible number of events will survive.
All of these signal and backgrounds events are generated at LO using MadGraph5-aMC@NLO with the CTEQ6L PDF. PYTHIA [65] is utilized for parton shower and hadronization. Delphes [66] is then employed to account for the detector simulations and MadAnalysis5 [67] for analysis, where the (mis-)tagging efficiencies and fake rates are assumed to be their default values, which is formulated as a function of the transverse momentum and rapidity of the jets. When generating the parton level events, we assume µ R = µ F to be the default event-by-event value. The anti-k t algorithm [68] with the jet radius of 0.4 is used to reconstruct jets. The high order corrections for the dominant backgrounds are considered by including a k-factor, which is 1.12 for W hj [69] and 1.3 for W jγγ [70,71], respectively. Here it should be mentioned that the k-factor for the LO cross section of σ th is chosen as 1.5 at the 14 TeV LHC [43]. In order to avoid the doublecounting issue of jets originated from matrix element calculation and the parton shower, we apply the MLM-matching implemented in MadGraph5-aMC@NLO [72]. In practice, for the background events of W hj, we include both processes pp → W hj and pp → W hj + j to form an inclusive dataset, and similarly for other backgrounds.
In our simulation, we generate 10 6 events for the signals and backgrounds respectively.
We first employ some basic cuts for the selection of events: where p T and η are the transverse momentum and the pseudo-rapidity of jets and leptons It should be mentioned that the pp → tt → thq process could also be considered as a

source of top plus a Higgs boson if the light quark is missed by the detector. It has been
shown that this additional contribution is very significant for detecting the λ tch couplings due to the suppressed production cross section for the cg → th process [43]. Therefore, we also consider this process when discussing the tch couplings. In our calculation, the FCNC couplings are chosen to be λ tuh = 0.1 and λ tch = 0.1, which are allowed by the low-energy experiments [58,59]. In Fig. 4, we show the transverse momentum distributions of two photons in the signal and backgrounds at 14 TeV LHC. Since the two photons in the signal and the resonant backgrounds come from the Higgs boson, they have the harder p T spectrum than those in the non-resonant backgrounds. Thus, we can apply the cuts p γ 1 T > 55 GeV and p γ 2 T > 25 GeV to suppress the non-resonant backgrounds. Since there are only one b jet and one lepton in the final states, it is easily to reconstruct the top quark transverse cluster mass, which is defined as where p T,ℓ and p T,b are the transverse momentums of the charged leptons and b-quark, respectively, and / p T is the missing transverse momentum determined by the negative sum of visible momenta in the transverse direction. In Fig. 5, we show the transverse mass Next we consider to utilize the invariant mass distributions to further suppress the background. Fig. 6 illustrates the normalized invariant mass distribution M γγ of the signal and backgrounds. We can see that the signals and the resonant backgrounds including the Higgs boson have peaks around 125 GeV. Thus we can further reduce the non-resonant backgrounds by the following cut: For a short summary, we list all the cut-based selections here: GeV and ∆R i,j > 0.4 (i, j = j, b, ℓ, γ); • Cut 1 means the basic cuts plus p γ 1 T > 55 GeV, p γ 2 T > 25 GeV; • Cut 2 means Cut 1 plus 120 GeV < M T (ℓ + b / E miss T ) < 190 GeV.
• Cut 3 means Cut 2 plus requiring the invariant mass of the diphoton pair to be in the range m h ± 5 GeV.
The cross sections of the signal and backgrounds after imposing the cuts are summarized in Table I. For the numbers of the cross sections as listed in the Table I, the FCNC couplings are chosen to be λ tuh = 0.1 and λ tch = 0.1. From Table I, we can see that after all these cuts, all the backgrounds are suppressed efficiently and the total production cross section for the backgrounds is about 9.4 × 10 −4 fb. The final production cross sections of thj, ttγ and ttγγ are all at the level of 10 −6 fb and thus they can be safely neglected.
However, the cross section of the process pp → tt → thq is comparable to that of cg → th.
As stated before, we should include these two processes when discussing the tch couplings.
In order to illustrate excluded detection potential regions of anomalous couplings to reach a given statistical significance, we define the statistical significance (SS) as [74]: where S and B are the signal and background cross sections and L is the integrated luminosity. Here we define the discovery significance as SS = 5 and exclusion limits as SS = 3.  In Fig. 7, we plot the excluded 3σ and 5σ discovery reaches in the plane of the integrated luminosity and the coupling parameter λ tqh . To observe the signal at the LHC, we here also require the minimum 5 events for the signal. From Fig. 7, we can see that the 5σ C.L. discovery sensitivity of λ tuh is 0.062, 0.049 and 0.034 when the integrated luminosity is 500, 1000 and 3000 fb −1 , respectively. For the thc couplings, the 5σ C.L. discovery sensitivity of λ tch is 0.065 when the integrated luminosity is 3000 fb −1 .
If no signal is observed, it means that the FCNC tqh couplings can not be too large.
The upper limits on the FCNC couplings λ tuh and λ tch can be respectively probed to 0.025 and 0.047 with L = 3000 fb −1 . These limits can be converted to the 3σ C.L. upper limits on the branching ratio Br(t → uh) = 0.036% and Br(t → ch) = 0.13%, respectively.
Compared with other phenomenological studies, we can see that our result is comparable with the sensitivity limits of LHC as Br(t → uh) < 5 × 10 −4 via multi-leptons channel with an integrated luminosity of 3000 f b −1 at √ s = 14 TeV [46].

IV. CHARGE RATIO FOR SIGNAL AND BACKGROUNDS
As we discussed in Sec. II, the useful handle on tagging th production in searches with leptonic decays of top quark is the enhanced abundance of positively charged leptons.
Due to the difference between the u-quark andū-quark PDF, the cross sections of th andth are different for the processes u(ū)g → t(t)h at the LHC. Since the efficiencies of lepton selection and fake charged lepton contamination are almost independent of charge, the top (anti-top) asymmetry can be directly translated in a corresponding lepton charge asymmetry. In this analysis, we define a ratio R = N + /N − as the number of events with positive charged lepton to the number of events with negative charge. For the signal and relevant backgrounds, the values of R are listed in Table II at 14 TeV LHC with the same basic cuts. Here we present three typical renormalization scale and factorization scale as µ R = µ F = µ 0 /2, µ 0 and 2µ 0 , respectively. One can see that, for λ tch = λ tuh = 0.1, the value of the charge ratio for the signal is insensitive to the renormalization and factorization scales, and it is significantly larger than that for the SM backgrounds. In Fig. 8 we show the dependence of the charge ratio R for the process pp → th on the various of ratio values λ tuh /λ tch at the 14 TeV LHC. One can see that for the case of λ tuh ≫ λ tch , the contribution from the tuh vertex can enhance the charge ratio, while for the case of λ tuh ≪ λ tch , the contribution from the tch vertex can significantly dilute the ratio.
Therefore, studying the charged ratio for lepton can be not only used to discriminate between signal and backgrounds, but also used to determine that the signal comes from the up initiated production channel and charm initiated production channel.

V. CONCLUSION
In the work, we investigated the process pp → th induced by the top-Higgs FCNC couplings at the LHC. We also studied the observability of top-Higgs FCNC couplings through the process pp → t(→ W + b → ℓ + νb)h(→ γγ) and proposed the charge ratio of signal and backgrounds of the charge lepton. From our numerical calculations and the phenomenological analysis we found the following points: 1. The cross section of pp → ug → th is larger than that for other process due to the larger PDF of the u-quark, which means that the sensitivity to the FCNC coupling λ tuh is better than λ tch .
2. We further studied the observability of top-Higgs FCNC couplings through the process pp → t(→ W + b → ℓ + νb)h(→ γγ) and found that in some parameter regions, the LHC may observe the above signals at the 5σ confidence level. Otherwise, the branching ratios Br(t → uh) and Br(t → ch) can be respectively probed to 0.036% and 0.13% at 3σ level at 14 TeV LHC with the integrated luminosity of 3000 fb −1 .
3. The charge ratio for the lepton in the signal is significantly larger than that for the SM backgrounds, which can be not only used to discriminate between signal and backgrounds, but also used to determine that the signal comes from the up initiated production channel and charm initiated production channel.