Probing anomalous top-Higgs couplings at the HL-LHC via $H\to WW^{\ast}$ decay channels

We study the prospects of probing the anomalous $tHq$~($q= u, c$) couplings via SS2L or 3L signatures at the High Luminosity (HL-LHC) run of the 14 TeV CERN collider. We focus on signals of the $tH$ associated production followed by the decay modes $t\to b\ell^{+}\nu_{\ell}$ and $H\to WW^{\ast}$, and $t\bar{t}$ production followed by the decay modes $t\to b\ell^{+}\nu_{\ell}$ and $\bar{t}\to H(\to WW^{\ast})\bar{q}$, where $\ell=e, \mu$. Based on two types of $H\to WW^{\ast}$ decay topologies, one assuming the semileptonic decay mode $H\to WW^{\ast}\to \ell^{+}\nu jj $ and the other the fully leptonic decay mode $H\to WW^{\ast}\to \ell^{+}\nu \ell^{-}\bar{\nu}$, we perform a full simulation for signals and backgrounds. It is shown that, at the future HL-LHC, the branching ratio $Br(t\to uh)~(Br(t\to ch))$ can be probed to $1.17~(1.56)\times 10^{-3}$ for the same-sign di-lepton channel and to $7.1~\times 10^{-4}~(1.39~\times 10^{-3})$ for the 3L channel at $3\sigma$ sensitivity.

Currently, the ATLAS and CMS collaborations have carried out searches [17][18][19][20][21] for tqH interactions with 7, 8 and 13 TeV data from the LHC. For example, using 13 TeV data, the ATLAS and the CMS experiments have studied the tqH FCNC processes in top quark pair events with H → γγ for ATLAS and H → bb for CMS. The resulting observed (expected) limits for Br(t→qH) at 95% Confidence Level (CL) have been found to be [19,20]: Very recently, a search for production of top pairs in which one top quark decays via t → qH is reported by the ATLAS Collaboration [21], with the subsequent Higgs boson decay to final states with at least one electron or muon. The upper limits on the branching fractions Br(t → Hc) < 0.16% and Br(t → Hu) < 0.19% at 95% CL are obtained (with expected limits of 0.15% in both cases). Apart from direct collider measurements, the upper limits of Br(t → qH) < 5×10 −3 and Br(t → qH) < 0.21% can be obtained by bounding the tqH vertex from the observed D 0 −D 0 mixing [22] and Z →cc [23], respectively. The upcoming project of the HL-LHC is expected to reach 3 ab −1 . Preliminary sensitivity studies for the HL-LHC suggest the upper bound on Br(t → qH) to become about 1.5×10 −4 at 95% CL by the ATLAS Collaboration [24]. Further, many phenomenological studies within model-independent methods have been performed from different channels [25][26][27][28][29][30][31][32][33]. In this work, we study the prospects of probing the anomalous tHq couplings by considering the processes of tH associated production and tt production at the HL-LHC. We analyze two kinds of final states through leptonic top quark decays and H →W W * , one with Same Sign 2-Lepton (SS2L) and the other with 3-Lepton (3L) topology, where the Higgs boson decays into a semi-leptonic (H →W W * →ℓ + νjj) or fully leptonic (H →W W * →ℓ + νℓ −ν ) mode. The advantage of these channels is that their final states including the SS2L or 3L topologies can be used to significantly suppress QCD backgrounds [34], which have not been fully studied in previous literature.
The organization of this paper is as follows. In Sec. II, we discuss two kinds of final states for the processes of tH associated production with the decay chain t→W + b→ℓ + νb and H →W W * as well as tt production with the decay chain t → ℓ + ν ℓ b andt → H(→ W W * )q. Then we discuss the HL-LHC sensitivity to the anomalous tHq couplings. We summarize in Sec. III.

Numerical calculations and discussions
The general Lagrangian for FCNC top interactions with the Higgs boson can be written as L=κ tuHt Hu+κ tcHt Hc+h.c., where the FCNC coupling parameters, κ tuH and κ tcH , are real and symmetric since we do not consider here the CP violating effects. We perform systematic Monte Carlo (MC) simulations and study the sensitivity to the anomalous tHq couplings through the associated tH and tt → tHq processes at HL-LHC. We first extract the relevant Feyn-man rules via the FeynRules package [35] and generate the events with MadGraph5-aMC@NLO [36]. The signal and backgrounds samples are simulated at parton level with the NN23LO1 Parton Distribution Function (PDF) set [37] and then passed through PYTHIA6.4 [38] and DELPHES 3 [39] for parton shower and detector simulations, with the MLM matching scheme [40] adopted. Finally, event analysis is performed by using MadAnal-ysis5 [41].

Analysis of the SS2L channel
For the final states including the SS2L topology, the signals are generated through the following processes, where ℓ=e,µ. The representative Feynman diagrams are shown in Fig. 1. For this channel, the typical signal is exactly two same-sign leptons plus at least three jets, with at least one jet identified as b-jet, and missing transverse energy. The main backgrounds are ttV (V = W,Z), W + W + jj and W + Zjj. The tt process, which has large cross section, may also contribute to background if a same-sign lepton pair comes from a B-hadron semi-leptonic decay in the b-jet. We do not consider other backgrounds from ttH, tttt, tri-boson events and tHj. They are neglected because the cross sections are all negligible after applying the selection cuts.
The cross sections of dominant backgrounds at Leading Order (LO) are adjusted to Next-to-LO (NLO) by means of K-factors, which are 1.04 for W + W + jj jets [42], 1.24 for ttW [43] and 1.39 for ttZ [44]. The dominant tt background is normalized to the NNLO QCD cross section of 953.6 pb [45]. For the tH production cross section, the K-factor is taken as 1.5 at the 14 TeV LHC [12].
The decay chain H →W W * →ℓνjj may result in soft leptons and light jets, especially when they are coming from an off-shell W boson. To analyze the signal sensi-  tivity, we thus employ the following basic cuts to select the events: Basic cuts: p T (ℓ) > 10 GeV, p T (j,b) > 15 GeV, |η ℓ,j,b |<2.5, where ℓ=e,µ.
In order to choose appropriate kinematic cuts, we plot in Fig. 2 examples of kinematic distributions for the signal and backgrounds. Based on these distributions, we impose a further set of cuts.
2) Cut-2: At least two jets in the events are required to be successfully reconstructed, i.e., N (j) ≥ 2. Among those reconstructed jets, there is at least one pair of jets which could come from a W boson either on-shell or offshell. Thus the invariant mass of the W boson is required to be M jj <90 GeV.
3) Cut-3: The invariant mass of M ℓ 2 jj is required to be smaller than 120 GeV. 4) Cut-4: Since the first lepton, ℓ 1 , is assumed to originate from the leptonically decaying top quark, the invariant mass of the b-jet and the leading lepton should be M bℓ 1 <140 GeV. 5) Cut-5: The scalar sum of transverse momenta, H T , is to be smaller than 250 GeV.
The effects of the cuts on the signal and background processes are illustrated in Table 1 for the SS2L channel, where the anomalous coupling parameters are taken as κ tuH = 0.1 or κ tcH = 0.1, while fixing the other to zero. From Table 1 we can see that, after all these cuts, the tt backgrounds for the SS2L channel with fake leptons from heavy-flavor jets or charge mis-identifications can be significant. Obviously, the non-prompt backgrounds may also be significant, where non-prompt leptons are from heavyflavor decays, mis-identified hadrons, muons from lightmeson decays or electrons from un-identified conversions of photons into jets. Recently, the CMS collaboration searched for SS2L signatures [46] and found that the overall non-prompt backgrounds are about 1.5 times the ttW background after all cuts. These non-prompt backgrounds are not properly modeled in our MC simulations. Therefore, for simplicity, we add a non-prompt background that is 1.5 times ttW [46] after selection cuts to the overall background. Accounting for the theoretical and experimental systematic uncertainties on the background predictions would certainly improve the reliability of the results, yet they can only be neglected in our simulation.
The dominant SM backgrounds are ttV (V = W,Z), ttH, W Z+ jets and tt. The multi-jet backgrounds (where jets can fake electrons) are not included since they are negligible in multi-lepton analyses [47].
In Fig. 3, we show the invariant mass distribution of M ℓ 2 ℓ 3 and M bℓ 1 from the signal and backgrounds at the 14 TeV LHC. To remove contamination from hadron decay chains including ℓ + ℓ − pairs and resonant Z bosons, we choose the invariant mass M ℓ 2 ℓ 3 cuts 12GeV<M (ℓ 2 ℓ 3 )<55 GeV.
Similarly, the invariant mass of the b-jet and the leading lepton, M bℓ 1 , should be smaller than 140 GeV. The effects of the cuts on the signal and background processes are illustrated in Table 2 for the 3L channel. One can see that significant backgrounds also come from the top pair production process with fake leptons or charge misidentifications.
Using the Poisson formula [48], we estimate the Signal Significance (SS) with fixed coupling parameters κ tqH and a given integrated luminosity L int . In Figs. 4 and 5, we plot the contours of SS = 3 and SS =5, respectively, for two channels in the plane of L int −κ tqH . It is clear that, for an integrated luminosity of 3000 fb −1 , the FCNC couplings κ tuH (κ tcH ) can be probed to 0.045 (0.052) and 0.035 (0.049) at 3σ statistical sensitivity for the SS2L and 3L channels, respectively. After neglecting the masses of light quarks, the branching ratio of t→qH is approximately given by [13,49] Br in terms of the Fermi constant G F and with x i = m i /m t (i = W, h). In our numerical calculation, the relevant SM input parameters are taken as [50]: Using Eq. (7), the limits can be translated in terms of constraints on the branching fractions of rare top decays. The 3σ CL upper limits on Br(t → qH) are Br(t → uH) = 1.17×10 −3 and Br(t → cH) = 1.56×10 −3 for the SS2L channel, and Br(t → uH) = 7.1×10 −4 and Br(t → cH) = 1.39×10 −3 for the 3L channel. The projected limits from different channels are summarized in Table 3. We can see from the table that our results are comparable with the sensitivity limits at the HL-LHC of Br(t → uH) < 0.036% via the H → γγ channel [29], Br(t → uH) < 0.05% via the multi-lepton channel and Br(t→uH)<0.02% via the di-photon channel [51].

Conclusions
The discovery of the 125 GeV Higgs boson opens the door to probe NP processes that involve Higgs boson associated production or decay. In this paper, we have investigated the signal of tH associated production via FCNC tqH couplings and tt production witht → Hq at the 14 TeV LHC. We focused on the final states includ-ing SS2L and 3L signals from the decay modes t→bℓ + ν ℓ , H → W W * → ℓ + νjj or H → W W * → ℓ + νℓ −ν . We have then shown that, at 3σ level, the branching ratios Br(t → uH) and Br(t → cH) are, respectively, about Br(t → uH) ≤ 1.17×10 −3 and Br(t → cH) ≤ 1.56×10 −3 for the SS2L channel, and Br(t → uH) ≤ 7.1×10 −4 and Br(t→cH)≤1.39×10 −3 for the 3L channel at the future HL-LHC.