Flavor Changing Heavy Higgs Interactions at the LHC

A general two Higgs doublet model (2HDM) is adopted to study the signature of flavor changing neutral Higgs (FCNH) decay $\phi^0 \to t\bar{c}+\bar{t}c$, where $\phi^0$ could be a CP-even scalar ($H^0$) or a CP-odd pseudoscalar ($A^0$). Measurement of the light 125 GeV neutral Higgs boson ($h^0$) couplings at the Large Hadron Collider (LHC) favor the decoupling limit or the alignment limit of a 2HDM, in which gauge boson and diagonal fermion couplings of $h^0$ approach Standard Model values. In such limit, FCNH couplings of $h^0$ are naturally suppressed by a small mixing parameter $\cos(\beta-\alpha)$, while the off-diagonal couplings of heavier neutral scalars $\phi^0$ are sustained by $\sin(\beta-\alpha) \sim 1$. We study physics background from dominant processes with realistic acceptance cuts and tagging efficiencies. Promising results are found for the LHC running at 13 or 14 TeV collision energies.


Introduction
The Standard Model (SM) is very successful in explaining almost all experimental data to date, culminating in the recent discovery of the long awaited Higgs boson at the CERN Large Hadron Collider (LHC) [1,2]. In the SM, all elementary particles acquire mass from a single Higgs doublet that generates spontaneous electroweak symmetry breaking (EWSB). All charged fermions have their masses and Yukawa couplings to the Higgs boson as correlated but free parameters. Furthermore, there are no flavor changing neutral currents (FCNC) mediated by gauge interactions, nor by Higgs interactions (FCNH), at the tree level. The most important goals of the LHC, at Run 2 and beyond, are the study of Higgs properties and the search for signals, direct or indirect, of new physics beyond the SM.
As the most massive particle ever discovered, the top quark might provide clues to better understand the mechanism of EWSB. A possible explanation for its heaviness could be provided by a special two Higgs doublet model for the top quark (T2HDM) [3], where it is the only fermion that couples to a Higgs doublet with a large vacuum expectation value (VEV). The second heaviest particle is the newly discovered Higgs boson (h 0 ). With m h 0 < m t , it * Corresponding author.  [4][5][6] for m h 0 120 GeV. If this decay is detected, it would indicate a large effective FCNH coupling of treelevel origins [7], or very large enhancement from beyond SM loop effects [6].
In this Letter, we study the discovery potential of the LHC in the search for heavy Higgs bosons H 0 or A 0 that decay into a top quark and a charm quark. The top quark then decays into a b quark, a charged lepton (e or μ), and a neutrino. Taking LHC Higgs data and B physics constraints into account, we evaluate production rates with full tree-level matrix elements for both signal and background. We optimize the acceptance cuts to effectively reduce the latter with realistic b-tagging and mistag efficiencies. Promising results are presented for the LHC with √ s = 13 TeV as well as 14 TeV.

Constraints from data
In this section, we apply the latest results from LHC Higgs measurements, as well as from B physics, to constrain the parameters ρ tt , ρ bb , ρ ct , ρ tc , and cos(β − α) of a general 2HDM that are relevant for observing flavor changing decays of heavy Higgs bosons at the LHC.

Constraints from ATLAS and CMS
Run 1 of LHC at √ s = 7 and 8 TeV has provided us with information on the couplings of the Higgs boson h 0 , by measuring the event rates relative to the SM signal strength. Even with our general 2HDM, the light Higgs boson h 0 constitutes a narrow resonance, and the signal strength for a production channel X and final state Y can be written as ATLAS and CMS often show the signal strength of measurements in two dimensions by grouping gluon fusion and tth production on one axis (ggF), and vector boson fusion and associated production on the other axis (VBF). These contours can be used to draw constraints on 2HDM's [22]. We follow a simpler approach and consider signal strengths for final states with the largest statistics, namely γ γ , Z Z * , W W * , and τ τ , where the dominant production mode is gluon fusion, as well as the signal strength for the bb final state from the associated production V h 0 with V = W or Z . Table 1 shows the average signal strengths obtained by the experimental groups in Run 1. We combine the ATLAS and CMS results and show both the combined values with their uncertainties in the last column.
To find the allowed regions of 2HDM parameter space compatible with ATLAS and CMS data, we take the discovered Higgs boson as the lightest CP even state (h 0 ) of a general two Higgs doublet model and scan over the following sets of parameters: Table 1 Signal strengths for the Higgs boson at the LHC. The last column is our combination.

The combined signal strength for
In SM, the most important contributions to the Higgs total width come from bb, W W * , gg, τ τ , cc and Z Z * . The same channels are expected to contribute in a general 2HDM. In our analysis, we include all the relevant parameters in Eq. (3) that affect the total width, gluon fusion cross section, and the associated Higgs production (V h 0 ). We cover a wide range of values for each free parameter, and require that all Yukawa couplings of the mass eigenstates (h 0 , H 0 , A 0 , H ± ) stay perturbative, and that the constraints given in Table 1 are satisfied. The signal strength for each production and decay channel can be expressed in terms of scale factors which are couplings of the Higgs boson to fermions and gauge bosons normalized to their Standard Model values [33]. These scale factors can then be expressed in terms of the parameters of a specific 2HDM.
Negative results from heavy Higgs searches provide us further insights on the parameter space of an extended Higgs sector. One of the strongest results comes from the search for a heavy Higgs decaying into the W W and Z Z final states, excluding a heavy Higgs boson with SM like couplings all the way up to 1 TeV [34]. We use these results to further constrain the parameter space of 2HDM.
In Fig. 1, we present the 68% (95%) confidence level (C.L.) regions in dark (light) color that are compatible with LHC constraints from the light Higgs boson (h 0 ) alone as well as constraints from both the light Higgs boson and the heavy Higgs boson (H 0 ) decaying into W W and Z Z [34] for a general 2HDM and a Type-II 2HDM. This figure shows that a large value of | cos(β − α)| is still a possibility within a general model for ρ tt < 1. This is due to the lack of a strong constraint on the b-quark coupling of the Higgs boson. To be consistent with the SM Higgs cross section from gluon fusion, a small value of cos(β − α) is favored for ρ tt 1 with λ htt ∼ λ SM htt = κ tt . Experimental data from Run 2 with higher energy and higher luminosity will provide much better guidance for parameters such as ρ tt and cos(β − α).

Constraints from B physics
The FCNH coupling ρ ct affects the This effect contributes to FCNC processes in down-type quark sector via H + and t loops. For simplicity, let us assume ρ ut is negligible.
Recasting the 2HDM-II expression [35], we estimate the modifications to the B q -B q (q = d, s) mixing amplitude (M where We adopt the following intervals from the Summer 2014 results by UTfit [36], The constraints from B d,s mixing data are shown in Fig. 2(a) on the (ρ tt , ρ ct ) plane with m H + = 500 GeV. Shaded regions are excluded by the 95% probability ranges in Eq. (6). The constraint from C B s (pink regions) is slightly tighter than the C B d exclusion (blue-shaded regions). Combining them with constraint from the CP-violating phase φ B d (light-green regions), we obtain the upper limit |ρ tt | 1.5, regardless of ρ ct . The parameter ρ ct is strongly constrained since its effect in Eq. (5) is enhanced by the CKM fac- Once ρ tt is fixed within this range, we obtain a constraint on ρ ct . For 0.5 |ρ tt | 1.5, we have |ρ ct | 0.06. Furthermore, the sizable phase in V cd /V td makes ρ ct sensitive to the CP-violating phase φ B d , even if ρ ct is real. For m H + = 300 (700) GeV, the constraints become: |ρ tt | 1.2 (1.8) regardless of ρ ct , and |ρ ct | 0.05 (0.09) for 0.5 |ρ tt | 1.2 (1.8).
We now turn to the b → sγ constraint. The H + -t loop affects this process via the Wilson coefficients C 7,8 at leading-order (LO), which are, at the matching scale μ 0 , given by Here, the operator basis and the definition of F Ref. [37]. We follow the procedure in Ref. [38] and calculate first the ratio linearly added. The constraint on ρ tt by C B s is also shown. For ρ tt ∼ κ t ∼ 1, ρ bb is constrained to be within −0.02 ρ bb 0.01. Note that this touches the region of |ρ bb | ∼ κ b ∼ 0.02. We set ρ ct = 0 as it is already strongly constrained by B d,s mixing. For m H + = 300 (700) GeV, the b → sγ constraint on ρ bb becomes: −0.009 (−0.03) ρ bb 0.008 (0.02) for ρ tt ∼ 1. Typically, B(B → X s γ ) constrains ρ bb more strongly than ρ tt , as the effect of ρ bb is enhanced by the chiral factor κ t /κ b = m t /m b in Eq. (7).
The contribution from charm loop has a mild dependence , s, b). In general, ρ tc may be very different from ρ ct . Since the charm quark in the loop is light and there is no CKM enhancement, the constraint on ρ tc is expected to be much weaker. The constraint on ρ tc has been analyzed in Ref. [38].
Additional flavor constraints can be obtained from K -K mixing (ρ ct 0.14) [38] and D-D mixing (|ρ tc ρ * tu | 0.02) [38] for m H m H + = 500 GeV. The value of (ρ tu ) is expected to be very small, thus B-B mixing provides a better limit for ρ tc .

Signal and background
We now discuss the prospects of discovering FCNH interactions at the LHC through H 0 and A 0 decays. The number of free parameters in a general 2HDM is too large for a comprehensive collider study of the FCNH signal, so we make some assumptions that are motivated by experiment. The latest experimental results point to a Higgs sector with the light CP even state behaving like the SM Higgs, indicating that cos(β − α) should be small. We consider sample cases with cos(β − α) = 0.1 and 0.2, which imply sin(β − α) ∼ 1.
In our case study, we choose the heavier states (H 0 , A 0 and H ± ) to be degenerate for simplicity, which is also in accordance with the decoupling limit [19], and we set λ 6,7 = 0 in the Higgs potential [9]. We also set tan β = 1 and choose m 2 To fix the Yukawa couplings, we assume that ρ tt = κ t while ρ bb = κ b . This is in good agreement with both B physics constraints as well as LHC Run 1 constraints. For the off-diagonal parts that generate the flavor changing signal, ρ ct is constrained to small values by B physics but we assume that ρ tc can have larger values. In the massless limit for charm, the signal cross section is, to a very good approximation, only a function of a single effective coupling ρ tc = 1 √ 2 ρ 2 tc + ρ 2 ct , but also very weakly depends ct . The contribution to the cross section from terms with ρ tc is at least 98% without cuts and more than 93% with all cuts for pp → H → bc ν + X with m H = 1 TeV, and it is even more dominating for a lower Higgs mass. With these experimentally motivated choices, gluon fusion is the dominant production mode for H 0 and A 0 states, and tt becomes the dominant final state at high mass (2m t < m φ 2 TeV). We display in Fig. 3 the branching fractions for (a) the heavier scalar H 0 , and (b) the pseudoscalar A 0 , as functions of Higgs mass with cos(β − α) = 0.1 and ρ tc = 0.24. The computer code 2HDMC [46] is employed to scan over |m 12 | ≤ 2 TeV and 0.1 ≤ tan β ≤ 50 with sets of parameters that satisfy potential stability, tree-level unitarity, and perturbativity. This gives rise to the "bands" in Fig. 3(a). We also display the branching fraction B(H 0 → tc) for the aforementioned choice of tan β and m 2 12 in our LHC case study with a dashed curve in Fig. 3(a). We note that with large branching fraction in most of the parameter space, H 0 → h 0 h 0 might offer great promise to discover Higgs pairs at the LHC.

Higgs signal
Our signal is the production of a heavy Higgs boson from gluon fusion, with subsequent flavor changing decays into a charm quark and a top quark, and the top decays semileptonically. More explicitly, we consider gg → φ 0 → tc +tc (φ 0 = H 0 or A 0 ), followed by tc → b νc with = e or μ. Unless explicitly specified, q generally denotes a quark (q) or an anti-quark (q) and represents a lepton ( − ) or anti-lepton ( + ). We calculate the matrix elements analytically, and compute the signal cross section with the parton distribution functions of MSTW2008 [47]. The factorization and renormalization scales are chosen to be μ F ,R = m φ . In addition, to estimate the NLO cross section for pp → φ 0 → tc +tc → b νc + X , we use the computer code HIGLU [48] to calculate σ (pp → φ 0 + X) , including both top and bottom quark loops to find a K -factor.

Standard model background
The dominant physics background to the final state of bj ν comes from W jj + W bb, as well as s-and t-channel single top (tb + t j). Another important background is tt production where either one of the two leptons is missed for both top quarks decaying semileptonically, or two of the four jets are missed when only one of top quarks decays semileptonically. We employ the programs MadGraph [49,50] and HELAS [51] to evaluate the exact matrix elements for the background processes. The factorization and the renormalization scales are chosen to be μ R,F = m W for W jj and W bb, μ R,F = m t for s-and t-channel single top, and μ R,F = √ŝ for tt. We use MCFM [52] to calculate the NLO K -factors for our background processes.

Mass reconstruction
Let us present our strategy for full reconstruction of each event with the help of intermediate on-shell particles. For each event, we require one b jet and one non-b jet, identified through b-tagging. In addition, we require a single isolated lepton and missing transverse energy from the neutrino in the semileptonic decay of the top quark in our FCNH signal. For lepton momentum p and neutrino momentum k, the invariant mass constraint for an on-shell W , (k + p) 2 = m 2 W , can be solved for the longitudinal component of the neutrino momentum (k z ), which is the only unknown in the event. We obtain two solutions If 2 < 0 hence k ± z complex, the event is vetoed. For 2 > 0 with k ± z real, we choose the solution that minimizes the reconstructed Systematics can be the limiting factor for new physics searches at high luminosities. Precise determination of the background needs to include systematics in experiments. Since our signal is a sharp peak over a smoothly falling background, the precise knowledge of background cross section at percent level is not required for a 5σ discovery. An uncertainty of 30% in the background estimation might shift the limit on g Htc by 10% without affecting our results.

Realistic acceptance cuts
To study the discovery potential, we employ three sets of realistic cuts and tagging efficiencies.
We consider a further powerful acceptance cut on non-b-tagged jet momentum. In the Higgs boson decay frame, the charm quark momentum from H 0 , A 0 → tc is approximately given by Since the Higgs boson from gluon fusion has little transverse momentum, the p T (c) distribution has both a kinematic cut-off and a peak at the above p c value. We require that the transverse momentum of the non-b-tagged jet satisfies 0.85 p c < p T (c) < 1.10 p c .
To simulate detector effects based on ATLAS [54] and CMS [55] specifications, we apply Gaussian smearing of momenta: ⊕ 0.01 (leptons) , (11) with individual terms added in quadrature (⊕). In LHC Run 1 at √ s = 8 TeV, the b-tagging efficiency ( b ) is taken to be 50%, the probability that a c-jet is mistagged as a b-jet ( c ) is 14% and the probability that any other jet is mistagged as a b-jet ( j ) is taken to be 1%. At the full CM energy ( √ s = 13 or 14 TeV) with L = 30 fb −1 , we follow the tagging and mistag efficiencies in the ATLAS Technical Design Report [54]: b = 60%, c = 14% and j = 1%. For the full CM energy ( √ s = 13 or 14 TeV) with HL of 300 fb −1 or 3000 fb −1 , the tagging and mistag efficiencies are taken to be b = 50%, c = 14% and j = 1%.

Discovery potential
We present the signal and background cross sections at the LHC for √ s = 8 TeV and 14 TeV in Fig. 4. All tagging efficiencies and K -factors discussed above are included. We observe that the largest contributions to the SM background come from singletop and W + jets processes, which is to be expected, since they can both produce very similar kinematics to our signal process. In The discovery region is the parameter space above the contours. Also shown is the future ATLAS sensitivity at 95% confidence level for t → ch 0 → cγ γ .
contrast, the tt background is substantially lower because of the requirement on the number of jets and leptons passing our cuts.
To estimate the discovery potential, we obtain the lower limit on σ S by requiring that the 99.4%-confidence-level (CL) upper limit on the background is smaller than the 99.4%-CL lower limit on the signal plus background [56] with statistical fluctuations. This leads to the condition, where σ S (B) is the signal (background) cross section and L the integrated luminosity. Choosing the parameter N = 2.5 corresponds to 5σ significance. For a large number of events (Lσ B 1), this requirement is equivalent to the statistical significance where N S (B) is the number of signal (background) events.
We show in Fig. 5

Conclusion
In a general two Higgs doublet model, there could be flavor changing neutral Higgs interactions with fermions. Strong limits exist for these FCNH interactions, except those involving the third generation quarks. It is of great interest to study the relation between the most massive elementary particle (the top quark) and the Higgs bosons. The LHC has discovered a Higgs boson lighter than the top, which makes the rare decay t → ch 0 kinematically possible. In a general 2HDM, the decay width of t → ch 0 is proportional to cos(β − α), while that of H 0 → tc is proportional to sin(β − α). Therefore, they are complementary to each other in the search for new physics beyond the Standard Model.
We investigated the prospects for discovering H 0 , A 0 → tc at the LHC, where the heavy scalar H 0 and pseudoscalar A 0 are produced via gluon fusion, which are facilitated by the extra tt couplings. The primary physics background comes from W jj, t j, Wbb, tb, and tt. Both signal and background processes are studied with realistic acceptance cuts as well as tagging and mistag efficiencies. Promising results have been found for the LHC with a center of mass energy of 13 TeV and 14 TeV. The FCNH decay of the heavy Higgs will be observable for cos(β − α) = 0.1 and ρ tc = 0.1 up to M H = 800 GeV with 3000 fb −1 of integrated luminosity. This result is robust against a small cos(β − α), independent of the t → ch 0 search, which becomes diminished. If c-tagging efficiency can be improved [57], the discovery potential of this FCNH signal will be greatly enhanced.
If ρ tc 0.5, B(H 0 → tc) can become comparable to B(H 0 → tt) or surpass it. Recently, it was suggested that next-to-leading order QCD and electroweak corrections might swamp the signal of Higgs decays into top quark pairs [58]. A recent analysis shows that H 0 → tt with SM couplings can be very difficult to observe at the LHC [59]. In that case the FCNH decay of H 0 , A 0 → tc +tc might offer a promising opportunity to observe the heavier Higgs bosons.
We have not emphasized the τ lepton sector. Recently, the CMS Collaboration reported unexpected τ μ events [60] that might be explained by neutral Higgs boson decay [61]. If this can be confirmed by the ATLAS Collaboration in the near future, or at LHC Run 2, it will be exciting new physics for FCNH interactions, and H 0 , A 0 → τ ± μ ∓ , unsuppressed by decoupling (i.e. small cos(β − α)), could help discover the exotic scalars.