Enhanced $thj$ signal at the LHC with $h\rightarrow \gamma\gamma$ decay and $\mathcal{CP}$-violating top-Higgs coupling

We study the observability of non-standard top Yukawa couplings in the $pp\rightarrow t(\rightarrow \ell \nu_\ell b ) h(\rightarrow \gamma\gamma)j$ channel at 14 TeV high-luminosity LHC (HL-LHC). The small diphoton branching ratio is enhanced when the $\mathcal{CP}$-violating phase, $\xi$, of the top-Higgs interaction is non-zero. When the modulus of the top-Higgs interaction assumes the SM value, $y_t=y_t^{SM}$, we find that the signal significance reaches $2.7\sigma$ $( 7.7\sigma)$ when $\xi= 0.25\pi$ $(0.5\pi) $. Furthermore, the different couplings modify the polarisation of the top quark, and can subsequently be distinguished via asymmetries in spin correlations with the final state leptons. The diphoton decay mode is found to be significantly more promising than the previously considered $pp\rightarrow t(\rightarrow \ell \nu_\ell b ) h(\rightarrow b\overline{b})j$ channel.


Introduction
Since the discovery of the Higgs resonance [1,2], the measured properties [3,4] of this boson have been consistent with a minimal Higgs sector. In particular, there has been compelling evidence that it carries a spin quantum number of zero [5][6][7][8] (for a discussion on a generic spin-2 impostor, see [9]). The next priority is to establish the CP properties of this Higgs boson, as this is important for pinning down new physics which may have cosmological implications. Currently, pure scalar couplings are favoured over pseudoscalar couplings to the electroweak gauge bosons [7,8]. However, a non-minimal Yukawa sector containing substantial pseudoscalar admixture is not yet excluded (cf. [10]) and requires further investigation.
The top-Higgs Yukawa coupling is the largest in the Standard Model (SM) and therefore plays an important role in electroweak symmetry breaking, notably in the context of Higgs vacuum stability [11][12][13][14]. Also, a CP-violating top-Higgs sector may provide additional sources of CP-violation that may have implications for the electroweak phase transition and baryogenesis [15,16]. Direct constraints on non-standard top Yukawa couplings must come from observation of processes where y t enters at the tree-level, with the leading contribution coming from Higgs production associated with a top pair, pp → tth (see e.g. [17][18][19][20][21][22]). Specifically, the prospect for the LHC to distinguish the scalar and pseudoscalar components of the top Yukawa coupling in this channel have been considered [23][24][25][26][27][28][29][30]. Given the relatively small pp → tth cross section (∼ 130 fb at 8 TeV [31]), current luminosity and analysis are not yet sensitivity enough to observe such signal. However, the ATLAS collaboration have set an upper limit on the signal strength µ tth = σ tth /σ SM tth to be µ tth < 3.9 at 95% C.L. limit by combining h → bb and h → γγ channels [32] and the CMS collaboration, a limit of µ tth ∈ [0.9, 3.5] by using all search channels [33].
This work focuses on thj production [34][35][36][37][38][39][40][41][42] because of its sensitivity to the CPviolating phase, ξ of top Yukawa coupling, in addition to the modulus, y t . This is explained by the interference between the contributing Feynman diagrams involving tth and W W h couplings [10,26,34,35,[43][44][45][46][47][48]. Increased |ξ| values enhance the pp → thj cross section [10,44] from its SM value of 18 fb at 8 TeV [43,[49][50][51] but reduce the corresponding pp → tth cross section [26]. Furthermore, the thj channel is important because the resulting top quark is polarised through the left-handed weak interaction involving a t-channel virtual W boson. The t-quark spin information is then inherited by its decay products because the top decays before it hadronises [52][53][54][55][56][57][58]. Top quark polarisation induced by the CPviolating top-Yukawa couplings will therefore manifest in the spin correlation of the top decay products [26,28,46,[59][60][61][62][63]. In particular, studies on non-trivial chiral structures in top couplings through t-polarisation have been proposed for single top processes (see e.g. [64][65][66][67][68][69][70][71]). We have subsequently considered the viability of using thj production at the LHC to probe the CP-violating phase in the h → bb channel [10]. On the other hand, the CMS collaboration have searched for the thj signal in the h → γγ channel [72]. This decay gains a more easily controlled QCD background compared to thebb channel, but at the expense of having significantly smaller cross section. Given that a combination of these channels will be required to achieve high signal significance, it is well motivated to consider the diphoton channel in this work.
This work will be organised as follows: in Sec. 2, the enhancement of the diphoton branching ratio through CP-violating top-Higgs couplings, and its consistency with current Higgs data is discussed; Sec. 3 will be concerned with the observability of the thj signal with scalar (ξ = 0), pseudoscalar (ξ = 0.5π) and mixed (ξ = 0.25π) top-Higgs interactions at 14 TeV high luminosity LHC (HL-LHC); the use of lepton spin correlation and asymmetries to distinguish the various phases is studied in Sec. 4; the results are summarised in the conclusion (Sec. 5). We remark that the photon polarisations of the diphoton decay [73,74] may provide information on the CP phase of the top-Yukawa coupling, but will not be considered in this work.

CP-violating Top-Higgs Sector and Enhanced hγγ Decay Rate
In this study, we investigate the CP-violating top Yukawa couplings using the phenomenological Lagrangian: where t and h are respectively the physical top quark and Higgs boson in the mass basis, y t ∈ R parameterises the magnitude of the tth interactions and ξ ∈ (−π, π] is the CPviolating phase. In the SM limit, y t takes the value y SM t := √ 2m t /v and ξ = 0, where v ≈ 246 GeV is the vacuum expectation value of the Higgs field. Such a non-standard top-Higgs sector may arise from various beyond SM models [28,[75][76][77][78][79]. The framework used in this work will be that of an effective field theory whereby phenomenological predictions can be made without adhering to specific models. Eqn. 2.1 may originate from an effective Lagrangian comprising gauge-invariant operators [80][81][82][83][84]: where α, β ∈ C are dimensionless parameters and Λ the new physics scale. After symmetry breaking with H = (0, v + h/ √ 2) T , it may be identified that y SM t = α + βv 2 /Λ 2 . The phase ξ may take the full range (−π, π], given that new physics enters at the TeV scale (Λ ∼ 10v) and |β| ∼ 1 [85].
An immediate consequence of non-standard top-Yukawa couplings are deviations of gg → h production and h → γγ decay rates from the SM. This has been considered in our previous work [10], where the current Higgs data was used to exclude values of |ξ| > 0.6π at 95% C.L (see also [26,83,86,87]). The corresponding 95% C.L limit allowed for y t /y SM t is 0.7-1.2 when ξ = 0, but decreases with ξ such that it becomes 0.4-0.6 for ξ = 0.5π. Strong bounds on CP-violating effects also come from low energy probes [88][89][90] such as electric dipole moments. However, these bounds on the pseudoscalar coupling depend on the light fermion Yukawa couplings, which are practically unobservable at the LHC, and therefore are not considered in this work. In the subsequent parts of this work, only the top Yukawa sector will be modified, but y t = y SM t will be assumed to focus on the effects of ξ.
We will now focus on the influence of non-standard top Yuakwa coupling in h → γγ, as it is the relevant decay mode the phenomenological study in Sec. 3 and Sec. 4. The associated effective operator (see e.g. [91][92][93]) for hγγ interactions can be written as: where α = e 2 /4π is the fine structure constant, F µν is the standard field strength for photon fields andF µν := 1 2 µνρσ F ρσ is its dual. Given that the CP-even and CP-odd parts do not interfere, the effective scalar (c γ ) and pseudoscalar (c γ ) coupling constants are obtained from the corresponding scalar and pseudoscalar decay rates, Γ S,P (h → γγ) (cf. Appendix A). At one loop order, the scalar part is dominated by t-quark and Wboson contributions whilst the latter is absent for the pseudoscalar part. Accordingly, the coupling constants are paramaterised in terms of the top Yukawa couplings as follows: The total diphoton decay rate resulting from our modified top-Higgs sector will then be parameterised as: It should be noted that the partial cancellation between the W -loop and t-loop contributions to the scalar component diminishes with increasing ξ. Since the pseudoscalar t-loop factor is larger than that of the scalar, and the interaction with different parities do not interfere, the decay rate will be maximally enhanced for ξ = 0.5π.

Observability at 14 TeV HL-LHC
The sensitivity of the Higgs associated single top production channel to the CP-violating top-Higgs couplings at 14 TeV HL-LHC was investigated through Monte Carlo simulations. This was carried out in the diphoton decay of the Higgs and semileptonic decay of the top: with j denoting the light jets and = e, µ. Three phases ξ = 0, 0.25π and 0.5π were studied as benchmark points, with y t assuming the SM value y SM t , as justified in Sec. 2. The small branching ratio Br SM (h → γγ) = 2.28 × 10 −3 is compensated by excellent resolution on invariant diphoton mass, m γγ at ATLAS and CMS. Continuous QCD backgrounds can therefore be efficiently suppressed by narrow mass window cut on the reconstructed Higgs mass, m h . Furthermore, it could be seen from Eqn. 2.6 that the decay rate may be enhanced to ∼ 1.3 (1.8) that of the SM for ξ = 0.25π (0.5π). The observability of such signal may be further improved, given that the CP-violating phase also enhances the thj production cross section [10]. The semileptonic decay mode of the top quark is chosen because the charged lepton is maximally correlated with the top spin [94] and will be used to measure the forward-backward asymmetry in Sec. 4.
The dominant backgrounds to the signal process in order of contribution are as follows: (B1) t(→ + νb)jγγ -this irreducible background has the same final state as the signal. However it is non-resonant and is expected to be efficiently suppressed through a window cut on m γγ .
(B2) t(→ + νb)t(→ bjj)γγ -one of the jets in the hadronically decaying top is misidentified, and the other two are missed in the detector. The diphoton pair coming from h → γγ will have a small contribution but are included in the analysis for each ξ.
(B3) W + (→ + ν)γγjj -one jet is mis-tagged as a b-jet and the other missed in the detector. Again, the photon pair may result from Higgs decay. This background is included in the analysis since it was demonstrated in [44] and [95] to be at least an order of magnitude lower than (B1) and (B2) after a window cut on m γγ .
The effective Lagrangian in Eq. 2.1 was implemented by FeynRules [96] with SM parameters taken from [97]. The signal and background matrix elements were generated by MadGraph 5 package [98] with default parton level cuts, and convolved with the CTEQ6L parton distribution function [99] using default dynamical renormalisation (µ R ) and factorisation (µ F ) scale. Parton showering was subsequently performed by Pythia [100] and jets were clustered via anti-k t algorithm [101] with a cone radius of ∆R = 0.7. Detector simulation was carried out by Delphes [102] where the (mis-)tagging efficiencies and fake rates assume their default values.
The signal analysis (cf. Tab. 1) begins with the basic selection criteria (C1) on transverse momenta and rapidities, based on the trigger capabilities and detector coverage at the LHC. In Fig. 1, it is evident that the p T spectra of the leading (γ 1 ) and subleading (γ 2 ) photons in the SM signal are more energetic than that of the corresponding backgrounds. The thj signals with ξ = 0.25π and 0.5π also exhibit the same behaviour. Furthermore, the resonant production of diphoton pairs in the signals leads to a peak near m h /2, allowing them to be separated from the non-resonant diphoton pairs in ttγγ and tjγγ through the p γ 1 T > 50 GeV and p γ 2 T > 25 GeV cut (C2) in Tab. 1.
Cuts    The cut (C3) on the invariant mass of the leading b-jet and lepton have been discussed in [10,44,95]. Given that both the b-jet and lepton should originate from the same tquark, their invariant mass should be less than the top mass. As the leading b-jet in ttγγ may also come from the hadronically decaying top, it is not surprising that this cut reduced the ttγγ background by ∼ 1/4 whilst having minimal effect on the signals and tjγγ. Lastly, Fig. 2 shows that the scalar signal has a relatively narrow diphoton invariant mass peak after (C3). The authors have verified that ξ = 0.25π and 0.5π exhibit a similar distribution. The invariant mass window cut |m γγ − m h | < 5 GeV (C4) is found to be the most effective, removing ∼ 1/4 of the signal events but the backgrounds by a factor of at least 16. Despite an increased h → γγ contribution in ttγγ for ξ = 0.25π and 0.5π, due to the enhanced Higgs-diphoton decay rates, Tab. 1 shows that the full ttγγ cross-section remains relatively similar to the ξ = 0 case after (C4).
[GeV]  The significance, S/ √ S + B at the end of HL-LHC (3000 fb −1 ), is expected to be the highest for the pure pseudoscalar case (ξ = 0.5π), with a value of 7.71σ. However it was pointed out in [10] that to remain consistent with the current Higgs data, the value of y t /y SM t has to decrease to at least 0.6 (cf. Sec. 2). The corresponding signal significance is expected to drop to ∼ 3.7 and systematic significance to ∼ 0.82. Similar to the bb channel, the mixed and pure scalar scenario remains less optimistic for observation.

Top Polarisation and Lepton Spin Correlation
The angular distribution of the lepton from a polarised top quark is given by [94]: where κ is the lepton spin analysing power, θ is the angle between the lepton momenta and spin quantisation axis of the top, as measured in the rest frame of the t-quark, and P t is the spin asymmetry. In this study, the top spin axis is taken to be the direction of the top quark in the laboratory frame. In order to reconstruct the top rest frame, the neutrino momentum was first determined from the on-shell condition of the W -boson [103,104]: where A W = m 2 W + 2p T · E T . The sign ambiguity is resolved via minimisation of: where Γ t is the SM top decay width. The lepton angular distributions θ for the CP-phases     Fig. 3 for the parton level and reconstructed level after cuts (C4). It is evident that in the SM (ξ = 0), the preferential direction for the lepton momentum in the top rest frame is opposite to the top's boost. The pure pseudoscalar (ξ = 0.5π) interaction changes the polarisation of the top through a tth vertex, such that the lepton direction becomes positively correlated with the top's boost. As expected, the mixed interaction (ξ = 0.25π) gives a slope that is intermediate between the pure scalar and pseudoscalar cases. The difference between the slopes become less prominent in the reconstructed case, reflecting the simulated effects of parton showering, reconstruction efficiencies and detector resolution. The differences between the angular correlations are quantified in terms of the lepton forward-backward asymmetry: A F B := σ(cos θ > 0) − σ(cos θ < 0) σ(cos θ > 0) + σ(cos θ < 0) , (4.4) and the significance [105,106] by: with ∆ σ and σ T being the numerator and denominator of the right hand side of Eqn. 4.4 respectively. From Tab. 2, it is observed that the top-Higgs interaction can be distinguished via A F B with the SM case reaching a value of -27% and a significance of 0.52 whilst that for the pseudoscalar case, 3.4% with a significance of 0.20.

Conclusion
In this work, we investigated the observability of the pp → t(→ + ν b)h(→ γγ)j at 14 TeV HL-LHC. The detector resolution on m γγ allows effective suppression of QCD background via a mass window cut, compensating for its small diphoton branching ratio. In addition, non-zero ξ enhanced the pp → thj production cross section and the h → γγ branching ratio, allowing the CP-violating top-Higgs couplings to be probed with signal significances of 1.4σ, 2.7σ and 7.7σ for scalar (ξ = 0), mixed (ξ = 0.25π) and pseudoscalar (ξ = 0.5π) interactions respectively, when y t = y SM t . Even when combined bb channel, it is likely that the significance is not sufficient for observation of the SM signal. Furthermore, the diphoton channel led to measurable differences in lepton spin correlation by modifying the t-quark polarisation and can be distinguished via the forward-backward asymmetries.
where τ h,i := m 2 h /4m 2 i , Q f is the charge of fermion f in units of electric charge of positrons N C = 1(3) are the colour factors for leptons (quarks). The scaling function may be found, for example in [109]: where f (τ ) is in terms defined as: (A.7)