Probing New Physics in Higgs Couplings to Fermions using an Angular Analysis

The standard-model Higgs boson couples to quarks through a parity-even scalar $Hq{\bar q}$ coupling. We show that the rare Higgs decay $H\to VZ$, where $V$ is a vector quarkonium state such as $J/\psi$ ($c{\bar c}$) or $\Upsilon(1S)$ ($b{\bar b}$), can be used to search for the presence of a parity-odd pseudoscalar $Hq{\bar q}$ coupling. Since both $V$ and $Z$ can decay to a pair of charged leptons, this presents an experimentally-clean channel that can be observed at the high-luminosity LHC or a future hadron collider. The P-even and P-odd $Hq{\bar q}$ couplings can be measured by analyzing the angular distribution of the final-state leptons.


Introduction
There is no doubt that the scalar particle of mass 126 GeV recently discovered at the LHC by the ATLAS and CMS Collaborations [1,2] is the Higgs boson. However, a crucial question remains: is this the Higgs boson of the standard model (SM), or do its properties indicate the presence of new physics (NP)? To this end, there have been numerous studies of the Higgs couplings to SM particles.
At tree level, the Higgs can decay to the W W * and ZZ * final states. These decays have been measured, with the result that the couplings of the scalar agree well with the theoretical predictions for the SM Higgs boson [3,4]. The ZZ * state is observed through its decay to four leptons, and as such it presents a clean measurement channel. The H → 4l decay process has therefore been used in several papers as a "golden channel" to look for NP [5][6][7][8]. At the loop level, the Higgs can also decay to γγ and Zγ. These decays can potentially probe the Higgs coupling to the top quark [9]. The measured rate for H → γγ agrees reasonably well with the SM prediction [3,4].
Although measuring the Higgs couplings in the bosonic decay channels takes priority, directly measuring its couplings in fermionic modes is also important. Indeed, one of the goals of the future LHC program is to precisely measure the Higgs' couplings to all SM fermions. However, this is challenging. Since the top quark is heavy, the favorable modes for measuring the Higgs' coupling to top quarks involve Higgs production in association with tt, a single t, or a singlet [10]. H → bb and H → τ + τ − [3] have been observed at the LHC, but a precise measurement of the Hbb and Hτ + τ − couplings will require further investigation. A direct measurement of Higgs' couplings to the first two generations of quarks is currently out of reach of experiments, though a search for H → µ + µ − was recently reported [11].
It seems clear that, in order to see evidence of NP in fermionic decay modes of the Higgs, a significant improvement in sensitivity is needed. One potential way of improving the sensitivity to NP is to study experimentally-clean modes such as those in which the final state includes leptons. Such modes are rare but often free from backgrounds. One possibility is the decay H → l + l − γ, where l represents an electron or a muon. This has been examined in Ref. [12]. Although this decay channel is extremely rare due to the small SM Hl + l − coupling, it can receive a significant contribution from the resonant production of a vector quarkonium state V , in which the V decays to an l + l − pair. Examples of such a state are J/ψ (cc) or Υ(1S) (bb). In this case, the decay H → V γ proceeds through the Hqq coupling. Higgs Yukawa couplings to the first-and second-generation quarks can also be probed through rare Higgs decays in which the final state consists of a QCD vector meson and an electroweak gauge boson. These channels were studied in Ref. [13] and deemed promising for observation at the high-luminosity LHC and future hadron colliders.
The direct decay H → qqγ, also known as the inverse Wilczek process [14], has been studied in Refs. [12,15]. Ref. [12] also considers the indirect decay process H → γγ * , in which the excited photon γ * then decays to a quarkonium state. The conclusion is that the interference between the direct and indirect processes sufficiently enhances the rate that H → V γ can be observed at the high-luminosity LHC. In principle, the study of this process will allow us to probe the NP properties of the Hqq coupling.
In the SM, the Hqq coupling c S is purely scalar (parity-even). In NP models, a parity-odd pseudoscalar coupling c P can be generated. The Higgs decays should therefore be studied with the aim of detecting the presence of c P . Now, the interference of the SM scalar and NP pseudoscalar couplings will lead to P-odd observables. The examination of such observables will give information about c P , in particular whether it is nonzero. However, this poses a problem. In H → V γ, the P-odd observable is the triple product where q is the difference between the momenta of the V and γ in the rest frame of the H, and ε * V and ε * γ are the polarizations of V and γ, respectively. But while ε * V can be measured by studying the momenta of the leptons in V → l + l − , ε * γ cannot be measured since the photon does not decay. Thus, the process H → V γ cannot be used to obtain information about c P .
Still, this also indicates how to resolve the problem. The photon in H → V γ must be replaced by a vector that does decay, so that its polarization can be measured. The most obvious process is H → V Z, with Z → l + l − . There are other possibilities, but this decay has the largest rate and is easiest to observe experimentally, due to the leptons in the final state. The process H → J/ψZ was studied in Ref. [17]. (Note also that nonstandard Hqq couplings can enhance the decay rate [16,18].) In this Letter, we show how to test for the presence of a nonzero c P by studying the angular distribution of H → V Z.
In Sec. 2, we introduce the P-odd Hqq coupling, and examine how it can arise in NP models. The matrix elements for H → V γ and H → V Z in terms of helicity amplitudes are discussed in Sec. 3. In Sec. 4, we show how to separate and measure the P-even and P-odd Hqq coupling from the helicity amplitudes using an angular analysis (full details are given in the Appendix). We conclude in Sec. 5.

Hqq Coupling
We write the Hqq coupling in the following form: Here c S and c P represent, respectively, the P-even scalar and P-odd pseudoscalar couplings of the Higgs to a pair of quarks. In the SM the coupling is purely scalar, so that c S = 1 and c P = 0.
Nonstandard Higgs couplings to fermions can arise in many theories beyond the SM. These couplings can be modified compared to the SM through mixing effects, when the SM Higgs boson mixes with other scalars, or through NP corrections to the Higgs-fermion vertex [18]. Higgs mixing effects are less interesting for our purposes as they can be first probed in Higgs decays to gauge bosons.
Modifications of the Higgs Yukawa couplings to fermions arising from dimension-six operators in an effective field-theory framework have been studied in several papers [19][20][21]. Below we focus on the up-type quark sector, but a similar analysis holds for down-type quarks. The relevant operators are Here the first term is the up-type Yukawa operator of the SM, while the second term is a dimension-six operator suppressed by the NP scale Λ. Q i and U i (i = 1, 2, 3) are, respectively, the left-handed quark doublets and right-handed quark singlets; H is the Higgs doublet, with H = iσ 2 H * . λ u and g u are generic complex 3 × 3 matrices in flavor space. Setting the Higgs field to its vacuum expectation value, we have H = (0, (v + h)/ √ 2) T . The mass and linear Higgs coupling matrices are then Here S u ij and A u ij are the scalar (Q i U j ) and pseudoscalar (Q i γ 5 U j ) couplings of the Higgs. In general, as we go from the gauge basis to the mass basis by diagonalizing λ u ij , flavor-changing neutral-current (FCNC) couplings of the Higgs will be generated. As in Ref. [21], we assume that λ u ij and g u ij are aligned so as to avoid FCNC; such an assumption can be justified in certain scenarios [22]. We further assume that the only significant corrections occur for the charm-quark couplings to the Higgs (or for the bottom-quark couplings to the Higgs in the down-type quark sector).
In Ref. [21] several different theoretical frameworks are considered that can lead to an Hqq coupling significantly larger than in the SM. These include a two-Higgs-doublet model with minimal flavor violation (MFV) [23][24][25][26], a general MFV [27] scenario with only one Higgs doublet, and composite models in which the Higgs field is realized as a pseudo-Nambu-Goldstone boson (pNGB). In the composite pNGB Higgs models, modifications of the Higgs couplings to up-type quarks are parametrized by the effective Lagrangian in Eq. (2), with Λ replaced by the global symmetry-breaking scale f , the "decay constant" of the pNGB Higgs [28,29]. Corrections to the Hcc coupling are considered in this framework in Ref. [30], and it is found that, for a fully-composite charm quark, a large enhancement of the coupling is possible. There are interesting attempts to understand the small light-quark masses in terms of suppressions from higher-dimensional operators constructed from the Higgs field [16,31]. These models can lead to large modifications of the Higgs couplings to the light fermions.
Finally, we note that the P-odd pseudoscalar coupling c P can be constrained from low-energy bounds on electric dipole moments under certain assumptions, but the constraints for the charm and bottom quark couplings are quite weak at present [32].
In H → V γ, the vector meson V is a qq pair. The Feynman diagrams for this decay are shown in Fig. 1. At tree level, the production of V γ involves the Hqq coupling. This is shown in the left-hand diagram of Fig. 1. At loop level, the vector can be produced from the decay of an off-shell neutral gauge boson γ * or Z * . However, the loop-level diagram shown on the right-hand side of Fig. 1 couples only indirectly to the quarks in V . As such, it does not give rise to a P-odd term in the amplitude. We therefore focus primarily on the tree-level diagram.
We begin by calculating the tree-level amplitude for H → qqγ. The final-state quark and antiquark then need to be dressed so that they form the vector quarkonium state V , where the relative motion between the q andq within V is small compared to the large momentum of V itself. This calculation can be done within the framework of non-relativistic QCD (NRQCD) [33,34] by expanding in powers of the small relative velocity. For our purposes, we stick to the leading order result in NRQCD where one neglects any relative motion between the quark and the antiquark, so that the tree-level invariant matrix element for H → V γ can be written as where ε * γ(V ) is the polarization of the photon (V ), k and p V are the four-momenta of the photon and V , respectively, and φ 0 is the wave function of the qq state at zero threemomentum. Since we neglect the relative motion of the quark and the antiquark in V , φ 0 can be considered real. Its magnitude can be measured directly in experiments from the quarkonium decay to a pair of leptons using the decay-rate formula Subleading NRQCD corrections give rise to a tiny phase in φ 0 [12], and also modify the coefficient of each term in Eq. (4). Ref. [35] contains a detailed discussion of the NRQCD corrections to the H → V γ amplitude. In our first attempt to probe new physics in this decay, we neglect the subleading contributions.
Equation (4) can be written in a more familiar form by going to the rest frame of the V . We can then define ε In the linear polarization basis, also known as the transversity basis, we have where Note that H ⊥ is proportional to c P , so that it can arise only if the pseudoscalar Hqq coupling is nonzero.
There are several things to notice about Eq. (6). First, there is no term involving the longitudinal polarization. This is because the final-state photon is on shell, and a massless particle has no longitudinal polarization. Second, the only P-odd observable in |M| 2 is the triple product (TP)k · ( ε * T V × ε * γ ). It arises due to the interference between the H and H ⊥ terms, and is proportional to c S c P . Third, and most important, the measurement of a nonzero value for the TP would indicate the presence of a NP pseudoscalar Hqq coupling c P .
However, this requires knowledge of the photon polarization ε * γ . Unfortunately, given that the photon does not decay, ε * γ cannot be determined. The upshot is that H → V γ cannot be used to extract information about c P .

H → V Z
The problem with H → V γ can be remedied by replacing the photon with a vector that does decay, so that its polarization can be measured. This naturally leads us to examine H → V Z. However, unlike the photon, the Z can couple to the Higgs at tree level. Thus, there is an additional tree-level contribution to this process, as shown in the middle diagram of Figure 2. Since this diagram also contributes to the indirect coupling of the Higgs to the quarks in V , just like the loop-level indirect-coupling diagram on the right, it does not generate a P-odd term in the H → V Z decay amplitude. In what follows we once again focus only on the direct-coupling diagram.
As before, we can write down the leading-order NRQCD tree-level invariant matrix element for the direct decay as where c V is the vector Zqq coupling in the SM. For up-type quarks c V = 1 − (8/3) sin 2 θ W , while for down-type quarks c V = 1 + (4/3) sin 2 θ W . Note that there is also an axial-vector Zqq coupling. However, its contribution to the matrix element for H → V Z vanishes to leading order in NRQCD. Once again, in the rest frame of the V , Eq. (8) takes a more familiar form. Letk represent the direction of the Z in this frame. With respect tok we can now define longitudinal and transverse components of both the V and Z polarizations. In the linear polarization (transversity) basis, we have where As in Eq. (7), H ⊥ is nonzero only if c P is nonzero.
As was the case for H → V γ, we can see from Eq. (9) that there is a P-odd TPk·( ε * T V × ε * T Z ) in |M| 2 due to the interference of the H ⊥ term with the H 0 or H terms. In this case, the TP is measurable since ε * T Z can be found by studying the decay products of the Z. Since H ⊥ is proportional to c P , the nonzero measurement of the TP is a clear signal of a nonzero c P .
Let us examine the helicity amplitudes in more detail. Consider the decay H → J/ψZ, which can be used to probe the direct coupling of the Higgs to cc. Using m H = 125 GeV, m Z = 91.2 GeV, and m J/ψ = 3.097 GeV, we find Thus, for the direct-coupling diagram, we see that the longitudinal piece of the amplitude, H 0 , is much smaller than a transverse piece, H . However, the magnitudes of the two transverse components, H and H ⊥ , can be comparable to one another if c P and c S are of a similar size. In addition, the indirect decay amplitude arising from the middle and right-hand diagrams in Fig. 2 can contribute to H 0 and H . These contributions have been evaluated in [36,37], and their effect is generally to increase the magnitudes of both H 0 and H . However, these diagrams do not contribute to H ⊥ and hence leave its structure unchanged.
The most complete study of H → V Z involves an angular analysis, which permits the extraction of H 0 , H and H ⊥ . This is discussed in the following section.

H → V Z: Angular Analysis
In the Higgs rest frame, the V and the Z are back to back. Since the Higgs is spinless, this decay distribution is isotropic. However, the angular information obtained in H → V Z when the V and Z each decay to a pair of leptons is sensitive to the helicity amplitudes H 0 , H and H ⊥ . The analysis of the decay of a scalar particle to a pair of vectors that subsequently decay to leptons has been studied in the context of meson decays in Refs. [38][39][40][41]. Here we apply this technique to the decay H → V Z.
In the rest frame of the decaying Higgs, we choose our coordinates such that the decay is along the z-axis. The subsequent decays of the V and the Z, each into a pair of leptons, can be characterized in terms of three angles: the polar angles θ V (Z) corresponding to the V (Z) → l + l − decay axes in the V (Z) rest frames, and the azimuthal angle φ between the two directions. Using the results of the Appendix, the differential decay rate for H → V Z can be written as a function of φ as follows: where Since Y is linear in H ⊥ , it is proportional to the P-odd coupling c P . Thus, the measurement of a nonzero Y gives a clear signal of NP in Higgs decays.
The advantage of performing an angular analysis with only φ is that it does not require the high statistics needed to perform a complete angular analysis. Y can be simply extracted in experiments as follows: Similarly, one can extract X using a different asymmetric integral over φ. Furthermore, since there are only two unknowns, c S and c P , the simultaneous measurement of X and Y allows one to obtain both c S and c P . Note that this holds even in the case that H receives a significant contribution from the indirect coupling of the Higgs to quarks via intermediate gauge bosons. Since information about such couplings is available entirely from the Higgs decay to gauge bosons, effectively c S and c P are still the only unknown parameters.
Alternatively, one can use the full angular distribution for H → (l + l − ) V (l + l − ) Z as a function of θ V , θ Z and φ to extract H 0 , H and H ⊥ . The derivation of the differential decay rate for H → V Z is given in the Appendix. The result presented there is model-independent, and simply describes a Higgs decay to a pair of spin-one particles, each of which subsequently decays to a pair of (massless) leptons. Combining the results in the Appendix with those in Sec. 2, we find where the W ′ s and Y ′ s, which are functions of θ V , θ Z and φ, are listed in Eq. (27). Asymmetric angular integrations over θ V , θ Z and φ can be used to separate the coefficients of the angular functions. The individual helicity amplitudes H 0 , H , and H ⊥ can then be obtained from a combined fit to these extracted coefficients. Since the three helicity amplitudes are functions of only two unknowns, c S and c P , one can solve for these unknowns, but with a certain redundancy. This shows that the full angular analysis provides additional cross checks for the validity of this formalism. Finally, we note that the angular analysis presented in this section is similar to that used to study H → Zl + l − in Refs. [5,7]. These papers consider the general distribution for H → Zl + l − , which can in principle include the contribution from H → V Z, with the V decaying to the lepton pair. However, in the SM the dominant contribution to this decay comes at tree level from H → ZZ * , with the off-shell Z * decaying to the lepton pair. Because angular momentum is conserved in both H → (l + l − ) V (l + l − ) Z and H → (l + l − ) Z (l + l − ) Z * , the expressions for the angular distributions for both processes are similar [8]. On the other hand, while the study of H → (l + l − ) Z (l + l − ) Z * sheds light on the coupling of the Higgs to gauge bosons, our primary objective is to study the couplings of the Higgs to fermions. As explained earlier, a decay distribution of H → (l + l − ) V (l + l − ) Z that is asymmetric in the azimuthal angle φ can arise only due to a P-odd direct coupling of the Higgs to the quarks in V .

Conclusions
Several new-physics scenarios suggest that the Higgs boson can couple to quarks through a dimension-six operator which is odd under parity. In this Letter, we discuss the consequences of such a P-odd pseudoscalar coupling on the decay processes H → V γ and H → V Z, in particular through triple-product (TP) correlations. Although the pseudoscalar Hqq coupling gives rise to a TP in H → V γ, it is not possible to retrieve information about the TP since the photon polarization cannot be measured. We show that this problem can be remedied by studying H → V Z, in which both V and Z decay to a pair of leptons. The dependence of the decay rate on the azimuthal angle between the planes of leptons from the two decays can be used to separate the P-even and P-odd couplings of the Higgs to the quarks.
Here λ X represents the helicity of the particle X, D J M,M ′ represents the Wigner D functions, which are the matrix elements of the rotation operator between eigenstates of angular momentum, and H represents the matrix elements for the Higgs decay. Although H depends on the helicities of both vectors, angular-momentum conservation requires that the two vectors have the same helicity. The helicity of a massive particle can take the values 0, ±1. Since the initial particle is spinless, its decay amplitude is spherically symmetric, justifying our arbitrary choice of coordinate axes.
We now allow the vector V and the Z to decay, each to an l + l − pair. The subsequent decay axis for the V decay can be characterized by a polar angle θ V with respect to the spin-quantization axis of the vector, chosen to be along the z-axis. The decay axis for the Z decay can be parametrized by a second polar angle (θ Z ) and an azimuthal angle (φ). The helicities of the final-state leptons can take the values ± 1 2 . However, the vector and axialvector currents in the electromagnetic and weak interactions of the SM require that the pair of massless leptons have opposite helicities. Thus, without loss of generality, the final-state helicity can be represented by ∆λ l = λ l − −λ l + , which can take the values ±1. The amplitude for the decay H → (l + l − ) V (l + l − ) Z depends on the leptonic helicity differences for the two pairs of final-state leptons (∆λ l and ∆λ ′ l ), and can be written as The angular distribution for H → (l + l − ) V (l + l − ) Z can be expressed as where From the above, X λλ ′ is symmetric under the exchange of λ and λ ′ . λ and λ ′ can each take the values 0 and ±1. Thus, we can write the six components as follows: The above result is completely general. However, it simplifies when we take into account certain properties of the V and Z decays. The decay to V → l + l − is electromagnetic. Since the electromagnetic interaction preserves parity, |V (V ) On the other hand, the amplitude for Z → l + l − can be written as, where c ′ V = 4 sin 2 θ W − 1 and c ′ A = 1. This implies that |V Using these, we can simplify our earlier results. For the V we find where N V = |V (V ) where N Z = |V (Z) − | 2 . Thus the angular distribution for H → (l + l − ) V (l + l − ) Z of Eq. (18) can be expressed as where Ω ++ (θ V , θ Z ) = 1 16 (1 + cos 2 θ V ) (1 + cos Ω 00 (θ V , θ Z ) = 1 4 sin 2 θ V sin 2 θ Z , Finally, it is also interesting to express all our results in the transversity basis defined by H = (H + + H − )/ √ 2 and H ⊥ = (H + − H − )/ √ 2. In this basis, we can rewrite Eq. (24) as dΓ d cos θ V d cos θ Z dφ = |H 0 | 2 W 00 (θ V , θ Z , φ) + |H | 2 W (θ V , θ Z , φ) + |H ⊥ | 2 W ⊥⊥ (θ V , θ Z , φ) where W 00 (θ V , θ Z , φ) = 1 4 sin 2 θ V sin 2 θ Z , W (θ V , θ Z , φ) = 1 16 (1 + cos 2 θ V )(1 + cos 2 θ Z ) + cos 2φ sin 2 θ V sin 2 θ Z , W ⊥⊥ (θ V , θ Z , φ) = 1 16 (1 + cos 2 θ V )(1 + cos 2 θ Z ) − cos 2φ sin 2 θ V sin 2 θ Z , (1 + cos 2 θ V ) cos θ Z ,