Constraints on top quark flavor changing neutral currents using diphoton events at the LHC

In this paper we show that the diphoton mass spectrum in proton-proton collisions at the LHC is sensitive to the top quark flavor changing neutral current in the vertices of $tu\gamma$ and $tc\gamma$. The diphoton mass spectrum measured by the CMS experiment at the LHC at a center-of-mass energy of 8 TeV and an integrated luminosity of 19.5 fb$^{-1}$ is used as an example to set limits on these FCNC couplings. It is also shown that the angular distribution of the diphotons is sensitive to anomalous $tu\gamma$ and $tc\gamma$ couplings and it is a powerful tool to probe any value of the branching fraction of top quark rare decay to an up-type quark plus a photon down to the order of $10^{-4}$. We also show that the $tu\gamma$ FCNC coupling has a significant contribution to the neutron electric dipole moment (EDM) and the upper bound on neutron EDM can be used to constrain the $tu\gamma$ FCNC coupling.


Introduction
The top quark with a mass of 173.34 ± 0.75 GeV [1] is the heaviest particle of the Standard Model (SM). With such a mass, the top quark has the largest Yukawa coupling to the Higgs boson and therefore measurement of its properties would provide a promising way to probe the electroweak symmetry breaking mechanism and new physics beyond the SM. New physics can show up either through direct production of new particles or indirectly via higher order effects. Observing indirect evidences is important as it provides hints to look for new physics before direct discovery. In the Standard Model (SM), the branching fractions of top quark rare decays t → qV , with q = u, c and V = γ, Z, g, are at the order of 10 −14 − 10 −12 [2]. Such branching fractions are extremely small and are out of the ability of the current and future collider experiments to be measured. Within the SM, such Flavor Changing Neutral Current (FCNC) transitions only occur at loop level and are strongly suppressed due to the Glashow-Iliopoulos-Maiani (GIM) mechanism [3]. On the other hand, it has been shown that several extensions of the SM are able to relax the GIM suppression of the top quark FCNC transitions due to additional loop diagrams mediated by new particles. Models, such as supersymmetry, two Higgs doublet models, predict significant enhancements for the FCNC top quark rare decays [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18]. As a result, the observation of any excess for these rare decays would be indicative of indirect effects of new physics. Many studies on searches for the top quark FCNC and other anomalous couplings have been already done [19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36].
In this paper, a direct search for the top quark FCNC interactions in the vertex of tqγ is discussed. Such interactions can be described in a model-independent way using the effective Lagrangian approach, which has the following form [37]: where the electric charges of the electron and top quark are denoted by e and eQ t , respectively and q ν is the four momentum of the involved photon, Λ is the cutoff of the effective theory, which is conventionally assumed to be equal to the top quark mass, unless we mention. In the FCNC Lagrangian in Eq.1, σ µν = 1 2 [γ µ , γ ν ] and the anomalous couplings strength is denoted by κ tqγ . Throughout this paper, no specific chirality is assumed for the tqγ FCNC couplings, i.e. λ v tqγ = 1 and λ a tqγ = 0. Within the SM framework, the values of κ tqγ , q = u, c, vanish at tree level. The leading order (LO) partial width of the top quark FCNC decay t → qγ, neglecting the masses of the up and charm quarks, has the following form [38]: and the LO width of t → bW + can be written as [38,39]: where α and V tb are the fine structure constant and the CKM matrix element, respectively. The sine of the Weinberg angle is denoted by s W and m t , m W are the top quark and W boson mass, respectively. The branching fraction of t → qγ is estimated as the ratio of Γ(t → qγ) to the width of t → bW + which takes the following form [38]: To obtain the above branching fraction, we set m t =172.5 GeV, α = 1/128.92, m W = 80.419 GeV and s 2 W = 0.234 in t → qγ and t → bW + widths. The tuγ and tcγ FCNC couplings have been studied in different experiments with no observation of any excess above the SM expectation up to now. In pp collisions at the Tevatron, the CDF experiment has set the following upper bounds on the branching fraction at the 95% confidence level (CL) [40]: This upper bound has been obtained using the study of the top quark decays in top quark pair production. Further searches for the anomalous tqγ couplings in electron-positron and electronproton colliders (LEP and HERA) have provided the following limits on the anomalous couplings at the 95% CL [41][42][43]: The ZEUS limit has been obtained under the assumption of m t = 175 GeV. The most stringent bounds on the tqγ FCNC interactions have been obtained by the CMS experiment at the LHC, using proton-proton collisions at a center-of-mass energy of 8 TeV, by studying the final state of single top quark production in association with a photon. The following upper bounds have been obtained on the anomalous couplings and the corresponding branching fractions at the 95% CL [44]: These limits has been obtained based on 19.1 fb −1 of integrated luminosity of data using only the muonic decay mode of the W boson in the top quark decay. All the above searches are based on final states containing at least a top quark. As the top quark has a short lifetime, it decays immediately (before hadronization). Therefore one has to reconstruct top quark from its decay products to be able to probe the tqγ couplings. This needs a careful attention to correctly select the final state objects, i.e. top quark decay products, and consider several sources of systematic uncertainties associated to each final state object in the detector. In this work, we propose instead to use diphoton events to probe the tqγ FCNC couplings which have less difficulties and challenges with respect to the events with top quarks in the final state.
The measurement of the diphoton invariant mass spectrum is one of the particular interests at the LHC as it is sensitive to several new physics models beyond the SM [45][46][47][48], being one of the most sensitive channels to the Higgs boson production at the LHC. On the other hand, the excellent mass resolution of the diphoton spectrum in the ATLAS and CMS detectors at the LHC provides the possibility for precise measurement of new signals above the SM expectation. Randall-Sundrum model [49] and large extra dimensions [50] are of the examples of the models which affect the diphoton differential cross sections. In this paper, we show that the presence of the FCNC anomalous coupling tqγ leads to significant change in the diphoton mass spectrum and the diphoton angular distribution. Using a mass spectrum measurement by the CMS experiment [45], we obtain bounds on the anomalous couplings κ tqγ . In addition it is shown that the diphoton angular distribution would be able to constrain the tqγ FCNC couplings strongly.
As an indirect way to probe the FCNC couplings, we calculate the effect of tuγ coupling to the neutron electric dipole moment (EDM) and show that the neutron EDM can receive significant contribution from the FCNC couplings. This paper is organized as follows. In Section 2, the details of the calculations and methods to constrain the tqγ FCNC couplings using the diphoton mass spectrum are presented. Section 3 is dedicated to present the the potential of the LHC to study the tqγ FCNC couplings using the angular distribution of the diphoton events. Finally, Section 4 concludes the paper. In the appendix A, using the upper bound on the neutron EDM, an upper limit on the anomalous tuγ is derived.

Diphoton: mass spectrum
In this section, we calculate the contribution of tqγ FCNC couplings to diphoton production at the LHC. Then, based on the measured diphoton mass spectrum by the CMS experiment [45], constraints on the anomalous couplings are derived.
Within the SM, the LO diphoton production proceeds through quark-antiquark annihilation. The tqγ FCNC couplings affect the diphoton production through the scattering of u, c,ū, andc quarks which proceed through t-channel as shown in Fig. 1. We calculate the leading order matrix element of diphoton production analytically for the Feynman diagrams shown in Fig.1. After averaging over the color and spin indices of the initial state partons and summing over the polarizations of the final state photons, the amplitude takes the following form: where for simplicity, we have assumed κ tuγ = κ tcγ = κ and s, t, u are the Mandelstam variables which can be written in terms of the scattering angle θ * in the center-of-mass frame as: t = − s 2 (1 − cos θ * ) and u = − s 2 (1 + cos θ * ). The first term in the above expression is the leading order amplitude describing the SM diphoton production, the second term is the interference between the SM and FCNC diagram, and the last term is the contribution of pure tqγ FCNC diphoton production. The interference term (SM+FCNC) is found to be constructive and the contribution of the third term, which is purely coming from FCNC, is smaller than the the interference term by a factor of ≈ 10 −3 . One of the characteristics of the LO SM is the enhancement of diphoton production at small angles as the production proceed through a t-channel virtual exchange.
In order to perform the signal simulation, the tqγ effective Lagrangian, Eq.1, is implemented into the FeynRules package [51] and then the model is exported to a UFO module [52] which is linked to MadGraph 5 [53,54]. Events are generated, describing the diphoton production at the LHC with the center-of-mass energy of √ s = 8 TeV. The LO parton distribution functions (PDFs) of CTEQ6L1 [55] are used as the input for the calculations and events generation. The renormalization and factorization scales are chosen to be equal and set to the default dynamic scales of the MadGraph generator. Pythia 8 [56] is used for parton showering and hadronization of the parton-level events. Finally, the detector-level effects are emulated by Delphes-3.3.2 package [57]. It includes a reasonable modeling of the CMS detector performances as described in [58].
In [45], the CMS collaboration has performed a search for diphoton resonances in high mass in proton-proton collisions at the center-of-mass energy of 8 TeV using an integrated luminosity of 19.5 fb −1 of data. The analysis searches for resonant diphoton production via gravitons in the Randall-Sundrum scenario with a warped extra dimension. According to the calculations presented above, the tqγ FCNC couplings affect the production of diphotons at the LHC. In this work, we follow the quite similar strategy to the CMS collaboration and use their result to probe the tqγ FCNC anomalous couplings.
In the CMS experiment analysis, two isolated photons with transverse energy (E T ) greater than 80 GeV within the pseudorapidity range of |η γ | < 1.4442, and with a diphoton-system invariant mass greater than 300 GeV are selected. In this region of the pseudorapidity, an excellent resolution for the photon energy is experimentally achieved. For the photons with E T ∼ 60 GeV and |η γ | < 1.4442, the energy resolution varies between 1% − 3% [59]. The used cuts for isolation and identification of the photons by the CMS collaboration lead to an efficiency of 86% for the photons with E T > 80 GeV and |η γ | < 1.4442. Small changes are seen in this efficiency when the E T and η of the photons change. In the current work, quite similar selection is employed for the analysis [45,46].
The background to the diphoton final state originates from SM diphoton production, γ+jet, and from dijet productions where one or two jets are misidentified as photons in the detector for the latter two background processes. Table 1 shows the number of observed events in data and the background prediction for several ranges of the diphoton mass spectrum [45]. The uncertainties presented in the Table 1 include both the statistical and systematic sources. The data and SM background expectation are found to be in agreement, considering the uncertainties on the predicted background and no significant excess over the SM background is found.
The values reported in Table 1 are used to probe tqγ anomalous couplings. As the measurement is compatible with the SM prediction, we set upper limit on the diphoton production cross section in the presence of anomalous couplings. Figure 2 shows the diphoton mass distribution at LO for the SM and SM+FCNC signal assuming κ tuγ = κ tcγ = κ = 0.1 obtained from the MadGraph simulation. The diphoton mass distribution at NNLO estimated based on the Monte Carlo program 2gNNLO [60, 61] is also shown in Fig.2 for comparison. 2gNNLO program calculates the production cross section of diphoton in hadron collisions to the accuracy of next-to-next-to-leading-order. As depicted, the presence of tqγ FCNC couplings lead to increase the diphoton cross section in the high invariant mass region. According to Table 1, the total number of observed data events above m γγ > 500 GeV is 333 events with the SM background prediction of 375. 8 ± 29.9 [45], where the SM diphoton production has been estimated based on the Monte Carlo program 2gNNLO. Assuming κ tuγ = κ tcγ = κ = 0.15, 99.4 ± 8.5 FCNC events are expected in this region for an integrated luminosity of 19.5 fb −1 of data. The uncertainty on the number of FCNC events includes the contributions coming from the choice PDFs, variations of renormalization and factorization scales, and the statistical uncertainty. The PDF uncertainty is obtained according to the PDF4LHC recommendation [62,63] using PDF sets CTEQ6L1 [55], NNPDF 3.0 [65], and MSTW 2008 [64]. The uncertainty originating from the variations of the scales has been estimated by varying the renormalization and factorization scales simultaneously by factors of 0.5 and 2.  We proceed to set an upper limit on the diphoton cross section in the presence of FCNC couplings. We compare the number of observed events in data and the expected events from SM in the region m γγ > 500 GeV. The limit at the 95% CL is set on the quantity σ s = (σ Total −σ SM )× A , where the whole diphoton production cross section (SM and FCNC signal) is denoted by σ Total and σ SM is the SM diphoton cross section. The FCNC signal acceptance is taken into account by the A term. The CL s technique [66] is used to calculate the upper limit on the cross section. An efficiency of 77.45% with an uncertainty of 10% is found for the FCNC signal. The observed and expected 95% CL upper limit on σ s are found to be 3.2 fb and 5.0 fb. The observed limit at the 95% CL and the FCNC signal cross section, σ s are shown in Fig. 3.  The 95% CL upper limit on the anomalous coupling parameter κ is the intersection of the observed limit on cross section with the theoretical cross section curve. The upper limit on σ s (3.2 fb) is corresponding to the upper limit of 0.153 on the anomalous coupling κ. This limit can be expressed to the upper limit on the branching fraction using Eq.4: The value obtained is comparable to the most stringent limits which has been obtained from the anomalous single top quark production in association with a photon by the CMS experiment (Eq.7) [44]. This provides a motivation for using this channel as a complementary technique for studying the tqγ FCNC interactions at the LHC experiments. A combination of this result with the results of other channels can lead to an improvement of the best limit.

Angular distribution of the diphoton system
In this section, we propose and use a diphoton angular variable to probe the tqγ anomalous couplings. In the SM, as the diphoton production proceeds through a t-channel exchange, the angular distribution peaks at cos θ * = 1, where θ * is the scattering angle in the center-of-mass frame of two partons. The scattering angle between two photons can also be expressed by the variable χ = e |ηγ 1 −ηγ 2 | . This variable has been used widely in searches for new physics such as searches for contact interactions, large extra dimensions, and excited quarks in dijet events in the Tevatron and LHC experiments [67][68][69][70]. It has been found that new phenomena affect this angular variable considerably and consequently is used to probe beyond SM. In order to produce the SM diphoton events, including the QCD next-to-leading order corrections and the contributions from the fragmentation processes, the Diphox (v 1.3.3) program [71] is used. The CT10 PDF set [72] is used as the input of the parton distribution functions and all the scales are set to m γγ . Figure 4 shows the distribution of the angular variable χ for the SM diphoton events with an invariant mass above 500 GeV and |η| < 1.442. The error bars represent the systematic uncertainties coming from the parton distribution functions and strong coupling constant α S . The shaded bars show the uncertainty from the theoretical scales in each bin of the angular distribution. The uncertainties arising from the scales variations are calculated by varying the factorization, renormalization, and fragmentation scales simultaneously by factors of 0.5 and 2. In each bin of the χ angular distribution, the uncertainty is calculated as the maximum difference between the new angular distributions and the distribution with the reference inputs. The uncertainty coming from limited knowledge on the choice of PDF is obtained using PDF4LHC recommendation [62,63]. The PDF sets CT10 [72], MSTW08 [64], and NNPDF 3.0 [65] are used to estimate the uncertainty on from PDFs and strong coupling constant α S is varied by 0.012 similar to the prescription adopted in [73].  The left plot in Fig. 5 shows the distributions of χ = e |ηγ 1 −ηγ 2 | as a function of the anomalous coupling κ. The distribution is normalized to unity since the sensitivity to FCNC couplings affects the angular distribution rather than normalization. This figure depicts the predicted SM distribution as well as the SM+FCNC with κ = 0.2 and 0.5. These distributions are after all selection cuts described previously requiring in addition that m γγ > 500 GeV. As seen, the presence of the anomalous FCNC couplings of tqγ changes the shape of the angular variable χ. Increasing the value of the anomalous coupling κ causes more events to be concentrated at small values of χ. It is notable that due to the detector acceptance cut applied on the photon pseudorapidity, χ varies from 0 to e 2×1.442 = 17.96.   In order to quantify the difference in the shape of χ for SM and FCNC, a ratio is defined as: where χ 0 is an arbitrary cut which is chosen in such a way that the best sensitivity to the FCNC couplings is achieved. The right plot in Fig. 5 shows the behavior of R versus the anomalous coupling κ for different choices of χ 0 . As the normalized distribution of χ depends on the cut on the diphoton invariant mass, the value of R varies with the cut on m γγ . Figure 6 shows the behavior of R for the SM and SM+FCNC for the χ 0 = 8 choice and different cuts on the minimum m γγ . The uncertainties in this plot includes both the statistical and theoretical uncertainties for the SM and only the statistical uncertainty for the SM+FCNC. The cut on m γγ can be chosen to optimize the expected sensitivity to κ. We define the statistical significance of the observable R as: where R SM χ 0 and R FCNC+SM χ 0 are the values of the ratio defined in Eq.10 with a choice of χ 0 for the SM and for the case of the presence of FCNC. The uncertainty on R SM χ 0 is denoted by ∆R SM χ 0 . Considering the theoretical and statistical uncertainties in the region of m γγ > 500, the value of χ 0 = 8 is found to provide the best sensitivity. The upper bounds at the 68% CL and at the 95% CL on the FCNC anomalous couplings including only statistical uncertainties are found to be: 68% CL : κ < 2.75 × 10 −2 corresponding to Br(t → qγ) < 1.56 × 10 −4 , 95% CL : κ < 3.91 × 10 −2 corresponding to Br(t → qγ) < 3.15 × 10 −4 , (12) and the 68% and 95% CL limits after including both the statistical and systematic uncertainties are: 68% CL : κ < 4.46 × 10 −2 corresponding to Br(t → qγ) < 4.10 × 10 −4 , 95% CL : κ < 6.26 × 10 −2 corresponding to Br(t → qγ) < 8.06 × 10 −4 .
From these results it can be concluded that the angular variable χ is able to provide additional sensitivity to tqγ anomalous coupling with respect to the diphoton mass spectrum. Further optimization on both m γγ and χ 0 is expected to improve the sensitivity to possible tqγ contribution to diphoton production at the LHC.

Summary and conclusions
Rare top quark decays through flavor changing neutral currents in the vertices of tqγ, tqZ, and tqg are particularly interesting as they are significantly sensitive to many extensions of the SM. The SM predictions for the branching ratios of these rare decay modes are expected to be unobservable at the LHC ( < 10 −12 ) while new physics models are able to enhance the branching fractions by several order of magnitudes. As a consequence, any observation of such processes would indicate new physics beyond the SM. In this paper, we propose a new indirect way to search for the tuγ and tcγ FCNC interactions. So far, these couplings have been directly studied by CDF, DELPHI, H1, ZEUS, and CMS experiments at colliders with at least a top quark in the final state of the collisions. In this work, we propose to use the diphoton invariant mass and angular differential distributions to probe tqγ FCNC couplings. Using a measured mass spectrum of diphoton at the LHC with the CMS experiment, an upper limit of 4.81 × 10 −3 is set on the branching fraction of t → qγ. Furthermore, we show that the angular variable χ = e |ηγ 1 −η γ 2 | would allow us to probe this branching fraction down to 8.2 × 10 −4 . These limits have been obtained based on the LO prediction of the tqγ FCNC contribution in diphoton productions at the LHC and are comparable with the ones recently obtained from the search for anomalous single top events by the CMS experiment.

A Electric dipole moment analysis
In this section, we obtain upper limit on the tuγ FCNC coupling using the present upper bound on the neutron electric dipole moment. Such an approach has been used in constraining the W boson electric dipole moment [74], top-Higgs non-standard interactions [35], and probing heavy charged gauge boson mass and couplings [36].
We calculate the contribution of the tuγ coupling to the neutron EDM using the effective interaction for the qqγ vertex. The most general effective vertex describing the interaction of a photon with two on-shell quarks can be written as [37]: where q is the four-momentum of the off-shell photon. The functions F 1v (q 2 ) and F 2v,2a (q 2 ) are called form factors which in the low energy limit q 2 → 0, they are physical parameters and be related to the static physical quantities according to the following relations: where Q q is the electric charge of a quark q, a q and q q are the magnetic dipole moment and electric dipole moment of a quark. The electric dipole moment term violates the P and CP invariance. Within the SM at tree level, d q and a q are zero and however non-zero values for d q and a q arise from higher order corrections. The SM prediction for the electric dipole moments of the quarks are extremely small and expected to be smaller 10 −30 e.cm [75][76][77][78].
Using the interactions described by Eq.14 at low energy, the tuγ FCNC coupling introduced by Eq.1, the induced CP violating amplitude coming from the tuγ FCNC interaction in Fig. 7 can be expressed as: After employing Dirac equation and Gordon identity, the above expression can be simplified to find the up quark EDM arising from tuγ FCNC coupling, which is the coefficient of σ µν q ν γ 5 . This integral on k is divergent, therefore a mass scale of Λ cut is introduced as an ultraviolet cutoff scale. After performing some algebraic manipulations and integration over k, the amplitude is found to be: [6xu(x + y)(x + y − 1) + 3(x + y) + xu − 1] + xu(x + y) 2(1 + xu(x + y − 1)) }, As x u ∼ 10 −5 , we take the limit of the amplitude for the case of x u → 0. We find the following form for the amplitude: Employing the effective coupling for uūγ coupling, we find the tuγ contribution to the up quark EDM: As seen, there are quadratic and logarithmic divergences to the up quark EDM from tuγ FCNC. However, there is a factor Λ 2 in the denominator which comes form the tuγ effective coupling and is the scale at which new physics effects is expected to appear. It is natural to assume that the cutoff scale of the loop divergences (Λ cut ) is equal to Λ which is the scale at which new physics effects are expected to show up, i.e. Λ cut = Λ. Using the non-relativistic SU(6) wave functions, the neutron EDM can be related to the up and down quark EDMs. The neutron EDM in terms of EDMs of quarks is written as [78]: where the up and down quarks EDMs are denoted by d u and d d and η describes the QCD higher order corrections and is equal to 0.61. The current experimental bound on the neutron EDM is d n < 2.9 × 10 −26 e.cm. [79,80]. The measured upper limit on the top quark EDM has been found to be d t < 10 −16 e.cm [81]. For example, by setting d t = 10 −16 , Λ = m t the upper bound of 0.0026 on κ tuγ is obtained. This is corresponding to an upper limit of the order of 10 −6 on Br(t → uγ).