Sensitivity of anomalous quartic gauge couplings via tri-photon production at FCC-hh

A direct investigation of the self-couplings of gauge bosons, completely described by the non-Abelian gauge symmetry of the Standard Model, is extremely valuable in understanding the gauge structure of the SM. Any deviation from the SM predictions on gauge boson self-coupling is to give a hint at the existence of a new physics beyond the SM, which is defined with a modification of the self-interactions using an effective field theory approach. In this paper, we present a detailed Monte Carlo study searching for anomalous quartic gauge dimension-8 couplings related to $\gamma\gamma\gamma\gamma$ and $\gamma\gamma\gamma Z$ vertices at the future hadron-hadron collider (FCC-hh) via tri-photon production at a 100 TeV center of mass energy with an integrated luminosity L$_{int}$=30 ab$^{-1}$. Events that have been parton showered and include detector effects are analyzed with a Toolkit for Multivariate Data Analysis (TMVA) using a boosted decision tree to help distinguish between signal and background events to achieve the best sensitivities on anomalous quartic gauge couplings. Our obtained results reveal that the limits on anomalous quartic gauge couplings $f_{T8}/\Lambda^{4}$ and $f_{T9}/\Lambda^{4}$ at 95\% C.L. without systematic errors are about three orders of magnitude stronger compared to the best current experimental limits reported by the ATLAS collaboration at the LHC. Considering a realistic systematic uncertainty such as 10\% from possible experimental sources, our obtained limits of anomalous quartic couplings get worse by about one order of magnitude compared to those without systematic uncertainty but are still two orders of magnitude better than those recently reported by ATLAS.


I. INTRODUCTION
The self-interactions of gauge bosons are defined by the non-Abelian structure of the electroweak (EW) sector within the framework of the SM.Therefore, precise measurements of their couplings play a key role in exploring new physics beyond the Standard Model (SM) as well as shedding light on the nature of the EW symmetry-breaking mechanism.Since there have been no deviations observed in the SM prediction of gauge boson self-couplings from the LHC data available so far, these couplings are used to search for indirect signs of new physics with the Effective Field Theory (EFT) approach as a model-independent tool.In this approach, new particles cannot be directly produced, and new physics emerges only as new interactions between known SM particles supplemented by operators higher than dimension-4 under the condition that the energy scale of the new physics is much higher than the center of mass energy [1,2].Assuming the Triple Gauge boson Couplings (TGC) and Higgs couplings are constrained by other processes with better experimental accuracy, such as di-boson and Higgs on-shell measurements, the EFT of new physics which modifies interactions between electroweak gauge bosons, includes 18 different dimension-8 operators [3,4].
Given that the precision of the aQGCs scales with the center of mass energy of the colliders, multi-TeV energy colliders become an ideal platform for studying the self-interactions of gauge bosons.One of the multi-TeV collider projects currently under consideration by CERN is the Future Circular Collider (FCC) facility, which would be planned to build in a 100 km tunnel and designed to deliver pp, e + e − and ep collisions [45].After completing the missions of the LHC and High-Luminosity LHC (HL-LHC) physics programmes, the FCC facility, which has the potential to search for a wide parameter range of new physics, will come to the fore as an energy frontier collider project.One of the FCC options, the FCC-hh, is designed to provide proton-proton collisions at the proposed 100 TeV centre-of-mass energy with an integrated luminosity of 30 ab −1 [46].
In this study, we investigate the sensitivity of dimension-8 effective operators related to the anomalous quartic γγγγ and γγγZ vertices through the tri-photon process at the FCC-hh collider.
Realistic detector effects are included in the production of the signal and background processes.
The Toolkit for Multivariate Data Analysis (TMVA) [51,52] and its implementation of the boosted decision trees (BDTs) method, one of the widely used and rapidly developed machine learning methods in particle physics, is used to improve the separation between signal and relevant backgrounds.After obtaining the sensitivities on aQGC couplings, we compare them with the current experimental limits and comment on the numerical results.

II. EFFECTIVE DIMENSION-8 INTERACTIONS FOR AQGC
aQGCs can be constructed by either linear or non-linear representations using an EFT framework.In the linear representation, the electroweak symmetry can be broken by the conventional SM Higgs mechanism while in the non-linear representation, the Higgs couplings are treated as additional free parameters.Since no deviations from the light SM Higgs predictions have been observed so far, it is more preferable to study an effective Lagrangian in a model-independent way for the anomalous quartic couplings, assuming that new physics beyond the SM still keeps SU (2) L × U (1) gauge invariance in linear representation [4,27,47,49].In order to construct a linearly parameterized effective Lagrangian for aQGC in the EFT framework, we start with the covariant derivative acting on the Higgs doublet field; This covariant derivative is normalized such that where W µν = W j µν σ j 2 and B µν = B µν are field strength tensors of the SU (2) and U (1) with following definition and τ i (i = 1, 2, 3) represent Pauli matrices.
The dimension-8 operators are separated into three classes: i) covariant derivatives of Higgs doublet only (O S,j ), ii) two field strength tensors and two derivatives of Higgs doublet (O M,j ) and iii) field strength tensors only (O T,j ).Therefore, the Lagrangian of the dimension-8 operators contributing to aQGCs can be written as where Λ is the scale of new physics, and f S,j , f M,j and f T,j represent coefficients of relevant effective operators.These coefficients are zero in the SM prediction.The explicit forms of 18 dimension-8 quartic gauge operators are given in Refs.[4,[47][48][49]: A complete list of corresponding quartic gauge boson vertices modified by dimension-8 operators is given in Table I.Due to phase space characterisation of our interested signal process pp → γγγ, the best sensitivity for anomalous quartic couplings of the γγγZ and γγγγ vertices is attainable.
Since only field strength tensor type operators (O T,x ) receive contributions to the γγγZ and γγγγ quartic vertices, we consider O T,8 and O T,9 couplings to represent all types of O T,x operators.
The explicit expression of the anomalous quartic γγγZ and γγγγ vertices as a function of where sine (cosine) of the weak mixing angle (θ W ) denotes by s W (c W ), G(f T 8 ) and G(f T 9 ) are defined as follows; The best 95% C.L. experimental limits on f T 8 /Λ 4 and f T 9 /Λ 4 , the anomalous quartic gauge Since the amplitude predicted by higher-dimensional operators will eventually violate unitarity with the increase at the center of mass energy (called the unitarity bound), it becomes even more important to avoid non-physical contributions from unitarity-violated regions.One of the methods for avoiding unitarity bound for the dimension-8 operators is to implement a dipole form factor, ensuring unitarity at high energies such as: where ŝ is the partonic center-of-mass energy, Λ F F represents the form factor cut-off scale.The Λ F F can be calculate with the form factor tool VBFNLO 2.7.1 [50] for a given input of anomalous quartic gauge boson coupling parameters.The VBFNLO utility determines the form factor using the amplitudes of on-shell VV scattering processes and computes the zeroth partial wave of the amplitude.The real part of the zeroth partial wave must be below 0.5, which is called the unitarity criterion.All channels have the same electrical charge are combined in addition to individual checks on each channel of the V V system.The calculated Unitarity Violation (UV) bounds using the form factor tool with VBFNLO as a function of higherdimensional operators considered in our study are given in Fig. 1.The unitarity is safe in the region that is below the line for each coefficient.
where σ SM is the SM cross section of γγγ production (Fig. assuming that only one anomalous quartic coupling is different from zero at a time, which gives the contributions of the γγγγ and Zγγγ vertices separately.In order to imitate the performance of a typical electromagnetic calorimeter and make sure that cross sections are free of infrared divergences, we applied a cut on the transverse momentum of photons in the final state as a p γ T > 25 GeV at the generator level. The results of Fig. 3 found clear support for cross section of the three photon production are most sensitive to f T 8 /Λ 4 coupling than to the f T 9 /Λ 4 coupling.Further novel finding form Fig. 3 is that the main contribution to the variation of the cross section with respect to both f T 8 /Λ 4 and f T 9 /Λ 4 comes from the γγγγ vertex and its interference with the SM rather than the Zγγγ vertex.We study dimension-8 operators via pp → γγγ signal process at FCC-hh since it is a clean probe for new physics.Although our analysis strategy relies mainly on the isolated three photons in the final state, the considered signal process suffers from the backgrounds from high-p T jets, which are misidentified as isolated photons even after imposing the tight identification and isolation requirements.Therefore following relevant backgrounds are considered; multi-jet photons (jjj), photon + jet (γjj), diphoton-jet (γγj).In order to perform detailed signal and background analysis, for γγγ (SM), γγj, γjj, and jjj background processes and five different positive values of each f T 8 /Λ 4 and f T 9 /Λ 4 couplings for signals 600.000 (or 600k) events are generated within MadGraph5 aMC@NLO using NNPDF23LO PDF set [56].Parton level events are then passed through to Pythia 8.20 [57] for parton showering and hadronization.Detector response is included in then hadronized events by Delphes 3.4.2[58] software with card namely FCC-hh.tclwhere detector response in the form of resolution functions and efficiencies are parametrised.Jets are reconstructed by using clustered energy deposits with FastJet 3.3.2[59] using anti-kt algorithm [60] where a cone radius is set as ∆R = 0.2 and p j T > 20 GeV.The probability of misidentifying a jet as a photon depends on the interaction between the jets and the detector.Presently, the probability of a hard scattering jet being inaccurately reconstructed as an isolated photon is minimal due to the exceptional angular resolution of existing calorimeters in LHC detectors.It is anticipated that the granularity of detectors in upcoming hadron-hadron colliders, which will employ detector parameters, will be 2-4 times more precise than those in current LHC detectors.Therefore we applied a constant jet-to-photon fake rate of 10 −3 to each jet in the γγj, γjj, and jjj background data samples.

IV. EVENT SELECTION AND ANALYSIS WITH BOOSTED DECISION TREES
Machine learning algorithms are increasingly popular in particle physics analysis due to their ability to solve particle classification and event selection problems and their usability for reconstruction, a regression task.Among these algorithms, the use of multivariate techniques in analyzing data from colliders has come to the forefront.The use of boosted decision trees (BDTs) in multivariate techniques offers new windows not only for the analysis of big data from particle colliders but also for the possibility of increasing signal significance by relying on the algorithm to separate the signal from the relevant background to identify new physics signal events.Therefore, we use the Boosted Decision Trees (BDT) method in the toolkit [61,62] for multivariate analysis (TMVA) for the analysis of the tri-photon process at the FCC-hh to obtain sensitivity on the dimension-8 effective operators.
Our multivariate analysis utilizes the BDT machine learning technique with 850 trees, a maximum depth of 3, minimum events at each final leaf of 2.5, a number of iterations to find the best split of 20, and a learning rate of 0.5.We use adaptive boosting (AdaBoost) mechanisms for training that signal events from the training sample that end up in a background node.We merge data featuring selected kinematic variables for both signal and background events into a unified data file.50% of the combined data is designated for training, while the remaining 50% is reserved for testing.
Event selection starts by requiring the presence of at least three photons in the signal and SM background processes, with generator level cuts on the final state particles.In order to make the most effective use of the kinematic differences between the signal and background processes, a Boosted Decision Tree (BDT) is trained using most of the available kinematic information from the event, given in Table II.Input variables to a multivariate BDT discriminant can be classified as the transverse momentum and pesudo-rapidity of three-p T ordered photons (γ 1 ,γ 2 and γ 3 ), the difference in the azimuthal angle and in pseudo-rapidity between pairs of photons, the distance between pairs of photons in the η-φ plane and the invariant mass of three photon system (m γ 1 γ 2 γ 3 ).
Since we try to well separate our signal from SM backgrounds with jets in the final state, the number of jets is considered in the BDT input variable list.However, the kinematic variables of the jets are not included.Among the multivariate BDT discriminant variables listed in Table II,  Invariant mass of reconstructed three photon system the invariant mass of the tri-photon system is selected as a target variable in BDT analysis.
Fig. 4 shows the BDT classifier response for the considered signal and total background events.
The first row in Fig. 4 corresponds to the BDT response, while the second row shows the signal efficiency as a function of background rejection, the so-called Receiver Operating Characteristic (ROC) curve.The left column in Fig. 4 corresponds to the BDT response and ROC curve for the signal with f T 8 /Λ 4 =0.03TeV −4 and all the other overwhelming backgrounds for trained samples, while the right column is for the signal with f T 9 /Λ 4 =0.05TeV −4 value.
From this figure, it is evident that the BDT performs well in separating the signal from the backgrounds.We consider 70% signal efficiency in the determination of an optimal cut on reconstructed BDT distributions.Since each signal, which depends on the different aQGC values of f T 8 /Λ 4 and f T 9 /Λ 4 couplings, is trained individually in the BDT analysis, we obtain a different BDT score for each signal and relevant SM backgrounds.Therefore, we apply the optimal BDT score obtained from the BDT response for each value of f T 8 /Λ 4 and f T 9 /Λ 4 aQGCs to the signal as well as relevant background processes to obtain the invariant mass distributions of three photon

systems.
The presence of effective operators results in a violation of unitarity at high energies; hence, it is necessary to restrict the application of parameterizing deviations from the SM predictions in terms of effective operator approximations to energy regimes where this violation does not occur.The method employed to avoid unitarity violation in this analysis is commonly known as the clipping method [18,22], which involves suppressing any effective field theory (EFT) contribution beyond an energy scale Λ F F .The energy scale, Λ F F for each aQGC parameter is calculated using VBFNLO after applying appropriate aQGC convention factors.
Then, the threshold of the clipping energy value for each aQGC parameter is applied to the invariant mass of m γγγ .In Fig. 5, the normalized invariant mass distribution of three photons system for the signal with f T 8 /Λ 4 =0.03TeV −4 (f T 9 /Λ 4 =0.05TeV −4 ) benchmark points and relevant backgrounds are shown on the left (on the right).These figures are normalized to the cross-section of each process times the integrated luminosity, L int = 30 ab −1 .Only one coupling (f T 8 /Λ 4 or f T 9 /Λ 4 ) at a time is varied from its SM value.

V. LIMITS ON ANOMALOUS QUARTIC GAUGE COUPLINGS
In order to calculate the median expected significance for discovery and exclusion of the f T 8 /Λ 4 and f T 9 /Λ 4 couplings, denoted as SS disc and SS excl , respectively, we use the expressions presented in ref. [63] where x = (S + B) 2 − 4δ 2 SB 2 /(1 + δ 2 B); S and B are the number of events obtained by integrating the normalized invariant mass distributions of the three photon system of the signal and total SM background, respectively, and δ is the systematic uncertainty.In the limit of δ → 0, these expressions can be simplified as Regions with a SS disc 5 (3) σ are categorized as discoverable regions, while regions with a SS excl   reported by ATLAS collaboration, we obtain a result more than three orders of magnitude better.
Even considering 10% systematic error, our results at 5σ and 95% C.L. are still two orders of magnitude better.

VI. CONCLUSIONS
The Future Circular Hadron Collider with its high center of mass energy and luminosity, as well as novel developments in detector technologies, comes in front of the search for clues to explain the physics beyond SM.Tri-photon production in the hadron colliders provides an ideal platform to search for deviations from SM since it is rare in the SM and involves only pure electroweak interaction contributions at tree level.
In this paper, we have studied anomalous quartic gauge couplings via tri-photon production at FCC-hh.FCC-hh is assumed to operate with a proton beam energy of 50 TeV and an integrated luminosity of 30 ab −1 .Among the aQGCs where the dimension-8 EFT operators contribute in the γγγZ and γγγγ quartic vertices, we have focused on aQGCs of O T,8 and O T,j opertors more sensitive to tri-photon production.Therefore, cross section is calculated as a function of pure anomalous aQGCs contribution and the interference between the SM process and anomalous aQGCs in order to show the sensitivity of aQGCs f T 8 /Λ 4 and f T 9 /Λ 4 to the pp → γγγ production.Since the signal final state has at least three photons, the background processes with the same final state as the signal as well as three dominant background processes relevant to our signal are considered and simulated at the detector level.We also investigate the violaiton arising from the unitarity in the presence of the dimension-8 aQGCs of our interest.To investigate the potential enhancement of sensitivity for f T 8 /Λ 4 and f T 9 /Λ 4 , we conducted a multivariate analysis that incorporated 17 kinematic variables such as transverse momentum and pesudo-rapidity of three-pT ordered photons (γ 1 ,γ 2 and γ 3 ), the difference in the azimuthal angle and in pseudo-rapidity between pairs of photons, the distance between pairs of photons in η − φ plane and the invariant mass of three photon system (m γ 1 γ 2 γ 3 ) as input to the TMVA package to perform multivariate analyses via the Boosted Decision Trees (BDT) algorithm.Finally, the threshold of the clipping energy value for each of the aQGC parameters is applied to the invariant mass distributions of the three photon systems.
We have reported limits for 3σ, 5σ and 95% C.L., as well as without and with δ → 10%

couplings of the O T 8
and O T 9 dimension-8 operators, are [-0.059;0.059] TeV −4 and [-0.13; 0.13] TeV −4 currently obtained from analysis of the electroweak production of Z(νν)γ in association with two jets by ATLAS collaboration at a center-of-mass energy of 13 TeV with an integrated luminosity 139 fb −1 [22], respectively.

Fig. 2 .
FIG. 2: The Feynman diagrams contributing γγγ production at hadron colliders (a) and (b) through anomalous quartic vector-boson interactions with marked green (c) as well as SM contribution 2(c)); σ γ , σ Z and σ γZ are pure anomalous cross sections due to γγγγ (Fig.2(a)), Zγγγ (Fig.2(b)) vertices and the interference between them, respectively; σ γ−SM and σ Z−SM indicate interference between SM processes and the anomalous contribution to the γγγ production.Numerical calculations of the cross section of pp → γγγ are performed in MadGraph5 aMC@NLO [53] by including effective vertices of γγγγ and Zγγγ aQGCs based on the Universal FeynRules Output (UFO) framework[54,55].In Fig.3, we show the contribution of each term in Eq. (7) separately as a function of f T 8 /Λ 4 and f T 9 /Λ4

TT
Transverse momentum of the second-highest-p T photon p γ3 Transverse momentum of the third-highest-p T photon η γ1 Pseudo-rapidity of the leading photon η γ2 Pseudo-rapidity of the second-highest-p T photon η γ3 Pseudo-rapidity of the third-highest-p T photon ∆|φ γ1γ2 | Difference in azimuthal angle between the leading and second-highest-p T photons ∆|φ γ1γ3 | Difference in azimuthal angle between leading and the third-highest-p T photon ∆|φ γ2γ3 | Difference in azimuthal angle between second-highest-p T photon and the third-highest-p T photon ∆|η γ1γ2 | Difference in pseudo-rapidity between the leading and the second-highest-p T photons ∆|η γ1γ3 | Difference in pseudo-rapidity between the leading and the third-highest-p T photon ∆|η γ2γ3 | Difference in pseudo-rapidity between the second-highest-p T and the third-highest-p T photon ∆R(γ 1 , γ 2 ) Distance between leading photon and sub-leading photon in η-φ plane ∆R(γ 2 , γ 3 ) Distance between sub-leading photon and third photon in η-φ plane ∆R(γ 1 , γ 3 ) Distance between leading photon and third photon in η-φ plane N j Number of jets in the event m γ1γ2γ3

FIG. 4 :
FIG. 4: The distribution of the BDT response (on the first row) and Receiver Operating Characteristic (ROC) curve of the BDT (on the second row) for signal (f T 8 /Λ 4 =0.03TeV −4 and f T 9 /Λ 4 =0.05TeV −4 ) and all relevant backgrounds.

1 . 4 FIG. 6 :
FIG.6:The SS disc and SS excl as a function of the f T 8 /Λ 4 and f T 8 /Λ 4 after applying an optimum BDT cut value with and without systematic uncertainty at FCC-hh with an integrated luminosity L int =30 ab −1 .
systematic uncertainty on dimension-eight anomalous quartic gauge couplings f T 8 /Λ 4 and f T 9 /Λ 4 with an integrated luminosity L int =30 ab −1 and a 100 TeV center of mass energy.Our obtained 95% C.L. limits on f T 8 /Λ 4 and f T 9 /Λ 4 couplings without systematic uncertainties by applying the clipping technique are three orders of magnitude stronger than the best limits recently reported by ATLAS, obtained with an integrated luminosity of 139 fb −1 at √ =13 TeV.Considering 10% systematic error, our obtained results at 5σ and 95% C.L. are still two orders of magnitude better than those recently reported by ATLAS.

TABLE I :
Quartic gauge boson vertices modified by the related dimension-8 operator are marked with

TABLE II :
The list of selected kinematical and reconstructed variables to be used in BDT.