Search for Anomalous Couplings in the Higgs Sector at LEP

Anomalous couplings of the Higgs boson are searched for through the processes e^+ e^- ->H gamma, e^+ e^- ->e^+ e^- H and e^+ e^- ->HZ. The mass range 70 GeV<m_H<190 GeV is explored using 602 pb^-1 of integrated luminosity collected with the L3 detector at LEP at centre-of-mass energies sqrt(s)=189-209 GeV. The Higgs decay channels H ->ffbar, H ->gamma gamma, H ->Z\gamma and H ->WW^(*) are considered and no evidence is found for anomalous Higgs production or decay. Limits on the anomalous couplings d, db, Delta(g1z), Delta(kappa_gamma) and xi^2 are derived as well as limits on the H ->gamma gamma and H ->Z gamma decay rates.


Introduction
The mechanism of spontaneous symmetry breaking is a cornerstone of the Standard Model of the electroweak interactions [1]. It explains the observed masses of the elementary particles and postulates an additional particle, the Higgs boson. Despite its relevance, experimental information on the Higgs boson is scarce and indirect. It leaves room for deviations from the Standard Model expectations such as anomalous couplings of the Higgs boson.
The Standard Model can be extended, via a linear representation of the SU(2) L × U(1) Y symmetry breaking mechanism [2], to higher orders where new interactions between the Higgs boson and gauge bosons become possible. These modify the production mechanisms and decay properties of the Higgs boson. The relevant CP-invariant Lagrangian terms are [3]: L eff = g Hγγ HA µν A µν + g (1) HZγ A µν Z µ ∂ ν H + g (2) HZγ HA µν Z µν + g (1) HZZ Z µν Z µ ∂ ν H + g (2) HZZ HZ µν Z µν + g (3) HZZ HZ µ Z µ + g (1) HWW HWW HW + µν W µν − , where A µ , Z µ , W µ and H are the photon, Z, W and Higgs fields, respectively, and X µν = ∂ µ X ν − ∂ ν X µ . The couplings in this Lagrangian are parametrized as [4][5][6]: HZZ = g m W 2 cos 2 θ W δ Z g (1) where g is the SU(2) L coupling constant, θ W is the weak mixing angle and m W and m Z represent the masses of the W and Z bosons, respectively. The five dimensionless parameters d, d B , ∆g Z 1 , ∆κ γ and δ Z constitute a convenient set to describe deviations in the interactions between the Higgs boson and gauge bosons. They are not severely constrained by electroweak measurements at the Z pole or at lower energies [3,7].
The couplings d and d B were introduced in Reference 4, while ∆g Z 1 and ∆κ γ also describe possible deviations in the couplings of W bosons with photons and Z bosons [5]. A search for anomalous Higgs production and decay with non-vanishing values of ∆g Z 1 or ∆κ γ is a complementary study to the analysis of triple-gauge-boson couplings in the e + e − → W + W − process. The parameter ξ 2 = (1 + δ Z ) 2 describes a global rescaling of all Higgs couplings and affects the Higgs production cross section, but not its branching fractions [6].
We search for a Higgs particle produced in the e + e − → Hγ and e + e − → e + e − H processes shown in Figures 1a and 1b. Their rates would be enhanced in presence of anomalous Hγγ and HZγ couplings. These processes probe Higgs masses, m H , up to the centre-of-mass energy of the collision, √ s. For m H < √ s − m Z , this analysis is complemented by the results from the L3 searches for the e + e − → HZ process [8,9], which are sensitive to anomalous HZZ and HZγ couplings, as shown in Figure 1c. The existence of Hγγ and HZγ couplings would lead to large H → γγ and H → Zγ branching fractions, which at tree level are zero in the Standard Model. These decay modes have complementary sensitivities and allow to probe a large part of the parameter space. In addition, the decay H → WW ( * ) would also be enhanced in the presence of anomalous HWW couplings.
The data used in this analysis were collected with the L3 detector [10] at LEP at √ s = 189− 209 GeV and correspond to an integrated luminosity of 602 pb −1 . Searches for anomalous Higgs production were previously performed, with data of lower energy and integrated luminosity, by L3 and other experiments [11,12]. Other non-standard Higgs searches performed at LEP are reported in [9,13]. The results reported in this Letter include and supersede those of Reference 11. For the e + e − → e + e − H process, only the H → γγ decay is studied. The H → bb decay was considered in the study of the e + e − → e + e − H and e + e − → Hγ processes for data collected at √ s = 189 GeV [11]. This decay is dominant for m H m Z , where H → γγ is strongly suppressed and H → Zγ is kinematically forbidden. At the centre-of-mass energies considered in this Letter, this region is efficiently covered by an interpretation of the results of the search for the e + e − → HZ process [8] and the H → bb decay is not considered here.

Analysis strategy
No dedicated selection is devised for the e + e − → HZ process and the limits obtained by L3 in the searches for the Standard Model Higgs boson and for a fermiophobic Higgs boson are interpreted in terms of anomalous Higgs couplings.
The analysis is performed as a function of m H in steps of 1 GeV. The H → γγ, H → Zγ and H → WW ( * ) decays probe the ranges 70 GeV < m H < 190 GeV, 95 GeV < m H < 190 GeV and 130 GeV < m H < 190 GeV, respectively. After the event selections described below, variables which depend on m H are built to discriminate signal and background. Finally, the number of events in a mass window around the m H value under study is compared with the Standard Model expectation and interpreted in terms of cross sections and anomalous couplings.  [11].

Data and Monte Carlo samples
To describe the e + e − → Hγ process we wrote a Monte Carlo generator which assumes a 1 + cos 2 θ H dependence of the differential cross section as a function of the cosine of the Higgs production angle, θ H . It includes effects of initial-state [14] and final-state [15] radiation as well as spin correlations and off-shell contributions in cascade decays such as H → Zγ → ffγ.
The e + e − → e + e − H process is interpreted as the production of a narrow-width spin-zero resonance in two-photon collisions, and modelled with the PC Monte Carlo generator [16].
The differential cross section of the process e + e − → HZ in the presence of anomalous couplings is taken from Reference 17. References 18 and 19 are used for the branching fractions and partial widths of a Higgs boson with anomalous couplings. The interference between the e + e − → HZ process in the Standard Model and in presence of anomalous couplings [17] is taken into account in the simulation. It is negligible for the e + e − → Hγ and e + e − → e + e − H cases.
Signal events are generated for 70 GeV < m H < 190 GeV, in steps of 20 GeV. More than 5000 signal events are generated for each value of m H and for each process under study. For intermediate values of the Higgs mass, the signal efficiency is interpolated between the generated values.
The L3 detector response is simulated using the GEANT program [25] which takes into account effects of energy loss, multiple scattering and showering in the detector. Time-dependent detector inefficiencies, as monitored during the data-taking period, are included in the simulations.

Event selection
All analyses presented in this Letter rely on photon identification. Photon candidates are defined as clusters in the electromagnetic calorimeter with a shower profile consistent with that of a photon and no associated track in the tracking chamber. To reduce contributions from initial-state and final-state radiation, photon candidates must satisfy E γ > 5 GeV and | cos θ γ |< 0.97, where E γ is the photon energy and θ γ its polar angle.
Events with hadronic decays of the Z and W bosons in the H → Zγ and H → WW ( * ) channels are pre-selected requiring high particle multiplicity and a visible energy, E vis , satisfying 0

The e + e − → Hγ → γγγ analysis
Events from the e + e − → Hγ → γγγ process are selected by requiring three photon candidates in the central region of the detector, | cos θ γ | < 0.8, with a total electromagnetic energy larger than √ s/2. Out of the three possible two-photon combinations, the one with a mass, m γγ , closest to the m H hypothesis under investigation is retained. As an example, Figure Table 3 for several m H hypotheses. The contamination from processes other than e + e − → γγ(γ), as estimated from Monte Carlo simulations, is found to be negligible. The signal selection efficiency is in the range 25% − 30%, depending on m H and √ s.

4.2
The e + e − → e + e − H → e + e − γγ analysis In the process e + e − → e + e − H, the final state e − and e + tend to escape detection at low polar angles, originating events with missing longitudinal momentum and missing mass. The selection requires two photon candidates from the H → γγ decay in the central region of the detector. A kinematic fit is performed assuming the missing momentum to point in the beam pipe and the visible mass of the event to be consistent, within the experimental resolution, with the m H hypothesis under investigation. The distribution of the χ 2 of the fit is shown in Figure 2b for m H = 130 GeV. Events are accepted as Higgs candidates if  Table 3 for several m H hypotheses. The background comes from e + e − → γγ(γ) events. The signal selection efficiency varies from 20% to 30%, with a smooth dependence on m H and √ s.

The e + e − → Hγ → Zγγ analysis
Pre-selected hadronic events with two isolated high energy photons are considered for the e + e − → Hγ → Zγγ analysis. Events are retained which have a recoiling mass, m rec , calculated from the four-momenta of the two photons, compatible with m Z : 80 GeV < m rec < 110 GeV.
The hadronic system is clustered into two jets with the DURHAM [26] algorithm and a kinematic fit, in which the jet angles are fixed and the jet energies can vary, is performed to improve the resolution on the reconstructed Z-boson mass. Of the two possible combinations of two jets and a photon, the one is retained with mass, m qqγ , closer to the m H hypothesis under investigation. The distribution of m qqγ is shown in Figure 2c for m H = 150 GeV. An event is considered as a Higgs candidate if |m qqγ − m H | < 15 GeV. The numbers of events observed and expected in the data sample at √ s = 192 − 209 GeV are shown in Table 3 for several m H hypotheses. The signal selection efficiency is around 22%. The background is dominated by resonant e + e − → Zγγ production (70%) with contributions from the e + e − → qq(γ) process and four-fermion final states.

4.4
The e + e − → Hγ → WW ( * ) γ analysis The energy of the photon in the e + e − → Hγ → WW ( * ) γ process depends on m H as E rec γ (m H ) = (s − m 2 H )/2 √ s. Pre-selected hadronic events are retained if they have a photon with energy compatible with the m H hypothesis under investigation, E rec γ (m H + 20 GeV) < E γ < E rec γ (m H − 20 GeV). If multiple photon candidates are observed, the photon is retained which has an energy closest to E rec γ (m H ). The rest of the event is clustered into four jets by means of the DURHAM algorithm.
A kinematic fit, in which the jet angles are fixed and the jet energies can vary, is performed to improve the resolution on the reconstructed W-boson mass. For m H > 2m W both W bosons are on-shell and the constraint that both invariant jet-jet masses be compatible with m W is included in the fit. For m H < 2m W one of the W bosons is off-shell and only one of the invariant jet-jet masses is required to be compatible with m W . The fit is repeated for all possible jet pairings and the pairing is chosen for which the χ 2 of the fit is minimal. An event is considered as a Higgs candidate if χ 2 < 6.0 for the hypothesis m H < 2m W or χ 2 < 15.0 for m H > 2m W .
The invariant mass of the four-jet system, m qqqq , estimates m H . Its distribution is presented in Figure 2d for m H = 170 GeV.
The numbers of events observed and expected in the data sample at √ s = 192 − 209 GeV are shown in Table 3 for several m H hypotheses. The signal selection efficiency is around 25%, for 150 GeV < m H < 170 GeV, decreasing to about 20% for masses out of this range. A small dependence on √ s is observed. The background is dominated by the processes e + e − → qq(γ) and e + e − → W + W − (γ), which is above 65% for m H > 150 GeV.

Cross sections limits
The results of all the analyses agree with the Standard Model predictions and show no evidence for a Higgs boson with anomalous couplings in the m H mass range under study. Upper limits on the product of the production cross sections and the corresponding decay branching fractions are derived [27] at the 95% confidence level (CL). The cross section of the e + e − → e + e − H process is proportional to the partial Higgs width into photons, Γ(H → γγ), and limits are quoted on Γ(H → γγ) × Br(H → γγ).
In order to combine data sets at different √ s values, a dependence of the type σ AC ( √ s) = ζ σ SM ( √ s) is assumed for the cross section of anomalous Higgs production, σ AC . The e + e − → Hγ production cross section in the Standard Model, σ SM , accounts for the dominant dependence on √ s while ζ is a parameter which does not depend on √ s. Limits on ζ are derived and interpreted as cross section limits at the luminosity-averaged centre-of-mass energy < √ s >= 197.8 GeV.
The cross section limits for the investigated processes are given in Figure 3 together with the expectations for non-zero values of the anomalous couplings.
6 Limits on anomalous couplings 6.1 Results from e + e − → HZ with H → ff or H → γγ The process e + e − → HZ, with H → ff, studied in Reference 8, is sensitive to anomalous HZZ and HZγ couplings in the Higgs production vertex. In addition, the process e + e − → HZ with H → γγ, object of the search for a fermiophobic Higgs [9], is sensitive to the Hγγ coupling in the decay vertex.
Limits on the coupling ξ 2 are derived from the results of our search for the Standard Model Higgs boson [8]. They are obtained by interpreting ξ 2 as a scale factor of the Higgs production cross section and are shown in Figure 4. They include the systematic uncertainties on the search for the Standard Model Higgs boson [8].
The limits on the couplings d, d B , ∆g Z 1 and ∆κ γ are extracted from the numbers of observed events, expected background and signal events reported in References 8 and 9. These limits are driven by the size of the deviations of the product σ AC × Br AC with respect to σ SM × Br SM , where Br AC and Br SM denote the Higgs branching ratios in the presence of anomalous couplings and in the Standard Model respectively. The ratios R = (σ AC ×Br AC )/(σ SM ×Br SM ) are shown in Figure 5 for H → ff and H → γγ, for m H = 100 GeV.
The H → ff and H → γγ channels have different behaviours with respect to the parameters d and d B , as these describe the Hγγ coupling. The parameters ∆g Z 1 and ∆κ γ describe the HZγ and HZZ couplings and hence affect only the Higgs production vertex in the e + e − → HZ process. They give similar deviations for both the H → ff and H → γγ channels.  Figure 6a is not excluded due to an excess of events observed in the e + e − → HZ search [28]. The e + e − → Hγ → γγγ and e + e − → e + e − H → e + e − γγ channels have a large sensitivity if the Hγγ coupling is large, i.e. when d sin 2 θ W + d B cos 2 θ W has a sizable value (Figures 6a  and 6b). On the other hand, the e + e − → Hγ → Zγγ process has a dominant role when the channel H → γγ is suppressed, which occurs for the couplings ∆g Z 1 and ∆κ γ in the mass region m Z < m H < 2m W (Figures 6c and 6d).

One-dimensional limits
The contribution from the e + e − → Hγ → WW ( * ) γ process to the limits presented in Figures  6a and 6c is small and restricted to m H ∼ 160 GeV. This happens since a large decay width for H → WW ( * ) corresponds to large values of d or ∆g Z 1 which also imply large widths for the competing modes H → γγ and H → Zγ.
The sensitivity of the analysis degrades rapidly when m H approaches the 2m W threshold, where the H → γγ and H → Zγ are no longer dominant, even in the presence of relatively large anomalous couplings.
Several sources of systematic uncertainties are investigated and their impact on the signal efficiency and background level is evaluated. The limited Monte Carlo statistics affects the signal by less than 2% and the background by 8% for the photonic channels and less than 4% for the hadronic channels. The accuracy of the cross section calculation for background processes adds less than 0.4% to the uncertainty in the background normalisation. The systematic uncertainty due to the selection procedure was estimated by varying the most important selection criteria and was found to be less than 1%. In particular, the effect of the limited knowledge of the energy scale of the electromagnetic calorimeter has a small impact in the limits.
The combined effect of the systematic uncertainties is included in the limits shown in Figure 6. It degrades the limits by at most 4%, slightly depending on the coupling and the Higgs mass hypothesis.
We verified that possible effects of angular dependence of the efficiency on the value of the anomalous couplings is negligible for the e + e − → HZ process. No such effects are expected for the e + e − → e + e − H and e + e − → Hγ processes.

Two-dimensional limits
Assuming the absence of large anomalous WWZ and WWγ couplings, i.e. ∆g Z 1 = ∆κ γ = 0 [29], the Hγγ and HZγ couplings are parametrized via the following subset of effective operators: where the dependence of g Hγγ and g (2) HZγ on the d and d B couplings is given by Equations 2 and 4. This Lagrangian is used to compute the maximal partial widths and branching fractions of the decays H → Zγ and H → γγ, allowed by the limits on d and d B . The results are presented in Figure 7 for two different Higgs masses, in the region of interest for Higgs searches at future colliders. The results are consistent with the tree level Standard Model expectations Γ(H → Zγ) ≈ Γ(H → γγ) ≈ 0.