Search for a Higgs boson decaying into gamma* gamma to ll gamma with low dilepton mass in pp collisions at sqrt(s) = 8 TeV

A search is described for a Higgs boson decaying into two photons, one of which has an internal conversion to a muon or an electron pair (ll gamma). The analysis is performed using proton-proton collision data recorded with the CMS detector at the LHC at a centre-of-mass energy of 8 TeV, corresponding to an integrated luminosity of 19.7 inverse femtobarns. The events selected have an opposite-sign muon or electron pair and a high transverse momentum photon. No excess above background has been found in the three-body invariant mass range 120<m[ll gamma]<150 GeV, and limits have been derived for the Higgs boson production cross section times branching fraction for the decay H to gamma* gamma to ll gamma, where the dilepton invariant mass is less than 20 GeV. For a Higgs boson with m[H] = 125 GeV, a 95% confidence level (CL) exclusion observed (expected) limit is 6.7 (5.9 +2.8/-1.8) times the standard model prediction. Additionally, an upper limit at 95% CL on the branching fraction of H to J/Psi gamma for the 125 GeV Higgs boson is set at 1.5E-3.


Introduction
The rare decay into the γ final state of the Higgs boson is a rich source of information that can enhance our understanding of its basic properties and probe novel couplings predicted by extensions of the standard model (SM) of particle physics. As illustrated in Fig. 1, this decay in SM has contributions from loop-induced H → γ * γ and H → Zγ diagrams (a, b, c), treelevel process H → with final-state radiation (d), and higher-order processes, known as box diagrams (e, f, g) [1][2][3][4]. Other contributions include H → V(qq)γ → γ, shown in Fig. 2, where V denotes a vector meson (J/ψ or Υ) that decays to [5][6][7]. The Higgs boson branching fraction to γ is dominated by the H → γ * γ and H → Zγ modes, while the contribution from the box diagrams is negligible [1]. In the muon channel, when the dilepton invariant mass, m , is greater than 100 GeV, final-state radiation in H → µµ starts to dominate [8]. In the three-body decay, H → γ, it is possible to investigate non-SM couplings by examining the angular distributions, and forward-backward asymmetry variables reconstructed from the γ final state [8,9].
The expected rates of the H → (Z/γ * )γ → γ processes compared to the rate of H → γγ decay, for a Higgs boson with mass m H = 125 GeV, are [10,11]: The H → γ * γ → eeγ decay is distinct from H → γγ followed by a conversion of a photon to an e + e − pair in the detector, which can become a background for H → γ * γ if photon conversions are not properly identified. Experimentally, the various contributions shown in Figs The ATLAS and CMS Collaborations at the CERN LHC have both performed a search for H → Zγ → γ decay with m above 50 GeV [12,13]. As a natural extension of those analyses, the current paper describes the first search for a Higgs boson Dalitz decay, H → γ * γ, where the γ * decays into a muon or an electron pair. The search is performed for a Higgs-like particle within the mass range between 120 and 150 GeV. In order to select the contribution from the Dalitz decay, we require m < 20 GeV. The µµγ topology is a clean final state with a mass resolution of about 1.6%, as measured from the simulated signal samples. The eeγ channel is challenging due to the low m that results in a pair of merged electron showers in the electromagnetic calorimeter (ECAL). Nevertheless, when the merged showers are reconstructed in the ECAL, a mass resolution of 1.8% is achieved. Important backgrounds include the irreducible contributions from the initial-and final-state photon radiation in Drell-Yan production, and Drell-Yan events with additional jets where a jet is misidentified as a photon.
In addition, a search is performed for H → (J/ψ)γ → µµγ decay for m H = 125 GeV, which is sensitive to the Higgs boson coupling to charm quark and a promising way to access the couplings of the Higgs boson to the second generation quarks at the LHC. In the SM this decay occurs through two main processes: direct coupling of the Higgs boson to charm (Fig. 2a), and the usual t/W loop, where the radiated γ * converts to a cc in a resonant state (Fig. 2b). The two amplitudes interfere destructively and the second one dominates [6,7]. For the SM Higgs boson with m H = 125 GeV, the branching fraction is predicted to be 2.8 × 10 −6 . A search by the ATLAS Collaboration for this decay is described in Ref. [14].
The results presented in this paper are based on proton-proton collision data recorded in 2012 with the CMS detector at a centre-of-mass energy √ s = 8 TeV, corresponding to an integrated luminosity of 19.7 fb −1 .

CMS detector and trigger
A detailed description of the CMS detector, together with a definition of the coordinate system used and the relevant kinematic variables, can be found in Ref. [15]. The central feature of the CMS apparatus is a superconducting solenoid, 13 m in length and 6 m in diameter, which provides an axial magnetic field of 3.8 T. Within the solenoid volume are a silicon pixel and strip tracker, the ECAL, and a hadron calorimeter (HCAL). Charged-particle trajectories are measured by silicon pixel and strip trackers, covering 0 ≤ φ ≤ 2π in azimuth and |η| < 2.5 in pseudorapidity. A lead tungstate crystal ECAL surrounds the tracking volume. It is comprised of a barrel region |η| < 1.48 and two endcaps that extend up to |η| = 3. A brass and scintillator HCAL surrounds ECAL and also covers the region |η| < 3. Iron forward calorimeters with quartz fibers, read out by photomultipliers, extend the calorimetric coverage up to |η| = 5. A lead and silicon-strip preshower detector is located in front of the ECAL endcaps. Muons are identified and measured in gas-ionization detectors embedded in the steel flux-return yoke outside the solenoid. The detector is nearly hermetic, allowing energy balance measurements in the plane transverse to the beam direction.
A two-tier trigger system selects collision events of interest for physics analysis. Two triggers are used in the current analysis. In the muon channel, the trigger requires a single muon and a photon, both with p T greater than 22 GeV. In the electron channel the γ * → ee process at low dielectron invariant mass mimics a photon at the trigger level. For this reason, a diphoton trigger is used in the electron channel, for γ + γ * final state. The trigger requires a leading (subleading) photon with p T > 26 (18) GeV. The diphoton trigger is inefficient for events with high dielectron invariant mass (m ee > 2 GeV) due to the isolation and shower shape requirements. The available dielectron triggers cannot be used to select events with 2 < m ee < 20 GeV because they also require isolation, and the p T requirement made on the subleading lepton is too high.

Event reconstruction
The photon energy is reconstructed from a sum of signals in the ECAL crystals [16]. The ECAL signals are calibrated and corrected [17], and a multivariate regression technique, developed for the H → γγ analysis [18], is used to determine the final energy of the photon [16]. The neighboring ECAL crystals with energy deposition are combined into clusters, and the collection of clusters that contain the energy of a photon or an electron is called a supercluster. Identification criteria are applied to distinguish photons from jets and electrons. The observables used in the photon identification criteria are: the isolation variables, the ratio of the energy in the HCAL towers behind the supercluster to the electromagnetic energy in the supercluster; the transverse width in η of the electromagnetic shower; and the number of charged tracks matched to the supercluster. The efficiency of the photon identification is measured using Z → ee data by reconstructing the electron showers as photons, and found to be 80 (88%) at a transverse energy > 30 (50) GeV and |η| < 1.44.
Muon candidates are reconstructed in the tracker and identified by the particle-flow global event reconstruction algorithm [19, 20] using hits in the tracker and the muon systems. This approach allows us to maintain a high efficiency independent of the dimuon invariant mass and to reconstruct muons with p T as low as 4 GeV. Muons from γ * → µµ internal conversions are expected to be isolated from other particles. A cone of size ∆R ≡ (∆η) 2 + (∆φ) 2 = 0.4 is constructed around the momentum direction of each muon candidate [21]. The relative isolation of the muon is quantified by summing the p T of all photons, charged and neutral hadrons within this cone, and then dividing by the muon p T . The resulting quantity, corrected for additional underlying event activity due to pileup events, is required to be less than 0.4 for the leading muon. The isolation requirement rejects misidentified leptons and background arising from hadronic jets. The ∆R(µµ) separation between the two muons is small due to their low invariant mass (as shown in Fig. 3) and high p T of the γ * in H → γ * γ decays. Hence, no isolation requirement is applied to the subleading muons as they are already within the isolation cone of the leading muons in most events. Dimuon identification and isolation efficiency of about 80% is obtained.
In the electron channel of the H → γ * γ → γ decay, the two electrons produced in the γ * → ee process are even closer to each other than in the muon channel, since the m is smaller (Fig. 3). Therefore, their energy deposits in the ECAL are merged into one supercluster giving rise to a unique signature. To identify these merged electrons, two tracks associated to the supercluster are required. A Gaussian sum filter (GSF) algorithm is used to reconstruct the electron tracks [22]. The supercluster energy must correspond to p T > 30 GeV and be located in the ECAL barrel (|η| < 1.44). The scalar sum p e 1 T + p e 2 T of the corresponding two GSF tracks must exceed 44 GeV. Both GSF tracks are required to have no more than one missing hit in the pixel detector in order to reduce the background from photons converting to e + e − in the detector material. A multivariate discriminator is trained to separate the γ * → ee objects from jets or single electrons. The input variables for the training include lateral shower shape variables, the median energy density in the event to take into account the pileup dependence, and the kinematic information from the supercluster and tracks. A combined reconstruction and selection efficiency of ∼40% is achieved for the signal. For comparison, the efficiency for a single isolated electron with similar p T is ∼88% [23].

Simulated samples
The description of the Higgs boson signal used in the search is obtained from simulated events. The samples for the Dalitz signal are produced at leading-order using the MADGRAPH 5 matrix-element generator [24] with the ANO-HEFT model [25], interfaced with PYTHIA 6.426 [26], for the gluon and vector boson fusion processes, and for associated production with a vector boson. Associated production with a tt pair is ignored because of its small contribution. The sample for H → (J/ψ)γ process is produced with the PYTHIA 8.153 generator [27], and reweighted to simulate 100% polarization of the J/ψ. The parton distribution function (PDF) set used to produce these samples is given by CTEQ6L1 [28]. The SM Higgs boson production cross sections are taken from Ref. [11]. The branching fractions for H → γ * γ are estimated using MCFM 6.6 [29] and for H → (J/ψ)γ are taken from Ref. [6]. For the SM Higgs boson in the mass range of 120-150 GeV, the H → γ * γ → µµγ (eeγ) branching fraction is expected to be between 2.0 (4.5) × 10 −5 and 3.3 (7.5) × 10 −5 for m below 20 GeV. The expected branching fraction for H → (J/ψ)γ is (2.8 ± 0.2) × 10 −6 for m H = 125 GeV, which is further suppressed due to the J/ψ meson decay to muons, B(J/ψ → µµ) = 0.059.
The simulation aims to include all known effects and the conditions of real data taking in CMS. Some residual differences between the data and simulation are taken into account by reweighting the simulated events with scale factors. Systematic uncertainties are assigned to cover imperfect knowledge of residual differences. Scale factors are implemented to match the distribution of primary vertices, the photon identification and isolation efficiency, and the muon isolation efficiency. No corrections are applied to the muon and electron identification and trigger efficiencies, but an uncertainty is assigned as described in Section 7.
The energy and momentum resolution of muons and photons in simulated events are corrected to match that in data. The energy scale of muons (photons) is corrected to that found in Z → µµ (ee) events. For the electrons, no resolution or scale corrections are applied because of their unique topology, and the absence of a data-driven method to derive those corrections. Therefore, we rely on the simulation of the γ * → ee process and assign uncertainties sufficient to cover any possible discrepancy in the scale and resolution between data and simulation.

Event selection
Events are required to pass the muon plus photon trigger in the µµγ final state and the diphoton triggers in the eeγ final state. The trigger efficiency for signal events after the selection m ℓℓ (GeV)  Distributions are shown for muon and electron channels, before and after selection. The invariant mass before selection is obtained from the leptons at the generator level, while after selection the reconstructed invariant mass is used. requirements described below is 85% (90%) in the muon (electron) channel, as measured from the simulated samples.
The muons (electrons) are required to be within |η| < 2.4 (1.44), while the photon is required to be within |η| < 1.44. The invariant mass of the γ system, m γ , is required to satisfy 110 < m γ < 170 GeV. The photon and dilepton momenta both must satisfy p T > 0.3 · m γ requirement, which is optimized for high signal efficiency and background rejection.
On average, there are 21 pp interactions within the same bunch crossing in the 8 TeV data, which result in about 16 collision vertices reconstructed in each event. The vertex with the highest scalar sum of the p 2 T of its associated tracks is taken to correspond to the primary interaction vertex. The primary vertex must have the reconstructed longitudinal position (z) within 24 cm of the geometric centre of the detector and the transverse position (x-y) within 2 cm of the beam interaction region. The lepton tracks from γ * → µµ (ee) are required to originate from the primary vertex, and to have transverse and longitudinal impact parameters with respect to that vertex smaller than 2.0 (0.2) mm and 5 (1) mm, respectively.
The muons must be oppositely charged, and have p T > 23 (4) GeV for the leading (subleading) lepton. The p T requirement on the leading muon is driven by the trigger threshold, and on the subleading muon by the minimum energy needed to reach the muon system, while maintaining high reconstruction efficiency. In the electron channel, no additional selection on p T of the GSF tracks is necessary, beyond those described in Section 3. Finally, in both muon and electron channels, the separation between each lepton and the photon is required to satisfy ∆R > 1 in order to suppress Drell-Yan background events with final-state radiation.
The dilepton invariant mass in the muon channel is required to be less than 20 GeV to reject contributions from pp → γZ and to suppress interference effects from the H → γZ process and the box diagrams shown in Fig. 1. Events with a dimuon mass in the ranges 2.9 < m µµ < 3.3 GeV and 9.3 < m µµ < 9.7 GeV are rejected to avoid the J/ψ → µµ and Υ → µµ contamination. In the electron channel the invariant mass, constructed from the two GSF tracks, is required to satisfy m ee < 1.5 GeV. The m distributions for simulated signal events are shown in Fig. 3 in the muon and electron channels.  In the search for the H → (J/ψ)γ → µµγ, both p γ T and p µµ T > 40 GeV are required, and the events are selected with 2.9 < m µµ < 3.3 GeV.
The observed yields after the event selection described above are listed in Table 1. In the electron channel, there is also a contribution from the H → γγ process due to unidentified conversions, which is about 15% of the H → γ * γ signal (0.2 events at m H = 125 GeV). This contribution is considered as a background to H → γ * γ, and negligible compared to the continuum background estimated from the fit to data described in the next section. Table 1: The expected signal yield and the number of events in data, for an integrated luminosity of 19.7 fb −1 . Signal events are presented before and after applying the full selection criteria described in the text. In the (J/ψ)γ sub-category only the J/ψ → µµ decay is considered, and the signal yield is a sum of two contributions: H → (J/ψ)γ → µµγ and H → γ * γ → µµγ, where the dimuon mass distribution is non-resonant.

Background and signal modeling
The background is modeled by fitting a polynomial function to the γ mass distributions in data. An unbinned maximum likelihood fit is performed over the range 110 < m γ < 170 GeV. Fig. 4 shows the m γ spectra, which are fitted with polynomial functions of fourth degree. The reduced χ 2 of the fits are 0.5 and 0.7 for the muon and electron channels, respectively. Even though the search is limited to 120 < m H < 150 GeV, the fits to the m γ spectra are performed over a wider range, giving a better modeling of the background, particularly at the edges of the search range. The degree of the polynomials is chosen following a procedure similar to the one described in Ref. [30]. This procedure ensures that the potential bias due to the background modeling is at least five times smaller than statistical uncertainty.
For the H → (J/ψ)γ search, where only the single Higgs boson mass hypothesis m H = 125 GeV is investigated, a fit to a polynomial of second degree is performed over the 110-150 GeV mass range (Fig. 5).
The signal model in all three cases is obtained from an unbinned fit to the mass distribution of the corresponding sample of simulated events to a Crystal Ball function [31] plus a Gaussian function.

Results
The data are used to derive upper limits on the Higgs boson cross section times branching fraction, σ(pp → H) B(H → γ * γ → γ) divided by that expected for a SM Higgs boson, for m < 20 GeV. No significant excess above background is observed in the full mass range, 120 < m H < 150 GeV, with a maximum excess of less than two standard deviations. In the electron channel a correction is made to account for the events that are removed by the requirement of m ee < 1.5 GeV due to the trigger and reconstruction inefficiencies described above.
The exclusion limits are calculated using the modified frequentist CL s method [32][33][34][35][36]. An unbinned evaluation over the full mass range of data is used. The uncertainty in the limit is dominated by the size of the data sample and systematic uncertainties have a small impact.
The systematic uncertainty in the limits results only from the uncertainty in the signal description, as the background is obtained from data and biases in the fitting procedure have been found to be negligible. A summary of the systematic uncertainties is given in Table 2. The uncertainty can be separated into the uncertainty resulting from theoretical predictions and from the uncertainty in detector reconstruction and selection efficiency.
Theoretical uncertainties come from the effects of the PDF choice on signal cross section, the missing higher-order calculations (scale) [38][39][40][41][42], and the uncertainty in the prediction on the Higgs boson decay branching fraction [4,11]. The uncertainty due to the muon reconstruction efficiency, 11%, is obtained from data using J/ψ → µµ events. It is dominated by the statistical uncertainty of the data sample. In the electron channel, the corresponding uncertainty, 3.5%, is obtained from simulation. The 11% uncertainty estimated for the muon identification efficiency is sufficiently small and it has no impact on our result, thus no simulation study was attempted, although it could greatly reduce the uncertainty. Pileup reweighting 0.8% Trigger efficiency, µ (e) channel 4 (2)% Muon reconstruction efficiency 11% Electron reconstruction efficiency 3.5% Photon reconstruction efficiency 0.6% m γ scale, µ (e) channel 0.1 (0.5)% m γ resolution, µ (e) channel 10 (10)% The expected and observed individual and combined µµγ and eeγ limits are shown in Fig. 6. The limits are calculated at 1 GeV intervals in the 120-150 GeV mass range. The median expected exclusion limits at 95% confidence level (CL) are between 6 and 10 times the SM prediction and the observed limit ranges between about 5 and 11 times the SM. The observed (expected) limit for m H = 125 GeV is 6.7 (5.9 +2.8 −1.8 ) times the SM prediction. The 95% CL exclusion limits on σ(pp → H) B(H → µµγ) for a narrow scalar particle without assuming the decay kinematics of a SM Higgs boson, in the muon channel, are shown in Fig. 7. The observed (expected) limit for m H = 125 GeV is 7.3 (5.2 +2.4 −1.6 ) fb. The total signal efficiency is 24% and almost independent of the dimuon invariant mass. In the electron channel, however, this efficiency depends on the dielectron mass, since it is strongly shaped by the selection. For this reason the corresponding limit in the electron channel is not evaluated.
Additionally, for the SM Higgs boson with m H = 125 GeV, we place an upper limit for a 2.9 < m < 3.3 GeV region in the muon channel: σ(pp → H) B(H → µµγ) < 1.80 fb, while the expected limit is 1.90 ± 0.97 fb. One can interpret this result as an upper limit on σ(pp → H) B(H → (J/ψ)γ → µµγ) and obtain for the branching fraction, B(H → (J/ψ)γ) < 1.5 × 10 −3 at 95% CL, which is about 540 times the prediction in Ref. [6]. The limit on the branching fraction at 90% CL is B(H → (J/ψ)γ) < 1.2 × 10 −3 . The number of events present in this m µµ mass window coming from the H → γ * γ → µµγ is large compared to the H → (J/ψ)γ → µµγ (as shown in Table 1). On the other hand it is small compared to the total background, hence it is considered as a part of the background when extracting the limit on B(H → (J/ψ)γ).

Summary
A search for a Higgs boson decay H → γ * γ → γ is presented. No excess above the background predictions has been found in the three-body invariant mass range 120 < m γ < 150 GeV. Limits on the Higgs boson production cross section times the H → γ * γ → γ branching fraction divided by the SM values have been derived. The observed limit for m H = 125 GeV is about 6.7 times the SM prediction. Limits at 95% CL on σ(pp → H) B(H → µµγ) for a narrow resonance are also obtained in the muon channel. The observed limit for m H = 125 GeV is 7.3 fb. Events consistent with the J/ψ in dimuon invariant mass are used to set a 95% CL limit on the branching fraction B(H → (J/ψ)γ) < 1.5 × 10 −3 , that is, 540 times the SM prediction for m H = 125 GeV.

Acknowledgments
We congratulate our colleagues in the CERN accelerator departments for the excellent performance of the LHC and thank the technical and administrative staffs at CERN and at other CMS institutes for their contributions to the success of the CMS effort. In addition, we gratefully acknowledge the computing centres and personnel of the Worldwide LHC Computing Grid for delivering so effectively the computing infrastructure essential to our analyses. Finally, we acknowledge the enduring support for the construction and operation of the LHC and the CMS detector provided by the following funding agencies:  [15] CMS Collaboration, "The CMS experiment at the CERN LHC", JINST 3 (2008) S08004, doi:10.1088/1748-0221/3/08/S08004.