Search for Lepton-Flavor-Violating and Lepton-Number-Violating tau to lhh' Decay Modes

We search for lepton-flavor-violating and lepton-number-violating tau decays into a lepton (l = electron or muon) and two charged mesons (h, h' = pion or Kaon) using 854 fb^{-1} of data collected with the Belle detector at the KEKB asymmetric-energy e^+e^- collider. We obtain 90% confidence level upper limits on the tau to lhh' branching fractions in the range (2.0-8.4)*10^{-8}. These results improve upon our previously published upper limits by factors of about 1.8 on average.


Abstract
We search for lepton-flavor and lepton-number-violating τ decays into a lepton (ℓ = electron or muon) and two charged mesons (h, h ′ = π ± or K ± ) using 854 fb −1 of data collected with the Belle detector at the KEKB asymmetric-energy e + e − collider. We obtain 90% confidence level upper limits on the τ → ℓhh ′ branching fractions in the range (
The Belle detector is a large-solid-angle magnetic spectrometer that consists of a silicon vertex detector (SVD), a 50-layer central drift chamber (CDC), an array of aerogel threshold Cherenkov counters (ACC), a barrel-like arrangement of time-of-flight scintillation counters (TOF), and an electromagnetic calorimeter comprised of CsI(Tl) crystals (ECL), all located inside a superconducting solenoid coil that provides a 1.5 T magnetic field. An iron fluxreturn located outside of the coil is instrumented to detect K 0 L mesons and to identify muons (KLM). The detector is described in detail elsewhere [9].
Particle identification is very important for this measurement. We use particle identification likelihood variables based on the ratio of the energy deposited in the ECL to the momentum measured in the SVD and CDC, shower shape in the ECL, the particle's range in the KLM, hit information from the ACC, dE/dx measured in the CDC, and the particle's time of flight. To distinguish hadron species, we use likelihood ratios, P(i/j) = L i /(L i +L j ), where L i (L j ) is the likelihood of the observed detector response for a track with flavor i (j). For lepton identification, we form likelihood ratios P(e) [13] and P(µ) [14] using the responses of the appropriate subdetectors.
We use Monte Carlo (MC) samples to estimate the signal efficiency and optimize the event selection. Signal and background event samples from generic τ + τ − decays are generated by KKMC/TAUOLA [15]. For signal, we generate the e + e − → τ + τ − process, in which one τ is forced to decay into a lepton and two charged mesons using a three-body phase space model, while the other τ decays following SM branching ratios. Background samples from BB and continuum e + e − → qq (q = u, d, s, c) processes are generated by EvtGen [16] while Bhabha and two-photon processes are generated by BHLUMI [17] and AAFH [18], respectively. In what follows, all kinematic variables are calculated in the laboratory frame unless otherwise specified. In particular, variables calculated in the e + e − CM frame are indicated by the superscript "CM." [ * ] Throughout this paper, charge-conjugate modes are implied; hence, the notation τ → ℓhh ′ includes both

EVENT SELECTION
We search for τ + τ − events in which one τ (the signal τ ) decays into a lepton and two charged mesons (h, h ′ = π ± or K ± ), and the other τ (the tag τ ) decays into one charged track, any number of additional photons and neutrinos.
For each candidate event we calculate the ℓhh ′ invariant mass (M ℓhh ′ ) and the difference of the ℓhh ′ energy from the beam energy in the CM frame (∆E). In the two-dimensional distribution of M ℓhh ′ versus ∆E, signal events should have M ℓhh ′ close to the τ -lepton mass (m τ ) and ∆E close to zero. From MC simulations, the dominant background for the τ → µhh ′ modes is from continuum and generic τ + τ − processes, while that for the τ → ehh ′ modes is from two-photon processes. Therefore, the event selection is optimized mode-by-mode since the backgrounds are mode dependent. This analysis includes 27% more data than in the previous one [11]. We reoptimized the selection criteria by reblinding the whole data set in the signal region until all selection criteria are finalized. The selection criteria are determined so that the figure of merit (FOM), defined by FOM=(Signal detection efficiency)/ √ No. of expected background events, is maximized. Here, the expected background is estimated using events from the simulated samples within the ±20σ M ℓhh ′ and ±5σ ∆E region on the M ℓhh ′ vs ∆E plane, where σ M ℓhh ′ /∆E is the resolution of M ℓhh ′ or ∆E, respectively, and details will be discussed later.
Candidate τ -pair events are required to have four tracks with zero net charge. All charged tracks and photons are required to be reconstructed within the fiducial volume defined by −0.866 < cos θ < 0.956, where θ is the polar angle with respect to the direction along the e + beam. Each charged track should have transverse momentum (p t ) greater than 0.1 GeV/c while each photon should have energy (E γ ) greater than 0.1 GeV. For each charged track, the distance of the closest point with respect to the interaction point is required to be less than 0.5 cm in the transverse direction and less than 3.0 cm in the longitudinal direction.
Using the plane perpendicular to the CM thrust axis [19], which is calculated from the observed tracks and photon candidates, we separate the particles in an event into two hemispheres. These are referred to as the signal and tag sides. The tag side is required to include a single charged track while the signal side is required to contain three charged tracks. We require one of three charged tracks on the signal side to be identified as a lepton. The electron and muon identification criteria are P(e) > 0.9 with momentum p > 0.6 GeV/c and P(µ) > 0.95 with p > 1.0 GeV/c, respectively. In order to take into account the emission of bremsstrahlung photons from the electron, the momentum of the electron candidate is reconstructed by adding to it the momentum of every photon within 0.05 radians of the electron track direction. The electron (muon) identification efficiency is 89% (81%) while the probability to misidentify a pion as an electron (a muon) is 0.7% (1.2%).
Charged kaons are identified by a condition P(K/π) > (0.6 − 0.9) for each mode, as shown in Table I, while charged pions are identified by the requirement P(K/π) < 0.6. Furthermore, we apply a proton veto for kaon candidates, P(p/K) < 0.6, to reduce protons incorrectly identified as kaons. The kaon (pion) identification efficiency is around 80% (88%) while the probability to misidentify a pion (kaon) as a kaon (pion) is below 10% (12%). In order to suppress background from photon conversions (i.e. γ → e + e − ), we require each h or h ′ candidate track to have P(e) < 0.1. Furthermore, we apply the condition P(µ) < 0.1 to suppress two-photon background from e + e − → e + e − µ + µ − .
Because a signal decay has no neutrino, the missing momentum is entirely due to neutrinos emitted from the tag side. The missing momentum p miss is defined as a difference between I: Selection criteria for kaon identification P(K/π) and magnitude of thrust (T ).
the sum of the e + and e − beam momenta and the vector sum of the momenta of all tracks and photons, where photons are measured in the ECL as clusters to which no charged tracks are associated. Since the direction of p miss should lie within the tag side of the event, the cosine of the opening angle between p miss and the charged track on the tag side in the CM system, cos θ CM tag−miss , is required to be greater than zero. If the track on the tag side is a hadron, we also require cos θ CM tag−miss < 0.85 for the τ → µhh ′ modes. This requirement reduces continuum background with missing energy due to neutrons or K 0 L 's since the masses of the neutron and K 0 L are larger than that of the neutrino. We also require that cos θ CM tag−miss < 0.96 for τ → ehh ′ modes. This requirement reduces Bhabha, inelastic vector meson-photoproduction, and two-photon background, since these processes produce electron in the tag-side in many cases and the electron can produce the radiated photons which result in missing momentum if they overlap with the ECL clusters associated with the tag-side track [20]. In addition, in order to ensure that the missing particles are neutrinos rather than photons or charged particles that leave the detector acceptance, we impose requirements on p miss : We require that |p t miss |, the magnitude of the transverse component of p miss , be greater than 0.5 GeV/c (0.7 GeV/c) for the τ → µhh ′ (ehh ′ ) modes, and that its direction point into the fiducial volume of the detector. For the τ − → e − π + π − mode only, we apply the tighter selection requirement |p t miss | > 1.5 GeV/c. To reject continuum, Bhabha and µ + µ − background, we require the magnitude of the thrust (T ) to be in the ranges given in Table I.
To suppress the BB and continuum background, we require that the number of photons on the tag side be n TAG γ ≤ 2 and n TAG γ ≤ 1 for decays with hadronic and leptonic tags, respectively. A leptonic tag is defined as P(e) > 0.1 or P(µ) > 0.1 while a tag is hadronic if the leptonic requirements are not satisfied. We allow at most one additional photon on the signal side. The reconstructed mass of the tag side, combining the photons with the charged track (assumed to be a pion mass) from the tag side, m tag , is required to be less than 1.0 GeV/c 2 to reduce the continuum background.
Photon conversions can result in large backgrounds when an electron is reconstructed as a hadron and still passes the electron veto. For the τ → ehh ′ modes, the e − h + and , assigning the electron mass to both tracks, are required to be greater than 0.2 GeV/c 2 to reduce photon conversion and other backgrounds.
For the τ → µhh ′ modes, a real muon track can come from a kaon decaying in the CDC (K ± → µ ± ν µ ). Therefore, we apply a kaon veto, P(K/π) < 0.6, for muon candidate tracks if the tag side track is a hadron (see Fig. 1 (a)). Another significant continuum background is from di-baryon production with a proton on the tag side. To suppress this background, we apply a proton veto, P(p/π) < 0.6 and P(p/K) < 0.6, as shown in Figs. 1 (b) and (c). For the τ − → µ − π + π − mode, we reduce the e + e − → µ + µ − γ background (with the photon converting into an electron-positron pair) by requiring that the invariant mass of the pion pair, evaluated by assuming the electron mass for both tracks, exceed 0.2 GeV/c 2 . In addition, we require the momentum of a muon in the CM system be less than 4.0 GeV/c and cos θ CM tag−miss < 0.97 if the track is a muon candidate with P(µ) > 0.1. For the τ → ℓπK modes, the background in the signal region is from τ − → π − π + π − ν τ events in which both the kaon and lepton candidates are misidentified. To reduce this background, we require the invariant mass of the three charged tracks on the signal side, M πππ , reconstructed by assigning the pion mass to the tracks, be larger than 1.52 GeV/c 2 (see Fig.2).
Finally, to suppress backgrounds from generic τ + τ − and continuum processes, we apply a selection based on the magnitude of the missing mass squared m 2 miss . The variable m 2 miss is defined as E 2 miss −p 2 miss , where E miss = E total −E vis , E total is the sum of the beam energies and E vis is the total visible energy. We apply different selection criteria depending on the type of a one-prong tag: there should be two emitted neutrinos (a single missing neutrino) if the tagging track is leptonic (hadronic). For the τ → ehh ′ , µππ and µKK modes, we impose the requirements −1.5 (GeV/c 2 ) 2 < m 2 miss < 1.5 (GeV/c 2 ) 2 for the hadronic tag and −1.0 (GeV/c 2 ) 2 < m 2 miss < 2.5 (GeV/c 2 ) 2 for the leptonic tag. For the τ → µπK modes, where the residual background after all selections is larger than in other modes, we require the following relation between p miss and m 2 miss : p miss > −8.0 × m 2 miss − 0.5 and p miss > 8.0 × m 2 miss − 0.5 for the hadronic tag and p miss > −9.0 × m 2 miss + 0.4 and p miss > 1.8 × m 2 miss − 0.4 for the leptonic tag; here p miss is in GeV/c and m miss is in GeV/c 2 (see Fig. 3). Typically, 75% of the generic τ + τ − background is removed by these m 2 miss requirements while 75% of the signal events are retained.

SIGNAL AND BACKGROUND ESTIMATION
For all modes, the M ℓhh ′ and ∆E resolutions are obtained from fits to the signal MC distributions, using an asymmetric Gaussian function that takes into account initial-state radiation. These Gaussians have widths as shown in Table II.
To evaluate the branching fractions, we use elliptical signal regions that contain ±3σ of the MC signal satisfying all selection criteria. The ellipse is defined as (1) where M 0 ℓhh ′ and ∆E 0 are the coordinates of the center of the ellipse and σ M ℓhh ′ (σ ∆E ) is the average of σ high M ℓhh ′ and σ low M ℓhh ′ (σ high ∆E and σ low ∆E ) in Table II. These elliptical regions are determined by scanning M 0 ℓhh ′ , ∆E 0 and θ to maximize the significance in MC simulation and obtain the highest sensitivity. Table III summarizes the signal efficiencies for each mode. We blind the data in the signal region until all selection criteria are finalized to avoid bias. Figures 4 and 5 show scatter plots for data and signal MC samples distributed over a ±20σ rectangular region in the M ℓhh ′ − ∆E plane for the τ → µhh ′ and ehh ′ modes, respectively. For the τ → µππ modes, the dominant background is from continuum processes while small background contributions come from generic τ + τ − events in the ∆E < 0 GeV and M µhh ′ < m τ region, which are combinations of a misreconstructed muon and two pions. For the τ → µKK modes, the dominant background sources are continuum and τ + τ − processes with a pion misidentified as a kaon. For the τ → µπK modes, the dominant background is from generic τ + τ − decays with combinations of a misidentified muon, a misidentified kaon and a real pion from τ − → π − π + π − ν. If a pion is misidentified as a kaon, the reconstructed mass from generic τ + τ − background can be greater than the τ lepton mass since the kaon mass is greater than that of the pion. For the τ → ehh ′ modes, the dominant background originates from two-photon processes, while the background from continuum and generic τ + τ − processes is small due to the low electron fake rate.
In order to estimate the background in the signal region, we use the number of data events observed in the sideband region inside a ±5σ ∆E band around ∆E = 0 GeV but excluding the signal region. The events in this band are projected onto the M ℓhh ′ axis. For the τ → µππ modes, the extrapolation to the signal region is performed by fitting the sideband of the projected M µππ distribution using the sum of an exponential and a first-order polynomial function for generic τ τ and continuum, respectively. For the τ → ehh ′ , µKK and µπK modes, the background remaining after all selections is small, and extrapolation to the signal region assumes that the background is linear as a function of M ℓhh ′ . The systematic uncertainty due to the estimation of the expected background includes the contributions due to the statistics of the background sample and the shape of the background distribution. By varying the assumptions about the background shape, we verify that this effect on the systematic uncertainty is less than 20% and is smaller than the background statistical error. The signal efficiency and the number of expected background events with its uncertainty, obtained by adding statistical and systematic uncertainties in quadrature for each mode, are summarized in Table III. After estimating the background, we open the blinded regions. (MeV/c 2 ) and σ high/low ∆E (MeV)). Here σ high (σ low ) is the standard deviation on the higher (lower) side of the peak. We observe one candidate event for each of the τ − → µ + π − π − and µ − π + K − modes, and no candidate events for the other modes. The numbers of events observed in the signal region are consistent with the expected background levels.
The dominant systematic uncertainties for this analysis come from the resolution in M ℓhh ′ and ∆E and from particle identification. We estimate the uncertainties from resolutions of M ℓhh ′ and ∆E due to the difference between data and MC samples to be (3.7 − 4.8)%. The uncertainties due to lepton identification are 2.2% and 1.9% from modes with an electron and a muon, respectively. The uncertainties due to hadron identification are 1.3% and 1.8% for pion and kaon candidates, respectively. The uncertainty due to the charged track finding is estimated to be 0.35% per charged track; therefore, the total uncertainty due to the charged track finding is 1.4%. The uncertainty due to integrated luminosity is estimated to be 1.4%. The uncertainties due to the trigger efficiency and MC statistics are negligible compared with the other uncertainties. All these uncertainties are added in quadrature, and the total systematic uncertainties for all modes are (5.5−6.7)%.

UPPER LIMITS ON THE BRANCHING FRACTIONS
Since no statistically significant excess of data over the expected background in the signal region is observed, we set upper limits on the branching fractions of the τ → ℓhh ′ modes using the Feldman-Cousins method [21]. The 90% C.L. upper limit on the number of signal events including systematic uncertainty (s 90 ) is obtained using the POLE program without conditioning [22] based on the number of expected background events, the number of observed events and the systematic uncertainty. The upper limit on the branching fraction , the number of expected background events (N BG ) estimated from the sideband data, the total systematic uncertainty (σ syst ), the number of observed events in the signal region (N obs ), 90% C.L. upper limit on the number of signal events including systematic uncertainties (s 90 ) and 90% C.L. upper limit on the branching fraction (B) for each individual mode.

(B) is then given by
where N τ τ is the number of τ + τ − pairs, and ε is the signal efficiency. The value N τ τ = 782 × 10 6 is obtained from the integrated luminosity and the cross section of τ -pair production, which is calculated in KKMC [23] to be σ τ τ = 0.919 ± 0.003 nb and σ τ τ = 0.875 ± 0.003 nb for 782 fb −1 at Υ(4S) and 72 fb −1 at Υ(5S), respectively. Table III summarizes the results for all modes. The upper limits for the τ → ehh ′ modes are in the range (2.0 − 3.7) × 10 −8 while those for the τ → µhh ′ modes are in the range (2.1−8.6)×10 −8 . These results improve upon our previously published upper limits [11] by factors of about 1.8 on average. This improvement results both from using a larger data sample and from the introduction of an improved rejection of specific backgrounds, such as di-baryon production in the continuum for the τ → µhh ′ modes, and improved kinematic event selections, for example, the M πππ requirement for the τ → ℓπK modes. Scatter plots in the M ℓhh ′ -∆E plane within a ±20σ region for the (a) τ − → µ − π + π − , (b) τ − → µ + π − π − , (c) τ − → µ − K + K − , (d) τ − → µ + K − K − , (e) τ − → µ − π + K − , (f) τ − → µ − K + π − , and (g) τ − → µ + π − K − modes. The data are indicated by the solid circles. The filled boxes show the MC signal distribution with arbitrary normalization. The elliptical signal regions shown by solid curves are used for evaluating the signal yield. The region between two horizontal solid lines excluding the signal region is used as a sideband.
limits by factors of about 1.8 on average. These more stringent upper limits can be used to constrain the space of parameters in various models of new physics. FIG. 5: Scatter plots in the M ℓhh ′ -∆E plane within a ±20σ region for a (a) τ − → e − π + π − , (b) τ − → e + π − π − , (c) τ − → e − K + K − , (d) τ − → e + K − K − , (e) τ − → e − π + K − , (f) τ − → e − K + π − , and (g) τ − → e + π − K − modes. The data are indicated by the solid circles. The filled boxes show the MC signal distribution with arbitrary normalization. The elliptical signal regions shown by solid curves are used for evaluating the signal yield. The region between two horizontal solid lines excluding the signal region is used as a sideband.