Search for lepton flavor violation in the Higgs boson decay at a linear collider

We discuss possibility of direct search for lepton flavor violation (LFV) in Yukawa interaction by measuring the bra ratio for the decay of the lightest Higgs boson (h0) into aτ–μ pair at a linear collider (LC). We study the significance of signal process, e+e− → Z∗ → Zh0 → Zτ±μ∓, against the backgrounds such as e+e− → Zτ+τ− → Zτ±μ∓+ missings. After taking appropriate kinematic cuts, the number of the background event is considerably reduced, so that the s be visible when the branching ratio of h0 → τ±μ∓ is larger than about 10 −4. In a minimal supersymmetric Standard Mod scenario, the effective coupling of h0τ±μ∓ can be generated at the loop level due to the slepton mixing. When supersym mass parameters are larger than TeV scales, the branching ratio can be as large as several times 10 −4. Therefore, the signal ca be marginally visible at a LC. In the general two-Higgs-doublet model, the possible maximal value for the branching h0 → τ±μ∓ can reach to a few times 10 −3 within the available experimental bound, so that we can obtain larger signific  2004 Elsevier B.V.Open access under CC BY license.


Lepton flavor violation (LFV) is a direct indication of new physics beyond the Standard Model (SM). It
In recent years, the Higgs-mediated LF-violating processes have been studied regarding the decay modes τ ± → µ ± µ ± µ ∓ [4][5][6], τ ± → µ ± η [7], and B s → µ ± τ ∓ [5]. Their branching fractions are being measured at current and forthcoming experiments at the B-factories [8] and CERN Large Hadron Collider (LHC) [9]. The Higgs-mediated LFV have also been investigated in muon processes: the muon-electron conversion in nucleus is studied in Ref. [10]. It will be explored at the Muon to Electron COnversion experiment (MECO) [11] and the PRISM Muon Electron conversion experiment (PRIME) based on the phase rotated intense slow muon (PRISM) [12]. All these processes are measured as a combination of contributions from the gauge boson mediation and the Higgs boson mediation.
In this Letter, we consider possibility of detecting the process of the lightest Higgs boson decaying into a pair of tau and muon, h 0 → τ ± µ ∓ , at a linear collider (LC). There LFV in Yukawa interaction can be directly studied by measuring the decay branching ratio of the Higgs bosons [6,[13][14][15] when they are found. In the minimal supersymmetric Standard Model (MSSM), the mass of the lightest Higgs boson is less than about 130 GeV. It is promising that such a light Higgs boson will be discovered at LHC. Then its properties such as the mass, the width, production cross sections, and decay branching ratios will be measured extensively. The precision study of the Higgs sector is one of the main purposes of a LC such as GLC, TESLA or NLC [16]. The lightest Higgs boson is produced mainly through gauge interactions at a LC. In the case of the nearly decoupling region [17], the production cross section for the lightest Higgs boson is much larger than that of the heavier ones. Therefore, the LF-violating Yukawa coupling can be better tested from the decay of the lightest Higgs boson than that of the extra (heavier) Higgs bosons.
At LHC, the extra Higgs bosons (H 0 , A 0 , H ± ) may also be detected in the MSSM and the THDM as long as their masses are not too large. The Higgs bosons are mainly produced through the Yukawa interaction, so that the production cross section of H 0 and A 0 can be sufficiently large to be detected especially for large tan β values, where tan β is the ratio of vacuum expectation values of two Higgs doublets. Therefore, the decays of H 0 and A 0 may be useful to explore the LF-violating Yukawa coupling [14]. The search of LFV via the Higgs boson decay at a hadron collider is suffered from huge backgrounds, and it should be required to pay much effort into the background reduction.
Magnitude of the LF-violating coupling in h 0 → τ ± µ ∓ is constrained by the results from the measurement of LFV in tau decay processes. The most stringent bound comes from the τ − → µ − η measurement [18]. In the framework of the MSSM, the theoretical prediction on the branching ratio of h 0 → τ ± µ ∓ can approach to the above experimental upper limit by adjusting the SUSY parameters [13]; i.e., Br(h 0 → τ ± µ ∓ ) ∼ several times 10 −4 . When all the SUSY parameters are as large as TeV scales, the LF-violating gauge-boson penguin diagram decouples from the experimental reach, while the LF-violating Yukawa coupling does not because they depend only on the ratio of the SUSY parameters. We can then avoid strong correlation between the LFV mediated by Higgs bosons and that by the gauge bosons. On the contrary, if the scale of the SUSY parameters is smaller than 1 TeV, the Higgs-mediated LF-violating coupling is strongly constrained from the experimental bounds on the gauge mediated LFV processes [5]. In such a case, the parameter choice which realizes Br(h 0 → τ ± µ ∓ ) ∼ O(10 −4 ) is already excluded by the data.
We evaluate the significance of detecting the signal for h 0 → τ ± µ ∓ at a LC. The Higgs boson with the mass around 120 GeV is mainly produced through the Higgsstrahlung mechanism e + e − → Zh 0 , when the center-of-mass energy √ s is lower than about 500 GeV. We can identify the signal event (τ ± µ ∓ Z) without measuring the tau lepton by using the information of the momenta for the outgoing Z boson and muon as well as the fixed beam energy √ s. The momentum of the Z boson is reconstructed from those of its leptonic ( + − with ± = e ± and µ ± ) as well as hadronic (jj ) products. The most serious irreducible background is e + e − → Zh 0 → Zτ + τ − with one of the tau leptons going to a muon and missings. The background can be suppressed by appropriate kinematic cuts with the expected resolution of the momentum of the Z boson from the decay channels into + − and jj and with the beam spread rate of √ s. We find that the significance S/ √ B can exceed 5 in the MSSM scenario when the SUSY parameters are taken to be as large as TeV scales. In the general THDM, the larger number of the signal events can be realized under the constraint from the perturbative unitarity [19,20], the vacuum stability [21] and available data. Therefore, the signal can be marginally detectable in the MSSM.
In Section 2, the possible enhancement of the decay branching ratio for the process h 0 → τ ± µ ∓ is discussed taking into account the current experimental data. We show a choice of the SUSY parameters that realizes a relatively large value of the effective h 0 µ ± τ ∓ coupling in the MSSM. In Section 3, we estimate the significance of detection for the signal against the backgrounds at a LC, taking into account appropriate kinematic cuts. The conclusions are given in Section 4.

Lepton flavor violating Yukawa coupling
The effective Lagrangian of Yukawa interaction for charged leptons in the THDM (including the MSSM) is described as are charged leptons with chirality L or R, Φ α (α = 1, 2) are neutral components of the two Higgs doublets with the hypercharge 1/2, and Y i (= m i / Φ 1 ) are the Yukawa coupling constants of i , respectively. In the MSSM, Φ 1 and Φ 2 correspond to H 0 d and H 0 * u , respectively [22]. With a nonzero value of ij (i = j ), the Yukawa interaction and the mass of charged leptons cannot be diagonalized simultaneously, so that the LF-violating Higgs couplings arise. The interaction corresponds to τ -µ or τ -e mixing is expressed [4,5,10,13] by with L i = e L or µ L , and the LF-violating parameter κ ij is given by where h 0 and H 0 are the CP-even Higgs bosons, A 0 is the CP-odd Higgs boson, α denotes the mixing angle of the CP-even Higgs bosons, and tan β is the ratio of the vacuum expectation values, tan β ≡ Φ 2 / Φ 1 . We define h 0 is lighter than H 0 (m h < m H ).
Let us discuss the branching ratio of the Higgs boson decaying into the LF-violating channel (τ ± µ ∓ ). We consider the situation that the main decay mode of the lightest Higgs boson is h 0 → bb. In addition, for a large tan β and sin(α − β) −1, the dominant decay modes of heavier Higgs bosons are those into a bb pair. In this case, the rate between the decay widths of the LF-violating process In our numerical evaluation, we calculate these branching ratios including all the decay modes; i.e., The LF-violating parameter |κ 32 | is constrained from the available data for LFV in tau decay processes. The most stringent bound is obtained from the τ − → µ − η measurement [18]. The other experiments such as τ ± → µ ± µ ± µ ∓ [23], τ ± → µ ± γ [24], and the muon anomalous magnetic moment give weaker bounds. The branching ratio of τ ± → µ ± η is given [4,5,7] by for tan β 1 and sin(α − β) −1, where G F is the Fermi constant and τ τ is the lifetime of the tau lepton. The present experimental bound is given by Br(τ − → µ − η) < 3.4 × 10 −7 (90% CL) [18], which yields for m A ∼ m H . The bound becomes relaxed for greater m A and smaller tan β values.
Next, we discuss theory predictions on the LFviolating parameter κ 32 in the framework of the MSSM. Nonzero values of ij then arise from the radiative correction due to the slepton mixing. They are calculated in the mass insertion method as ij ≡ [4][5][6]10,13], with (9) where α and α 2 are fine structure constants of U(1) Y and SU(2) L symmetries, M 1 and M 2 are the soft-SUSY-breaking masses for gauginos, µ is the SUSYinvariant Higgs mixing parameter, and ml Li , mẽ Ri and mν Li are the left-and right-handed charged slepton and sneutrino masses of the ith generation, respectively. The off-diagonal element of the slepton mass matrix is expressed by (∆m 2 l L ) ij , (i = j ). 2 The functions I 3 and I 4 are defined as (11) 2 We here assume the situation in which the origin of LFV is only the mixing of the left-handed slepton. The formulas which include the mixing of the right-handed slepton are shown in Ref. [13]. .
Unlike the photon-mediation, the LF-violating Yukawa coupling does not decouple for large values of the SUSY parameters. It depends only on the ratio of the SUSY parameters. For instance, by assuming where R ≡ µ/m S . Therefore, magnitude of |κ 32 | 2 becomes greater as R is larger. for Case 2 with m A = 280 GeV, respectively. We note that these extreme choices are not excluded by the condition of theory consistencies, such as color breaking, positiveness of eigenvalues of squark and slepton mass matrices.
In Fig. 1, the decay branching ratio for the process h 0 → τ ± µ ∓ is shown as a function of m A at tan β = 60. We take the other SUSY parameters so that the  8) is also plotted as a dotted curve for each case. The branching ratio Br(h 0 → τ ± µ ∓ ) can reach to 7×10 −4 and 2 × 10 −4 for Case 1 and Case 2, respectively. In a wide region of m A , the branching ratio can be as large as 10 −4 for both cases.
In the THDM, the parameters ij in Eq. (1) can be taken freely within the experimental constraints and conditions from perturbative unitarity [19,20] and vacuum stability [21]. The experimental bound on |κ 32 | can be weakened by considering the large value of m A (> 150 GeV) and smaller tan β (< 60). Therefore, much larger values of |κ 32 | are allowed in the THDM than those in the MSSM, especially for lower tan β values.

Search for LF-violating Higgs decays at a linear collider
Let us consider the LF-violating Higgs decay h 0 → τ ± µ ∓ at a LC in the situation where the heavier Higgs bosons nearly decouple from the gauge bosons; i.e., sin(α − β) −1. The lightest Higgs boson then approximately behaves as the SM one. The main production modes of the lightest Higgs boson at a LC are the Higgsstrahlung e + e − → Z * → Zh 0 and the W fusion e + e − → (W + * ν e )(W − * ν e ) → h 0 ν eνe . For a light h 0 with the mass m h ∼ 120 GeV, the former production mechanism is dominant at low collision energies ( √ s < 400-500 GeV), while the latter dominates at higher energies. For our purpose, the Higgsstrahlung process is useful because of its simple kinematic structure. The signal process is then e + e − → Z * → Zh 0 → Zτ ± µ ∓ . We can detect the outgoing muon with high efficiency, and its momentum can be measured precisely by event-by-event. The momentum of the Z boson can be reconstructed from those of its leptonic + − ( ± = e ± and µ ± ) or hadronic (jj ) decay products. Therefore, we can identify the signal event without measuring tau momentum directly, as long as the beam spread rate for √ s is sufficiently low.
Depending on the Z decay channel, the signal events are separated into two categories, jj τ ± µ ∓ and + − τ ± µ ∓ . The energy resolution of the Z boson from hadronic jets jj is expected to be 0.3 √ E Z GeV and that from + − is 0.1 √ E Z GeV [16], where E Z represents the value of the Z boson energy in units of GeV. We assume that the detection efficiencies of the Z boson and the muon are 100%, the rate of the beam energy spread is expected to be 0.1% level [16], the muon momentum is measured with high precision and the mass of the lightest Higgs boson will have been determined in the 50 MeV accuracy [16]. We also expect that the effect of the initial state radiation is small for the collider energies that we consider ( √ s ∼ 250-300 GeV). Taking into account all these numbers, we expect that the tau momentum can be determined indirectly within 3 GeV for jj τ ± µ ∓ and 1 GeV for + − τ ± µ ∓ .
Let us evaluate the number of the signal event. We assume that the energy √ s is tuned depending on the mass of the lightest Higgs boson: i.e., we take the optimal √ s to product the lightest Higgs boson through the Higgsstrahlung process. (It is approximately given by The production cross section of e + e − → Zh 0 is about 220 fb for m h = 123 GeV. Then, we obtain 2.2 × 10 5 Higgs events if the integrated luminosity is 1 ab −1 . When |κ 32 | 2 is 8.4 × 10 −6 , about 118 events of jj τ ± µ ∓ and 11 events of + − τ ± µ ∓ can be produced. Next, we consider the background. For the signal with the Higgs boson mass of 120 GeV, the main background comes from e + e − → Zτ + τ − . The number of the Zτ ± µ ∓ event from e + e − → Zτ + τ − is estimated about 3.6 × 10 4 [25]. Although the number of the background events is huge, we can expect that a large part of them is effectively suppressed by using the following kinematic cuts: (i) the muon from the Higgs boson should have high energies larger than √ s/4, while those of the muon from the other parent are normally smaller. Therefore, we impose the cut E µ > √ s/4. (ii) The invariant mass M µτ distribution of the signal event (which is reconstructed from the information of the beam spread rate of √ s as well as the momenta of the outgoing muon and the Z boson) should be located at the mass of the lightest Higgs boson, while that of the background is widely distributed. By taking only events which satisfy |M µτ − m h | < Max[Γ h , ∆m h , ∆M µτ ], the background events are expected to be considerably reduced, where Γ h (∼ 40 MeV for m h = 120 GeV) is the natural width of h 0 , ∆m h is the experimental uncertainty of m h (∼ 50 MeV), and ∆M µτ is the uncertainty of the recoil invariant mass M µτ . We here assume that ∆M µτ is 1 GeV for the Z → + − channel and 3 GeV for Z → jj . The irreducible background comes from the process shown in Fig. 2(b): the Higgs boson decays into a tau pair, and one of the tau decays into a muon and missings (e + e − → Zh 0 → Zτ + τ − → Zτ ± µ ∓ + missings). We cannot distinguish the signal event h 0 → τ ± µ ∓ with the event of Fig. 2(b) when the muon emitted from the tau lepton carries the similar momentum to that of the parent, because it leaves the same track on the detector as the signal event. We refer this kind of the background as the fake signal. In the following, we estimate the number of the fake signal. As the branching ratio for h 0 → τ + τ − is about 0.1, the initial number of the background event for jj τ ± µ ∓ +missings is calculated to be about 5200, and that for + − τ ± µ ∓ + missings is to be 500. Since the signal includes the two-body decay of the Higgs boson h 0 → τ ± µ ∓ , its muon energy distribution shows the mono-energetic spectrum. On the other hand, that of the background, h 0 → τ + τ − → τ ± µ ∓ + missings, is the continuous spectrum. The energy and angular distribution of the muon from the tau lepton in the lab frame is calculated as where γ τ ≡ m h /(2m τ ) and β τ ≡ 1 − 1/γ 2 τ are the boost factors from the tau-rest frame to the Higgsrest frame, γ h ≡ E h /m h and β h ≡ 1 − 1/γ 2 h are the boost factors from the Higgs-rest frame to the lab frame, θ hµ is the angle between momenta of the Higgs boson and the muon, and x is defined as the ratio of the energy of the muon and that of the parent tau lepton, x ≡ E µ /E τ . Eq. (15) can be derived from the differential cross section for τ − → µ − ν τνµ in the tau-rest frame by making the boost twice. In the boost from the Higgs-rest frame to the lab frame, we take the approximation in which the muon is emitted to the forward direction of the tau lepton. The number of events of the fake signal can be evaluated as where N initial Zµτ is the initial number of the background event for Zµτ with Z → jj or Z → + − , x max is the maximal value of x which is given by , and parameter δx depends on the uncertainty of the tau momentum, δ(E τ ); We find that the number of the fake signal strongly depends on the precision of the tau momentum determination. We expect that it is attained with the similar precision to that of the Higgs boson mass reconstructed by the recoil momentum. We here take the uncertainty of the tau momentum as 3 GeV for jj τ ± µ ∓ and as 1 GeV for + − τ ± µ ∓ . Finally, we estimate the statistical significance (S/ √ B ) for each channel (see Fig. 3). The number of the fake events is evaluated by Eq. (16), which is 460 for jj τ ± µ ∓ and 15 for + − τ ± µ ∓ . Therefore, in Case 1 where |κ 32 | 2 is 8.4 ×10 −6 with m h = 123 GeV, the significance can become 5.5 and 3.0 for jj τ ± µ ∓ and + − τ ± µ ∓ at m A = 350 GeV, respectively, taking into account the constraint from the τ − → µ − η result given in Eq. (8). The combined significance can reach to 6.3. In Case 2 where |κ 32 | 2 is 3.8 × 10 −6 with m h = 123 GeV, the number of the signal becomes smaller, and the combined significance amounts to be as large as 2.0 at m A = 280 GeV.

Summary and discussions
We have discussed detecting the lepton flavor violating decay mode of the Higgs boson h 0 → τ ± µ ∓ at a LC. The effective coupling of h 0 τ ± µ ∓ is induced at one loop in the MSSM due to the slepton mixing. We have studied the situation where the typical scale of supersymmetric parameters is as large as TeV scale. The magnitude of the effective h 0 τ ± µ ∓ coupling can then be substantially large. Consequently, the number of the signal event via e + e − → Zh 0 → Zτ ± µ ∓ can be large enough to be detected after the background is suppressed by kinematic cuts. The signal can be marginally visible in the MSSM when the effective h 0 τ ± µ ∓ coupling becomes enhanced due to the large ratio of µ and m S , where m S is the typical scale of the soft-breaking mass.
When m S is greater than the TeV scale, the LFviolating processes associated with gauge bosons such as τ − → µ − γ , τ − → e − γ and µ − → e − γ are suppressed. In addition, the LF-violating processes including the Higgs mediation such as τ − → µ − η, τ − → µ − µ + µ − and µ − N → e − N as well as the flavor changing processes such as b → sγ are suppressed when m A is greater than about 300 GeV [26]. On the other hand, the branching ratio for h 0 → τ ± µ ∓ does not decouple for large m S as long as the ratio µ/m S is not small. Therefore, in such a case, the decay h 0 → τ ± µ ∓ at a LC can be a complementary process to test the Higgs mediated LF-violating coupling.
We comment on the case of the general framework of the THDM. Unlike the MSSM, the mixing angle α is independent of tan β and m A . For larger values of m A , the bound from τ − → µ − η can be relaxed by the factor of m 4 A (cf. Eq. (8)), whereas the branching ratio of the h 0 → τ ± µ ∓ can remain to be larger than 10 −3 within the available experimental and theoretical constraints. Therefore, the number of the signal in the THDM can be by a few order of magnitude larger than the possible value in the MSSM.
In our analysis, we have used the bound on the LF violating coupling |κ 32 | from the current data of τ − → µ − η. In near future, if the bound becomes strong by a few factor, the number of the signal becomes reduced by the same factor. We have assumed some important numbers which are associated with the machine property for the collider and the detector of a LC experiment. The estimation of the number of the signal events and the reduction of the background events largely depend on the detection efficiencies of Z boson and the muon, the resolution of the momenta for them, the rate of beam energy spread of the e + e − collision and the initial state radiation. Our assumption for these numbers might be rather optimistic. On the other hand, the significance can be improved when direct detection of the tau lepton is taken into account. In any case, a more realistic simulation analysis is necessary to determine feasibility of the signal.