Tests of a Family Gauge Symmetry Model at 10^3 TeV Scale

Based on a specific model with U(3) family gauge symmetry at $10^3$ TeV scale, we show its experimental signatures to search for. Since the gauge symmetry is introduced with a special purpose, its gauge coupling constant and gauge boson mass spectrum are not free. The current structure in this model leads to family number violations via exchange of extra gauge bosons. We investigate present constraints from flavor changing processes and discuss visible signatures at LHC and lepton colliders.


Introduction
In the current flavor physics, it is a big concern whether flavors can be described by concept of "symmetry" or not. If the flavors are described by a symmetry (family symmetry), it is also interesting to consider that the symmetry is gauged. (For an earlier work of gauge SU(3) symmetry, for example, see Ref. [1].) Most models with a family gauge symmetry have been introduced for the purpose of understanding mass spectra and mixings of quarks and leptons. However, it is difficult to exclude such models by the present and near future experiments, because in most models the gauge coupling constant g f and gauge boson masses are free parameters. In the present paper, we pay attention to a specific model with a U(3) family gauge symmetry which was proposed by one of the authors (YS) [2,3]. In contrast to the conventional U(3) family gauge model, the present model has been introduced to explain the charged lepton spectrum with high precision. Therefore, the gauge coupling constant g f is fixed with respect to the standard electroweak gauge coupling constants as g f /2 = e = g 2 sin θ W , and the mass spectrum of the gauge bosons is also fixed (see eq.(8) below). As a result, we can give definite predictions, which may allow these gauge bosons to be clearly detected or excluded in forthcoming experiments.
First, let us give a short review: "Why do we need a family gauge symmetry?" In the charged lepton sector, we know that an empirical relation [4] is satisfied with the order of 10 −5 with the pole masses, i.e. K pole = (2/3)×(0.999989±0.000014) [5], while it is only valid with the order of 10 −3 with the running masses, i.e. K(µ) = (2/3) × (1.00189 ± 0.00002) at µ = m Z . In conventional mass matrix models, "mass" means not "pole mass" but "running mass." Why is the mass formula (1) so remarkably satisfied with the pole masses? This has been a mysterious problem as to the relation (1) for long years. Recently, a possible solution to this problem has been proposed by one of the authors (Y.S.) [2,3]: The deviation of K(µ) from K pole is caused by a logarithmic term m ei log(µ/m ei ) in the running mass. It was advocated that a family symmetry is gauged, and that the logarithmic term in the radiative correction to K(µ) due to photon is canceled by that due to family gauge bosons.
(This does not mean m ei (µ) = m pole ei .) In order that cancellation works correctly, the lefthanded lepton field ψ L and its right-handed partner ψ R should be assigned to 3 and 3 * of U(3) [6], respectively, differently from the conventional assignment [1] The assignment (ψ L , ψ R ) = (3, 3 * ) can induce interesting observable effects. In the conventional assignment, a family gauge boson A i j couples to a current component ( In general, the currents (2) cause the violation of individual family number N f by |∆N f | = 2.
The influence of the family number violation is determined by the family gauge coupling constant g f and each family gauge boson mass m f ij ≡ m(A j i ). Here, for simplicity, the family current structure has been presented by a field ψ as a representative of quarks u and d and leptons e and ν. For example, the charged lepton current component (J ρ ) 2 1 is given by This causes an e (or µ) lepton-number-violating process e − +e − → µ − +µ − through the effective current-current interaction where ). In order to realize the cancellation mechanism between photon and family gauge bosons, g f should be related to the electric charge e as where g 2 is the gauge coupling constant of SU(2) L . In [2,3] a speculation is given that the relation (5) may originate from unification of SU(2) L and family U(3) gauge symmetries at 10 2 -10 3 TeV scale; the level of tuning of the unification scale required in this scenario is estimated to be a factor of 3 to match the present experimental accuracy of eq. (1). This model of charged lepton sector has been constructed in the context of an effective field theory with a cut-off scale Λ ∼ 10 3 -10 4 TeV, assuming this unification scenario and incorporating the family U(3) gauge symmetry. The masses of A j i are predicted to be in the 1 − 1000 TeV range.
Thus, the ratio of the coefficients of the four-Fermi contact interactions is given by Here In this model, Yukawa coupling constants Y ef f e of the charged leptons are effectively given by where Φ e is a scalar with (3, 3) of family U(3)×O(3) symmetries. (Here, the family U(3)×O(3) symmetries originate from a U(9) family symmetry [3], and only U(3) gauge symmetry can contribute to the radiative correction of the running masses of charged leptons below the cut-off scale Λ, at which the charged lepton mass relation (1) is given exactly.) In other words, the VEV matrix Φ e is given as such an idea for the charged lepton masses is found in Ref. [7] related to the mass formula (1).] Then, the gauge symmetry U(3) is completely broken by Φ e = 0, so that the gauge boson masses m f ij are related to the charged lepton masses as [3] ( The mass spectrum (8) is essential in this model. For example, if we assume (Y ef f e ) j i ∝ k (Φ e ) k i (Φ e ) j k , we cannot obtain the relation (8). It is assumed that other scalar VEV's with non-zero family charge, if they exist, have much smaller magnitudes than Φ e , such that they do not affect the family gauge boson spectrum. This is crucial to protect the cancellation mechanism within the present scenario.
The purpose of the present paper is to discuss how to test this family gauge symmetry within the above model. We note that this model is incomplete, e.g. the quark and neutrino sectors are not included, anomaly of the family gauge symmetry is not canceled. 1 We focus only on the family gauge interactions, which are fairly independent of the details of the model. We examine the interactions with |∆N f | = 2 via the gauge boson A 1 2 . In the next section, we estimate a lower bound of its mass m f 12 from the experimental limit on the branching ratio of a rare kaon decay K + → π + µ − e + , assuming that the quarks are assigned to multiplets of the U (3) × O(3) family gauge group in the same way as the charged leptons. 2 We also discuss K 0 -K 0 mixing and muonium into antimuonium conversion. (For a review of searches for signatures with |∆N f | = 2, see, for example, Ref. [8].) In Sec. 3, we investigate possible signatures in collider experiments, such as e − + e − → µ − + µ − production. Since the mass of the lightest gauge boson A 1 1 may take a value within 1-10 TeV range, we may expect a production p + p → A 1 1 + X → (e + e − ) + X at LHC. We estimate the production cross section and decay rate. Finally, Sec. 4 is devoted to a summary.

Lower bounds for the gauge boson masses
First, in order to see more details of the characteristic current structure (2), we discuss the flavor changing neutral currents relevant for µ and e. According to eq. (2), the current (J ρ ) 2 1 can be written as where (J V ) ρ = (1/2)(μγ ρ e −ēγ ρ µ) and (J A ) ρ = (1/2)(μγ ρ γ 5 e +ēγ ρ γ 5 µ). The vector current J ρ V and axial current J ρ A have CP = −1 and CP = +1, respectively. However, this does not mean that the effective current-current interactions cause CP -violating interactions. In fact, the current (J ρ ) 1 2 is written as so that the effective current-current interaction is CP conserving: Next we discuss rare kaon decays. Note that, in this model, the family number i = (1, 2, 3) is defined as (e 1 , e 2 , e 3 ) = (e, µ, τ ) in the charged lepton sector. If we assume (d 1 , in the down-quark sector, the gauge boson masses m f 12 can be constrained by the rare kaon decay searches. In general, a down-quark mass matrix M d is not necessarily diagonal in the diagonal basis of the charged lepton mass matrix M e . For simplicity, we assume that M d is Hermitian and consider only a d-s mixing where the down-quark mass matrix M d is given in the flavor basis in which the charged lepton mass matrix M e is diagonal, and M d is diagonalized as where the first, second and third terms have CP = −1, +1 and +1, respectively. Note that the vector current is independent of the mixing angle θ. (However, this is valid only with the mixing matrix eq. (11).) As an example of the s-d current, let us discuss a decay of neutral kaon into e ± + µ ∓ . In eq. (12), only the second term is relevant to a neutral kaon with spin-parity 0 − , which has CP = +1. Since the observed neutral kaons K S and K L have CP = +1 and CP = −1, respectively, in the limit of CP conservation, we must identify the second term in eq. (12) as K S (not K L ). Hence, a stringent lower limit of m f 12 cannot be extracted from the present experimental limit [5] BR(K L → e ± µ ∓ ) < 4.7 × 10 −12 .
The results are listed in Table 1. In the present model, the mass m f 33 of the heaviest gauge boson A 3 3 is predicted in the 10 2 -10 3 TeV range. On the other hand, the lower bound of m f 33 estimated from K + → π + e − µ + is 300 TeV as seen in Table 1. Therefore, the lower bound of each gauge boson listed in Table 1 seems to be almost near to its upper bound. In other words, the mass values given in Table 1 suggest that experimental observations of family gauge boson effects soon become within our reach. If we consider, however, a more general mixing of the down-type quarks, we obtain suppression factors to the above branching ratios. In this case, constraints to the gauge boson masses become looser.
A constraint on m f 12 can also be obtained from the observed value of the K 0 -K 0 mixing. The prediction for the K 0 -K 0 mixing in the present model is more sensitive to the mixing of the down-type quarks than for the rare kaon decays. Even with the simple ansatz eq. (11), the prediction depends on the value of θ. Hence, first we present the prediction in the nomixing case (θ = 0) as a reference for small mixing, and afterwards we discuss the case with a general down-type quark mixing. In contrast to the ( under the vacuum saturation approximation, which should be compared with in the conventional case. With eq. (15) we find a lower bound for m f 12 of order 10 3 TeV, which serves as a reference for small down-type quark mixing. We note that this bound is much more stringent than the values listed in Table 1 (although it may still not completely rule out the model if we take into account uncertainties in the estimate of the unification scale in the model).
If we take into account a general mixing of the down-type quarks, the prediction for the K 0 -K 0 mixing can be either larger or smaller. In particular, in the case that the mixing matrices U dL and U dR are complex, without specific tuning of the matrices, generally a very stringent constraint is imposed from the CP violation in the K 0 -K 0 mixing: m f 12 10 5 TeV [9], which rules out the present model. On the other hand, there exists a parameter region (parametrized by a set of continuous parameters), where the contribution to the K 0 -K 0 mixing vanishes. Even if we restrict the mixing matrices to real (orthogonal) matrices, such solutions exist with rather simple forms. For instance, in the case U dR = 1 and (s θ ≡ sin θ, c θ ≡ cos θ for ∀θ), the induced four-Fermi operator for the K 0 -K 0 mixing vanishes due to the characteristic form of the family gauge interactions. 3 In general (but restricting to orthogonal mixing matrices to circumvent constraints from the CP violation), if the mixing induces a coupling of the d-s current to the lightest gauge boson A 1 1 , the K 0 -K 0 mixing tends to be more enhanced and the bounds for the gauge boson masses tend to be severer. For certain choices of the mixing matrices, [e.g. U dR sufficiently close to 1 and U dL to eq. (17)], the induced four-Fermi operators are suppressed, and the lower bound for m f 12 can be reduced much below 10 3 TeV.
Let us briefly discuss bounds from the observed D 0 -D 0 mixing. In order to predict contributions of family gauge boson exchanges to the D 0 -D 0 mixing, we need to know the mixing matrices for the up-type quarks U uL and U uR . Of these, U uL is related to U dL by V CKM = U † uL U dL , where V CKM is the Cabibbo-Kobayashi-Maskawa (CKM) matrix, while U uR is unknown. Naively  17), there always exist U dR which suppress the induced four-Fermi operator for the D 0 -D 0 mixing, although we have not found particularly simple forms for the combination U uL and U uR . 4 We present a detailed analysis of the effects of the quark mixing in our future work. We also note that if the CKM quark mixing originate from VEV's of scalar fields (with non-trivial U (3) charges) other than Φ e , in general they may contribute to mixings of family gauge bosons, and therefore they would receive a tight constraint from the experimental data for the K 0 -K 0 mixing. This is, however, highly dependent on the model of the quark sector, in comparison to the constraints analyzed above. 5 We summarize here our standpoint with respect to the constraints on the gauge boson mass from the quark sector, namely from the charged kaon decays, K 0 -K 0 mixing, and D 0 -D 0 mixing. The severe constraint from the CP violation in the K 0 -K 0 mixing shows that CP phases in the down-type quark mixing U dL and U dR are absent or do not contribute to the K 0 -K 0 mixing, for the model to be viable. A simple possibility is to constrain U dL and U dR to be real, and this will be assumed in the rest of our analysis. The constraints from the K 0 -K 0 mixing and D 0 -D 0 mixing indicate that m f 12 10 3 TeV, without tuning of the mixing matrices. These bounds, however, can be lowered to order 10 2 TeV (roughly the expected size of this family gauge boson mass) in a non-negligible region of the parameter space of the mixing matrices. In order to reduce m f 12 to a much lower mass range, naively it seems to require considerable fine tuning of the mixing matrices. Nevertheless, given the simple forms of the down-type quark mixing eq. (17), we may as well keep our mind open for a possibility that Nature indeed conspires to realize such a case.
As seen above, the bounds for m f 12 extracted from the quark sector are quite dependent on the structure of the quark mixing matrices. By contrast, a strict bound can be extracted from a purely leptonic process independently of the quark sector, since the interactions of the charged leptons with the family gauge bosons are completely fixed. In passing, let us comment on the leptonic processes µ → 3 e and µ → eγ. The effective interaction (10) include only |∆N f | = 0 and |∆N f | = 2 terms, whereas these processes have |∆N f | = 1. Hence, these processes can occur only through family mixing in quark loops. They are dependent on the quark mixing matrices; furthermore, the constraints from these processes are looser than other quark-mixing dependent ones which we considered above. Therefore, we do not discuss µ → 3 e and µ → eγ any further. Here, we consider the muonium into antimuonium conversion M (µ + e − ) → M (µ − e + ), which has |∆N f | = 2. The total M M conversion probability P M M (B) under an external magnetic field B is given by , where E M and E M are the energies of M and M , respectively, λ is the bound muon decay width, and δ is defined by M |H M M |M which is proportional to (G f 12 / √ 2)/πa 3 (a is the electron Bohr radius). Here, the effective interaction describing M M conversion is given by eq.(4). This has the same (V − A)(V + A) form as the one corresponding to a dilepton model [10], and the formulation in this case has been investigated by Horikawa and Sasaki [11] in detail. It predicts P M M (0) ≃ (3/2)δ 2 /λ 2 and δ = −8(G f 12 / √ 2)(1/πa 3 ). It follows that This constraint is looser than the constraints listed in Table 1 or from the K 0 -K 0 /D 0 -D 0 mixing. However, since the down-quark mixing matrices U dL and U dR are unknown at present apart from the CKM matrix, we would like to emphasize the importance of observations in the pure leptonic processes, independently of the bounds from the rare kaon decays. In this respect, we expect that future experiments will improve the bounds given in eq. (19).

Search for signatures at collider experiments
Next, we investigate possible signatures of the current-current interaction with |∆N f | = 2 at collider experiments. Although a top-top production at LHC (via u+u → t+t) is very attractive, the cross section ∼ 10 −6 pb at √ s = 14 TeV and for m f 13 = 10 2 TeV would be too small to detect the signal. The cross section for e − + p → µ − + X amounts to σ ∼ 10 −5 pb at E p = 7 TeV and E e = 400 GeV for m f 12 = 50 TeV, which would also be difficult to detect, because of a large background e − + p → µ − + ν e +ν µ + p with σ ∼ 10 −1 pb.

Angular distribution of muon cos θ
Cross section  Table 1, which assume the down-type quark mixing eq. (11). The most clean reaction with |∆N f | = 2 is e − + e − → µ − + µ − . This reaction is expected at an optional experiment at a future e + e − linear collider. The current structure in this model shows that this reaction takes place only between invertedly polarized electron pairs e − L e − R . This aspect is useful for discriminating this model from others using the polarized e − beams. We obtain the differential cross section  d, s, b). The method is practically the same as that for Z ′ boson. [For reviews of Z ′ , see, for instance, Refs. [13]. In particular, the highest limit of Z ′ mass from direct searches is about 1 TeV, which is much smaller than the bounds on m f 11 in Table 1.] In conventional Z ′ models, Z ′ couples to fermions of all flavors, whereas the A 1 1 boson couples only to the first generation, i.e., A 1 1 → e + e − , ν eνe , uū, dd. The total decay width and the branching ratio are given, respectively, by which are different from those of conventional Z ′ models. Since we presume that A 1 1 has a mass larger than O(1 TeV), it is not expected to find A 1 1 at Tevatron. On the other hand, we may expect productions of A 1 1 at LHC. In Fig.2, we show the cross section σ(p p → A 1 1 X → e + e − X) = σ(pp → A 1 1 X) · BR(A 1 1 → e + e − ) for √ s = 7 TeV and 14 TeV. The cross sections are calculated with CalcHEP [14] implementing eq. (2) and with the CTEQ6L code [15] for the parton distribution function. When we reconstruct dilepton invariant masses m(l + l − ), if we observe a peak in m(e + e − ) but no peak in m(µ + µ − ), this will be a signal of the new gauge boson A 1 1 . (This feature is unchanged even with up-quark mixing.) The dominant backgrounds in the A 1 1 search, after moderate event selection cuts, are Drell-Yan dielectrons [16]. Table 2 lists S/ √ N as a measure of A 1 1 discovery reach for m f 11 ≤ 3 TeV. Estimates of backgrounds within a window of ±4Γ Z ′ ≈ ±Γ A 1 1 before any cut are taken from [16]. Comparing to the analysis given there, we anticipate that, with an integrated luminosity of 10 fb −1 , m f 11 up to several TeV would be within discovery reach. However, we leave a detailed study to our future work.

Summary
At present, the cancellation mechanism based on U(3) family gauge symmetry is the only known one as a possible explanation for K(µ) = K pole . Therefore, tests of the model are urgently required.
In this model, the family number i = (1, 2, 3) is defined as (e 1 , e 2 , e 3 ) = (e, µ, τ ) in the charged lepton sector. Once we fix the mass matrix (or the mixing matrix) of the down-type quarks in this basis, we can extract constraints on the family gauge boson masses from the rare kaon decay searches and from the observed value of the K 0 -K 0 mixing. Similarly if we fix the up-type quark mixing, we can extract constraints from the D 0 -D 0 mixing. The very stringent bounds from the CP violation in the K 0 -K 0 mixing rule out contributions from CP phases in the down-type quark mixing matrices to this process. Hence, we restrict our analysis to the real (orthogonal) down-type quark mixing matrices. Generally (without tuning of the mixing matrices) we find m f 12 10 3 TeV from the K 0 -K 0 and D 0 -D 0 mixing. However, m f 12 ∼ O(10 2 TeV) is also viable in a non-negligible range in the parameter space of the mixing matrices, which is consistent with the bounds from the rare kaon decay searches. We also find that, with certain simple forms of the down-type quark mixing matrices, the contribution of the family gauge bosons to the K 0 -K 0 mixing vanishes. Strictly speaking, if we allow for an arbitrary quark mixing, we cannot constrain the gauge boson masses from these experimental data, since there exist solutions, for which all these processes are suppressed. A quark-mixing independent bound is obtained from a purely leptonic process, muonium-antimuonium conversion, whose current lower bound reads m f 12 > 1.6 TeV. More sensitive tests will come from an upgrade of this experiment or from the process e − L e − R → µ − µ − at ILC. Furthermore, if the lightest gauge boson A 1 1 happens to exist below several TeV, we expect to observe a peak in m(e + e − ) but no peak in m(µ + µ − ) at LHC. These searches may uncover an interesting possibility.
One may suspect that the bounds from the K 0 -K 0 mixing are too severe for the new physics signals to be observed at LHC and/or ILC. We note, however, that at present our knowledge on the structure of the quark mixing matrices is rather limited, and a conservative attitude would be to rely on the current bounds from the purely leptonic process M (µ + e − )-M (e + µ − ). In this regard, we stress that, although the production rate of A 1 1 's at LHC depends on our assumption of the up-and down-quark mixing, once they are produced, the family dependent appearance of a peak among the purely leptonic decay channels is independent of the assumption and is a unique prediction of the present model.