Approaching a strong fourth family

A heavy fourth family is an example of new physics which is well defined and familiar in some respects, but which nevertheless has radical implications. In particular it eliminates a light Higgs description of electroweak symmetry breaking. We discuss an early signal for heavy quarks at the LHC in the form of an excess of"$W$-jets", and as well show how $W$-jets may be useful in the reconstruction of the heavy quark masses. We argue that fourth family quarks can be distinguished from vector-like quarks of a similar mass at roughly the same time that a same sign lepton signal becomes visible. Given the large mass of the fourth neutrino we describe how a picture for neutrino mass emerges in the absence of right-handed neutrinos, and how it suggests the existence of a remnant flavor gauge symmetry. Based on talk given at"Second Workshop on Beyond 3 Generation Standard Model -- New Fermions at the Crossroads of Tevatron and LHC", January 2010, Taipei Taiwan.


Motivation
A recent result from the Tevatron places a lower bound of 338 GeV on the mass of the b quark of a fourth family [1]. This lower bound is narrowing the allowed mass window for fourth family quarks, since the upper bound allowed by partial wave unitarity is about 600 GeV [2]. Nevertheless, depending on whether the masses are at the low or high end of this range a fourth family could still be characterized as "light" or "heavy". The implications are profoundly different. The response to the discovery of a light fourth family may be "Who ordered that?", while the discovery of a heavy family could cause a realization as in "So that is the way nature works!" A light fourth family is introduced into the standard model in the hope that it can co-exist with the Higgs boson. The new family significantly modifies the running of the quartic Higgs coupling as reflected by the two new terms in µdλ/dµ ∝ λy 2 q − y 4 q + ... involving the new large Yukawa couplings. These terms must be carefully balanced to keep λ(µ) finite and positive at some higher scale µ = Λ, and this translates into a smaller allowed range of the Higgs mass [3]. The Yukawa couplings y q (µ) also tend to quickly run into trouble at higher scales. Perhaps these issues can be side-stepped if the Yukawa and Higgs couplings instead approach a nontrivial ultraviolet fixed point [4].
More troublesome is the direct contribution to the Higgs mass from the loop of fourth family fermions. If we are to take this seriously then there must be some new physics which acts to cut off this quadratically divergent contribution. Assuming this effective cutoff is somewhat above the heavy quark masses, then these quarks act to shift the Higgs mass by Thus as the heavy quark masses approach 400 GeV, the Higgs mass in the absence of fine tuning is pushed up to the cutoff. When the Higgs mass is pushed up to the scale where new physics must enter, then the original Higgs description is no longer useful. When a fourth family becomes clearly "heavy" then it clearly cannot co-exist with a light Higgs. 1 When m q ≈ 600 GeV then the Goldstone bosons of electroweak symmetry breaking couple maximally strongly to the t , b . The self-interactions of the Goldstone bosons will also be strong and so strong interactions rather than a weak Higgs exchange will unitarize W W scattering. A Higgs not only loses meaning but it is also no longer needed, since the strong interactions can cause dynamical symmetry breaking and thus yield the Goldstone bosons directly. Following the cue of QCD, we can expect that a bilinear fermion condensate takes the place of the Higgs vacuum expectation value. 2 We also need an alternative to Yukawa couplings for the generation of quark and lepton masses. In the presence of electroweak symmetry breaking fermion condensates of size Λ 3 ew , appropriate 4-fermion operators can feed mass down to other quarks and leptons. The origin of these 4-fermion operators imply the existence of new scales of flavor physics above the TeV scale. If such a scale is Λ f l then the resulting quark or lepton mass is of order Λ 3 ew /Λ 2 f l . What then causes fermions to develop the needed condensates? A new confining gauge interaction, technicolor, is one possibility, but another is to have new strong broken gauge interactions. It is for this latter possibility that we can identify the fermions developing condensates with the fourth family, since by definition a sequential fourth family does not feel any new unbroken gauge interaction.
The idea that new broken gauge interactions are involved with electroweak symmetry breaking is also economical. We have just mentioned that new flavor interactions well above the TeV scale are responsible for feeding mass to light quarks and leptons. The gauge symmetry broken near a TeV may then just be a remnant of this larger flavor gauge symmetry, where the latter only partially breaks at the higher scale. The remnant flavor interactions must yield effective 4-fermion operators that are strong enough to produce the electroweak scale condensates. Thus broken flavor gauge interactions may hold the key to both electroweak symmetry breaking and quark and lepton masses.
We return to the point that a heavy fourth family is incompatible with a light Higgs. This provides strong motivation to search for a fourth family, given that the experimental signatures of a fourth family at the LHC are more accessible than those of a light Higgs. It then becomes intriguing to wonder whether it is possible to rule out a light Higgs before the search for it gets seriously under way. The answer to this question involves another question. How easy is it to tell a sequential quark of a fourth family from a "vector quark" (vectorlike in the sense that its mass preserves electroweak symmetry)? Unlike sequential quarks, vector quarks are compatible with and are sometimes even motivated by a light Higgs. Vector quarks have unconstrained masses and they have decay modes involving the Z as well as the W . We shall consider a method to distinguish vector quarks from sequential quarks in the next section.

Searches
We first would like to consider what the early indications of a sequential fourth family at the LHC may look like. We shall assume that the dominant decay modes are We shall also assume that a b-tagger is not yet operational in the early searches. We can consider signatures that are indirect in the sense that they do not involve a full mass reconstruction of the heavy quark. A well studied method of this type is the search for same sign where the backgrounds are believed to be small [6,1]. But same sign leptons could also be a signal of other types of new physics, and so this search needs to be supplemented by other approaches. A strategy to be explored here is to focus on the excess of boosted and isolated W 's, that arise from both b b → W + W − W + W − bb and t t → W + W − bb. The hadronic decay of such W 's give rise to W -jets, and they have been studied in the context of the mass reconstruction of the heavy quarks [7,8,9]. Here we shall study the excess of W -jets directly by looking for a peak in the jet invariant mass distribution. To suppress backgrounds one of the other W 's is required to decay leptonically. In each event we shall consider the jet with the largest jet mass to construct the jet mass distribution. Event generation and simulation details are as follows. Signal and background events are generated by Madgraph [10] and Alpgen [11] respectively, along with Pythia [12] using tune 108 (D6) with CTEQ6L1. The primary backgrounds of interest are tt + jets and W + jets, and MLM jet matching is used with p T min = 100 and 150 GeV respectively. Detector simulation is performed with PGS [13] modified to use the anti-k T jet algorithm and using CMS settings. 3 We present results at √ s = 10 TeV for m q = 600 and at √ s = 7 TeV for m q = 450 GeV. Results are always normalized to a luminosity of 1 fb −1 .
We use a K factor of 1.5 for both the pair production of heavy quarks and the tt + jets background, as indicated by the NLO and NLL cross sections in [15]. These cross section results do not directly apply to the relevant tt + jets background, where H T is large compared to the tt threshold, but the study in [7] comparing MC@NLO to Alpgen suggests that K = 1.5 is reasonable. Our W + jets background is dominated by 2 or more jets; for W + 2 jets a K factor somewhat less than unity was found [16], and for W + 3 jets the K factor may be even less [17]. Nevertheless we set K = 1 for this background.
For event selection we require three or more jets with p T > 100 GeV and |η| < 2.5, as well as an L defined as an isolated lepton with p T > 15 GeV or missing E T > 200 GeV. We focus on the jet with largest invariant mass in each selected event. This jet is required to be isolated, separated by ∆R > 1 from the other p T > 100 GeV jets and L, and if so its mass is used to form the histogram.
There are two quantities that need to be adjusted to enhance the signal to background ratio: the value of H T min for the H T cut and the jet resolution parameter R. The optimal H T min is about 2m q . R should be large enough to make it likely that a single jet can capture the hadronic decay products of a W produced in a heavy quark decay. Thus the optimal R also depends on m q , becoming smaller for larger m q . Therefore the observed effect that the variation of H T min and R has on the signal can provide information about m q .
The signal events are shown in Fig. (1) where it is seen that the t t and b b production contribute about equally to the strong peak at the W mass. b b production has a larger high jet mass tail due to the presence of the t in the final state. Fig. (2) shows the combined signal and background for H T min = 2m q = 1200 GeV and R = 0.7. Signal to background is degraded for a smaller H T min , as seen in Fig. (3) where H T min = 1000 GeV. The procedure of optimizing the results as a function of H T min will thus help to determine m q . In Fig. (4) R is increased to 0.8, and this shows the sensitivity of the shape of the jet mass distribution to this quantity. Increasing R moves the distribution towards higher values as expected, but the signal remains strong. We see that a strong W mass peak does not show up in the background events. This is the key for isolating signal from background since it makes possible some type of side-band subtraction. It reflects the fact that this method is only really effective for identifying W 's that are both boosted and isolated. There are boosted W 's decaying hadronically in the tt + jets background, but they are typically not isolated due to a neighboring b jet. For the W + jets contribution the W is required to decay leptonically to satisfy event selection, so while QCD jets can fluctuate into large invariant masses this does not produce a W peak. The W + jets background could also be significantly reduced when b tagging becomes available. Given the success of identifying W -jets from the signal events, we see why W -jets may be useful for a direct reconstruction of the heavy quark masses [7,8,9]. We can show this here for the present set of parameters, and including both the t and the b contributions. The basic idea is to combine a W -jet with one other jet. Events are selected to have only three jets having p T > 150 GeV and |η| < 2.5, labelled J W , J 1 , J 2 . The W -jet, J W , has a jet mass within 12 GeV of M W . With L defined as before we require that L be ∆R > 1 away from J W , J 1 , J 2 . We consider all possible ways of assigning objects to [J W , J 1 , J 2 , L]. In each case we let the invariant mass of [J W , J 1 ] contribute to the histogram. Note that for the events which contribute, they each contribute at least twice to the histogram. This technique helps to avoid biasing the background contributions, thus suppressing a false peak in the background that often arises in traditional attempts at full event reconstruction.  7) shows the peak in the reconstructed heavy quark mass for the signal events. The t t contribution gives a more realistic peak while the b b contribution is broader and extends to invariant masses below m q . This is understandable given that the J 1 jet for the latter case is attempting to capture the hadronic decay of the t, where the t is often not boosted enough for this to be effective. It is of interest to contrast the t t dominance in the heavy quark mass peak to the roughly equal t t and b b contributions in the previous W mass peak. The relative strength of the W and q mass peaks could then in principle be used as evidence for the existence of both the t and the b . Fig. (8) shows the combined signal and background while Fig. (9) shows the same for different choices of parameters. We comment here on our use of large jet resolution parameters, R in the range of 0.7 to 0.9, and the possibility that this may imply sensitivity to multiple parton interactions and the underlying event. To test this we changed the underlying event model in Pythia in the generation of the signal events. We changed from tune 108 (old model) to tune 326 (new model) and we found that this hardly degrades the signal. We also turned off the multiple parton interactions altogether, in which case the W and heavy quark mass peaks are only somewhat sharper.
We have described methods that will allow a fairly early discovery of heavy quarks with standard charged current decays. The next question is how to distinguish such sequential quarks from vector quarks. To illustrate the issues we consider a vector doublet of quarks Q = (T, B) because this most closely resembles the sequential quarks q = (t , b ). The mixing of the vector quarks with t and b happens through Yukawa couplings Since the Goldstone bosons are identified with the longitudinal modes of the gauge bosons, this mixing implies the following decay modes.
As indicated by the color coding, Y t (Y b ) controls the branching fraction into modes involving t (b). But regardless of the relative size of Y t and Y b , and for T and B of similar mass, the mixing as described implies that the proportions of W , Z, and H produced will be close to 1 : 1/2 : 1/2. This is also true of other varieties of vector quarks and top partners.
This leads us to compare the production of like-sign leptons to the production of pairs of leptons that arise due to Z decay. The former can arise when at least three W 's are produced, as happens when at least one of the two heavy quarks decays to W t. Thus like-sign leptons can occur for both sequential and vector quarks, while leptons from Z decay occur only for vector quarks. For suitable event selection the backgrounds for both processes can be made small. We therefore need only count the signal events, and we consider the case √ s = 10 GeV and m q = 600 GeV with 1 fb −1 .
Event selection for leptons from Z is: H T > 1 TeV, [2 isolated leptons and E / > 100 GeV] or [3 isolated leptons], and M (e ± e ∓ ) or M (µ ± µ ∓ ) within 4 GeV of the Z mass. The requirement of missing energy or an extra lepton eliminates the Z +jets background. This leaves W Z +jets as the largest of the small backgrounds, and this could be reduced further with a b-tag. Event selection for like-sign leptons is: H T > 1 TeV, 2 isolated same-sign leptons, E / > 50 GeV and M ( ± ± ) > 100 GeV. The backgrounds for this process are considered in [6,1].
We find for the number of events, without a b-tag: ± ± Z → + − sequential quarks 7 0.7 vector quarks 1 to 7 6.3 W Z + jets 0 1.5 The 1 to 7 corresponds to Y t /Y b varying from 0 to ∞. These results indicate that sequential quarks could be distinguished from vector quarks at roughly the same time that a new same sign lepton signal is detected. Thus when heavy quark masses have been reconstructed or inferred and if leptons from Z are not observed at the rate at least comparable to same sign leptons, then we will have good evidence for new quarks belonging to a fourth family.

Implications
The existence of a fourth family would cast a new light on the question of neutrino mass. A sufficiently heavy fourth neutrino is required and is at first sight strange; this has even held back the consideration of a fourth family. A large neutrino mass is also often taken to imply that a large Dirac mass is present (with or without a separate Majorana mass for the right-handed neutrino). This bias may also be faulty since there may well be no right handed neutrinos, at least at or below the TeV scale. In particular no known gauge symmetry protects the ν R 's from receiving some mass much larger than a TeV, from whatever dynamics existing at higher scales. When we discussed the origin of mass for a heavy fourth family we described a new strong interaction that coupled to all members of the fourth family. Thus the fourth neutrino mass could arise in a way similar to the other fourth family fermions, and in the absence of ν R 's this would imply a Majorana mass for ν Lτ . In any case a fourth neutrino mass similar to the other fourth family members is not in itself unnatural, and the real question is why the other neutrinos are so light.
Before pursuing that question we can look at the implications for electroweak precision observables. The effect of the Majorana mass for a purely left-handed neutrino shows up in the last term in each of the following contributions to S and T [5,20].
The presence of Λ ν (just somewhat larger than m ν ) is associated with the dynamical nature of the mass and the fact that the mass function falls off quickly above this scale.  (5) and (6).
In the absence of a light Higgs, a contribution to T of the order 0.5 is needed. Since T leptons is decreased by the new term, one needs a larger m τ /m ν to obtain T leptons ≈ 0.5. One is forced to the right side of the plot which decreases S leptons further as shown, and this can thus cancel a larger portion of S quarks . Larger values of m τ and m ν can also produce this effect, and the result is that any constraint on the relative size of m t and m b is lessened. Thus while there was never a serious conflict between a fourth family and electroweak precision observables [18,19,5,20,3], a ν Lτ with a Majorana mass makes a fourth family even easier to accommodate [5,20].
Let us return to the question of light neutrino masses by comparing them to their associated charged lepton masses. We relate the much lighter neutrino masses to our hypothesis that they are purely left handed. Consider the following operators (which can be written in an SU (2) L × U (1) Y symmetric form) that feed down mass from the τ to a light charged lepton e (or µ) and a light neutrino ν e (or ν µ ). 1 The main point is that there is no 4-fermion operator that can feed mass from τ (or t or b ) to the ν e . Then for example if Λ f l ≈ 200 TeV and τ L τ R ≈ (300 GeV) 3 , the resulting illustrative lepton masses are m e ≈ 0.7 MeV and m ν ≈ 0.002 eV. We note that the 6-fermion operator could result from 4-fermion operators that involve right-handed neutrinos after the latter are integrated out. Thus it is quite consistent for the ν R masses to be similar to the highest flavor scale, denoted here by Λ f l . 4 The large Majorana mass for right handed neutrinos is a natural order parameter for the breaking of the original flavor gauge symmetries, including a possible SU (2) R . We can contrast this situation to the standard see-saw with Higgs, which requires a much larger hierarchy between the ν R masses and other masses (or alternatively some very small Yukawa couplings).
There is in fact a 4-fermion operator that could feed down mass from the ν Lτ to the light neutrinos, of the form ν Lτ ν Lτ ν L ν L . This can also be written in an SU (2) L ×U (1) Y symmetric form and the ν L here can be the e, µ or τ neutrino. But this operator can be eliminated, or strongly suppressed, if (fourth family lepton number) − (third family lepton number) is a symmetry, or a very good approximate symmetry of the flavor physics. The operators in (7) and (8) are allowed by this symmetry, while it is broken by the ν Lτ dynamical mass. This symmetry may correspond to a broken gauge symmetry, in which case the associated gauge boson may be a TeV scale remnant of the original flavor gauge symmetry.
Similarly we consider 4-fermion operators that can feed down mass from the fourth family quarks to the third family quarks.
Here we note the effect of an approximate symmetry defined by the charges: This is (fourth family axial quark number) − (third family axial quark number) and it may be the quark part of the same remnant flavor gauge symmetry already mentioned. The q masses also contribute to its breaking. To the extent that it is a symmetry of the flavor physics it would suppress the two operators in (9) relative to the two operators in (10). This is advantageous since the t mass operator in (10) has suppressed impact on the Zbb coupling; two insertions of this operator are needed due to its LRLR form [21]. We mention in passing that the massive gauge boson associated with the remnant flavor gauge symmetry has its own implications for LHC physics [22]. We have hopefully made it clear why a fourth family is interesting, and especially so if its masses are large enough to imply new strong interactions. Despite the presence of strong interactions, the standard weak decays of the heavy quarks make them quite easy to find or to rule out at the LHC. Any modification of the production cross section due to the strong interactions should not change this conclusion. A crucial step will then be to distinguish fourth family quarks from vector quarks, for example in a manner we have described. The confirmation of heavy sequential quarks would bring an end to the long "Era of the Higgs". It would cause the focus of the LHC, which has been to find the Higgs, to suddenly shift. And it would also cause the focus of theorists to suddenly shift. Theorists would see that new strong dynamics is showing up at the TeV scale, right where it is needed. This would be a serious blow to anthropic thinking, and the concept of naturalness may return as an important criteria in the construction of physical theories.