Hunting for sterile neutrino with future collider signatures

We study the feasibility to observe sterile neutrino at the high energy colliders with direct production channels through $e^+e^-$, $ep$ collision, and indirect production channels through decays of heavy meson, baryon and Higgs. For $e^+e^-$ collision, the $e^+e^-\to\bar{\nu}_e N$ channel is explored with new signal selection method which tends to be efficient for light $m_N$, the constraints of active-sterile mixing $|U_{eN}|^2$ at the SuperKEKB, CEPC and ILC are expected to reach better lower limits than current experiments. For $ep$ collision, We investigate the heavy sterile neutrino production through a new channel via proton bremsstrahlung, i.e., $e^-\gamma \to NW^-$, hundreds of GeV heavy sterile neutrino can be probe and new limit on mixing is given. For heavy hadrons decay, the lepton-number-violating decays of $\Lambda_c,\ \Xi_{c},\ \Xi_{cc}$ and $\Lambda_b$ are explored via an intermediate on-shell Majorana neutrino in GeV scale. The branching fractions and the constraints for $|U_{\ell N}|^2$ are given, and hence may put new limits on this mass region. The ${\rm Higgs} \to W\mu\mu\pi$ channel is also considered to test massive neutrino within Higgs sector.


I. INTRODUCTION
The experimental observation of neutrino oscillation has conclusively shown small but non-zero neutrino mass.Since neutrinos are massless in the Standard Model (SM), the mass origin has become an important portal to physics beyond the standard model (BSM).There are generally three theoretical hypotheses: see-saw mechanism [1][2][3], radiative generation mass [4] and extra-dimensions [5].Originally, the smallness is resorted to the presence of extra field in see-saw models with far beyond electroweak energy mass scale.There are also models where extra fields are not so heavy [6], leaving the open possibility for sterile neutrinos in eV to TeV scale, and hence are feasible for collider searches.The reason for the smallness is yet not fully understood, but the very existence of neutrino mass may indicate the existence of a right-handed gauge-singlet (sterile) neutrino N R .The Dirac or Majorana nature of N R can be identified by neutrinoless double-beta decay (0νββ) [7] of nucleus or other W * ,± W * ,± → ℓ ± ℓ ± induced processes, which led to lepton number violation (LNV) with ∆L = 2.
Now in this article, we will give several complementary investigations to the previous works by studying the feasibility of collider test for sterile neutrino through: The rest of the paper is organized as follows.In Sec.II, we present the direct searches for sterile neutrino at the e + e − , ep colliders.In Sec.III, we investigate the indirect channels for sterile neutrino via heavy particles decay.The last section is reserved for summary and conclusions.

II. DIRECT PRODUCTION
A. e + e − collision In the presence of one or several sterile neutrinos, active neutrinos in the flavor base are a mixture of the light and heavy sterile neutrinos in mass eigenstates.The lagrangian of interaction terms between sterile neutrino and gauge boson, Higgs boson in mass eigenstates are where g = e sin θ W , θ W is the weak Weinberg mixing angle with sin 2 θ W = 0.231, U ℓm and V ℓm are the Pontecorvo-Maki-Nakagawa-Sakata (PMNS) matrix elements [33,34], the charge conjugated state is defined as ψ c = C ψT , the left hand projection operator In this section, we discuss the possible production of the sterile neutrino through e + e − collision, and its subsequent decay.In this line, we address the dependency on active-sterile mixing |U ℓN | 2 for the different colliders.There are generally two production channels, one is an annihilation channel through Z boson (s-channel), another is given by the exchange of a W boson (t-channel).We note that the sterile neutrino can be also produced via e + e − annihilation into Higgs, which is largely suppressed by the tiny coupling between electron and Higgs.
Among the previous researches, various channels have been explored at next generation high energy lepton colliders, e.g., e + e − [35][36][37][38], µ + µ − [39], e − e − [40], and see [41,42] for review.In Refs.[35,40], ℓjj + / E signal is proposed to reconstruct heavy neutrino N, and signals are required that the invariant mass of ℓjj is near m N or the invariant mass of jj is near m W .In this work, we find that the cut on open angle Here, we study the sterile neutrino produced via s-channel and t-channel at the SuperKEKB, the Super Tau-Charm Facility (STCF), the Circular Electron Positron Collider (CEPC) and the International Linear Collider (ILC), where the final sterile neutrino is reconstructed by µπ-channel for light N and by ℓjj-channel for heavy N.The analyses in this section and the rest sections are based on Feynman diagram through interaction vertex in (1), we develop a private package to perform numerical integration and simulate kinematic of produced particle with the help of CUBA [43].Here and the rest of this article, one sterile neutrino N is supposed, one can easily extend our analysis to multi sterile neutrino models.
According to gauge-interaction lagrangian in (1), the canonical matrix element square takes the form: where G F is the weak interaction Fermi constant with The cross section is straightforward where the first two 1 2 are spin-polarization average factors of electron and positron, 1 2s and 1 8πs are the flux and two-body final state phase space factor.
The direct search for sterile neutrino is considered at the future lepton colliders, the SuperKEKB, the STCF, the CEPC and the ILC, each with its own physics focus on .We note that the s-channel can be largely enhanced at the Z-pole running for the CEPC and the ILC, the mainly contribution can be regarded as on-shell Z boson decay [22], and |U µN |2 can be constrained via e + e − → Z * → Nν µ [44][45][46].
As estimated in [47,48], the total decay width for Dirac sterile neutrino is set to be where U eN ≈ U µN ≈ U τ N is set for the universality consideration in the practice computation, see Refs.[48][49][50].For Majorana sterile neutrino, At the STCF and the SuperKEKB, we construct the signal via N → µπ plus missing energy, the main background comes from e + e − → W * W * → µν µ π which is negligible due to double weak coupling at low center-of-mass energy 2 , hence we neglect this background in our analysis.At the CEPC and the ILC, the signal is composed of µjj plus missing energy, and there tend to be small open angle between the signal lepton and jets for more  According to our numerical results, see FIG. 3, the canonical cross sections are sub- caused by signal cuts and definition of Γ N .The current experiment limits labeled "PS191" [87] and "CHARM" [89] are the constraints from beam-dump experiments; the constraint labeled "DELPHI" [22] is given by Z 0 decay, the labels "CMS ′ 18-TL" [29] and "CMS ′ 18-DL" [28] are the trilepton and dilepton searches from direct N decay at the CMS detector, the labels "CMS ′ 22-DV" [23] and "ATLAS ′ 22-DV" [24] are the displaced vertex searches at the CMS and ATLAS detector, and the label "L3" [30] is the result from N e → e + W search at LEP.
The label "Non-Unitarity" [32] is the constraints from non-unitarity effects in lepton mixing via dim-6 Weinberg operator at tree level.
fb to tens of pb with the increase of √ s from the STCF to the ILC.Considering that the integrated luminosities of those facilities are large, the signatures for GeV sterile neutrino can be well explored and hence make a constraint for active-sterile mixing

The constraint is estimated by solving
Ns √ Ns+N B ≈ 1.7, which means signal significance at the 95% confidence level [44], where N s/B is the signal/background events number.At the STCF, the center-of-mass energy may reach 7 GeV, the lower-limit of |U eN | 2 will be 10 −3 ∼ 10 −4 at 0.3 ∼ 2 GeV with integrated luminosity of 5 ab −1 .As for the SuperKEKB, the constraints can be further extended due to its high luminosity, and hence puts new complementary limits to current experiments.At the high energy electron-positron colliders, e.g., the CEPC and ILC, the physics potential for sterile neutrino can be extended to hundreds of GeV with |U eN | 2 sensitivities of 10 −3 ∼ 10 −6 , which will give better lower limits than "CMS ′ 18-TL" [29], "CMS ′ 18-DL" [28], "L3" [30] and "Non-Unitarity" 3 constraint as indicated by previous research [35][36][37][38].

B. ep collision
Heavy sterile neutrino can be also probe at the future electron proton (Ion) colliders, like the Large Hadron electron Collider (LHeC) and the Electron-Ion Collider (EIC).There are extensive investigations on this topic through e − + q → N + q ′ channel within W-exchange [36,52,53].As indicated in Refs.[38,54], where the γ-W * mechanism is proposed using electron bremsstrahlung, the γ-W * channel through proton bremsstrahlung can also provide tests on heavy sterile neutrino for its relative clear signatures and negligible backgrounds.
In this section, we explore the production mechanism of sterile neutrino in the context of γ-W * interaction at the future ep colliders, where the photon is produced via proton bremsstrahlung [55], the distribution function of photon is given in FIG. 4.
The photon from proton bremsstrahlung tends to less energetic compared with electron bremsstrahlung one, while the collider signature is relative clear due to the unbroken proton (several light jets are generated via broken proton and the signal jets could get submerged in complex jets background).
The energy spectrum of proton bremsstrahlung photon can be well formulated in Weizsacker-Williams approximation (WWA-p) [55] where µ p = 2.79, a = 4.96, z = 1 + a 4 x 2 1 − x and The total cross section can be expressed as the convolution of the e − + γ → N + W − cross section with the photon distribution function, where the Mandelstam variables are define as s  The amplitude square for e − γ → NW − is Here, we consider the production of sterile neutrino through e − γ → NW − channel at the future electron-proton collider, like the LHeC.For the events reconstruction, N → µjj channel is adopted, W boson is reconstructed via hadronic channel with width.The current experiment constraint labeled "DELPHI" [22] is given by Z 0 decay, the labels "CMS ′ 18-TL" [29] and "CMS ′ 18-DL" [28] are the trilepton and dilepton searches from direct N decay at the CMS detector, and the label "L3" [30] is the result from N e → e + W search at LEP.The label "Non-Unitarity" [32] is the constraints from non-unitarity effects in lepton mixing via dim-6 Weinberg operator at tree level.
In FIG. 6 the mass of sterile neutrino is in GeV, kink structure might exhibit at given energy point in the spectrum, see FIG. 7 for diagrammatic sketch.In the previous researches, focus are put on Nucleus or π, K beam-dump experiments, in which π, K are stopped, e.g., by plastic scintillator inside a homogeneous magnetic field [59].While at the SuperKEKB, B meson can be produced nearly at rest, thus may be applied to kink search.
and then the produced D meson is reconstructed via K + nπ channel, Therefore, the low sensitivity limit for |U ℓN | 2 may reach 10 −7 for mass region 1 ∼ 4 GeV.What deserves mentioning here is that the decay length of N may be long than detector size hence the directly reconstructed signal of N will suppressed, the direct search of N may be limited, so the kink method will provide another way to search sterile neutrino.A similar strategy can be used in the BES-III via Ψ(3770) → D D where D meson is produced approximately at rest.The 0νββ decay ratios for charged meson (π, K, D (s) , B (s,c) ) are tiny, due to severe suppression either due to the small mass ratio like

B. Meson Decay
are lack of experiment interest at present.For example, the K + → π − +µ + µ + branching ratio for heavy sterile neutrino is [61]: Nevertheless, for a sterile neutrino lies between m π + m µ ≤ m N ≤ m K − m µ , the branching ratio can be greatly enhanced by resonant effect [62][63][64][65].In the narrow resonance approximation, the decay rate can be formulated as In this way, the branching ratio can be easily estimated by the product of leptonic ratio of meson and N → ℓ + M ratio, which is estimated to be several percents.The number of B/D meson at LHC is about 10 12 /10 13 , thus provides experimental tests for sterile neutrino in mass region 1 ∼ 4 GeV.The resonant mechanism can be also applied in baryon semileptonic decay, i.e., B 1 → B 2 + ℓN(→ ℓπ), the branching ratio is factorized as, where Γ B 1 (N ) is the total decay width of initial meson (N).The secondary decay width is well known, where we set m π = 139 MeV, f π = 130.4 MeV, m µ = 105 MeV for numerical analysis.The Feynman diagram for this process is shown in FIG. 9, the amplitude can be formulated as where the CKM matrix elements are taken as: The hadronic transition matrix elements are parameterized in terms of six invariant form factors, where u(B 1/2 ) are Dirac spinors of the initial/final baryon with q Note that a considerable number of heavy baryons can be produced at the LHC, hence it may provide opportunity for experiment searches of the sterile neutrino.Experimental limits from the searches of |∆L| = 2 processes can be reinterpreted as constraints versus m N , with considering the detection efficiencies in TABLE VII.Precise computation of the detection efficiencies requires fully simulated decay-specific Monte Carlo samples, that is reconstructed in the same manner as real data and with a simulation of the full detector, which is out of the range of this paper.We note that the decay width for N is proportional to |U ℓN | 2 and m 5 N in Eq.( 4), a light neutrino with small mixing will fly long before decay, hence the decaying probability of the neutrino within detector will be exponentially suppressed, where , τ N is the lifetime of the neutrino, and L is the length of detector.Considering the rapidity and transverse momentum are 3 and 4 GeV within typical acceptance of LHCb detector, and the detector length is set to be 12 m [69] which requires N decays before the calorimeter system (It provides the identification of leptons and hadrons), the decay length effects are considered in the following analysis.
In contrast, the sizable decay length between N production and decay vertexes will in turn improve the efficiency of events reconstruction [23], if partially reconstruction strategy is adopted.For the fully reconstruction, as indicated in previous Λ b/c studies [48,70], e.g., a complete Λ c event is required in Λ b → Λ c + µµπ decay, while for partially reconstruction method, e.g., one need only tag a baryon (for instance proton produced in Λ c decay) despite its type.The two methods are compared in the following analyses.
via the exchange of Majorana neutrino with kinematically allowed mass, Within this mass region, the narrow width approximation [47] is valid due to Γ N ≪ m N .For numerical evaluation, we use the form factors 73] in the references respectively, the canonical branching fractions Br(B 1 →B 2 +µµπ) are given in FIG.11.

TABLE VI:
The estimated production numbers of heavy baryons at the LHCb in 13 TeV with an accumulated luminosity of 50 fb −1 .At the LHC, the production number of heavy baryons can be estimated by fragmentation fractions of c-quark and b-quark with The production cross section of Ξ + c [78] is measured to be 14.9 µb, the production number is estimated to be N(Ξ + c ) = 2 × 50 fb −1 × 14.9 µb = 1.49× 10 12 .The production cross section of Ξ ++ cc can be estimated via its decay fraction versus Λ + c [79], = 2.22 × 10 −4 , and the fraction for Br(Ξ In the fully reconstruction strategy, we explore the constraints of |U µN | 2 from the ex- The reconstruction channels and detection efficiencies of heavy baryons Λ c , Λ b , Ξ ++ cc , Ξ + c , Ξ 0 c four-body decay.The branching fractions of secondary decay chain are adopted from [66].The detection efficiency for Λ b/c is adopted according to estimation of previous researches [48,70], the efficiencies for Ξ + c , Ξ 0 c are estimated from similar decay channels at LHCb [81].Due to the experimental investigation of new discovered double-charmed Ξ ++ cc is poor, hence we set the detection efficiency approximately to be 1 × 10 −4 .
values of the cross sections and efficiencies, the constraints for |U µN | 2 versus m N at the 95% confidence level are shown in FIG.11 at the LHCb in 13 TeV with an integrated luminosity of 50 fb −1 .The final state baryons Λ, Ξ 0 , c → pK − π + channels respectively, the branching fraction are listed in TABLE VII.The branching fraction for heavy baryon four-body decay reach its maximal value just above µπ threshold 252MeV, while the decay probabilities are highly suppressed by exponential factor, hence only moderate constraints for |U µN | 2 are made.For Λ c → Λµ + µ + π − channel, the maximal branching fraction can reach several part-per-thousand, and the Λ c can be numerously produced at LHCb, the lower limit for |U µN | 2 can reach 10 −4 ∼ 10 −5 at 0.25 < m N < 1 GeV through full reconstruction strategy.The fraction of charm quark fragment into Ξ c is smaller compare with Λ + c , and the branching fractions for are in part-per-thousand, hence the lower limits for |U µN | 2 are suppressed by 1 ∼ 2 magnitude at the same mass region, around 10 −3 ∼ 10 −4 .We also explore the decay channels for double-charmed baryon Ξ ++ cc via although the branching fractions are not small, the constraints are weak for less produced number and inefficiency.As pointed in previous research [48,70], the bottom baryon Λ 0 b can also provide unique test with broad  at the 95% confidence level.The current experiment limits labeled "PS191" [87], NA3 [88], BEBC [93], FMMF [94] are the constraints from beam-dump experiments, the label "NuTeV" [95] is the result from direct N decay search, the label "LHCb ′ 14-B" is the constraints from B meson decay search, the constraint labeled "DELPHI" [22] is given by Z 0 decay, and the labels "CMS ′ 22-DV" [23] is the displaced vertex searches at the CMS detector.i.e., N(B 1 )Br(B 1 → B 2 + µµπ)P N ǫ par ef f = 3.09, where ǫ par ef f is set to be 10% due to the high efficiency to tag µ ± and π ± , the constraints for |U µN | 2 is given in Fig. 11.As a full baryon event is avoided and the reconstruction efficiency is highly enhanced, the lower limits for |U µN | 2 can be greatly improved.For Λ 0 b → Λ + c µ − µ − π + channel, a new bound around 10 −5 ∼ 10 −6 could be set for 2 ∼ 3 GeV.Considering a high luminosity and much yields of Λ b in High-Luminosity LHC (HL-LHC), the constraints will be further extended.

D. Higgs Decay
The collider searches for massive sterile neutrino via Higgs boson decay channels also provide a interesting test, especially for heavier mass as the Yukawa coupling is proportional to fermion mass.In the references [82][83][84], sterile neutrino is explored through direct coupling like Higgs → νN.Nevertheless, if the sterile neutrino mass lie much below Higgs mass, the coupling strength will dramatically suppressed, in this case the Higgs → W W (→ µµπ) channel may be also test window.Furthermore, in the direct Higgs decay channel one missing energy is required in the event construction; while in the Higgs → W W (→ µµπ) channel, final state products can be fully reconstructed and the |∆L| = 2 same-sign µµ event exhibits absolutely new physics beyond SM.
For Higgs decay, we investigate the Higgs → Wµµπ process with relative clear signal, the constraint of mixing parameter is also given.The low sensitivity limits of |U ℓN | 2 versus sterile neutrino mass from channels of this article and present experimental limits are given in FIG. 14 -15.
would be another efficient way to exclude the background, especially for m N below m W .For instance, the decay products µj 1 j 2 of a rest N (e.g., m N ∼ √ s) tend to form a large open angle; while for light N (highly boosted), µj 1 j 2 tend to stay along N flying direction, and hence form a small open angle.However, in the background channel, e.g., e + e − → W * W * → µν + j 1 j 2 , µν and j 1 j 2 are produced by different W bosons, which is less energetic, hence the open angle between µj 1 j 2 will be large.In this way, a open angle cut might be efficient way to select signals especially for light N. Aside from the investigation at the future high energy lepton colliders, the SuperKEKB can also provide unique test for sterile neutrino in this channel, which can put new lower limit for sterile neutrino mixing |U eN | 2 .

FIG. 2 :
FIG. 2: The cross sections of sterile neutrino N production in Drell-Yan(Z) and W-exchange channels at the e + e − collider and the background e + e − → W W → µjj + / ν in different open angle cut θ µjj .Here the basic cut is adopted and the charge conjugate is proposed.

FIG. 3 :
FIG.3:The cross section of sterile neutrino N production in Drell-Yan(Z) and W-exchange channels at the e + e − collider and the |U eN | 2 sensitivity at the 95% confidence level.Here for light mass, N → µπ is used; while for heavy mass, N → µjj is adopted.The inflections are

W ) 2
with the Källen function λ(x, y, z) ≡ (x − y − z) 2 − 4yz; the first two 1 2 are spin-polarization average factors of electron and photon; and two-body final state phase space factor.

FIG. 5 :
FIG.5:The Feynman diagrams for heavy sterile neutrino production through e − γ → N W − channel, where the photon is produced via proton bremsstrahlung.

FIG. 6 :
FIG. 6: Left: The canonical production cross section of e − γ → N W − at the LHeC with basic cuts.Here the photon is produced through proton bremsstrahlung.Right: The sensitivities of sterile and active neutrino mixing |U eN | 2 at the 95% confidence level with integrated luminosity of 1 ab −1 at the LHeC.Here N → µ − jj and W − → jj are taken into account for N and W reconstruction.The inflection near m W is caused by the definition of sterile neutrino total , the canonical cross section for e − γ → NW − and |U eN | 2 sensitivity at the LHeC are given.According to our analysis, the canonical production cross section for e − γ → NW − will reach tens of fb if we probe a sterile neutrino in the electroweak energy mass scale.The cross section decreases to several fb for m N ∼ 500 GeV, and hence provides tests for heavy sterile neutrino in this region.The sensitivity of activesterile neutrino mixing |U eN | 2 with 100 < m N < 500 GeV is estimated to be 10 −3 ∼ 10 −4 level, therefore extends the current limit to a new level.III.INDIRECT PRODUCTION A. Kink search via B meson semileptonic decayA massive sterile neutrino may be also investigated by searching kinks in lepton energy spectrum of the B meson semileptonic decay, see earlier nucleus β-decay spectrum kink search[58].The mechanism is straightforward, in three-body decay B → D + ℓν, the maximal lepton energy at B meson rest frame is

FIG. 11 :
FIG. 11: The canonical branching fractions of sterile neutrino exchange four-body heavy baryon decay, and the sensitivities of |U µN | 2 at the LHCb with an integrated luminosity of 50 fb −1

mass region 0. 3 ∼
3.0 GeV.As we can see, the lower limits for heavy baryon decay within full reconstruction method can only reach current bound around 2 GeV.In the partially reconstruction strategy, see FIG.10, a distance of several millimeters, which is the typical flying length of B/D hadrons at LHC, is required from collision vertex to baryon decay vertex.Due to the fact that light and weak mixing sterile neutrino will fly long, the N decay vertex is generally far away from baryon decay vertex, e.g., in meters for m N = 1 GeV with |U ℓN | 2 = 10 −4 .To avoid background from B/D meson decay, at least one baryon is need in baryon decay vertex.In the "N Decay Vertex", similar decay signal due to K → π + µν can be avoided by requiring the direction of p π + p µ point to "Baryon Decay Vertex".At the 95% confidence level,

TABLE I :
The center-mass energies and integrated luminosities of current and future e + e − colliders.The integrated luminosity is estimated by 10 34 cm −2 s −1 ∼ 1 ab −1 for 10 years

TABLE II :
The cross sections for background σ(e + e − → W W → µν µ jj) at the CEPC with

TABLE III :
[51]cross sections for σ(e + e − → νN )Br(N → µjj) and its background σ(e + e − → W W → µν µ jj) at the CEPC and the ILC after no cut, basic cut and signal cut.The background cross section without cut is in agreement with prediction in[51].The θ min is 160 • (270 • ) for m N = 50 (100) GeV at the CEPC; while for ILC, the θ min is set to be

TABLE IV :
The leptonic decay fraction of K, D, D s , B, B c , the e + ν e branching ratio are tiny due to small electron mass.