1 Introduction

With the discovery of Higgs scalar [1, 2] at the Large Hadron Collider (LHC), identifying the nature of this scalar, which reveals the origin of electroweak symmetry breaking (EWSB), is one of prior tasks at this facility. There are various scenarios in the literature that naturally explain the EWSB, among which supersymmetry (SUSY) has attracted much attention since the discovery. Nevertheless, the prospective on the minimal supersymmetric standard model (MSSM) is rather pessimistic. Some reasons for this include that the mass parameters related to the third-generation squarks are probably at least of order several TeVs [3, 4], and also signals of neutralino dark matter are not yet observed at any dark matter direct detection facilities [5, 6].

The status of MSSM motivates a diversity of extensions, of which vectorlike (VL) fermions are interesting because of a few features as follows.

  • First, the VL fermions are one of viable extensions which retain the perturbative unification [7, 8], similar to a well-known example – the next-to-minimal supersymmetric model (NMSSM).

  • Second, the issue of naturalness imposed by the Higgs mass in the MSSM can be resolved by new radiative correction to the Higgs mass due to the VL fermions of order TeV scale [9,10,11]. It differs from the NMSSM, as in the later one the correction occurs at the tree level.

  • Last, the stringent constraint on the neutralino dark matter from dark matter direct detection will be modified due to be presence of neutral fermions in the VL sector.

This paper is devoted to study the minimal VL model as described in Table 1. We will explore this model by focusing on the VL leptons therein, the matter content of which is composed of one singlet and two doublets. A key point in this model is that the VL leptons directly couple to the Higgs doublets rather than mix with the SM leptons, which suggests that they may play important roles in both the Higgs physics and dark matter.

The paper is organized as follows. In Sect. 2, we introduce the minimal VL model, derive the EWSB conditions, and calculate mass spectrum in the electroweak sector for our purpose. Section 3 is devoted to estimate effects on the Higgs physics, where the one-loop radiative correction to Higgs mass and the modification on Higgs self coupling are presented in Sects. 3.1 and  3.2, respectively. Section 4 is devoted to discuss the dark matter physics in the presence of VL leptons, where in Sect. 4.1 we will show the VL dark matter is excluded by the large Yukawa coupling as required by the Higgs mass, whereas in Sect. 4.2 the parameter space of neutralino dark matter is expanded due to the relax of Higgs mass constraint. In Sect. 5, we briefly discuss the collider constraints on the VL leptons. Finally, we conclude in Sect. 6.

2 The model

In the minimal VL model, the matter content contains a singlet N, two down-type triplets D and \(\bar{D}\), and two VL doublets L and \(\bar{L}\) with hyper charge 1/2 and \(-1/2\), respectively. They constitute a \(\mathbf {5}_{V}\) and a \(\bar{\mathbf {5}}_{V}\) representation of SU(5) consistent with grand unification [7, 8].

The superpotential for the VL electroweak sector reads as,

$$\begin{aligned} {} W_{VL}=kH_{u}\bar{L}N-hLH_{d}N+M_{L}\bar{L}L+\frac{1}{2}M_{N}N^{2} \end{aligned}$$
(1)

where \(M_{L,N}\) refer to VL masses and k and h are Yukawa coupling constants. The superpotential for MSSM is given by,

$$\begin{aligned} {} W_{\text {MSSM}}=Y_{u}q\bar{u}H_{u}+Y_{d}q\bar{d}H_{d}+Y_{e}l\bar{e}H_{d}+\mu H_{u}H_{d}. \end{aligned}$$
(2)

Although the matter content in Eq. (1) is similar to that of NMSSM with two additional doublets [12], they are different as what follows. Following our purpose as mentioned above, we assume that the conservative R-parities of \(\mathbf {5}_{V}\) and \(\bar{\mathbf {5}}_{V}\) that contain L and \(\bar{L}\) are the same as those of the \(\mathbf {5}_{H}\) and \(\bar{\mathbf {5}}_{H}\) that include the Higgs doublets \(H_{u}\) and \(H_{d}\). Moreover, in order to ensure that the low-energy effective superpotential is precisely described by Eqs. (1) and (2), we impose a \(Z_2\) parity, under which N, \(\mathbf {5}_{V}\) and \(\bar{\mathbf {5}}_{V}\) referring to the VL sector are odd but \(\bar{\mathbf {5}}_{M}\), \(\mathbf {10}_{M}\), \(\mathbf {5}_{H}\) and \(\bar{\mathbf {5}}_{H}\) referring to the MSSM sector are even. The \(Z_2\) parity is spontaneously broken by the vacuum expectation values (vevs) in the VL sector. Since the triplets D and \(\bar{D}\) affect the phenomenological analysisFootnote 1 only in terms of their interaction with heavy triplet fields in \(\mathbf {5}_{H}\) and \(\bar{\mathbf {5}}_{H}\), we will simply neglect them.

Table 1 TeV-scale VL models and their representations under \(\mathrm{SU(5)}\)
Full size table

Apart from the superpotential, the soft mass Lagrangian \(\mathcal {L}_{\text {soft}}\) is also expanded as,

$$\begin{aligned} {}&-\mathcal {L}_{\text {soft}}\supset -\mathcal {L}^{\text {MSSM}}_{\text {soft}} +m^{2}_{L}\mid L\mid ^{2}+m^{2}_{\bar{L}}\mid \bar{L}\mid ^{2}+m^{2}_{N}\mid N\mid ^{2}\nonumber \\&\quad +\left( kA_{k}H_{u}\bar{L}N-hA_{h}LH_{d}N+\text {H.c}\right) , \end{aligned}$$
(3)

where \(m_{L,N,D}\) and \(A_{k,h}\) represent soft mass parameters, and \(\mathcal {L}^{\text {MSSM}}_{\text {soft}}\) contains MSSM soft masses such as

$$\begin{aligned}{} -\mathcal {L}^{\text {MSSM}}_{\text {soft}}= m^{2}_{H_{u}}\mid H_{u}\mid ^{2}+m^{2}_{H_{d}}\mid H_{d}\mid ^{2}+(bH_{u}H_{d}+\text {H.c}). \end{aligned}$$

Following the notation and conventions in [13], we express the Higgs and VL doublets as, respectively,

$$\begin{aligned} {} H_{u}= & {} \left( \begin{array}{c} H^{+}_{u} \\ H^{0}_{u}\\ \end{array} \right) =\left( \begin{array}{c} H^{+}_{u} \\ \upsilon _{u}+\frac{1}{\sqrt{2}}(H^{0}_{uR}+iH^{0}_{uI})\\ \end{array} \right) \nonumber \\ H_{d}= & {} \left( \begin{array}{c} H^{0}_{d} \\ H_{d}^{-}\\ \end{array} \right) =\left( \begin{array}{c} \upsilon _{d}+\frac{1}{\sqrt{2}}(H^{0}_{dR}+iH^{0}_{dI}) \\ H^{-}_{d}\\ \end{array} \right) \end{aligned}$$
(4)

and

$$\begin{aligned} {} L= & {} \left( \begin{array}{c} E^{+} \\ \eta \\ \end{array} \right) =\left( \begin{array}{c} E^{+} \\ v_{L}+\frac{1}{\sqrt{2}}(\eta _{R}+i\eta _{I})\\ \end{array} \right) \nonumber \\ \bar{L}= & {} \left( \begin{array}{c} \bar{\eta } \\ E^{-} \\ \end{array} \right) =\left( \begin{array}{c} \bar{v}_{L}+\frac{1}{\sqrt{2}}(\bar{\eta }_{R}+i\bar{\eta }_{I}) \\ E^{-}\\ \end{array} \right) \end{aligned}$$
(5)

Similarly, we write singlet N as

$$\begin{aligned} {} N=n+\frac{1}{\sqrt{2}}(N_{R}+iN_{I}). \end{aligned}$$
(6)

Mass parameters \(\upsilon _{u,d}\), \(v_{L}\), \(\bar{v}_{L}\) and n denote vevs.

These vevs are determined by the minimization of scalar potential V as given by

$$\begin{aligned} {} V=V_{F}+V_{D}-\mathcal {L}_{\text {soft}} \end{aligned}$$
(7)

where

$$\begin{aligned} {} V_{F}= & {} \mid \mu H_{d}+k\bar{L}N\mid ^{2}+ \mid \mu H_{u}-hLN\mid ^{2}\nonumber \\&+ \mid M_{L}\bar{L}-hH_{d}N\mid ^{2}+ \mid kH_{u}N+M_{L}L\mid ^{2}\nonumber \\&+\mid kH_{u}\bar{L}-hLH_{d}+M_{N}N\mid ^{2},\nonumber \\ V_{D}= & {} \frac{g^{2}_{1}+g^{2}_{2}}{8}(\mid H_{u}\mid ^{2}-\mid H_{d}\mid ^{2}+\mid L\mid ^{2}-\mid \bar{L}\mid ^{2})^{2}\nonumber \\&+\frac{g^{2}_{2}}{2}\mid H_{u}H^{\dag }_{d}+L\bar{L}^{\dag }\mid ^{2}. \end{aligned}$$
(8)

Here, \(g_{1}\) and \(g_2\) refers to \(U(1)_Y\) and \(SU(2)_{L}\) gauge coupling constant, respectively. Substituting Eqs. (4) and (6) into Eq.(7) gives rise to the EWSB conditions:

$$\begin{aligned} {} 0= & {} F_{N}(-k\bar{v}_{L})+ F_{\bar{L}} (kn)+F_{H_{d}}\mu -kA_{k}n\bar{v}_{L}\nonumber \\&+m^{2}_{H_{u}}\upsilon _{u}-\frac{1}{2}M^{2}_{Z}\cos 2\beta \upsilon _{u}(1-\delta )-b\cot \beta \upsilon _{u}\nonumber \\ 0= & {} F_{N}(hv_{L})+ F_{L} (-hn)+F_{H_{u}}\mu +hA_{h}nv_{L}\nonumber \\&+ m^{2}_{H_{d}}\upsilon _{d}+\frac{1}{2} M^{2}_{Z}\cos 2\beta \upsilon _{d}(1-\delta )-b\tan \beta \upsilon _{d} \nonumber \\ 0= & {} h^{2}n^{2}v_{L}+F_{N}(h\upsilon _{d})+hA_{h}n\upsilon _{d}\nonumber \\&+m^{2}_{L}v_{L}-\frac{1}{2}M^{2}_{Z}\cos 2\gamma v_{L} \delta \nonumber \\ 0= & {} k^{2}n^{2}\bar{v}_{L}+F_{N}(-k\upsilon _{u})-kA_{k}n\upsilon _{u}\nonumber \\&+m^{2}_{\bar{L}}\bar{v}_{L}+\frac{1}{2}M^{2}_{Z}\cos 2\gamma \bar{v}_{L} \delta \nonumber \\ 0= & {} F_{N}M_{N}+F_{L} (-h\upsilon _{d})+F_{\bar{L}}(k\upsilon _{u})+n(k^{2}\bar{v}^{2}_{L}\nonumber \\&+h^{2}v^{2}_{L})-kA_{k}\upsilon _{u}\bar{v}_{L}+hA_{h}\upsilon _{d}v_{L}+ m^{2}_{N}n, \end{aligned}$$
(9)

where \(F_i\) refer to the F-terms related to chiral superfield \(i=\{N, L, \bar{L}, H_{u,d}\}\). For simplicity, we have neglected small cross terms from D-terms. In Eq. (9), \(\tan \beta \), \(\tan \gamma \) and \(\delta \) are defined as

$$\begin{aligned} {} \tan \gamma= & {} \frac{v_{L}}{\bar{v}_{L}}, ~~~~~~~~~\delta =\frac{v^{2}_{L}+\bar{v}^{2}_{L}}{(\upsilon /\sqrt{2})^{2}}, \nonumber \\ \tan \beta= & {} \frac{\upsilon _{u}}{\upsilon _{d}},~~~~ 1-\delta =\frac{\upsilon ^{2}_{u}+\upsilon ^{2}_{d}}{(\upsilon /\sqrt{2})^{2}}, \end{aligned}$$
(10)

with the expectation value for the weak scale \(\upsilon \simeq 246\) GeV.

With the conditions of EWSB described in Eq. (9), one can directly derive the scalar and fermion mass spectrum in the electroweak sector. We refer the reader to Appendix A and B for scalar mass spectrum such as CP-even, CP-odd, and CP-charged scalars, and fermion mass spectrum such as neutralinos and charginos, respectively. In the next section, we will discuss constraints on the model parameters from precision measurements on the Higgs couplings.

3 Higgs physics

3.1 Higgs couplings

The precision measurement on the Higgs couplings is one of major tasks at the LHC, for it reveals the pattern of EWSB in the sense that different new physics models predict different sets of Higgs couplings. The LHC has verified the SM-like Higgs coupling to SM fermions such as b [14] and \(\tau \) [15] as well as couplings to SM vector bosons such as W [16]. These measurements tell us to what extension [17,18,19] \(\upsilon \) is deviated from its SM reference value. Such data will be further improved in higher level of precision at the future HL-LHC [20] or ILC [21] in preparation.

Table 2 LHC constraints from global fits to SM-like Higgs couplings [22] and weak scale \(\upsilon \) [19] at \(68\%\) CL. Mass parameter is in unit of GeV
Full size table

The Higgs couplings to SM particles in our model are given as,

$$\begin{aligned} {} y_{t}= & {} \frac{m_{t}}{\upsilon } \frac{\mathcal {O}_{11}}{\sqrt{1-\delta }s_{\beta }} , \nonumber \\ y_{b}= & {} \frac{m_{b}}{\upsilon } \frac{\mathcal {O}_{12}}{\sqrt{1-\delta }c_{\beta }}, \nonumber \\ y_{\tau }= & {} \frac{m_{\tau }}{\upsilon } \frac{\mathcal {O}_{12}}{\sqrt{1-\delta }c_{\beta }} , \nonumber \\ y_{W}= & {} \frac{2M^{2}_{W}}{\upsilon }\left[ \sqrt{1-\delta }(s_{\beta }\mathcal {O}_{11}+c_{\beta }\mathcal {O}_{12}) +\sqrt{\delta }(s_{\gamma }\mathcal {O}_{13}+c_{\gamma }\mathcal {O}_{14})\right] , \nonumber \\ y_{Z}= & {} \frac{M^{2}_{Z}}{\upsilon }\left[ \sqrt{1-\delta }(s_{\beta }\mathcal {O}_{11}+c_{\beta }\mathcal {O}_{12}) +\sqrt{\delta }(s_{\gamma }\mathcal {O}_{13}+c_{\gamma }\mathcal {O}_{14})\right] , \end{aligned}$$
(11)

where the constant coefficients are the SM values.Footnote 2 \(\mathcal {O}_{ij}\) are components of the orthogonal matrix \(\mathcal {O}\) which diagonalizes the CP-even mass matrix \(\mathcal {M}^{2}_{S}\) in Appendix A. In terms of \(\mathcal {O}_{ij}\) the SM-like Higgs scalar \(h_1\) can be written as,

$$\begin{aligned} {} h_{1} =\mathcal {O}_{11} H^{0}_{uR} +\mathcal {O}_{12} H^{0}_{dR} + \mathcal {O}_{13} \eta _{R} +\mathcal {O}_{14} \bar{\eta }_{R} +\mathcal {O}_{15} N_{R}.\nonumber \\ \end{aligned}$$
(12)

Unlike in the case of two Higgs doublets, where it is convenient to parameterize the couplings of SM-like Higgs scalar in term of two free angles \(\beta \) and \(\alpha \), in our model there are five new parameters \(m_{N}\), \(m_{L,\bar{L}}\) and \(A_{k,h}\) and three new EWSB conditions as shown in Eq. (9). It implies that in our case there are four free parameters related to the Higgs couplings, which are subject to the constraints from the global fits to the LHC measurements on the Higgs couplings as summarized in Table 2. In the following analysis, we choose the set of parameters \(\tan \beta \), \(\mathcal {O}_{11}\), \(\mathcal {O}_{12}\) and \(\delta \) for later discussions.

The best fit value for \(\delta \) has changed from the earlier 244 GeV in Ref. [17] to 233 GeV in Ref. [18] and 231 GeV in Ref. [19]. The latest constraint on \(\delta \) in Table 2 suggests the following parameter range,

$$\begin{aligned} {} 0\le \delta \le 0.22, \end{aligned}$$
(13)

which implies that the EWSB is dominated by the Higgs doublet vevs. So, it is rational to ignore the small \(\sqrt{\delta }\) term in the scaling factors \(k_{W}\) and \(k_{Z}\) in Table 2. With these approximations, we obtain the constraint on \(\mathcal {O}_{11}\) and \(\mathcal {O}_{12}\) in Table 2

$$\begin{aligned} {} 0.78 \le \mathcal {O}^{2}_{11}+\mathcal {O}^{2}_{12}\le 1.0, \end{aligned}$$
(14)

after we perform a numerical scan within the region \(10\le \tan \beta \le 50\).

3.2 Higgs self coupling

Apart from the precision measurements on the Higgs couplings above, a probe of Higgs self interaction is also useful in identifying the nature of EWSB. In the SM, the tri-Higgs and quartic Higgs coupling reads as, respectively,

$$\begin{aligned} {} \lambda ^{\text {SM}}_{3h}= & {} \frac{3m^{2}_{h}}{\upsilon },\nonumber \\ \lambda ^{\text {SM}}_{4h}= & {} \frac{3m^{2}_{h}}{\upsilon ^{2}}, \end{aligned}$$
(15)

where the self coupling \(\lambda _{3h}\) is a key parameter to determine the SM Higgs pair production at LHC [24,25,26,27,28], which mainly arises from gluon gluon fusion. The cross section for Higgs pair production is altered in new physics such as MSSM and NMSSM even at tree level [24, 26, 29,30,31,32].

Fig. 1
figure 1

The contour of \(\lambda _{3h}/\lambda ^{\text {SM}}_{3h}\) as function of \(\delta \) and \(\mathcal {O}_{11}\) for \(\tan \beta =20\) and \(\mathcal {O}_{12}=0.05\). The shaded region satisfies all constraints in the Table.2, where triangle region on the right side of green curve is in the reach of HL-LHC (14 TeV, 3\(\text {ab}\) \(^{-1}\)) [20]

Full size image

In our case, both \(\lambda _{3h}\) and \(\lambda _{4h}\) receive contributions from \(V_F\) and \(V_D\), which are given by,

$$\begin{aligned} {} \lambda _{3h_{1}}= & {} \frac{3M^{2}_{Z}}{\upsilon }\left[ \sqrt{1-\delta }(\mathcal {O}^{2}_{11}-\mathcal {O}^{2}_{12})(s_{\beta }\mathcal {O}_{11}-c_{\beta }\mathcal {O}_{12})\right. \nonumber \\&\left. +\sqrt{\delta }(\mathcal {O}^{2}_{13}-\mathcal {O}^{2}_{14})(s_{\gamma }\mathcal {O}_{13}-c_{\gamma }\mathcal {O}_{14})\right] ,\nonumber \\ \lambda _{4h_{1}}= & {} \frac{3M^{2}_{Z}}{\upsilon ^{2}}\left[ (\mathcal {O}^{2}_{11}-\mathcal {O}^{2}_{12})^{2}+(\mathcal {O}^{2}_{13}-\mathcal {O}^{2}_{14})^{2}\right] . \end{aligned}$$
(16)

From Eq. (14), one observes that the components \(\mathcal {O}_{11}\) and \(\mathcal {O}_{12}\) constitute most of SM-like Higgs within region with moderate or large \(\tan \beta \). In contrast, the other components \(\mathcal {O}_{1j}\) (\(j=3-5\)) can be ignored. In Eq. (16), we have neglected contributions to \(\lambda _{3h}\) and \(\lambda _{4h}\) due to \(V_F\) which are at least of order \(\mathcal {O}^{2}_{1j}\). As expected, the results in Eq. (16) reduce to those of MSSM under the limit \(\delta \rightarrow 0\),

$$\begin{aligned} {} \lambda ^{\text {MSSM}}_{3h_{1}}= & {} \frac{3M^{2}_{Z}}{\upsilon }\cos (2\alpha )\sin (\alpha +\beta ),\nonumber \\ \lambda ^{\text {MSSM}}_{4h_{1}}= & {} \frac{3M^{2}_{Z}}{\upsilon ^{2}}\cos ^{2}(2\alpha ). \end{aligned}$$
(17)

The probe of Higgs self interaction is thus useful in discriminating this model from the others.

Following the constraints on \(\mathcal {O}_{11}\) and \(\mathcal {O}_{12}\) in the Table 2, we show in Fig. 1 the contour of \(\lambda _{3h}/\lambda ^{\text {SM}}_{3h}\) for \(\tan \beta =20\) and \(\mathcal {O}_{12}=0.05\), with the shaded region satisfying all constraints in the Table 2. Note the ratio \(\lambda _{3h}/\lambda ^{\text {SM}}_{3h}\) is more sensitive to scaling factor \(\delta \) and \(\mathcal {O}_{11}\) rather than \(\mathcal {O}_{12}\) for large \(\tan \beta \). Under the limit \(\delta \rightarrow 0\), the ratio approaches to the maximal value \(\lambda _{3h}^{\text {max}}/\lambda ^{\text {SM}}_{3h}\) \(\sim 0.52\). With \(\delta \ne 0\), a suppression on the ratio appears. We find that the triangle region on the right side of green curve is in the reach of HL-LHC (14 TeV, 3\(\text {ab}\) \(^{-1}\)) [20], whereas the left side will be still invisible in the foreseeable future.

3.3 Higgs mass

The soft mass parameters in Eq. (3) lead to mass splittings between the fermion and its scalar partner masses in VL superfield L, \(\bar{L}\) and N. It can be extracted via evaluating the one-loop correction to effective potential [23],

$$\begin{aligned} {} \Delta V= 2\sum ^{3}_{i=1} [F(M^{2}_{b_{i}})-F(M^{2}_{f_{i}})], \end{aligned}$$
(18)

with \(F(x)=x^{2}[\ln (x/Q^{2})-3/2]/64\pi ^{2}\). For simplicity, we consider the universal soft masses \(m_{L}=m_{\bar{L}}=m_{N}=m\), universal VL masses \(M_{L}=M_{\bar{L}}=M_{N}=M\) and large \(\tan \beta \) limit. Meanwhile, we take the good approximation \(v_{L}=\bar{v}_{L}=0\). The neutral scalar mass squared matrix, which determines the mass eigenvalues \(M^{2}_{b_{i}}\) in Eq. (18), is given by (see Eq. (A1))

$$\begin{aligned} {} M^{2}_{b}\simeq \left( \begin{array}{ccc} M^{2}+m^{2} &{} 0 &{} kM\upsilon _{u} \\ *&{} M^{2}+m^{2}+k^{2}\upsilon ^{2}_{u} &{} -kX_{k}\upsilon _{u} \\ * &{} * &{} M^{2}+m^{2}+k^{2}\upsilon ^{2}_{u} \\ \end{array}\right) \nonumber \\ \end{aligned}$$
(19)

where \(X_{k}\simeq M+A_{k}-\mu \cot \beta \). The neutral fermion mass squared matrix, which gives the mass eigenvalues \(M^{2}_{f_{i}}\) in Eq. (18), reads as (see also Eq. (B3))

$$\begin{aligned} {} M^{2}_{f}\simeq \left( \begin{array}{ccc} M^{2}+k^{2}\upsilon ^{2}_{u} &{} 0 &{} kM\upsilon _{u} \\ *&{} M^{2} &{} kM\upsilon _{u} \\ * &{} * &{} M^{2}+k^{2}\upsilon ^{2}_{u} \\ \end{array} \right) , \end{aligned}$$
(20)

Substituting the mass eigenvalues in Eqs. (19) and (20) into Eq. (18), one obtains the one-loop correction to SM-like Higgs mass

$$\begin{aligned} {}&\Delta M^{2}_{h} \simeq \frac{1}{4\pi ^{2}}k^{4}\upsilon ^{2} \sin ^{4}\beta \left[ \ln (x)+\frac{1}{8}\left( \frac{\lambda (y)}{x}-\lambda _{*}\right) \right. \nonumber \\&\qquad \qquad \left. - \frac{5}{192}\left( \frac{\lambda ^{2}(y)}{x^{2}}-\lambda ^{2}_{*}\right) \right] \end{aligned}$$
(21)

where \(x=1+m^{2}/M^{2}\), \(y=X^{2}_{k}/M^{2}\) and \(\lambda _{*}=\lambda (1)\), with \(\lambda (y)\) defined as

$$\begin{aligned} {} \lambda (y)=4+2y+2\sqrt{y^{2}+4y}. \end{aligned}$$
(22)

As clearly seen in Eq. (21), \(\Delta M^{2}_{h}\rightarrow 0\) under the degenerate mass limit \(x\rightarrow 1\) and \(y \rightarrow 1\), where the scalar and fermion masses are degenerate. Conversely, large \(\Delta M^{2}_{h}\) is expected in the situation with large x and certain value of y.

Fig. 2
figure 2

The contour of Higgs mass with \(\Delta M_{h}=\sqrt{(125~\text {GeV})^2-M^{2}_{Z}}\) as function of Yukawa coupling constant k and ratio \(x=1+m^{2}/M^{2}\) that describes the mass splitting between scalars and fermions in the VL electroweak sector. The red and black curves refer to the minimal and non-minimal VL model, respectively

Full size image

Shown in Fig. 2 is the contour of Higgs mass with correction \(\Delta M_{h}=\sqrt{(125~\text {GeV})^2-M^{2}_{Z}}\). The red curves represent the minimal VL model, which are compared with the non-minimal VL model [9] as shown in black curves. In individual case, the dashed and solid one refers to the small- and large-mixing effect, respectively. There are a few comments in order. (i) \(X_{k}/M\), which is fixed by the condition of maximal-mixing effect, is equal to 12x/5 and \(2(3x-1)\) in the minimal and non-minimal VL model, respectively. (ii) The discrepancies between the two models are small (large) in the situation with maximal (small)-mixing effect. (iii) All of four patterns in Fig. 2 favor large value of x. This suggests a large mass splitting (\(m/M\sim 10\)) between the scalars and fermions in the VL electroweak sector, if the Yukawa coupling k is required to be in the perturbative region in high energy scales. For VL fermions of order \(\sim 1\) TeV, the VL scalars have mass of order \(\sim 10\) TeV, which implies that the later ones have no role to play either at the LHC or dark matter experiments.

4 Dark matter

In our model, the number of neutral fermions is seven, with four from the MSSM neutralino sector (for review, see, e.g. [33]) and the other three from the VL sector. The mass matrix for them is shown in Eq. (B3), where the complexity can be relatively reduced by taking the decoupling limit \(v_{L}=\bar{v}_{L}=0\) as favored by precise measurements on the Higgs couplings. Moreover, a small but nonzero n is sufficient to yield desired decays in the VL electroweak sector, with the first-order approximation to which the mass matrix in Eq. (B3) is further divided into two separate parts—the neutralino mass matrix A and the VL mass matrix B.

4.1 VL dark matter

For VL dark matter \(\tilde{\chi }^{0}_{1}\) is decomposed as,

$$\begin{aligned} {} \tilde{\chi }^{0}_{1}=N^{*}_{15}\tilde{\bar{\eta }}+N^{*}_{16}\tilde{\eta }+N^{*}_{17}\tilde{N}, \end{aligned}$$
(23)

where for simplicity we have neglected the neutralinos in Eq. (B3). The mass matrix \(M_{\chi }\) is reduced to matrix B. Since all of scalars in the VL sector are decoupled, the Lagrangian in Eq. (1) most relevant for this situation is given by,Footnote 3

$$\begin{aligned} {}&\mathcal {L}_{\text {VL}}\supset k(H^{+}_{u}\tilde{E}^{-}-H^{0}_{u}\tilde{\bar{\eta }})\tilde{N}-h(H^{0}_{d}\tilde{\eta }-H^{-}_{d}\tilde{E}^{+})\tilde{N}\nonumber \\&\quad \quad -\frac{g}{4c_{W}}\overline{\tilde{\eta }}\gamma ^{\mu }\gamma _{5}\tilde{\eta } Z_{\mu } +\frac{g}{4c_{W}}\overline{\tilde{\bar{\eta }}}\gamma ^{\mu }\gamma _{5}\tilde{\bar{\eta }} Z_{\mu }\nonumber \\&\quad \quad -\frac{g}{\sqrt{2}}\overline{\tilde{E}^{+}}\gamma ^{\mu }P_{L}\tilde{\eta }W^{+}_{\mu }+\frac{g}{\sqrt{2}}\overline{\tilde{E}^{-}}\gamma ^{\mu }P_{R}\tilde{\bar{\eta }}W^{-}_{\mu }. \end{aligned}$$
(24)

From Eq. (24), the VL \(\tilde{\chi }^{0}_{1}\) annihilation is mediated by Z boson, SM Higgs \(h_{\text {SM}}\) and the charged fermion \(\tilde{E}^{\pm }\), with SM final states \(\bar{f}f\), ZZ, WW and \(h_{\text {SM}}h_{\text {SM}}\).

Fig. 3
figure 3

Constraints on the parameter space of VL dark matter with \(k=1.2\) and \(\tan \beta =20\). We have shown the relic density of dark matter (red), the latest Xenon1T SI limit [40] (green dotted), LUX SD constraint [41] (green solid), Z invisible decay limit [43] (blue solid), and Higgs invisible decay limit [18] (blue dotted) simultaneously. The contours of VL dark matter masses in unit of GeV are shown in gray dotted curves. Regions on the top of green dotted curve, on the left side of green solid curve, and below the blue curves are excluded

Full size image

Equation (24) also indicates that this dark matter model is a specific example of singlet-doublet dark matter [34,35,36,37,38,39]. The key feature in our model is that the coupling \(k\simeq 1.1-1.2\) due to the observed Higgs mass. The other coupling h can be neglected for \(\tan \beta \ge 10\), in which case we are left with two mass parameters \(M_{L}\) and \(M_{N}\). Using these mass parameters, one obtains the dark matter couplings to Higgs and Z [39] given as respectively,

$$\begin{aligned} {}&c_{h}\simeq -\frac{k^{2}}{2}\frac{m_{\tilde{\chi }^{0}_{1}}\upsilon }{M^{2}_{L}+\frac{k^{2}}{2}\upsilon ^{2}+2M_{N}m_{\tilde{\chi }^{0}_{1}}-3m^{2}_{\tilde{\chi }^{0}_{1}}},\nonumber \\&c_{Z}\simeq -\frac{k^{2}}{2} \frac{M_{Z}\upsilon (m^{2}_{\tilde{\chi }^{0}_{1}}-M^{2}_{L})}{(m^{2}_{\tilde{\chi }^{0}_{1}}-M^{2}_{L})^{2}+\frac{k^{2}}{2}\upsilon ^{2}(m^{2}_{\tilde{\chi }^{0}_{1}}+M_{L}^{2})}. \end{aligned}$$
(25)

Shown in Fig. 3 are the constraints on the parameter space projected to the two-parameter plane of \(M_{L}-M_{S}\) with \(k=1.2\). The red curve therein refers to the relic density of VL dark matter \(\Omega h^2\simeq 0.12\). We show the latest limits [40, 41] on the spin-independent (SI) and spin-dependent (SD) dark matter-p/n scattering cross sections in green dotted and green solid plots, respectively, where we have used the results \(\sigma _{SI}\simeq c^{2}_{h}\times (2.11\times 10^{3})\) zb and \(\sigma ^{n}_{SD}\simeq c^{2}_{z}\times (8.97\times 10^{8})\) zb. In order to be complete, indirect constraints are also shown in Fig. 3. Since the VL leptons do not directly mix with the SM leptons, constraints such as lepton flavor violations are weaker than the electroweak precision measurements [42] on the Z boson and Higgs decay widths. In Fig. 3, the experimental bounds \(\Delta \Gamma _{Z} \le 2\) MeV [43] and \(\Delta \Gamma _{h} \le 0.16 \Gamma _{h}\) [18] are shown in blue solid and blue dotted curves, respectively, parameter regions below which are excluded.

The combination of these limits exclude parameter regions on the top of green dotted curve and on the left side of green solid curve. It turns out that while VL dark matter with mass larger than 200 GeV may be still consistent with the LUX SD limit, it is totally excluded by the latest Xenon1T limit. The main reason for it is that large \(k\sim 1.2\) and large \(\tan \beta \) lead to too large \(c_h\) in Eq. (25) to evade the Xenon1T limit.Footnote 4 In the above analysis, we have handled the dark matter relic density by an analytic treatment similar to Ref. [39] instead of numerical calcualtions such as MicrOMEGAs [44]. In particular, we have used our previous results in Ref. [45].

4.2 Neutralino dark matter

Although the VL dark matter is excluded, it still has a role to play in the case of neutralino dark matter. A neutralino dark matter \(\tilde{\chi }^{0}_{i}\) mainly arises from following components:

$$\begin{aligned} {} \tilde{\chi }^{0}_{i}=N^{*}_{i1}\tilde{B}^{0}+N^{*}_{i2}\tilde{W}^{0}+N^{*}_{i3}\tilde{H}^{0}_{d}+N^{*}_{i4}\tilde{H}^{0}_{u}, \end{aligned}$$
(26)

whose relic density is affected by the VL sector in the following ways.

Firstly, \(\tilde{\chi }^{0}_{1}\) pair may annihilate into the VL final states such as \(\tilde{E^{+}}\tilde{E^{-}}\), through either the s-channel exchange of Z boson/Higgs scalar \(h_i\) or the t- and u-channel exchange of VL scalars \(E^{\pm }\). However, this channel is not kinetically allowed, since the masses of \(\tilde{E^{\pm }}\) are larger than \(m_{\tilde{\chi }^{0}_{1}}\) when \(\tilde{E^{\pm }}\) share the same R-parity with the neutralino \(\tilde{\chi }^{0}_{1}\). Moreover, the t and u-channel exchange of VL scalars \(E^{\pm }\) are both suppressed by large VL scalar masses.Footnote 5

Second, the VL sector can mediate new Feynman diagrams for neutralino dark amtter annihilation:

  • Fermions \(\tilde{E}^{\pm }\) mediate t- or u- channel Feynman diagrams for neutralino annihilation into \(W^{\pm }\), with the neutralino-\(\chi ^{\pm }_{E}\)-W vertex given by

    $$\begin{aligned} {} \mathcal {L}\supset -\frac{g}{\sqrt{2}}W^{-}_{\mu }( N_{61}P_{L}-N_{51}P_{R})\overline{\tilde{\chi }^{0}_{1}} \gamma ^{\mu }\chi ^{+}_{E}+\text {H.c}. \end{aligned}$$
    (27)

  • Fermions \(\tilde{\eta }\) and \(\tilde{\bar{\eta }}\) mediate t- or u-channel Feynman diagrams for neutralino annihilation into Z bosons, with the neutralino-\(\tilde{\eta }/\tilde{\bar{\eta }}\)-Z vertex read as

    $$\begin{aligned} {} \mathcal {L}\supset -\frac{g}{4c_{W}}Z_{\mu }\overline{\tilde{\chi }^{0}_{1}}\gamma ^{\mu }\gamma _{5}\left( N_{61}\tilde{\eta }-N_{51}\tilde{\bar{\eta }}\right) \end{aligned}$$
    (28)

  • Fermions \(\tilde{\eta }\), \(\tilde{\bar{\eta }}\) and \(\tilde{N}\) mediate t- or u-channel Feynman diagrams for neutralino annihilation into hh, with the neutralino-\(\tilde{\eta }/\tilde{\bar{\eta }}/\tilde{N}\)-\(h_{\text {SM}}\) vertex in the MSSM modified by

    $$\begin{aligned} {}&\mathcal {L}\supset \frac{1}{\sqrt{2}}(-kN_{51}s_{\beta }+hN_{61}c_{\beta })\overline{\tilde{\chi }^{0}_{1}}\tilde{N}h_{\text {SM}}\nonumber \\&\quad \quad +\frac{1}{\sqrt{2}}N_{71}(-ks_{\beta }\tilde{\bar{\eta }}+hc_{\beta }\tilde{\eta })\overline{\tilde{\chi }^{0}_{1}}h_{\text {SM}} \end{aligned}$$
    (29)

Equations (27) to (29) determine the deviation from ordinary neutralino, which rely on mixings between \(\tilde{\chi }^{1}_{0}\) and VL sector as described by components \(N_{i1}\) with \(i=5-7\). With the mass parameter \(m\sim \) several TeVs, the conditions of EWSB in Eq. (9) imply that \(\langle n\rangle \) is at most of order \(\sim 10\) GeV and \(\delta \) is less than \(\sim 0.01\). Thus, \(N_{i1}\) reaches to its maximal values in the case of higgsino-like \(\tilde{\chi }^{1}_{0}\).

Fig. 4
figure 4

In the case of VL doublet-dominated correction, the ratio \(\delta \sigma _{\text {ann}}/\sigma _{\text {ann}}\) as function of higgsino-like neutralino mass \(m_{\tilde{\chi }^{0}_{1}}\) for \(-h=k=1.2\), \(\mid M_{L}-m_{\tilde{\chi }^{0}_{1}}\mid < M_{Z}\), and different values of \(n=\{5,15, 25, 30\}\) GeV, where \(\sigma _{\text {ann}}\simeq 3\times 10^{-27}\text {cm}^{3}/s\)

Full size image

We divide the effects on higgsino-like \(\tilde{\chi }^{1}_{0}\) into the VL doublets- and singlet-dominance. In the case of VL doublet-dominated correction, \(M_L\) is smaller than \(M_{N}\). The effect on the relic density of higgsino-like \(\tilde{\chi }^{0}_{1}\) mainly arises from the neutralino-\(\tilde{\eta }/\tilde{\bar{\eta }}\)-Z vertex, which also contributes to the SD cross section. The first-order solution to the reduced \(4\times 4\) mass matrix in Eq. (B3) only composed of Higgs doublets and VL doublets is given by

$$\begin{aligned} {} N^{\tilde{H}_{d(u)}}_{61}\simeq & {} -kn/(\mu +M_{L}), \nonumber \\ N^{\tilde{H}_{d(u)}}_{51}\simeq & {} -(+) kn/(\mu +M_{L}), \nonumber \\ N^{\tilde{H}_{d(u)}}_{71}\simeq & {} 0, \end{aligned}$$
(30)

for \(h= -k\) and \(M_{N}>M_{L}\).

Substituting Eq. (30) into Eq. (28), we show in Fig. 4 the maximal correction to dark matter annihilation cross section \(\sigma _{\text {ann}}\) that occurs in the mass region \(\mid M_{L}-m_{\tilde{\chi }^{0}_{1}}\mid < M_{Z}\) for \(k=1.2\) and different values of \(n=\{5, 15, 25, 30\}\) GeV. In the mass range of \(m_{\tilde{\chi }^{0}_{1}}=250\)–1000 GeV, the ratio \(\delta \sigma /\sigma _{\text {ann}}\) varies from \(6\%\) (\(n=30\) GeV) to less than \(0.1\%\) (\(n=5\) GeV). It is sensitive to n since \(\delta \sigma \) is proportional to factor \((n/\mu +M_{L})^{4}\). In the case of singlet-dominated correction, similar order of \(\delta \sigma /\sigma _{\text {ann}}\) is expected.

The modifications on the neutralino relic density, the SD and the SI cross sections, which are in percent level, suggest that the parameter space of neutralino dark matter is mainly expanded due to relaxing the Higgs mass constraint e.g. by lowering the values of \(\tan \beta \) or gaugino masses in the case of universal soft masses.

5 Collider detection

Collider detections on VL electroweak fermions depend on their interactions with SM particles. If mixed with SM leptons such as e, \(\mu \) or \(\tau \), VL leptons can be constrained by mixing-relevant processes at collider constraints [48,49,50]. These constraints depend on both the VL lepton mass and its mixing with the SM leptons. In our case, the VL electroweak fermions \(\chi _{E}^{\pm }\) (\(\tilde{\eta }\), \(\tilde{\bar{\eta }}\)) directly mix with charginos (neutralinos) rather than SM leptons. The strategies of searching them at colliders thus differ from the case of lepton mixings.

Table 3 The SM backgrounds for productions of VL electroweak fermion pairs at the LHC, where \(E^\mathrm{miss}_{T}\) refers to missing transverse momentum
Full size table

The VL electroweak fermions can be produced via chargino (neutralino) decay when their masses are beneath chargino (neutralino) masses. Alternatively, they just imitate the electroweak productions of charginos (neutralinos). The main difference between these two processes are that the cross section for the former case depends on both the VL lepton masses and their mixings with chargino or neutralinos, while the cross section for the later one mainly relies on the VL electroweak fermion masses. Here, we stick to the later case, where the VL electroweak fermion pairs decay and are produced in terms of SM Z-, W- and Higgs-boson. Under the basis of gauge eigenstates as shown in Eq. (24), the interaction vertexes of these processes are fixed by the SM gauge interactions, so the numbers of events for VL lepton pairs are mainly sensitive to their mass parameters \(M_L\) and \(M_{N}\). We show in Table 3 the backgrounds for the productions of various VL electroweak fermion pairs at the LHC, where mass bounds on neutralinos and charginos can be found. \(m_{\tilde{\chi }^{\pm }_{1}}\) or \(m_{\tilde{\chi }^{0}_{2}}\) up to \(\sim 550\) GeV is excluded for \(m_{\tilde{\chi }^{0}_{1}}\) less than \(\sim 100\) GeV, and they are obviously relaxed when \(m_{\tilde{\chi }^{0}_{1}}\) is above 100 GeV [53, 54].

The mixing effects in the VL electroweak sector are described by the mass matrix B in Eq. (B4), which is similar to the higgsino-bino benchmark scenario. See the reduced matrix A with wino decoupled in Eq. (B4). We can infer the implications of LHC limits above to the parameter space of \(M_{L}-M_{N}\) by introducing a ratio of signal strengths \(\hat{\mu }\). For example, for the process \(pp\rightarrow \tilde{\chi }^{\pm }_{E}\tilde{\chi }^{0}_{2}+\) jets \(+\) leptons [53, 54] we have

$$\begin{aligned} {} \hat{\mu }=\frac{\sigma (pp\rightarrow \tilde{\chi }^{\pm }_{E}\tilde{\chi }^{0}_{2}+\cdots )}{\sigma (pp\rightarrow \tilde{\chi }^{\pm }_{1}\tilde{\chi }^{0}_{2}+\cdots )} \cdot \frac{\text {Br}(\tilde{\chi }^{\pm }_{E}\rightarrow W^{\pm } \tilde{\chi }^{0}_{1})}{\text {Br}(\tilde{\chi }^{\pm }_{1}\rightarrow W^{\pm } \tilde{\chi }^{0}_{1})}\cdot \frac{\text {Br}(\tilde{\chi }^{0}_{2}\rightarrow Z \tilde{\chi }^{0}_{1})}{\text {Br}(\tilde{\chi }^{0}_{2}\rightarrow Z \tilde{\chi }^{0}_{1})},\nonumber \\ \end{aligned}$$
(31)

where \(\tilde{\chi }^{0}_{1}\) in the numerators (denominators) refers to the lightest neutral fermion in the VL (higgsino-bino) sector. In the parameter regions with \(M_{N}<M_{L}\), where \(\tilde{\chi }^{0}_{1}\sim \tilde{N}^{0}\), \(\tilde{\chi }^{0}_{2}\sim \tilde{\eta }^{0}\) and \(\tilde{\chi }^{\pm }_{E}\sim \tilde{E}^{\pm }\), the number of events for \(\tilde{\chi }^{\pm }_{E}\tilde{\chi }^{0}_{2}\) pair can be of same order as that of \(\tilde{H}_{u,d}^{\pm }\tilde{B}^{0}\). So the pattern of LHC limits projected to the plane of \(M_{N}-M_{L}\) is similar to Fig. 8(d) in Ref. [54]. In the parameter regions with \(M_{N}>M_{L}\), where \(\tilde{\chi }^{0}_{1}\sim \tilde{\eta }^{0}\), \(\tilde{\chi }^{0}_{2}\sim \tilde{N}^{0}\), and \(\tilde{\chi }^{\pm }_{E}\sim \tilde{E}^{\pm }\), the mass splitting between \( \tilde{E}^{\pm }\) and \(\tilde{\chi }^{0}_{1}\) is small similar to the case of Higgsino-like \(\tilde{\chi }^{\pm }_{1}\) and \(\tilde{\chi }^{0}_{1}\). In this situation, the decays [55] of charged fermion \(\tilde{\chi }^{\pm }_{E}\rightarrow \tilde{\chi }^{0}_{1}W^{*\pm }\rightarrow \tilde{\chi }^{0}_{1} \cdots \) are rather sensitive to the mass splitting.

6 Conclusion

In this paper, we have studied the minimal VL model motivated by the grand unification. The key feature in such model is that the VL electroweak sector couples to the Higgs doublets rather than mixes with the SM leptons. Therefore, they can play important roles both in the phenomenologies of Higgs physics and dark matter.

For the Higgs physics, we have used the LHC bounds on the Higgs coupling constants to constrain the vevs in the VL electroweak sector, the magnitude of which are found to be small. Consequently, the magnitudes of the mixing effects between the Higgs doublets and the VL electroweak sector controlled by the vevs are small as well, leading to small deviation in the Higgs self coupling from the MSSM expectation. Moreover, we used the observed Higgs mass to constrain the other model parameters in the VL sector. The fit reveals that both large Yukawa coupling constant \(k\sim 1.1\) and large mass splitting between scalar and fermion masses in the VL sector are required.

For the dark matter phenomenology, w have verified that the large Yukawa coupling k excludes the possibility of VL dark matter. For it gives rise to too large dark matter coupling to Higgs to evade the Xenon1T limit. Nevertheless, they are still useful to reduce the tension on the neutralino dark matter by relaxing the constraint from Higgs mass.

Finally, in order to be complete we briefly discussed the constraints on the VL electroweak fermions at the LHC. Unlike in the 4th generation leptons which directly mix with SM leptons, the VL electroweak fermions can be either produced via the decays of charginos and neutralinos, or they imitate the electroweak productions of charginos and neutralinos. The prospect of the later class at the high luminosity-LHC will be explored elsewhere [56].