Existence, nonexistence, and multiplicity of solutions for the fractional p&q\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$p\&q$\end{document}-Laplacian equation in RN\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$\mathbb{R}^{N}$\end{document}

In this paper, we study the existence, nonexistence, and multiplicity of solutions to the following fractional p&q\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$p\&q$\end{document}-Laplacian equation: 0.1(−Δ)psu+a(x)|u|p−2u+(−Δ)qsu+b(x)|u|q−2u+μ(x)|u|r−2u=λh(x)|u|m−2u,x∈RN,\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document} $$\begin{aligned} &(-\Delta)^{s}_{p}u + a(x)\vert u\vert ^{p-2}u +(-\Delta)^{s}_{q}u + b(x)\vert u\vert ^{q-2}u +\mu(x)\vert u\vert ^{r-2}u \\ &\quad= \lambda h(x)\vert u \vert ^{m-2}u,\quad x\in {\mathbb {R}}^{N}, \end{aligned}$$ \end{document} where λ is a real parameter, (−Δ)ps\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$(-\Delta)_{p}^{s} $\end{document} and (−Δ)qs\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$(-\Delta )_{q}^{s} $\end{document} are the fractional p&q\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$p\&q$\end{document}-Laplacian operators with 0<s<1<q<p,r>1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$0< s<1<q<p, r>1$\end{document} and sp<N\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$sp< N$\end{document}, and the functions a(x),b(x),μ(x)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$a(x), b(x),\mu(x)$\end{document}, and h(x)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$h(x)$\end{document} are nonnegative in RN\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}${\mathbb {R}}^{N}$\end{document}. Three cases on p,q,r,m\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$p,q,r,m$\end{document} are considered: p<m<r<ps∗\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$p< m< r< p_{s}^{*}$\end{document}, max{p,r}<m<ps∗\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$\max\{p,r\}< m< p_{s}^{*}$\end{document}, and 1<m<q<r<ps∗\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$1< m< q< r< p_{s}^{*}$\end{document}. Using variational methods, we prove the existence, nonexistence, and multiplicity of solutions to Eq. (0.1) depending on λ,p,q,r,m\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$\lambda, p,q,r,m$\end{document} and the integrability properties of the ratio hr−p/μm−p\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$h^{r-p}/\mu^{m-p}$\end{document}. Our results extend the previous work in Bartolo et al. (J. Math. Anal. Appl. 438:29-41, 2016) and Chaves et al. (Nonlinear Anal. 114:133-141, 2015) to the fractional p&q\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$p\&q$\end{document}-Laplacian equation (0.1).


Introduction and the main result
In this paper, we study the existence, nonexistence, and multiplicity of solutions to the following fractional p&q-Laplacian equation: where (-) s p and (-) s q are the fractional p&q-Laplacian operators with  < s <  < q < p, r >  and sp < N . The nonlinearity f (x, u) = λh(x)|u| m- uμ(x)|u| r- u can be seen as a competitive interplay of two nonlinearities. The coefficients a(x), b(x), μ(x), h(x) are assumed to be positive in R N , and other exact assumptions will be given further.
The fractional t-Laplacian operator (-) s t with  < s <  < t and st < N is defined along a function ϕ ∈ C ∞  (R N ) as |ϕ(x)ϕ(y)| t- (ϕ(x)ϕ(y)) |x -y| N+ts dy, ∀x ∈ R N , (  .  ) where B ε (x) := {y ∈ R N : |x -y| < ε}; see [-] and the references therein. When p = q, Eq. (.) is reduced to the fractional p-Laplacian equation and when s = , Eq. (.) is the p&q-Laplacian equation Equation (.) comes from a general reaction-diffusion system where D(u) = |∇u| p- +|∇u| q- . This system has a wide range of applications in physics and related sciences such as biophysics, plasma physics, and chemical reaction design. In such applications, the function u describes a concentration, and the first term on the right-hand side of (.) corresponds to the diffusion with a diffusion coefficient D(u), whereas the second one is the reaction and relates to sources and loss processes. Typically, in chemical and biological applications, the reaction term f (x, u) is a polynomial of u with variable coefficients [, ]. The solution of (.) has been studied by many authors; for example, see [, , , -] and the references therein. In the literature cited, the authors always assume that the potentials a(x), b(x) satisfy one of the following conditions: It is well known that one of assumptions (A  ), (A  ), and (A  ) guarantees that the embedding W ,t (R N ) → L r (R N ) is compact for each t ≤ r < t * = tN N-t with  < t < N . As far as we know, there are few papers that deal with a general bounded potential case for problem (.). Now let us recall some advances of our problem. Pucci and Rădulescu [] first studied the nonnegative solutions of the equation where A(x, ∇u) acts like the p-Laplacian, max{, p} < m < min{r, p * }, and the coefficients ω and h are related by the integrability condition By imposing a strong convexity condition of the p-Laplacian type on the potential of A, the authors extend completely the result of []. Moreover, Autuori and Pucci [] proposed two open questions: the deletion of the restriction max{, p} < m and the replacement of (.) by the assumption that Later, Autuori and Pucci [] studied the existence and multiplicity of solution to the following elliptic equation involving the fractional Laplacian: where λ > ,  < s < , s < N ,  < m < min{r,  * s },  * s = N/(N -s), and (-) s is the fractional Laplacian operator. The coefficients ω and h are related by condition (.). The authors proved the existence of entire solutions of (.) by using a direct variational method and the mountain pass theorem.
More recently, Xiang et al.
Up to now, it is worth noting that there is much attention on equations like (.), (.), and (.) with  < m < r. From the papers mentioned, it is natural to ask whether the existence, nonexistence, and multiplicity of solutions to Eq. (.) is admitted if  < r < m < p * s and  < m < r < p * s ? Clearly, equations like (.), (.), and (.) are contained in (.). In this paper, motivated by [, ], we will answer this interesting question, extend the p&q-Laplacian (.), which has been studied deeply in [, ], to the fractional p&q-Laplacian equation (.), and investigate the existence, nonexistence, and multiplicity of solutions depending on λ and according to the integrability properties of the ratio h r-p /μ m-p .
For this purpose, we apply a version of symmetric mountain pass lemma in []. Also, we adapt some ideas developed by Pucci et al. [] and Xiang et al. []. Note that although the idea was earlier used for other problems, the adaptation to the procedure to our problem is not trivial at all since the parameters r, m satisfy  < r < m and we must consider our problem for a suitable space, and so we need more delicate estimates and new technique. Our results, which are new even in the canonical case p = q = , generalize the main results of [, ] in several directions. Furthermore, we weaken the conditions in those papers, and assumptions (A  )-(A  ) are not necessary for our results.
In order to state our main theorems, we recall some fractional Sobolev spaces and norms. The fractional Sobolev space This space is endowed with the natural norm whereas [u] s,t denotes the Gagliardo seminorm given by The spaces X p and X q denote the completion of C ∞  (R N ) with respect to the norms is a uniformly convex Banach space, and there exists a positive constant S  = S  (N, t, s) such that where t = p or q, t * s = tN N-ts is the fractional critical exponent, and S r is a constant depending on s, r, t, N . For convenience, we denote S t * s by S  . Consequently, the space Y t is continuously Clearly, from definitions (.) and (.) and assumption (H  ), we see that Let J(u) : E → R be the energy functional associated to Eq. (.) defined by where the norms · X p and · X p are defined by (.). From the embedding inequalities (.) and assumptions (H  )-(H  ) below, we see that the functional J is well defined and J ∈ C  (E, R) with Throughout this paper, we let  < s <  < q < p with sp < N . Our main results are as follows.

Theorem . Assume (H  ) and
Then Eq. (.) with λ >  admits infinitely many solutions u n ∈ E with u n →  in E.
Remark . From Theorem ., we know that it still remains an open problem to verify whether λ  = λ  . In addition, the nonlinear function f (x, u) = λh(x)|u| m- uμ(x)|u| r- u with p < m < r fails to satisfy the Ambrosetti-Rabinowitz condition. Furthermore, for s =  in (.), our results and context are more general than those in [, ].
The paper is organized as follows: In Section , we give some preliminaries, will set up the variational framework for problem (.), and prove that the functional associated to (.) satisfies the (PS) c condition. The proofs of Theorems . and . are given in Section . Finally, Section  is devoted to the proof of Theorem ..

Preliminaries
To prove our main results, we need to establish some lemmas. Proof We first choose a constant β >  such that u n E ≤ β for all n ≥ . If (H  ) is satisfied, then for any ε > , there exists R  >  such that Then, it follows from the Hölder inequality and Lemma . that, for R ≥ R  , By Lemma ., up to a subsequence, we obtain u n → u strongly in L m (B R  ) and u n (x) → u(x) a.e. in B R  as n → ∞. Thus h(x)|u n (x)u(x)| m →  a.e. in B R  as n → ∞. Similarly, for each measurable subset ⊂ B R  , we have Since h(x) ∈ L γ (R N ), we obtain that the sequence {h(x)|u n (x)u(x)| m } is uniformly integrable and bounded in L  (B R  ). Furthermore, an application of the Vitali convergence theorem gives Then the conclusion that u n → u strongly in L m (R N , h) follows from (.) and (.). If (H  ) is satisfied, then for any ε > , there exists R  >  such that Similarly, we can derive (.). Then combining (.) with (.), we have u n → u in L m (R N , h).

Lemma . Let (H  ) and one of assumptions (H  ) and (H  ) hold. If {u n } is a bounded (PS) c sequence of the functional J defined by (.), then the functional J satisfies (PS) c condition.
Proof Let {u n } be a (PS) c sequence, that is, Since the sequence {u n } is bounded in E, there exists a subsequence, still denoted by {u n }, such that where t = p or q. We now prove that u n → u in E. Let ϕ ∈ E be fixed and denote by T ϕ the linear functional on E defined by where A ϕ (v) and B ϕ (v) are the linear functionals defined by respectively. Clearly, by the Hölder inequality, T ϕ is also continuous, and Furthermore, the fact that u n u weakly in E implies that lim n→∞ A u (u nu) = lim n→∞ B u (u nu) = , and so On the other hand, as n → ∞, we have From (.) and Z n ≥ , we obtain, for large n, Note that, by Lemma ., P n →  as n → ∞.
Let us now recall the well-known vector inequalities: for all ξ , η ∈ R N , where c p and C p are positive constants depending only on p. Assume first that p > q ≥ . Then by (.) we have u nu p p,a ≤ c p n and Similarly, we have u nu q q,b ≤ c q n and By (.) and (.) we see that Then the application of (.) yields In conclusion, u n → u in E as n → ∞.
Finally, it remains to consider the case  < p < . By (.) there exists β >  such that u n E ≤ β for all n ≥ . Now from (.) and the Hölder inequality it follows that and where we have applied the inequality (x + y) (-p)/ ≤ x (-p)/ + y (-p)/ for all x, y ≥  and  < p < , (.) and D p = C p β p(-p)/ . Similarly, for  < q < , we have with D q = C q β q(-q)/ . Then, by (.), (.), and (.) we get with some C  > . Then (.) and (.) imply that u n → u in E as n → ∞. Therefore, J satisfies the (PS) c condition, and we complete the proof of Lemma ..
Lemma . Under the assumptions of Theorem ., suppose that u ∈ E is a nontrivial weak solution of (.). Then there exists λ  >  such that λ ≥ λ  .

Lemma . Under the assumptions of Theorem ., the functional J is coercive in E.
Proof where c  = (r) where S  is the embedding constant in (.). So, it follows from (.)-(.) that For fixed R  >  and for any τ >  and ω > , we decompose B R  = X ∪ Y ∪ Z as follows: On the other hand, since u ∈ E, we have that J(u) ≥ d, which shows that J(u) = d. Therefore, u is a global minimum for J, and hence it is a critical point, namely a weak solution of (.).
Lemma . Under the assumptions of Theorem ., there exists λ  >  such that for all λ > λ  , Eq. (.) admits a global nontrivial minimum u  ∈ E of J with J(u  ) < .
Proof Clearly, J() = . Consider the constrained minimization problem Let u n be a minimizing sequence of (.), which is clearly bounded in E, so that we can assume, without loss of generality, that it converges weakly to some u  ∈ E with u  m m,h = m and Thus, J(u  ) = λ λ <  for any λ > λ  , and This completes the proof.
Next, we show that if λ > λ  , then problem (.) admits a second nontrivial weak solution e = u  by the mountain pass theorem.
Lemma . Suppose that assumptions (H  )-(H  ) are satisfied. Then, for all e ∈ E and λ > , there exist α >  and ρ ∈ (, e E ) such that J(u) ≥ α for all u ∈ E with u E = ρ.
Proof Let u ∈ E. From (H  ), (.), and (.) with t = p we obtain Then, where H = h γ , u E = ρ, and so that Thus, we finish the proof of Lemma ..
Since λ >  and m > max{p, r}, there exists R = R(E  ) > ρ such that J(u) <  for u ∈ E  and u E ≥ R. Therefore, all conditions are verified. Then an application of Theorem . in [] shows that Eq. (.) admits infinitely many solutions u n ∈ E with J(u n ) → ∞ as n → ∞. This completes the proof of Theorem ..

Proof of Theorem 1.4
In this section, we give a proof of Theorem .. The main tool for this purpose is the following symmetric mountain pass lemma. First, we introduce the concept of genus.

Definition . []
Let E be a Banach space, and A a subset of E. The set A is said to be symmetric if u ∈ E implies -u ∈ E. For a closed symmetric set A that does not contain the origin, we define the genus γ (A) of A as the smallest integer k such that there exists an odd continuous mapping from A to R k \ {}. If such k does not exist, then we define γ (A) = ∞. We set γ (∅) = . Let k denote the family of closed symmetric subsets A of E such that  / ∈ A and γ (A) ≥ k.

Lemma . [] (Symmetric mountain pass lemma) Let E be an infinite-dimensional
We choose small d >  such that the cube D(d) ⊂  := B d  (x  ). Note that h(x) >  in D(d).
Fix k ∈ N arbitrarily. Let n ∈ N be the smallest integer such that n N ≥ k. We divide D(d) equally into n N small cubes, denoted D i ,  ≤ i ≤ n N , by planes parallel to each face of D(d).
The edge of D i has the length of z = d n . We construct new cubes E i in D i such that E i has the same center as that of D i . The faces of E i and D i are parallel, and the edge of E i has the length d n . Then, let the functions ψ i (x) ∈ C  (R N ),  ≤ i ≤ k, be such that supp(ψ i ) ⊂ D i , supp(ψ i ) ∩ supp(ψ j ) = ∅ (i = j), (.)