A CLASS OF GENERALIZED QUASILINEAR SCHR¨ODINGER EQUATIONS

. We establish the existence of nontrivial solutions for the following quasilinear Schr¨odinger equation with critical Sobolev exponent: where V ( x ) : R N → R is a given potential and l,h are real functions, λ ≥ 0, α > 1, 2 ∗ = 2 N/ ( N − 2), N ≥ 3. Our results cover two physical models l ( s ) = s α 2 and l ( s ) = (1 + s ) α 2 with α ≥ 3 / 2.

1. Introduction. In this paper, we consider the following quasilinear Schrödinger equation: where V (x) : R N → R is a given potential and l, h are real functions, λ ≥ 0, α > 1, N ≥ 3. Solutions of (1) are related to standing wave solutions of Schrödinger equation where z : R × R N → C, W : R N → R is a given potential and ρ is a real function. For l(s) = s, (2) was used for a superfluid film equation in plasma physics, which was introduced in Kurihara [14]. In the case l(s) = (1 + s) 1/2 , (2) models the self-channeling of a high-power ultra short laser in matter [15]. Besides, equation (2) also appear in plasma physics and fluid mechanics [16], in dissipative quantum mechanics [11], in the theory of Heisenberg ferromagnetism and magnons [13,22] and in condensed matter theory [20]. See also [10,6] for more physical backgrounds. Let l(s) = s  (1), we obtain the following corresponding equation of elliptic type: If α = 2, equation (3) has been studied extensively recently. See, for example, [21,7,17,18,19,1,2,23,25] for λ = 0 (i.e., the subcritical case) and [9,26,29,27,28] for λ = 0 (i.e., the critical case). For general α > 0, by using a minimization argument, Liu and Wang in [17] studied the following quasilinear Schrödinger equation with subcritical growth where α > 1/2. They proved the existence of a solution for a sequence of λ n → ∞ and a sequence of λ n → 0 provided 4α ≤ p + 1 < 2α2 * . In [1] and [2], Adachina and Watanable considered the following changing of unknown variable v = f −1 (u) which was introduced in [18] for α = 1 in (5) (see also [7]), where f is defined by ODE: Applying the change of variable (6), the quasilinear equation (5) is reduced to a semilinear equation. Then, by using variational techniques, the existence of unique solution and multiple positive solutions were established. In the mathematical literature, few results are known on (4). For α = 1, in [8], Bouard, Hayashi and Saut proved the global existence and uniqueness of small solutions in transverse space dimensions 2 and 3. But, they did not studied the existence of standing waves. For standing waves and λ = 0, we refer to [25].
Our goal in this paper is to study the existence of nontrivial solutions for quasilinear Schrödinger equation (1) with critical exponent. Without loss of generality, in what follows, we set λ = 1 in (1).
The poteintial V : R N → R is continuous and satisfies: The nonlinearity h : R → R is continuous, we also suppose the following hypotheses: dt. Now, we may state our main result: has a nontrivial solution if either N ≥ 4α + 2 and µ > 2α or 3 ≤ N < 4α + 2 and µ > α2 * − 1.
In order to prove our Theorems, motivated by the arguments used in [26,25], we first use a change of variable to reformulate the problem by a semilinear problem which the associated functional is well-defined in the Sobolev space H 1 (R N ) and satisfies the geometric hypotheses of the mountain pass theorem. Using this fact, we obtain a Palais-Smale sequence converging weakly to a solution v. In order to prove that v is nontrivial, arguing by contradiction that v = 0, we combine Lions's compactness lemma together with some classical arguments used by H. Brezis and L. Nirenberg [5] to establish that the Palais-Smale sequence has a nonvanishing behaviour. Finally, a translated Palais-Smale sequence converges to a nontrvial critical point of an associated functional at infinity. Then, this critical point is used to construct a path related to mountain pass theorem to find a contradiction.
In this paper, we shall work on the space H 1 (R N ) with the norm .

2.
Proof of Theorem 1.1. We observe that (1) is the Euler-Lagrange equation of the following functionals: where g(s) = 1 + 2[sl (s 2 )] 2 . From the variational point view, the first difficulty associated with (8) is to find an appropriate function space where it is well defined. To overcome this difficulty, as in [24,25], we make change of variables as the following: Then, after the changing of variables, I can be written by the following functional Under the hypotheses ( If u is a nontrivial solution of (7), then it should satisfy We show that (10) is equivalent to Indeed, if we choose ϕ = 1 g(u) ψ in (10), then we get (11). On the other hand, since (11), we get (10). Therefore, in order to find the nontrivial solutions of (7), it suffices to study the existence of the nontrivial solutions of the following equation We collect some properties of the change of variable G −1 (t).
which implies the result. Now, we establish the geometric hypotheses of the mountain pass theorem.

YAOTIAN SHEN AND YOUJUN WANG
By Lemma 2.1− (3) and (h 2 ), we get Thus, for ε > 0 sufficiently small, there exists a constant C ε > 0 such that Then, we have Therefore, by choosing ρ 0 small, we get (1) when Let e = tϕ with t large enough, we get the result.
In consequence of Lemma 2.3 and of Ambrosetti-Rabinowitz mountain mass theorem [3], for the constant and for any ψ ∈ C ∞ 0 (R N ), where o n (1) → 0 as n → ∞. Now, we consider the function Therefore, by (h 3 ), (14) and (16), we have On the other hand, by Sobolev imbedding inequality, we have In the case {x : |v n (x)| ≤ 1}, since g(v n ) is nondecreasing, we get . We shall prove that J (v) = 0, that is, v is a weak solution of (12). To prove this, it suffices to show for ∀ψ ∈ C ∞ 0 (R N ) that Analogous to the arguments in [26], by the Lebesgue Dominated Theorem, we have Hence, v is a weak solution of (1). If v ≡ 0, then Theorem 1.1 is proved. Otherwise, we further establish the Palais-Smale sequence has a nonvanishing behaviour. To this end, we state the result which provides an appropriate estimate on the minimax level.
Lemma 2.5. The minimax level c satisfies where S is the best constant of the embedding D 1,2 (R N ) → L 2 * (R N ).

Proof. It suffices to show that there exists
We follow the strategy used in [5]. First, we choose a cut-off function It is known that w ε (x) satisfies the equation −∆u = u 2 * −1 in R N and Thus, if we define the function v ε (x) = ψε(x) |ψε| 2 * , then, by (21) and (22), as ε → 0, we have .

Since lim
t→∞ J(tv ε ) = −∞, there exists t ε > 0 such that J(t ε v ε ) = max t>0 J(tv ε ). We claim that there exist ε 0 > 0 and positive constants t 0 , t 1 > 0 such that t 0 ≤ t ε ≤ t 1 for every 0 < ε < ε 0 . First, we prove that t ε is bounded from below by a positive constant. Otherwise, we could find a sequence ε n → 0 such that t εn → 0. Up to a subsequence (still denote by ε n ), we have t εn v εn → 0. Therefore, 0 < c ≤ sup t≥0 J(t εn v εn ) = J(0) = 0, which is a contradiction. On the other hand, similar to the arguments used in [26] and Lemma 2.1-(8), we have where C > 0 is independent of ε. Since v ε is uniformly bounded for 0 < ε < ε 0 , the claim is proved. Now, by Lemma 2.2, we have
Moreover, the condition α ≥ 3/2 is necessary for (50). Then We will show that γ (s) ≤ 0 for all s ≥ 0. It suffices to prove that .