Symbol Error-Rate Analysis of OSTB Codes and Linear Precoder Design for MIMO Correlated Keyhole Channels

Analytical expressions for the symbol error rate (SER) of MIMO systems are important from both design and analysis point of views. This paper derives exact and easy to evaluate analytical expression for the SER of an orthogonal space-time block coded (OSTBC) MIMO system with spatially correlated antennas, in which the channel signal propagation suffers from a degenerative effect known as a keyhole. These expressions are valid for complex valued correlations between antenna elements. Numerical results obtained by simulations are presented to confirm the validity of the analytical SER expressions for several multilevel modulation schemes. Subsequently, a procedure for designing a linear precoder which exploits the knowledge of the channel correlation matrix at the transmitter to enhance the performance of the above system is given. To this end, the exact SER expression is minimized by using a gradient descent algorithm. To demonstrate the performance improvements achievable with the proposed precoder, numerical results obtained in several design examples are presented and compared.


INTRODUCTION
The benefits of using multiple transmit and receive antennas and space-time coding for wireless communication are well known [1]. In principle, the capacity of such multiple-input multiple-output (MIMO) channel can be much higher than a single-input, single-output (SISO) channel. The capacity of a MIMO channel is maximum when each transmit and receive antenna pair forms a statistically independent transmission path, a condition which is nearly met in a rich scattering environment. Space-time codes (STC) can be used to exploit the spatial diversity available in such a channel to realize high-rate data transmission [1]. However, in realistic situations, the channel capacity is degraded by spatial correlations at the transmitting and/or receiving antenna arrays, due to, for example, insufficient spacing of antenna elements in the arrays, the existence of few dominant scatters, or small AOA spreading [2]. Furthermore, recent studies, both theoretical and experimental, have shown that, in addition to antenna correlation, the MIMO channel capacity can also be decreased by so-called keyhole effects in wave propagation [3,4]. In particular, a keyhole effect reduces the rank of the MIMO channel matrix to one, even in the absence of any spatial correlation in the channel [4]. In a keyhole MIMO channel, the signals from the transmit antenna array are constrained by the physical structure of the propagation environment to pass through a keyhole before reaching the receive antenna array (Figure 1(a)). Such keyhole effects have been observed in both indoor and outdoor propagation environments [5,6].
In previous work, the capacity of MIMO keyhole channels is studied in [3,[7][8][9]. In [10], expressions for the exact symbol error rate (SER) of orthogonal space-time block codes (OSTBC) over a double Rayleigh fading keyhole channel are derived for M-PSK and M-QAM modulation schemes. In a subsequent work [7], similar analysis is also carried out for keyhole channels with Nakagami-m fading. In another related work, [11] derives expressions for the pairwise error probability (PEP) of MIMO systems with keyhole channels. However, in [7,10,11], the channel is assumed to be spatially uncorrelated. Similar analysis for MIMO keyhole channels with spatial correlations has not 2 EURASIP Journal on Wireless Communications and Networking been reported so far. It is well known that if the channel correlation matrix is known at the transmitter (e.g., as channel side information (CSI)), a linear precoder matched to the channel correlation structure can be used at the MIMO channel input to improve the system performance (see, e.g., [12][13][14][15][16]). However, the aforementioned work assumes Rayleigh or Ricean fading and is not applicable to double Rayleigh fading as in the case of channels with keyholes. Formally, a keyhole MIMO channel can be modeled by a matrix which is the outer product of two complex Gaussian vectors [4]. This implies that each coefficient in the channel matrix is a product of two independent complex Gaussian variables (and hence the name double Rayleigh fading). Since keyhole effects lead to the degeneration of the channel matrix [3,4], it is particularly important to exploit CSI available at the transmitter using a properly designed precoder to enhance the MIMO system performance.
In contrast to previous work, we consider in this paper the SER analysis and the design of linear precoders which minimize the average SER of double Rayleigh fading MIMO channels with spatial correlations. More specifically, we derive easy, to evaluate, exact analytical expressions for the average SER of systems with M-PSK, M-PAM, and M-QAM signaling and maximum likelihood (ML) detection. These closed-form SER expressions are useful in performance analysis of such systems and can be easily evaluated with computing software, using readily available functions. Numerical results are presented in this paper to confirm that our analytical expressions agree with those obtained by simulation of the OSTBC-MIMO system. Based on the analytical expressions for SER, we then present a design of linear precoders which minimize the average SER by exploiting the knowledge of the channel correlation matrix (available at the transmitter either as prior knowledge or via a feedback channel from the receiver), under a constraint on the total average transmitted power. In our formulation, the antenna correlations are allowed to be complex valued [17]. In the general case, we consider it is difficult to find a closedform solution for the minimum SER (MSER) precoder. As such, the MSER precoder matrix is found by a constrained gradient descent minimization method, based on the derivatives of the average SER. Experimental results are presented to demonstrate the performance improvements achieved by the proposed MSER precoder. In these experiments, MIMO keyhole channels with both real and complex correlation matrices have been considered.
The expressions and the optimal precoder design derived in this paper are novel in the following aspects. Different to [10], the SER expression derived in this paper applies to MIMO keyhole channels with spatial correlations at both transmit and receive antennas. While the SER expressions in [10], which apply only to uncorrelated channels, do not generalize to our case, we show that in the absence of spatial correlations the SER expressions we derive reduce to those in [10]. Note that, for the uncorrelated channel considered in [10], the precoding problem we consider is not relevant. Also, different to [14,15], the SER expressions and the optimal precoders derived in this paper account for the keyhole effect, that is, each channel coefficient is a  product of two complex Gaussian random variables. While there is no direct relationship between these expressions, they correspond to two extreme cases of a double scattering channel with n s ≥ 1 scatterers [4]. In particular, when n s = 1, the channel exhibits the keyhole effect considered in this paper. When n s →∞, the MIMO channel is the singlescattering case considered in [14,15]. It should be noted that neither the results in this paper nor those in [14,15] readily generalize to the case 1 < n s < ∞.
The rest of this paper is organized as follows. Section 2 describes the keyhole channel model and the OSTBC-MIMO system model under consideration. Section 3 derives the SER expressions for this system, where M-PSK, M-PAM, and M-QAM modulation schemes are considered. Section 4 then presents an algorithm for designing an MSER linear precoder for the system. In Section 5, numerical results are presented to demonstrate the accuracy of the given SER expressions and the performance gains achievable with the proposed precoder designs. Finally, concluding remarks are given in Section 6. Some proofs are provided in the appendices.

Channel model
The gain matrix H of a double Rayleigh fading keyhole MIMO channel with n t transmit antennas and n r receive antennas can be represented by the outer product of two independent complex Gaussian vectors h r (size n r × 1) and h t (size 1 × n t ) representing transmit and receive Rayleigh fading, respectively, [4], that is, H = h r h t . However, each vector is assumed to be correlated so that h t = h tw R 1/2 t and h r = R 1/2 r h rw , where R t is the n t × n t transmit antenna correlation matrix, R r is the n r × n r receive antenna correlation matrix, and h tw ∼ CN (0 1×nt , I nt ) and h rw ∼ CN (0 nr ×1 , I nr ) are independent identically distributed (iid) complex Gaussian vectors of sizes 1 × n t and n r × 1, respectively, (here, R 1/2 t and R 1/2 r are the unique square roots of Hermitian positive semidefinite matrices R t and R r , resp., [18]). Note that each element of H is a product of two independent Gaussian variables. Note also that the rank of H is necessarily one.

System model
The communication system considered in this paper is shown in Figure 1(b). An orthogonal space-time block (OSTB) encoder operates on a block of input symbols x = (x 1 , . . . , x K ), where x k ∈ A and A is the complex valued modulation signal set with |A| = M. In this paper, M-PSK, M-PAM, and M-QAM signal sets will be considered. Let the output of the OSTB encoder C be a B × N matrix, where B is the space dimension and N is the time dimension. The elements of C are linear combinations of x 1 , . . . , x K and their complex conjugates. The OSTB codeword is precoded using a n t ×B matrix F to produce the transmitted codeword Z = FC. Thus, the transmission rate is R = K/N. The average power of a transmitted codeword is P t = E{Tr (ZZ H )}. From the orthogonality property of the OSTBC [21], it follows that where P x = E{|x k | 2 } is the average power of modulation signal set, and a is a constant that depends on the particular OSTBC in use. For example, a = 1 for G 2 [21], and a = 2 for G 4 in [19]. The codeword Z is transmitted through the MIMO keyhole channel H whose output is a n r ×N matrix Y = HZ+ V, where V is the n r × N complex channel noise matrix. The elements of V are independent complex Gaussian variables with iid real and imaginary parts of variance N 0 /2. The receiver decodes channel output using a ML detector. It is assumed that the channel matrix remains constant for the duration of a codeword (quasi-static channel), and that the receiver has the knowledge of F and H to be used in ML decoding. While H is not known to the transmitter, the antenna correlation matrices R t and R r are assumed to be known.

MGF OF OUTPUT SNR AND SER EXPRESSIONS
We use the moment generating function-(MGF) based approach [23] to find the average output SER of the abovedescribed system, assuming that the precoder matrix F is given. Our main goal in this section is to find an easy to evaluate expression for average SER. To this end, we first find an expression for the MGF of the output signal to noise ratio (SNR). When the space-time block code is orthogonal, the MIMO system is equivalent to a SISO system, in which the SNR at the input of the ML detector is given by [10]: where where is an n t × n t Hermitian positive semidefinite matrix. Using eigenvalue decomposition, let where U q is the unitary matrix of eigen vectors and Λ q is the diagonal matrix of whose diagonal elements are eigenvalues λ Then, (5) can be expressed as where h tw = h tw U q is also an iid complex Gaussian vector of size 1 × n t . Hence, it follows that where { h tw } i denotes the ith element of h tw . In a similar manner, where λ (r) j , j = 0, . . . , n r − 1 are the eigenvalues of the matrix R r and h rw is an iid complex Gaussian vector of size n r × 1. Note that if the channel is uncorrelated and F = I nt , then y t and y r are chi-square distributed random variables with 2n t and 2n r degrees of freedom, respectively. This special case was considered in [10]. On the other hand, in general, the MGF of y t is given by [24, equation (14)]: A similar expression exists for φ yr (s). In this paper, we mainly focus on the case in which the eigenvalues of both Q and 4 EURASIP Journal on Wireless Communications and Networking R r are distinct. However, the following derivation may be generalized to the case of eigenvalues with multiplicity, by using the appropriate partial fraction expansion, as discussed at the end of this section. For the case of distinct eigenvalues, the partial fraction expansion yields where Let the corresponding set of coefficients for φ yr (s) be β j , j = 0, . . . , n r − 1. Hence, the pdf of y r can be found by inverse Laplace transform as The pdf of Y = y t y r (where y t and y r are independent) is given by [25, page 141] Then, it follows that the MGF of Y is where This integral is readily available as a built-in function in software packages such as Matlab and Mathematica. Given this result, the MGF of the output SNR, γ = ηY can be obtained as φ γ (γ) = φ Y (ηs). The average SER, P S , of several modulation schemes over the correlated keyhole channel can now be derived using the MGF of the random variable γ [23]. For example, the conditional SER of M-PSK given the instantaneous SNR γ is [23, (8.23)] where g MPSK = sin 2 (π/M). For convenience, define Then, the average SER of M-PSK signaling can be expressed as where ξ 1 = ηg MPSK . Similar expression can be obtained for M-PAM and M-QAM using [23, (8.5) and (8.12)] which are summarized below: where

Note
When Q and/or R r have repeated eigenvalues, the partial fraction expansions of φ t (s) and φ r (s) have to be appropriately modified. We now outline the solution for this case. Suppose Q has n 1 distinct eigenvalues with λ (q) i repeating l qi times, and R r has n 2 distinct eigenvalues with λ (r) j repeating l r j times. For this general case, the same procedure as above can be used to show that the MGF of Y is given by (see where ψ(m, m − k + 1; z) is the confluent hypergeometric function [26, (9.211-4)], Γ(·) is the gamma function, and α im , β jk are partial fraction coefficients (procedures for finding which are well known). This expression can now be used to generalize the SER expressions (21)- (23). It can be shown that when l qi = 1 for all i (n 1 = n t ) and l r j = 1 for all j (n 2 = n r ), (24) reduces to (16). Other interesting cases are when either transmit antennas or receive antennas are uncorrelated, in which case either R t or R r is the identity matrix. Also, when both R t and R r are identity matrices (no channel correlation), then it is easy to show that (24) reduces to [10, (6)].

PRECODER OPTIMIZATION
Given R t and R r , our goal is to find the optimal precoder matrix F opt which minimizes the average SER P S (F), subject to a constraint P max on the average transmitted power P in (1). This constrained optimization problem can be solved by minimizing the Lagrangian [27]: P. Yahampath and A. Hjørungnes 5 where μ > 0 is the Lagrange multiplier and P max = P max /(aKP x ). It is generally difficult to find directly the minimum of this function, by setting the partial derivatives to zero. Hence, we resort to gradient descent minimization to find locally optimal values for F and μ, starting from an initial value F 0 [27]. In general, the solution of a constrained minimization problem such as (25) by gradient descent requires gradient projection, [27,Chapter 11]. However, the specific nature of the constraint (1) allows a simpler approach. To this end, let D F * P S (F) be the 1 × n t B vector of partial derivatives of P s (F) with respect to the elements of vec(F * ), where D F * denotes the matrix derivative operator [28,Definition 2]. We can perform gradient decent according to where τ > 0 and μ i is chosen so that the updated solution satisfies the power constraint, by solving By (27) can be written as This equation yields two positive values for μ i for which the updated precoder in (26) satisfies the power constraint, provided that the step size τ is chosen sufficiently small (see Appendix B). The choice of the smaller value for μ i out of the two possible solutions results in the proper convergence of the gradient descent algorithm (see Figure 8). In each iteration i, we first find μ i by solving (28) and then F i+1 using (26). The convergence of the solution can be decided when F i+1 − F i F becomes less than a prescribed threshold. In the following, we derive closed-form expressions for D F * P S (F) for different modulation schemes.
While the SER expressions derived in the previous section are simple to evaluate for a given F, the elements of the precoder matrix appear only indirectly (in the eigenvalues of Q) in these expressions. Consequently, they are not easily differentiable with respect to the precoder matrix. However, a slight reformulation of the given expressions readily eliminates this difficulty as follows. First, we express (14) as φ Y (s) = E yr {φ yt (sy r )}. Then, from (18) it follows that where ξ 1 = ηg MPSK . Now using [24, equation (14)], (9) can be rewritten as Hence, where G(θ, x, ξ) I nt + (ξx/sin 2 θ)Q and we have obtained SER of M-PSK as a function Q (and hence as an explicit function of F). However, note that the evaluation of this expression requires the explicit evaluation of a double integral, which is avoided in (21) by using the exponential integral. Similar expressions can be derived for M-PAM and M-QAM using [23, (8.5) and (8.12)] and the same procedure as above, which are summarized below: It can now be seen that computing D F * P S (F) for (31)-(33) simply involves computing the n t B × 1 derivative vector of the form (see Appendix C): where R t = (R 1/2 t ) * R 1/2 t . Then, by defining D F * P S (F) for M-PSK, M-PAM, and M-QAM can expressed, respectively, as

NUMERICAL RESULTS AND DISCUSSION
In this section, we present numerical results to verify the analytical SER expressions derived in Section 3 and to demonstrate the benefit of the proposed MSER precoders.
In order to confirm the analytical SER expressions derived in Section 3, we compare them with SER estimated by simulating the underlying MIMO system. While the expressions we have derived are valid for a general channel correlation matrix, we here present experimental results obtained for the case of exponential correlation model [29] widely used in the literature. In this case, the transmit and receive The results in Figure 2 are for a 2 × 2 MIMO system based on the OSTBC G 2 from [19] (Alamouti code) for which a = 1. In this case, B = n t = n r = 2, K = N = 2, that is, rate is one. Figure 3 shows the results for a 4 × 2 MIMO system based on the OSTBC C 4 from [20] for which a = 1 and B = N = n t = 4, n r = 2, K = 3, that is, rate is 3/4. Note that no precoding has been used in these cases (F set to a scaled identity matrix). Since spatial correlation in the channel degrades the SER performance, it is also of interest to compare the SER of an OSTBC over a correlated keyhole channel with that over a keyhole channel with no spatial correlation. Such a comparison for the 2 × 2 MIMO system is shown in Figure 4 which shows that the degradation of performance due to spatial correlation in the given keyhole channel can be quite significant at high SNR (exponential antenna correlation model is used). Also included here is the SER over an6 Rayleigh fading channel (no keyhole effect and no antenna correlation). In particular, these curves clearly highlight the loss of diversity (as indicated by the slope of the curves [1]) in the MIMO system due to the combined effect of the keyhole propagation and the antenna correlation.
Next, we investigate the performance achievable with MSER precoders found using the proposed design algorithm. Figure 5 shows the performance of the above-described 4 × 2  MIMO system with a linear precoder as function of SNR (only 8-PSK and 16-QAM performance is shown, but similar improvements were also observed for 8-PAM). As before, the results in Figure 5 are obtained for the case of exponential correlation at both transmit and receive antennas so that both R t and R r have real elements. Since our derivation P. Yahampath and A. Hjørungnes  Figure 5: Performance improvements achieved with optimal precoding in a 4 × 2 MIMO system based on OSTBC C 4 from [20] and the exponential channel correlation model.
is valid for a complex Hermitian and positive semidefinite correlation matrix, we also carried out precoder designs for a 4 × 4 MIMO system based on G 4 from [21], in which the transmit antennas have the exponential correlation matrix while the receive antennas have the complex correlation matrix [22]: where A = 0.3773 + j0.5411, B = 0.0673 − j0.8081, and C = −0.6821 + j0.6512.
The relevant results are presented in Figure 6. The results in both Figures 5 and 6 clearly demonstrate the effectiveness of the MSER precoder designed by the proposed algorithm. The effect of the precoder is to shift the SER versus SNR curve downwards, without changing its slope at high SNR. Thus, the precoder does not change the diversity order of the system. (The slope of SER versus SNR curve at high SNR determines the diversity order of the system. This asymptotic slope depends on the rank of the channel correlation matrix.) The fact that precoding is less useful at high SNRs is generally known [13].In our particular case, it can be shown that in the limit E s /N 0 →∞, the MSER precoder is proportional to the identity matrix (see Appendix D), that is,  Figure 6: Performance improvements achieved with optimal precoding in a 4 × 4 MIMO system based on OSTBC G 4 from [21]. The transmit antenna correlation follows the exponential model while the receive antenna correlation is modeled by a general complex correlation matrix from [22].
However, at low to moderate channel SNR, the optimal precoder is seen to improve the system performance, the improvement being dependent on the degree of spatial correlation in the channel coefficients. It was also observed that at low SNR, the precoder matrix obtained by the design algorithm had nearly identical elements in all the cases we considered. If the MSER precoder matrix has equal elements, it can be shown that there exists an equivalent solution (which is also optimal in the MSER sense) of the form (see Appendix E): where u is an n t × 1 vector and 0 is a n t × 1 vector with all elements equal to zero. Then, from the discussion in [13, Section IV-A], it follows that the resulting precoding scheme is also equivalent to beamforming in the direction of u. Note that if the MSER precoder matrix is known to have equal elements, it can be directly determined by using the power constraint.
Note that the result in Appendix E shows the existence of multiple local minima. Since the given descent algorithm may converge to a local minimum, one can choose the best design among multiple designs obtained with random initializations for the precoder. In our experiments, the best solution obtained in this manner was comparable to the one obtained with the diagonal initialization which is the scaled (to satisfy the power constraint) identity matrix. It is also possible to use a relaxation-type algorithm in which the SNR is reduced from a higher value (for which the diagonal  Figure 8: A graphical interpretation of the solutions of (28). The gradient vector D F * P s (F i ) + μ i vec(F i ) depends on μ i . Provided that the step-size τ is small enough, (28) yields two possible values for μ i for which the updated precoder F i+1 satisfies the power constraint. The two possible solutions for the updated precoder are given by OA and OA . The use of smaller μ i yields the desired updated precoder OA for which the gradient descent converges.
precoder is nearly optimal) to the desired value in steps, and to progressively optimize the precoder to each SNR value. In all our designs, the gradient descent minimization algorithm converged rapidly to a stable solution. A typical example is shown in Figure 7 which shows the convergence of both the precoder matrix and the Lagrange multiplier. Thus, this algorithm can be used to adaptively update the precoder matrix based on, for example, the estimates of the channel correlation matrix obtained via measurements. Typically, the channel correlation matrix changes much slower than the channel matrix itself and hence can be estimated periodically at the receiver and be fed back to the transmitter for adapting the precoder matrix.

CONCLUDING REMARKS
Exact analytical expressions were derived for SER of M-PSK, M-PAM, and M-QAM modulated OSTBCs over a MIMO spatially correlated keyhole channel. A general complex correlation matrix has been assumed in the derivations. These expressions are easy to compute using numerical software and have been verified by Monte Carlo simulations. The given analytical expressions have been used to design MSER linear precoders based on the knowledge of channel correlation matrix available as CSI at the transmitter. Using simulation experiments, it has been demonstrated that the proposed MSER precoder can significantly reduce the error probability of a MIMO system operating on a fading channel degraded by a keyhole effect.
An important extension to this work includes precoder design for more general multiple scattering fading channels [5]. In particular, an interesting case is the double scattering model [4] (of which the keyhole channel is a special case).

B. SOLUTION OF (28)
Clearly, all coefficients of (28) are real. A simplified graphical interpretation of the solution to this equation is shown in Figure 8. Note that f i is the updated solution for the precoder matrix, without considering the power constraint, that is, when μ i = 0. This solution satisfies the power constraint as an inequality if Tr(f i f H i ) < P max . In this case, the rescaling of vec(F i ) = f i to satisfy the power constraint will result in a solution with a lower SER. On the other hand, if Tr(f i f H i ) − P max > 0, we have to choose μ i so that vec(F i ) satisfies the power constraint. Thus, we consider this case.
It is easy to show that . It, then, follows that (28) has real positive roots only if Otherwise the updated solution cannot satisfy the power constraint. This scenario is also evident from Figure 8. Thus, the step-size τ for gradient descent must be chosen sufficiently small, in order to ensure the convergence (however, an overly small step-size results in slow speed of convergence). In Figure 8, OA and OA are the two candidate solutions for the updated precoder F i+1 , corresponding to the roots of the quadratic (28). The desired solution is OA (corresponding to the smaller μ i ), which ensures the convergence of the gradient descent algorithm. This can be seen form the fact that OA , which corresponds to the larger μ i , will be opposite to F i+1 closer to the convergence point, where F i+1 ≈ F i .

C. PROOF OF (34)
Consider the matrix derivative of the form: , ( C . 1 ) where G = I nt + κQ (κ is a constant) and Q = R 1/2 t FF H R 1/2 t . Note that the derivative vector on the right-hand side has size n t B × 1, see [28, Table III]. Now, using [28,  where we use the fact that Tr(AB) = vec T (A T )vec(B). Also, d vec(G) = κd vec(Q) and from [28,  where K nt,B is the commutation matrix of size n t B × n t B [30]. For n t × B matrix F, the commutation matrix satisfies K nt,B vec(F) = vec(F T ). Let R t (R 1/2 t ) * R 1/2 t . Then, using [28, (4)] , we obtain c(F) where we use the fact that K nt,nt (F ⊗ R t ) = ( R t ⊗ F)K nt,B [30,Theorem 3.1]. From this, the result in (34) follows.

D. PROOF OF (38)
Consider SER expression (31)   and hence L(F , μ) = L(F, μ), that is, the objective function (25) is unchanged by a unitary transformation of F. Next, consider the singular value decomposition (SVD) of F = UΘV H , where U and V are unitary matrices and Θ is a diagonal matrix of singular values of F. If F has identical elements, then it has rank one and hence only a single nonzero singular value θ 1 , that is, Θ = diag(θ 1 , 0, . . . , 0). Therefore, it can be seen that F = FV has the form F = (u 0 · · · 0), where u is a n t × 1 vector and 0 is the n t × 1 matrix of zero elements.