EURASIP Journal on Applied Signal Processing 2005:6, 820–827 c ○ 2005 Hindawi Publishing Corporation Improved Max-Log-MAP Turbo Decoding by Maximization of Mutual Information Transfer

The demand for low-cost and low-power decoder chips has resulted in renewed interest in low-complexity decoding algorithms. In this paper, a novel theoretical framework for improving the performance of turbo decoding schemes that use the max-log-MAP algorithm is proposed. This framework is based on the concept of maximizing the transfer of mutual information between the component decoders. The improvements in performance can be achieved by using optimized iteration-dependent correction weights to scale the a priori information at the input of each component decoder. A method for the o ﬄ ine computation of the correction weights is derived. It is shown that a performance which approaches that of a turbo decoder using the optimum MAP algorithm can be achieved, while maintaining the advantages of low complexity and insensitivity to input scaling inherent in the max-log-MAP algorithm. The resulting improvements in convergence of the turbo decoding process and the expedited transfer of mutual information between the component decoders are illustrated via extrinsic information transfer (EXIT) charts.


INTRODUCTION
Since the discovery of turbo codes [1], there has been renewed interest in the field of coding theory, with the aim of approaching the Shannon limit. Furthermore, with the proliferation of wireless mobile devices in recent years, the availability of low-cost and low-power decoder chips is of paramount importance. To this end, several techniques for reducing the complexity of the optimum MAP decoding algorithm [2] have been proposed. Examples include the log-MAP, max-log-MAP, and SOVA algorithms [3,4,5]. In the case of the latter two algorithms, the reduction in complexity is accompanied by some degradation in error correction performance. This issue has been addressed by a number of authors in the context of turbo decoding schemes.
In [6], the performance degradation caused by the SOVA algorithm is attributed to an incorrect scaling of the extrinsic information, in addition, to nonzero correlation between the intrinsic and extrinsic information at the component decoder outputs. Performance improvements were demonstrated through the use of correction factors computed as a function of soft-output statistics of the component decoders.
The degradation caused by the max-log-MAP algorithm was addressed in [7,8]. Performance gains were achieved by scaling of the extrinsic information at the component decoder outputs. The value of the scaling factor was derived empirically and is iteration independent.
In this paper, a novel theoretical framework for improving the performance of turbo decoding schemes that use the max-log-MAP algorithm is proposed. The convergence behaviour of turbo decoding schemes has been recently quantified by using extrinsic information transfer (EXIT) charts [9]. An EXIT chart essentially illustrates the transfer of mutual information between the component decoders as a function of the encoder polynomials and the signal-to-noise ratio. It has been shown that the turbo decoding performance is strongly linked to an increase in mutual information at each decoding step. This suggests that the optimum strategy for mitigating the degradations resulting from any suboptimal decoding algorithm should maximize the mutual information at the outputs of the component decoders. It is shown here that effective maximization of mutual information can be achieved for the max-log-MAP algorithm through scaling of a priori information by iteration-specific correction weights. Such scaling essentially corrects the bias in the a priori information that results from the max-log approximation in the previous component decoder.
The offline calculation of the correction weights is developed in Section 4. Sections 2 and 3 provide the necessary background, and Section 5 presents simulation results demonstrating the performance gains achieved by the proposed technique.
It is shown that the performance of a turbo decoder using the max-log-MAP algorithm with the proposed correction scheme approaches that of a turbo decoder using the optimum log-MAP or MAP algorithms. This is achieved at the expense of only two additional multiplications per systematic bit per turbo iteration. Furthermore, the insensitivity of the max-log-MAP algorithm to an arbitrary scaling of its input log-likelihood ratios is maintained.

TURBO DECODING
Consider the received signal, r t = x t + n t , at the output of an AWGN channel at time instant t, where x t ∈ {+1, −1} is the transmitted binary symbol (corresponding to the encoded bit b t ∈ {1, 0}) and n t is zero-mean Gaussian noise of variance E{n 2 t } = N 0 . Then the log-likelihood ratio (LLR) of the transmitted symbol is defined as where P{A} represents the probability of event A. We also consider, without loss of generality, a parallel concatenated turbo encoding process of rate 1/3 at the transmitter. This consists of two 1/2 rate recursive systematic convolutional (RSC) encoders separated by an interleaving process, resulting in transmitted systematic symbol x t,0 and parity symbols x t,1 and x t,2 . The corresponding signals at the output of the channel (input of the decoder) may then be expressed as L c (x t,0 ), L c (x t,1 ), and L c (x t,2 ). Figure 1 depicts the turbo decoding procedure whereby decoding is performed in an iterative manner via two softoutput component decoders, separated by an interleaver, with the objective of improving the estimates of x t,0 from iteration i to iteration i + 1. The first decoder generates extrinsic information L (i) e (x t,0 ) on the systematic bits, which then serves as a priori informationL (i) a (x t,0 ) for the second decoding process. The symbol "∼" denotes interleaved quantities.
The maximum a posteriori probability (MAP) algorithm is the optimum strategy for the decoding of RSC codes, as it results in a minimum probability of bit error. However, due to its high computational complexity, the MAP algorithm is usually implemented in the logarithmic domain in the form of the log-MAP or max-log-MAP algorithms. While the former is mathematically equivalent to the MAP algorithm, the latter involves an approximation which results in even lower complexity, albeit at the expense of some degradation in performance [3,4,5]. For purposes of brevity, the expressions presented in this paper are written for the first component decoder, with obvious extensions to the second component decoder.

Log-MAP algorithm
The log-MAP algorithm is the log-domain implementation of the MAP algorithm and operates directly on LLRs. Given the LLRs for the systematic and parity bits as well as a priori LLRs for the systematic bits, the log-MAP algorithm computes new LLRs for the systematic bits as described below: [1] t (l , l) +β t (l) whereγ [q] t (l , l) is the logarithm of the probability of a transition from state l to state l of the encoder trellis at time instant t, given that the systematic bit takes on value q ∈ {0, 1} and M is the total number of states in the trellis. Note that the new information at the decoder output regarding the systematic bits is encapsulated in the extrinsic information term L e (x t,0 ). Coefficientsᾱ t (l) andβ t (l) are forward-and backward-accumulated metrics at time t. For a data block of τ systematic bits (x 1,0 · · · x τ,0 ) and the corresponding parity bits (x 1,1 · · · x τ,1 ), these coefficients are calculated as follows.

Max-log-MAP algorithm
The complexity of the log-MAP algorithm can be further reduced by using the max-log approximation log(e δ1 + e δ2 ) ≈ max(δ 1 , δ 2 ) for evaluating (2). Clearly, this results in biased soft outputs and degrades the performance of the decoder. Nevertheless, the max-log-MAP algorithm is often the preferred choice for implementing a MAP decoder since it has the added advantage that its operation is insensitive to a scaling of the input LLRs. Using the max-log-MAP algorithm, the LLRs for the systematic bits can be calculated as The application of the max-log approximation implies that if the inputs of the decoder are scaled by a certain factor, thenᾱ t (l),β t (l), andγ [q] t (l , l), and consequently the output L(x t,0 ), are all equally scaled by the same factor. In other words, the decoding process becomes linear, and as a result, knowledge of the channel noise variance N 0 is not required for correct scaling of the decoder inputs. This is in contrast to the case of the log-MAP algorithm, where the decoder output is a nonlinear function of its input, and therefore a reliable estimate of N 0 is essential for the computation of LLRs at the decoder inputs.

EXIT CHARTS
The performance and convergence behaviour of turbo codes can be analysed using extrinsic information transfer (EXIT) charts, as proposed in [9]. The idea is to visualize the evolution of the mutual information exchanged between the component decoders from iteration to iteration. EXIT charts operate under the following assumptions. (a) The a priori information is fairly uncorrelated from channel observations. This is valid for large interleaver sizes. (b) The extrinsic information L e (x t,0 ) has a Gaussian-like distribution, as shown in [10] for the MAP decoder.
An EXIT chart consists of a pair of curves which represent the mutual information transfer functions of the component decoders in the turbo process. Each curve is essentially a plot of a priori mutual information I a against extrinsic mutual information I e for the component decoder of interest. Here, the mutual information is a measure of the degree of dependency between the log-likelihood variables L a (x t,0 ) or L e (x t,0 ) and the corresponding transmitted bit x t,0 . The mutual information takes on values between 0 for no knowledge and 1 for perfect knowledge of the transmitted bits, dependent on the reliability of the likelihood variables. The terms I a and I e are related to the probability density functions (pdfs) of L a (x t,0 ) and L e (x t,0 ), the signal-to-noise ratio E b /N 0 , and the RSC encoder polynomials. If the component decoders are identical, the two curves are naturally mirror images. The required pdfs can be estimated by generating histograms p(L a ) and p(L e ) of L a (x t,0 ) and L e (x t,0 ), respectively, for a particular value of E b /N 0 where E b denotes the energy per information bit. This can be achieved by applying a priori information modelled as L a (x t,0 ) = µ a x t,0 + n a,t , t = 1, . . . , τ, to the input of a component decoder and observing the output L e (x t,0 ) for a coded data block corresponding to τ information bits. The random variable n a,t is zero-mean Gaussian with variance E{n 2 a,t } = σ 2 a such that σ 2 a = 2µ a . The latter is a requirement for L a (x t,0 ) to be an LLR. The mutual information I a may then be computed as where p a = p(L a |x t,0 = −1) + p(L a |x t,0 = +1). Similarly, I e can be computed as where p e = p(L e |x t,0 = −1) + p(L e |x t,0 = +1). The resulting pair (I a , I e ) defines one point on the transfer function curve. Different points (for the same E b /N 0 ) can be obtained by varying the value of σ 2 a . Having derived the transfer functions, we may now observe the trajectory of mutual information at various iterations of an actual turbo decoding process. At each iteration, mutual information is again computed as in (10) and (11), however the a priori LLR, L a (x t,0 ), at the input of the component decoder is no longer a modelled random variable but corresponds to the actual extrinsic LLR generated by the previous component decoding operation. Figures 2 and 3 illustrate EXIT charts with trajectories of mutual information for the log-MAP and max-log-MAP algorithms, respectively. The "snapshot" trajectories correspond to turbo decoding iterations for a specific coded data block. The 1/2 rate (punctured) turbo encoder consists of two component RSC encoders, each operating at 1/2 rate with a memory of 4 and octal generator polynomials  (G r , G) = (1 + D + D 4 , 1 + D + D 2 + D 3 + D 4 ), where G r denotes the recursive feedback polynomial [9,11]. Note that while the mutual information trajectory for the log-MAP algorithm in Figure 2 fits the predicted transfer function, the trajectory in Figure 3 clearly indicates the impact of numerical errors resulting from the max-log approximation: the trajectory stalls after only the first iteration and the turbo decoder is unable to converge at the simulated E b /N 0 of 1 dB.

MAXIMUM MUTUAL INFORMATION COMBINING (MMIC)
The poor convergence of the turbo decoder using the maxlog-MAP algorithm is due to the accumulating bias in the extrinsic information caused by the max(·) operations. Since extrinsic information is used as a priori information, L a (x t,0 ), for the next component decoding operation and is combined with channel observations L c (x t,0 ), as shown in (4), this bias leads to suboptimal combining proportions in the decoder. To correct this phenomenon, the logarithmic transition probabilities at the ith iteration may be modified as follows: In other words, the bias of the a priori information can be corrected by scaling it by a factor w (i) a at the ith iteration, as depicted in Figure 4. This correction procedure for the maxlog-MAP algorithm is far less complex than the correction function employed in the log-MAP algorithm. Furthermore, and perhaps more importantly from a practical point of view, the corrected max-log-MAP algorithm remains insensitive to    an arbitrary scaling of the LLR values at its input, thereby eliminating the need to estimate the noise variance at the channel output. From observations of the EXIT charts in the previous section, it is evident that rapid convergence of the turbo process relies on the effective exchange of mutual information between the component decoders. It may be inferred that the optimum value for the weight factor w (i) a is that which maximizes the mutual information between the term ζ (i) c (x t,0 ) and the vector of "uncorrupted" LLRs λ (i) t for each component decoder and at each iteration i. Using vector notation, ζ (i) t may be modelled as where ε (i) t represents the contributions of channel noise plus the numerical approximation error inherent in the max-log-MAP algorithm. Given variances and modelling v (i) t as a Gaussian random variable, the differential and conditional entropies of By definition [12], the mutual information can be written as and the optimized weight factors can then be derived as Setting z = (R (i) ε ) T/2 w (i) , we arrive at the Rayleigh quotient problem [13] with solutions where eig max (A) is the eigenvector of A corresponding to its largest eigenvalue. The scalar k is chosen such that the second element of w (i) OPT , that is, the weight factor of L (i) c (x t,0 ), equals unity. Inspection of (13) to (22) reveals that the weights are functions of the iteration index, the error correcting capabilities of the component decoders (i.e., encoder polynomials), and the signal-to-noise ratio. The optimized weights w (i) OPT can be computed or "trained" offline based on time-averaged estimates of correlation matrices R (i) λ+ε and R (i) ε derived over a sufficiently long data block corresponding to τ encoded information bits. Specifically, assuming ergodicity, Furthermore, the vector λ (i) t of "uncorrupted" LLRs may be written as so that where Finally, assuming that vectors ε (i) t and λ (i) t are uncorrelated, one may derive R (i) ε as R (i) λ+ε − R (i) λ . The above training procedure should be performed under E b /N 0 conditions that are typical at the bit error rate range of interest.

SIMULATION RESULTS
Two different turbo encoders are considered at the input of an AWGN channel. The first 1/2 rate (punctured) turbo encoder consists of two 1/2 rate component RSC codes of memory 4, with polynomials (G r , G) = (1+D+D 4 , 1+D+D 2 +D 3 + D 4 ) and an interleaver size of 10 5 bits. The second 1/2 rate (punctured) turbo encoder is that specified for UMTS [14] and consists of two 1/2 rate component RSC codes of memory 3, with polynomials (G r , G) = (1 + D 2 + D 3 , 1 + D + D 3 ). Here, interleaver sizes of 5114 bits and 1000 bits are investigated. The former is the maximum block size specified for high-speed downlink packet access (HSDPA) in UMTS. Table 1 shows the optimized weight factors derived offline for each iteration of the two turbo decoders at E b /N 0 of 1.0 and   0.7 dB, respectively. The impact of the combining scheme of (12) on the mutual information trajectory of the first turbo decoder is indicated in Figure 5. In comparison to the original trajectory of Figure 3, turbo decoding with MMIC and the max-log-MAP algorithm does not stall and is able to converge almost as well as turbo decoding with the log-MAP algorithm.
This is achieved at the expense of only two additional multiplications per iteration per systematic bit. Figure 6 shows the BER performance of the first turbo decoder after 6   iterations with an interleaver size of 10 5 bits. The results show that the proposed MMIC scheme significantly improves the performance of the turbo decoder. Figures 7 and 8 show the BER results for the UMTS turbo decoder after 6 iterations with different interleaver sizes. Again, the performance of the turbo decoder using the max-log-MAP algorithm and MMIC approaches that of the turbo decoder using the optimum log-MAP algorithm. The performance difference can be reduced down to only 0.05 dB at a BER of 10 −4 .

CONCLUSIONS
The theoretical framework for a maximum mutual information combining (MMIC) scheme was proposed as a means to improve the performance of turbo decoders whose component decoders use the max-log-MAP algorithm. The convergence behaviour of such turbo decoders was investigated by using extrinsic information transfer (EXIT) charts. The combining scheme is achieved by iteration-specific scaling of the a priori information at the input of each component decoder in order to maximize the transfer of mutual information to the next component decoder, as suggested by the EXIT charts. The scaling corrects the accumulated bias introduced by the max-log approximation. A method for offline computation of the scaling factors was also described. It was shown that the proposed combining scheme significantly improves the performance of a turbo decoder using the max-log-MAP algorithm to within 0.05 dB of a turbo decoder using the optimum log-MAP or MAP algorithms. The improved decoder retains the low complexity and insensitivity to input scaling which are inherent advantages of the max-log-MAP algorithm.
Bernard Mulgrew received his B.S. degree in 1979 from Queens University, Belfast. After graduation, he worked for 4 years as a Development Engineer in the Radar Systems Department, BAE Systems (formerly Ferranti), Edinburgh. From 1983-1986, he was a Research Associate in the Department of Electronics & Electrical Engineering, the University of Edinburgh, studying the performance and design of adaptive filter algorithms. He received his Ph.D. and was appointed to a lectureship in 1987. He currently holds the BAE Systems/Royal Academy of Engineering Research Chair in multisensor signal processing and is the Head of the Institute for Digital Communications Edinburgh. His research interests are in adaptive signal processing and estimation theory and in their application to communications, radar, and audio systems. He is a coauthor of three books on signal processing and over 50 journal papers. He is a Fellow of the IEE, a Fellow of the Royal Society of Edinburgh, a Member of the IEEE, and a Member of the Audio Engineering Society.