EURASIP Journal on Applied Signal Processing 2005:6, 928–941 c ○ 2005 M. Adrat and P. Vary Iterative Source-Channel Decoding: Improved System Design Using EXIT Charts

The error robustness of digital communication systems using source and channel coding can be improved by iterative source-channel decoding (ISCD). The turbo-like evaluation of natural residual source redundancy and of artificial channel coding redundancy makes step-wise quality gains possible by several iterations. The maximum number of profitable iterations is predictable by an EXIT chart analysis. In this contribution, we exploit the EXIT chart representation to improve the error correcting/concealing capabilities of ISCD schemes. We propose new design guidelines to select appropriate bit mappings and to design the channel coding component. A parametric source coding scheme with some residual redundancy is assumed. Applying both innovations, the new EXIT-optimized index assignment as well as the appropriately designed recursive nonsystematic convolutional (RNSC) code, allows to outperform known approaches to ISCD by far in the most relevant channel conditions.


INTRODUCTION
The design and development guidelines for today's digital communication systems are inspired by the information theoretic considerations of C. E. Shannon. His fundamental statements indicate that, in order to find the most error resistant realization of a communication system, the transmit, respectively, receive operations are in principle separable into source coding and channel coding. However, the achievement of the global optimum using this two-stage process is possibly subject to impractical computational complexity, to unlimited signal delay, and to stationary source signals. Taking realistic constraints of real-world communication systems into account, a separate treatment of source and channel coding usually inflicts a loss of optimality. Joint sourcechannel coding allows to narrow the gap to the global optimum.
The present contribution addresses a novel concept for joint source-channel coding. A new method is proposed to This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
improve the error robustness of existing or emerging digital mobile communication systems like GSM (global system for mobile communications) or UMTS (universal mobile telecommunications system), or the digital audio/video broadcasting systems (DAB/DVB). In these systems the source coding part extracts characteristic parameters from the original speech, audio, or video signal. Usually, these source codec parameters exhibit considerable natural residual redundancy such as a nonuniform parameter distribution or correlation. The utilization of this residual redundancy at the receiver helps to cope with transmission errors.
Besides several other concepts utilizing residual redundancy at the receiver to enhance the error robustness, two outstanding examples are known as source-controlled channel decoding (SCCD) [1,2,3,4] and as softbit source decoding (SBSD) [5]. On the one hand, SCCD exploits the natural residual redundancy during channel decoding for improved error correction. On the other hand, softbit source decoding performs error concealment. SBSD can reduce the annoying effect of residual bit errors remaining after channel decoding.
The error concealing capabilities of SBSD can be improved if artificial redundancy is added by channel coding. In practice, however, the optimal utilization of both, the artificial channel coding redundancy and the natural residual source redundancy, is not feasible due to the significantly increased complexity demands. Therefore, a lowcomplexity approximation has recently been proposed in terms of iterative source-channel decoding (ISCD) [3,6,7,8,9,10,11].
In an ISCD scheme a soft-input/soft-output (SISO) channel decoder and a (derivative of a) softbit source decoder are concatenated. The first decoder exploits the artificial redundancy which has explicitly been introduced by channel encoding, and the second one mainly utilizes the natural mutual dependencies of the source codec parameters due to their residual redundancy. The reliability gains due to both terms of redundancy are exchanged iteratively in a turbolike process [12,13,14]. In literature, the reliability gains are also referred to as extrinsic information. This information can usually be extracted from the soft-output values provided by any SISO decoder.
In order to evaluate the number of iterations allowing noteworthy improvements of error robustness a powerful analysis tool has recently been proposed in terms of extrinsic information transfer (EXIT) charts [15,16]. This method had already been applied to an ISCD scheme in [10,11]. However, the EXIT chart representation of ISCD schemes also reveals some new design and development guidelines which are the topic of this paper. This paper is organized as follows. In Section 2, we give a comprehensive review of iterative source-channel decoding (ISCD). Next, we define a new, clear classification of ISCD approaches into serially and parallel concatenated schemes. Afterwards, we apply the EXIT chart analysis to ISCD in Section 3. Based on this EXIT chart analysis we develop new design guidelines for ISCD schemes in Section 4, which provide higher error correcting/concealing capabilities. Finally, the improved error robustness of these schemes is demonstrated by simulation.

System overview-transmitter site
At time instant τ a source encoder extracts a set u τ of M scalar source codec parameters u µ,τ from a short segment of the original speech, audio, or video signal (see Figure 1). The index µ = 1, . . . , M denotes the position within the set u τ = (u 1,τ , . . . , u M,τ ). For instance, in GSM speech communication the set u τ comprises the coefficients of a linear filter describing the spectral envelope of a 20 millisecond segment of a speech signal as well as some parameters repre-senting the excitation of this filter. Each value u µ,τ , which is continuous in magnitude but discrete in time, is individually quantized by 2 Kµ reproduction levelsū (i) µ with i = 0, . . . , (2 Kµ − 1). The reproduction levels are invariant with respect to τ and the whole quantizer code-book is given by To each index i of a quantizer reproduction levelū (i) µ specified at time instant τ, a unique bit pattern x µ,τ of length K µ is assigned. The complete frame of M bit patterns x µ,τ specified at time instant τ is denoted as x τ = (x 1,τ , . . . , x M,τ ). A particular data bit of the bit pattern x µ,τ is addressed by an additional index κ written in parentheses, that is, x µ,τ (κ) with κ = 1, . . . , K µ . For convenience, in the following we assume that the code-books U µ ofū (i) µ are the same for all parameters u µ,τ in the set u τ , that is, U µ = U and K µ = K for all µ = 1, . . . , M.
A (source-related) bit interleaver Φ scrambles the set of data bits x τ tox τ using a deterministic mapping. In GSM, for example, the data bits are rearranged according to their individual importance with respect to the subjective speech quality. The reordering performs some kind of classification for unequal error protection. This helps to cope with annoying artifacts in the reconstructed speech signal if residual bit errors remain after channel decoding.
If there is no danger of confusion, the following notation will always refer to the deinterleaved domain. That means, even though bit interleaving changes the actual position of x µ,τ (κ) in the sequence of data bitsx τ , we keep the notation. The interleaver might be sized such that T + 1 consecutive sets x τ with τ = Λ − T, . . . , Λ are rearranged in common. To simplify notation, such time series of sequencesx τ are also denoted by the compact expressionx Λ Λ−T . The deterministic mapping Φ of the bit interleaver has to be designed in a way that (at the receiver site) the reliability gains resulting from softbit source decoding and from channel decoding can be considered as independent. Thus, the (source-related) bit interleaver plays a new key role in ISCD schemes (namely providing independent reliability gains) as compared to the original purpose of unequal error protection as in GSM.
As the reliability gain of source coding is due to the residual redundancy of source codec parameters u µ,τ , independence will be ensured if channel encoding is performed over bits x µ,τ (κ) of (more or less) mutually independent bit patterns x µ,τ , for example, across different positions µ. Such channel encoding may be realized either on a single-bit sequencex τ at time τ, or with respect to the interleaver Φ on multiplex τ with τ = Λ − T, . . . , Λ. Channel codes of code rate r expand the sequencesx Λ Λ−T of bit patterns to a sequence y Λ Λ−T of code bits y(ζ) with ζ = 1, . . . , (1/r) · (T + 1) · M · K. Note, if terminated convolutional codes of rate r and memory J are applied, there exist (1/r) · J additional code bits. If channel encoding of the systematic form is assumed, the individual data bits x µ,τ (κ) of x τ are present in the code sequence y Λ Λ−T . In real-world communication systems a second (channel-related) interleaver is placed after channel encoding to cope with burst errors on the transmission link. This kind of L(z µ,τ (κ)|x µ,τ (κ)) (if channel coding is of systematic form)
interleaver is assumed to be sized sufficiently large so that the equivalent transmission channel can be considered as memoryless and AWGN (additive white Gaussian noise).

Transmission model for binary phase shift keying
At the receiver, reliability information about the single data bits x µ,τ (κ) is generated from the possibly noisy received sequence z Λ Λ−T corresponding to y Λ Λ−T . In this respect, it is most convenient to express reliability information in terms of log-likelihood ratios or short L-values, for example, [13]. For instance, if the transmission channel is considered to be AWGN, the channel-related L-value is given by [13] L z(ζ) | y(ζ) = for all y(ζ). The term E s denotes the energy per transmitted BPSK-modulated (binary phase shift keying) code bit y(ζ) and N 0 /2 the double-sided power spectral density of the equivalent AWGN channel. The possibly noisy received value z(ζ) ∈ R denotes the real-valued counterpart to the originally transmitted BPSK-modulated code bit y(ζ) ∈ {−1, +1}. Time variant signal fading can easily be considered as well. For this purpose, a factor a has to be introduced on the right-hand side of (1). The specific probability distribution (e.g., Rayleigh or Rice distribution) of the random process a represents the characteristics of the signal fading. However, in the following we neglect signal fading, that is, a = 1 constantly.

Receiver model
The aim of the iterative source-channel decoding algorithm is to jointly exploit the channel-related L-values of (1), the artificial channel coding redundancy as well as the natural residual source redundancy. The combination yields a posteriori L-values L(x µ,τ (κ) | z Λ 1 ) for single data bits x µ,τ (κ) given the (entire history of) received sequences z τ with τ = 1, . . . , Λ (see Figure 2). This a posteriori L-value can be separated according to Bayes' theorem into four additive terms, if a memoryless transmission channel (and channel encoding of the systematic form, see below) is assumed: The first term in (2) represents the channel-related Lvalue of the specific data bit x µ,τ (κ) under test. Of course, this term is only available if channel encoding is of the systematic form. In this case, the data bit x µ,τ (κ) corresponds to a particular code bit y(ζ) and thus, the channel-related L-value L(z µ,τ (κ) | x µ,τ (κ)) is identical to one of the L-values determined in (1). Note, with respect to the correspondence of x µ,τ (κ) and y(ζ), we used two different notations for the same received value, that is, z µ,τ (κ) = z(ζ). If channel encoding is of the nonsystematic form, the term L(z µ,τ (κ) | x µ,τ (κ)) cannot be separated from L(x µ,τ (κ) | z τ 1 ). In this case it can be considered to be L(z µ,τ (κ) | x µ,τ (κ)) = 0 in (2) constantly. 1 The second term in (2) represents the a priori knowledge about bit x µ,τ (κ). Note, this a priori knowledge comprises natural residual source redundancy on bit-level. Both terms in the first line mark intrinsic information about x µ,τ (κ).
In contrast to these intrinsic terms, the two terms in the second line of (2) gain information about x µ,τ (κ) from received values other than z µ,τ (κ). These terms denote so-called extrinsic L-values which result from the evaluation of one of the two particular terms of redundancy. In the following, whenever the magnitude of these extrinsic L-values increases by the iterations we refer to this as reliability gain.

Soft-input/soft-output channel decoder
The SISO channel decoder (CD) in Figure 2 determines extrinsic information L [ext] CD (x µ,τ (κ)) mainly from the artificial redundancy which has explicitly been introduced by channel encoding. For this purpose, the SISO decoder combines the channel-related soft-input values L(z(ζ) | y(ζ)) for the code bits y(ζ) with a priori information L(x µ,τ (κ)) about the data bits x µ,τ (κ). The valid combinations are precisely described by the channel encoding rule. The a priori information L(x µ,τ (κ)) can be improved by additional a priori information which is provided by the other constituent decoder in terms of its extrinsic information L [ext] SBSD (x µ,τ (κ)) (feedback line in Figure 2). These L [ext] SBSD (x µ,τ (κ)) are usually initialized with zero in the first iteration step. As the determination rules of extrinsic L-values L [ext] CD (x µ,τ (κ)) of channel decoding are already well known, for example, in terms of the log-MAP algorithm [13,17], we refer the reader to literature.

Softbit source decoder
The second decoder in the ISCD scheme is a (derivative of a) softbit source decoder (SBSD) [5]. The softbit source decoder determines extrinsic information mainly from the natural residual source redundancy which typically remains in the bit patterns x µ,τ after source encoding. Such residual redundancy appears on parameter-level, for example, in terms of a nonuniform distribution P(x µ,τ ), in terms of correlation, or in any other possible mutual dependency in time 2 τ. The latter terms of residual redundancy are usually approximated by a first-order Markov chain, that is, by the conditional probability distribution P(x µ,τ | x µ,τ−1 ). These source statistics can usually be measured once in advance for a representative signal data base.
After several iterative refinements of L [ext] CD (x µ,τ (κ)) and L [ext] SBSD (x µ,τ (κ)) the bit-level a posteriori L-values of (2) are utilized for estimation of parametersû µ,τ . For this purpose, at first parameter-oriented a posteriori knowledge is determined and secondly combined with quantizer reproduction levels to provide the parameter estimatesû µ,τ . Parameteroriented a posteriori knowledge like P(x µ,τ | z Λ 1 ) can easily be measured either from the bit-wise a posteriori L-values of (2) or from the intermediate results of (B.5) (see Appendix B), for example, by The term C denotes a constant factor which ensures that the total probability theorem is fulfilled. Thus, if the minimum mean squared error (MMSE) serves as fidelity criterion, the 2 For convenience, we neglect any possibly available mutual dependency in position µ like cross-correlation of adjacent parameters u µ,τ and u µ−1,τ . However, it is straightforward to extend the following formulas such that mutual dependencies in position µ can be exploited by ISCD as well [11]. individual estimates are given by [5] If a delay is acceptable, that is, T + 1 > 1, (4) performs interpolation of source codec parameters due to the look-ahead of Λ − τ parameters. Otherwise, if T + 1 = 1 and Λ = τ, (4) performs parameter extrapolation.

Realization schemes
In connection with the turbo principle, typically two different realization schemes have to be regarded. If two constituent encoders operate on the same set of bit patterns (either directly on x τ or on the interleaved sequencex τ ), this kind of turbo scheme is commonly called a parallel code concatenation. A parallel code concatenation implies that at the receiver site, channel-related knowledge is available about all code bits of both decoders. In contrast to this, in a serially concatenated turbo scheme the inner encoder operates on the code words provided by the outer one. If the inner code is of the nonsystematic form, no channel-related information is available to the outer decoder.
In ISCD schemes, the constituent coders are the source and the channel encoder while the respective decoders are the channel decoder and the "utilization of source statistics" block (see Figure 2). With respect to the above considerations the amount of channel-related information which is available at both SISO decoders allows a classification into parallel, respectively, serially concatenated ISCD schemes.
(i) Parallel concatenated ISCD scheme. We define an ISCD scheme to be parallel-concatenated if channel-related information is available about all code bits to both constituent decoders. This is the case if channel encoding is of the systematic form.
(ii) Serially concatenated ISCD scheme. If channel encoding is of the nonsystematic form, channel-related knowledge is only available to the inner decoder. The outer decoding step, that is, the utilization of residual redundancy, strongly depends on the reliability information provided by the inner channel decoder.
From the above definition it follows that all formerly known approaches to ISCD, for example, [6,7,8,9,10], have to be classified as parallel concatenated as in these contributions channel codes of the systematic form are used. However, albeit our definition sometimes the denotation serial concatenation has been used as a source and a channel encoder are arranged in a cascade.

CONVERGENCE BEHAVIOR
In order to predict the convergence behavior of iterative processes, in [15,16] a so-called EXIT chart analysis has been proposed. By using the powerful EXIT chart analysis, the mutual information measure is applied to the input/output relations of the individual constituent SISO decoders. Figure 3 shows a generalization of the input/output out (x µ,τ (κ)) relations of decoders in case of a parallel ISCD scheme (compare to "channel decoder" and "utilization of source statistics" in Figure 2). On the one hand, the information exhibited by the overall a priori L-value, and on the other hand, the information comprised in the extrinsic L-values after soft-output decoding is closely related to the information content of the originally transmitted data bits x µ,τ (κ). For convenience, we define the simplified notations: (i) I [apri] quantifies the mutual information between the data bit x µ,τ (κ) and the overall a priori L-value In (x µ,τ (κ)), (ii) I [ext] denotes the mutual information between x µ,τ (κ) and the extrinsic information L [ext] Out (x µ,τ (κ)).
If needed, an additional subscript "CD", respectively, "SBSD" will be added to differentiate between channel decoding and softbit source decoding. The upper limit for both measures is constrained to the entropy H (X) (the data bit x µ,τ (κ) is considered to be a realization of the random process X). Note, the entropy H (X), respectively, the mutual information measures I [apri] , I [ext] depend on the bit position κ. To simplify matters, in the following we consider only the respective mean measures which are averaged over all bit positions κ = 1, . . . , K.

Extrinsic information transfer characteristics
The mutual information measure I [ext] at the output of the decoder depends on the input configuration. The channelrelated input value L(z µ,τ (κ) | x µ,τ (κ)), is mainly determined by the E s /N 0 value (compare to (1)). For the overall a priori input value L(x µ,τ (κ)) + L [ext] In (x µ,τ (κ)) it has been observed by simulation [15,16] that this input can be modeled by a Gaussian distributed random variable with variance σ 2 L = 4/σ 2 n (with σ 2 n = N 0 /2) and mean µ L = σ 2 L /2·x µ,τ (κ). As both terms depend on a single parameter σ 2 L , the a priori relation I [apri] can directly be evaluated for arbitrary σ 2 L by numerical integration. Thus, the EXIT characteristics T of SISO decoders are defined as [15,16] If specific settings for I [apri] , respectively, σ 2 L and for E s /N 0 are given, I [ext] is quantifiable by means of Monte-Carlo simulation. Note, in case of a serially concatenated ISCD scheme, the EXIT characteristic (5) of the outer SBSD becomes (more or less) independent of the E s /N 0 value because While the EXIT characteristics of various channel codes have already been extensively discussed, for example, in [15,16], in the following we extend our investigation here to the EXIT characteristics of SBSD [10,11]. Figure 4 depicts EXIT characteristics of SBSD if either the nonuniform distribution of the source codec parameters u µ,τ or additionally correlation is exploited. The u µ,τ are modeled by a first-order Gauss-Markov process with correlation ρ = 0.0 or ρ = 0.9 and quantized by a Lloyd-Max quantizer using K = 3 (Figures 4a and 4d), 4 (Figures 4b and 4e), or 5 bits/parameter (Figures 4c and 4f). As index assignment serves natural binary (Figures 4a-4c), respectively, an EXIToptimized mapping (Figures 4d-4f) as proposed in Section 4.

EXIT characteristics of softbit source decoding
Each subplot shows 16 simulation results for the case where SBSD is applied to a parallel-concatenated ISCD scheme. The lower subset of 8 EXIT characteristics is determined for an uncorrelated and nonuniformly distributed parameter, that is, ρ = 0.0. The upper subset of 8 EXIT characteristics results if in addition correlation is utilized, for example, ρ = 0.9. Due to correlation, more information about x µ,τ (κ) is available and thus, mutual information I If in a parallel-concatenated ISCD scheme the channel quality decreases utterly, then the channel-related L-values become negligibly small, that is, L(z µ,τ (κ) | x µ,τ (κ)) ≈ 0. This resembles a serially concatenated ISCD scheme where the outer softbit source decoder is (more or less) independent of the E s /N 0 value. Thus, the dashed curves in the different subplots are valid for both situations: for a very bad channel condition like E s /N 0 = −100 dB in case of a parallel ISCD scheme as well as for all channel conditions E s /N 0 in a serially concatenated scheme.
The simulation results depicted in all subplots reveal the same two apparent properties. Firstly, for a fixed but arbitrary parameter configuration, all curves merge in a single point if I SBSD = H(X) ≈ 1 bit even if the information at the a priori input can be considered as error free, that is, I [apri] SBSD ≈ 1 bit. Thus, perfect reconstruction of the data bit x µ,τ (κ) by solely studying the extrinsic output L-value (B.5) of SBSD is impossible.
Moreover, in case of the natural binary index assignment, it can be stated for all EXIT characteristics that the mutual information at the output increases approximately linear with the mutual information at the input. Thereby, the slope is usually rather flat. The EXIT characteristics for the EXIT-optimized bit mapping are discussed in more detail in Section 4.1.

Theoretical upper bound on I [ext]
SBSD For every configuration of index assignment, correlation ρ, quantizer code-book size 2 K , and look-ahead Λ−τ the maximum mutual information value I [ext] SBSD,max can also be quantified by means of analytical considerations [10,11]. Whenever the input relation I [apri] SBSD increases to H (X) (or the channel quality is higher than E s /N 0 ≈ 10 dB), the terms Θ(x [ext] µ,τ ), α τ−1 (x µ,τ−1 ), and β τ (x µ,τ ) of (B.5) (see Appendix B) are generally valued such that all summations in the numerator and denominator degenerate to single elements. In consequence, the theoretically attainable L [ext] SBSD (x µ,τ (κ)) are given for all possible combinations of After the discrete probability distribution of all attainable values L [ext] SBSD (x µ,τ (κ)) is quantified, the evaluation of mutual information between L [ext] SBSD (x µ,τ (κ)) and x µ,τ (κ) provides the upper bound for I [ext] SBSD,max (averaged over all κ = 1, . . . , K).  Table 1 summarizes the upper bounds for the example situations with K = 3, 4, 5 bits/parameter and some frequently used index assignments: natural binary, folded binary, and Gray-encoded bit mapping. To simplify matters, softbit source decoding is restricted to parameter extrapolation, that is, Λ − T = 1 and Λ = τ. Thus, the evaluation of L [ext] SBSD (x µ,τ (κ)) of (6) reduces to an evaluation of all combinations of x [ext] µ,τ , x µ,τ−1 . The theoretical upper bounds for natural binary confirm the corresponding simulation results of Figure 4. Compared to folded binary and Gray-encoded, the natural binary bit mapping provides higher I [ext] SBSD,max for all configurations of quantizer code-book size and correlation.
Recently an advanced bit mapping for ISCD has been proposed by Hagenauer and Görtz [18]. By considering simplified constraints like single-bit errors and by neglecting parameter correlation, the optimization is realized such that the best possible parameter signal-to-noise ratio (SNR) between the original codec parameter u µ,τ and its reconstructionû µ,τ is reached. If the theoretical upper bound I [ext] SBSD,max is evaluated for this SNR-optimized mapping, further substantial gains of I [ext] SBSD,max can be observed for most configurations (see Table 1).
The theoretical upper bounds I [ext] SBSD,max for the EXIToptimized bit mapping are discussed in more detail in Section 4.1.

EXIT chart of iterative source-channel decoding
The combination of the two EXIT characteristics of both soft-output decoders in a single diagram is referred to as EXIT chart [16]. The main contribution of EXIT charts is that an analysis of the convergence behavior of a concatenated scheme is realizable by solely studying the EXIT characteristics of the single components. Both EXIT characteris-tics are plotted into the EXIT chart considering swapped axes because the extrinsic output of the one constituent decoder serves as additional a priori input for the other one and vice versa (see Figure 2). Figure 5 shows an exemplary EXIT chart of a parallel approach to iterative source-channel decoding for a channel condition of E s /N 0 = −3 dB. The source codec parameters u µ,τ are assumed to exhibit correlation of ρ = 0.9. The parameters are quantized by a Lloyd-Max quantizer using K = 4 bits/parameter each, and natural binary serves for index assignment. Thus, the EXIT characteristic of SBSD is taken from Figure 4b. For channel encoding a rate r = 1/2, memory J = 3 recursive systematic convolutional (RSC) code with generator polynomial G = (1, (1 + D 2 + D 3 )/(1 + D + D 3 )) is used.
Usually, the best possible error correcting/concealing capabilities of an iterative source-channel decoding process are limited by an intersection of both EXIT characteristics [10].

DESIGN OF IMPROVED ISCD SCHEMES
The primary objective of iterative turbo-algorithms is to gain as much information from the refinements of extrinsic Lvalues L [ext] SBSD (x µ,τ (κ)) and L [ext] CD (x µ,τ (κ)) as possible. This goal implies that the intersection of the EXIT characteristics of the constituent decoders is located at the highest possible (I [ext] CD , I [ext] SBSD ) pair 3 in the EXIT chart. 3 In the two-dimensional (I SBSD )) 2 . In this sum, the term Ψ −1 (·) denotes the inverse function to Ψ(·) which is an approximation I [apri] = Ψ(σ 2 L ) for the numerical integration mentioned in Section 3.1 [16]. Thus, an ISCD scheme with improved error correcting/concealing capabilities might be given if the (I [ext] CD , SBSD ) pair is maximized. Next, this maximization will be realized in a two-stage process. Firstly, we propose a new concept on how to design an optimal index assignment. For this purpose the highest possible I [ext] SBSD,max value serves as optimality criterion. Secondly, we search for an appropriate channel coding component which ensures that the EXIT characteristic of CD crosses that one of SBSD at the highest possible I [ext] CD .

Optimization of the index assignment
In a first straightforward approach, the theoretical upper limit I [ext] SBSD,max has to be evaluated for all 2 K ! possible assignments of 2 K -bit patterns x µ,τ to the valid quantizer reproduction levelsū (i) µ with i = 0, . . . , (2 K − 1) of quantizer codebook U. That specific realization of all examined assignments which provides the maximum value for I [ext] SBSD,max marks the optimal mapping. Of course, such a full search (FS) is only manageable if the size of the quantizer code-book U is reasonably small, that is, K ≤ 3 bits/parameter. Otherwise, if K ≥ 4 bits/parameter, a full search is almost impossible because there exist 2 K ! ≥ 2 4 ! = 2.09E + 13 different assignments.
For the optimization of the index assignment with K ≥ 4 bits/parameters, we propose a low-complexity approximation which resembles 4 the binary switching algorithm (BSA) [19]. Starting from an initial index assignment (e.g., the natural binary mapping), that bit pattern which is assigned tō u (i) µ with i = 0 is exchanged on a trial basis with every other bit pattern for the indices j = 0, . . . , (2 K − 1) (including the unmodified arrangement i = j). From the 2 K possible arrangements, that combination is selected for further examination which provides the maximum I [ext] SBSD,max . Afterwards, this kind of binary switching is repeated for the other indices i = 1, . . . , (2 K − 1). Whenever a rearrangement provides a higher I [ext] SBSD,max value, the iterative search algorithm is restarted with i = 0, that is, the last-determined rearrangement serves as new initial index assignment. Usually, after several iterative refinements a steady-state is reached. The finally selected arrangement serves as EXIT-optimized index assignment. Some examples are listed in Table 2 in Appendix A.
The highest I [ext] SBSD,max values for the EXIT-optimized mappings are also listed in Table 1. Compared to the classical index assignments like natural binary, folded binary, and Gray-encoded, the extrinsic mutual information at the output of the softbit source decoder has significantly been increased by the optimization. Notice that the EXIT-optimized mapping found by the BSA approximation may only be considered as a local optimum. As shown for K = 3 bits/parameter, the global optimum obtained by the full search is usually more powerful.
In addition, the theoretical analysis also reveals substantial gains in I [ext] SBSD,max over the SNR-optimized mapping [18]. The key advantage over this approach is that correlation of the source codec parameters u µ,τ can easily be taken into account during the optimization process. As a consequence, the gap in I [ext] SBSD,max between the SNR-optimized mapping and the EXIT-optimized mapping increases with higher terms of correlation. The major drawback is that the instrumental quality measure parameter SNR is not explicitly included in the optimization. Moreover, the bounds I [ext] SBSD,max do not comprise information about the adverse effects of different mappings on instrumental quality measures like the parameter SNR. In certain situations, a higher parameter SNR might be available even if the I [ext] SBSD,max is smaller. Thus, it has to be confirmed by simulation if the EXIT-optimized bit mapping is able to provide a noteworthy gain in error robustness (see Section 5).

Optimization of the channel coding
component of ISCD So far, all known approaches to iterative source-channel decoding, for example, [6,7,8,9,10], consider channel codes of the systematic form and therefore, these ISCD schemes are concatenated in the parallel way. It is most common to use recursive systematic convolutional (RSC) codes of code rate r = 1/2. Due to the systematic form, one of the generator polynomials of the matrix G = (1, F(D)/H(D)) is fixed to 1, and due to the recursive structure, the second generator polynomial consists of a feed-forward part F(D) and a feedback part H(D). The term D denotes the one-tap delay operator and the maximum delay, that is, the maximum power J of D J in F(D), respectively, H(D), determines the constraint length J + 1 of the code. There exist 2 J+1 possible realizations for the feed-forward part F(D) and 2 J possible realizations for the feed-back part H(D). The number of possible realizations of H(D) is lower than that of F(D) because the present feed-back value is usually directly applied to the undelayed input value, that is, the term D 0 = 1 is always given in H(D). Thus, F(D) and H(D) offer (in maximum) 2 J+1 × 2 J combinatorial possibilities to design the EXIT characteristic of a rate r = 1/2, memory J RSC code. The effective number of reasonable combinations is even smaller, because in some cases F(D) and H(D) exhibit a common divisor and thus, the memory of the RSC encoder is not fully exploited. We expect improved error correcting/concealing capabilities from ISCD schemes if the RSC code is replaced by a recursive nonsystematic convolutional (RNSC) code. These ISCD schemes are serially concatenated. At the same code rate r and constraint length J + 1 such RNSC codes offer higher degrees of combinatorial freedom. As the matrix G(D) = (F 1 (D)/H(D), F 2 (D)/H(D)) exhibits two feedforward parts F 1 (D) and F 2 (D) and one feed-back part H(D), there exist (less than) 2 J+1 ×2 J+1 ×2 J reasonable combinations. The RNSC code degenerates to an RSC code if either F 1 (D) or F 2 (D) is identical to H(D).
Hence, in our two-stage optimization process for improved ISCD schemes we have to find the most appropriate combination of F 1 (D), F 2 (D), and H(D). The EXIT characteristic of the RNSC code with this specific combination will guarantee that the intersection with the EXIT characteristic of SBSD is located at the highest possible (I [ext] CD , I [ext] SBSD ) pair. Remember, in the first step of this process I [ext] SBSD,max had been maximized by an optimization of the index assignment.
However, even if in a real-world system the constraint length J + 1 is limited to a reasonably small number, for example, due to computational complexity requirements, the search for the globally optimal combination of F 1 (D), F 2 (D), and H(D) might enlarge to an impractically complex task. For instance, if the constraint length is limited to J + 1 = 4 (as done for the simulation results in Section 5), there are (in maximum) 2048 combinatorial possibilities and thus, 2048 EXIT characteristics need to be measured. To lower these demands, we propose to carry out a presearch by finding some of the best possible RSC codes, that is, we alter F 2 (D) and H(D) and fix F 1 (D) = H(D). This requires (in maximum) only 128 measurements. Moreover, the effective number can even be reduced to some ten. After having found some of the best possible RSC codes, for each of these combinations of F 2 (D) and H(D) the formerly fixed F 1 (D) is altered. In total, a few hundred of EXIT characteristics need to be measured to find the (at least) locally optimum RNSC code.
In the next section, it will be demonstrated by simulation that due to the higher degrees of combinatorial freedom, the usage of RNSC codes instead of RSC codes reveals remarkable benefits for the error correcting/concealing capabilities of iterative source-channel decoding.
Finally, we have to remark that due to the nonsystematic form of RNSC codes there is no channel-related reliability information available about the data bits. The additional information which is given in the extrinsic L-values L [ext] SBSD (x µ,τ (κ)) and L [ext] CD (x µ,τ (κ)) due to the higher intersection in the EXIT chart must be (at least) higher than the information content of L(z µ,τ (κ) | x µ,τ (κ)) of (2).

SIMULATION RESULTS
The error correcting/concealing capabilities and the convergence behavior of the conventional parallel approach to ISCD and the new improved serial approach using the EXIToptimized index assignment as well as channel codes of the nonsystematic form will be compared by simulation. Instead of using any specific real-world speech, audio, or video encoder, we consider a generic model for the source codec parameter set u τ . For this purpose, M components u µ,τ are individually modeled by first-order Gauss-Markov processes with correlation ρ = 0.9. The parameters u µ,τ are individually quantized by a scalar 16-level Lloyd-Max quantizer using K = 4 bits/parameter each.
After the natural binary 5 index assignment (parallel ISCD scheme), respectively, after the EXIT-optimized index assignment (serial ISCD scheme), a pseudorandom, sufficiently large-sized bit interleaver Φ of size K × M × (T + 1) = 2000 serves for spreading of data bits. For convenience, with respect to K = 4 bits/parameter, we set M = 500 and T + 1 = 1. In practice, a smaller M might be sufficient if bit interleaving Φ is either realized jointly over several consecutive parameter sets or if an appropriately designed (nonrandom) bit interleaver is applied. Here, pseudorandom bit interleaving is realized according to the so-called S-random design guideline [14]. A random mapping is generated in such a way that adjacent input bits are spread by at least S positions. To simplify matters, the S-constraint is given by S = 4 positions.
For channel encoding terminated memory J = 3 recursive (non-)systematic convolutional codes are used. In case of the parallel ISCD scheme it turns out that the RSC code with G = (1, (1 + D 2 + D 3 )/(1 + D + D 3 )) is best suited. Notice, the same channel code has been standardized for turbo channel 5 We use the natural binary index assignment as reference instead of folded binary or Gray-encoded, because in line with our optimization criterion in Section 4, natural binary reveals the highest I SBSD,max values (see Table 1). decoding in UMTS. In case of the new serial ISCD scheme, an RNSC code with the same constraint length and with G = ((1+D 2 +D 3 )/(1+D+D 2 +D 3 ), (1+D+D 3 )/(1+D+D 2 +D 3 )) provides the best results. For termination, J = 3 tail bits are appended to each block of 2000 data bits which force the encoder back to zero state. The overall code rate of both ISCD schemes amounts to r = 2000/4006. A log-MAP decoder which takes the recursive structure of RSC, respectively, RNSC codes into account [12,13] serves as component decoder for the channel code. Figures 6a-6d show the EXIT charts of the different approaches to ISCD either with or without the innovations proposed in Section 4. Each EXIT chart is measured for a particular channel condition E s /N 0 .

Convergence behavior-EXIT charts
In the remainder, that specific approach to parallel ISCD using natural binary index assignment and the RSC channel code is referred to as reference approach (Figure 6a). The EXIT characteristic of SBSD is taken from Figure 4b, but with swapped axes. Both EXIT characteristics specify an envelope for the so-called decoding trajectory [10,11,15,16]. The decoding trajectory denotes the step curve, and it visualizes the increase in both terms of extrinsic mutual information I [ext] CD , respectively, I [ext] SBSD being available in each iteration step.
Decoding starts with the log-MAP channel decoder while the a priori knowledge amounts to I If the natural binary index assignment is exchanged by the EXIT-optimized mapping as proposed in Section 4.1, then the EXIT characteristic of SBSD has to be replaced by the corresponding curve of Figure 4e. Due to the higher I [ext] SBSD,max , the intersection in the EXIT characteristics is located at a remarkably higher (I [ext] CD , I [ext] SBSD ) = (0.96, 0.85). This intersection can be reached quite closely by the decoding trajectory after 6 iterations.
In a third approach to ISCD (Figure 6c), the RSC channel code of the reference approach is substituted by an RNSC code of the same code rate r and constraint length J + 1 as motivated in Section 4.2. As the new channel coder is of the nonsystematic form, the EXIT characteristic of SBSD has to be replaced too because channel-related reliability information will not be available for the outer softbit source decoder Finally, both innovations will be introduced to the reference at the same time (Figure 6d). In order to illuminate the particular features of this approach the channel condition is reduced to E s /N 0 = −4 dB. It can be seen that the EXIT characteristic of the RNSC channel code matches very well to the EXIT characteristic of SBSD. Both characteristics span a small tunnel through which the decoding trajectory can pass. Up to 10 iterations reveal gains in both terms of extrinsic mutual information. The highest possible (I [ext] CD , I [ext] SBSD ) pair (0.97, 0.85) is higher than for all the other approaches mentioned heretofore. This is even true although the channel quality had been decreased by ∆E s /N 0 = 1 dB (E s /N 0 = −4 dB instead of −3 dB).
In certain situations the decoding trajectory exceeds the EXIT characteristic of SISO channel decoding. The reason is that the distribution of the extrinsic L-values L [ext] SBSD (x µ,τ (κ)) of SBSD is usually non-Gaussian in particular if no channelrelated reliability information is given. Thus, the model which was used to determine the EXIT characteristics of SISO channel decoding (see Section 3.1) does not hold strictly anymore. However, even if the precise number of required iterations cannot be predicted from the EXIT chart, the intersection of the EXIT characteristics still remains to be the limiting constraint for the iterative process.

Error robustness-parameter signal-to-noise ratio
The simulation results in Figure 6e depict the parameter signal-to-noise ratio (SNR) of the originally determined source codec parameter u µ,τ and the corresponding estimatê u µ,τ as a function of the channel quality E s /N 0 . For the first basic considerations, we use the same system configuration as for the reference approach introduced before, that is, we apply the natural binary index assignment and the RSC channel code. For every approach to ISCD, the number of iterations is chosen such that the best possible error robustness is reached in the entire range of E s /N 0 = [−5, 0] dB. A higher number of iterations does not yield any noteworthy increase/decrease in the parameter SNR. The lowest curve shows the error robustness of a conventional noniterative receiver using SISO channel decoding and classical source decoding by hard decision and table lookup. If this hard decision (HD) source decoder is replaced by a conventional softbit source decoder [5] the utilization of residual redundancy permits to outperform the classical approach significantly. The maximum gain in parameter SNR amounts to ∆ SNR = 8.76 dB at a channel condition of E s /N 0 = −2.5 dB. Notice, the latter approach resembles an ISCD scheme without any iteration.
A turbo-like refinement of the extrinsic information of both SISO decoders makes further substantial quality improvements possible. Mainly one additional iteration reveals remarkable quality gains in terms of the parameter SNR by up to ∆ SNR = 3.96 dB at E s /N 0 = −2.5 dB. No notewor-thy larger improvements in error robustness are achievable by higher numbers of iterations as can be confirmed by the EXIT chart analysis (see, e.g., Figure 6a with E s /N 0 = −3 dB). However, in the entire range of channel conditions the reference approach to iterative source-channel decoding is superior to (or at least on a par with) the noniterative schemes marked dash-dotted.
As proposed in Section 4 the EXIT chart representation can be used to optimize the index assignment and/or the channel coding component in view of the iterative evaluation. If either of both innovations (each optimized for E s /N 0 = −3.0 dB) is introduced, further remarkable quality improvements can be realized in the most interesting range of moderate channel conditions. Compared to the reference approach, additional gains in parameter SNR of ∆ SNR = 4.54 dB are determinable at E s /N 0 = −3.0 dB if the natural binary index assignment is replaced by the EXIToptimized mapping. The gain amounts to ∆ SNR = 1.43 dB at E s /N 0 = −3.0 dB if the RSC code is substituted by the RNSC code. A quality degradation has to be accepted in case of heavily disturbed transmission channels.
If both innovations are introduced at the same time, almost perfect reconstruction of the source codec parameters becomes possible down to channel conditions of E s /N 0 = −3.8 dB. If the channel condition becomes worse, the parameter SNR drops down in a waterfall-like manner. The reason for this waterfall-like behavior can be found by the EXIT chart analysis (see Figures 6d). As long as the channel condition is better than E s /N 0 = −4.5 dB, there exists a tunnel through which the decoding trajectory can pass to a relatively high (I [ext] CD , I [ext] SBSD ) pair. If the channel becomes worse, the tunnel disappears and the best possible (I [ext] CD , I [ext] SBSD ) pair takes relatively small values. In view of an implementation in a real-world cellular network like the GSM or UMTS system, the E s /N 0 of the waterfall region might be a new design criteria which has to be guaranteed at the cell boundaries. Here, a handover might take place and the loss of parameter SNR in channel qualities of E s /N 0 < −4.5 dB is not relevant anymore.
Finally, it has to be mentioned that the combination of the SNR-optimized mapping [18] with an RNSC code to a serially concatenated ISCD scheme also reveals remarkable improvements in error robustness. However, the EXIToptimized mapping remains to be more powerful as correlation of the source codec parameters can be included in the optimization process.

CONCLUSIONS
In this contribution, the error robustness of iterative sourcechannel decoding has significantly been improved. After a new classification of ISCD into parallel and serially concatenated schemes has been defined, EXIT charts are introduced for a convergence analysis. Based on the EXIT chart representation, novel concepts are proposed on how to determine a powerful index assignment and on how to find an appropriate channel coding component. It has been demonstrated by example that both innovations, the EXIT-optimized index assignment as well as the RNSC channel code, allow substantial quality gains in terms of the parameter SNR in the most interesting range of channel conditions. Formerly known parallel approaches to ISCD are outperformed by far by the new serial arrangement. Table 2 summarizes the EXIT-optimized bit mappings for various quantizer code-book sizes 2 K and correlation ρ = 0.9.

B. EXTRINSIC L-VALUE OF SBSD
The determination rules for the extrinsic L-value L [ext] SBSD (x µ,τ (κ)) of SBSD have been derived in [8,9,10,11]. They will briefly be reviewed next. At the end, a slight modification is proposed which allows to omit a quality loss due to an approximation.
With respect to the defined size of the interleaver Φ, throughout the refinement of bit-wise log-likelihood values T + 1 consecutive bit patterns x µ,τ (with τ = Λ − T, . . . , Λ) of a specific codec parameter are regarded in common. In consequence, the forward recursion does not need to be recalculated from the very beginning α 0 (x µ,0 ) in each iteration.
All terms x Λ−T−1 µ,1 , which are scheduled before the first interleaved bit pattern x µ,Λ−T , will not be updated during the iterative feedback of extrinsic information and can be measured once in advance.