EURASIP Journal on Applied Signal Processing 2005:6, 942–953 c ○ 2005 W. Zhong and J. Garcia-Frias

We propose a coding scheme based on the use of systematic linear codes with low-density generator matrix (LDGM codes) for channel coding and joint source-channel coding of multiterminal correlated binary sources. In both cases, the structures of the LDGM encoder and decoder are shown, and a concatenated scheme aimed at reducing the error floor is proposed. Several decoding possibilities are investigated, compared, and evaluated. For different types of noisy channels and correlation models, the resulting performance is very close to the theoretical limits.


INTRODUCTION
The introduction of turbo codes [1] and low-density parity check (LDPC) codes [2,3,4,5] has been one of the most important milestones in channel coding during the last years. Provided that the information block lengths are long enough, performance close to the Shannon theoretical limit can be achieved for different channel environments. However, in practical applications, complexity issues have to be carefully considered, since both schemes present either high encoding or high decoding complexity. Specifically, for the case of turbo codes the encoding complexity is very low, but the decoding complexity is high. Compared with turbo codes, standard LDPCs present a higher encoding complexity, but the decoder is simpler.
In this paper, we first show that it is possible to achieve a channel coding performance comparable to that of standard This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
This work was partially supported by NSF Grant CCR-0311014 and prepared through collaborative participation in the Communications and Networks Consortium sponsored by the US Army Research Laboratory under the Collaborative Technology Alliance Program, Cooperative Agreement DAAD19-01-2-0011. The US Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation thereon. LDPC and turbo codes by utilizing systematic linear codes with low-density generator matrices [6] (LDGM codes 1 ). LDGM codes present a complexity advantage over standard LDPC and turbo codes. Specifically, because of the sparseness of the generator matrix, the amount of processing required in the encoder is linear with the block size and similar to that of turbo codes. Moreover, since the parity check matrix of systematic LDGM codes is also sparse, such codes are in fact a subset of LDPC codes and can be decoded in the same manner and with the same complexity as standard LDPC codes. Notice, however, that in order to facilitate the development of the paper, we will derive the decoding algorithm utilizing the graph corresponding to the generator matrix.
In the second part of the paper, we will focus on the use of LDGM codes to perform joint source-channel coding of correlated sources, which includes the case of pure source coding as a particular case. The problem of coding for multiterminal correlated sources has important practical applications (e.g., in the context of sensor networks [9]). Since compression and joint source-channel coding of correlated sources can be considered as a problem of channel coding with side information [10,11], the use of powerful channel codes, such as LDGM codes, should produce very good results in this context. The first schemes for compression of multiterminal correlated sources using channel codes were proposed in [12,13,14], and the use of turbo-like codes in this problem was proposed in [15,16]. However, although as shown in [17,18] the separation principle between source and channel coding holds for transmission of correlated sources over separated noisy channels, it is not straightforward to implement a practical system based on this concept. One of the reasons is that, in spite of some previous work (see [14,19] and the references therein), the problem of designing good practical codes for correlated sources from a source coding perspective is still open. Moreover, the separation between source and channel coding may lead to catastrophic error propagation. Previous work in joint source-channel coding using iterative decoding schemes (turbo and LDPC codes) for the cases in which only one source is corrupted by noise (and the special case of joint source-channel coding for single sources) can be found in [20,21,22,23,24]. The case in which both sources are transmitted through separated noisy channels has appeared only in [25,26], where turbo codes were proposed to perform joint source-channel coding of correlated sources.
The contents of this paper are as follows: Section 2 introduces systematic LDGM codes and presents the decoding algorithm in relation with that of standard LDPC codes. The proposed concatenated scheme, required to eliminate the error floor in both channel coding and joint source-channel coding, is introduced in Section 3. Section 4 contains simulation results for the case of channel coding with LDGM codes over binary symmetric channels (BSC), and additive white gaussian noise (AWGN) and Rayleigh fading channels. The rest of the paper deals with the problem of joint sourcechannel coding of correlated sources. The theoretical limits for this problem are presented in Section 5, and the rationale for using turbo-like codes in this context is explained in Section 6. Section 7 specifies the proposed system and the correlation model utilized in this paper for joint sourcechannel coding of correlated sources. Simulation results are provided in Section 8. Finally, Section 9 concludes the paper, and a comparison between the decoding complexity of LDGM and turbo codes is provided in the appendix.

Encoding
Systematic LDGM codes are linear codes with a sparse generator matrix, G = [I P] with P = [p km ] and thus the corresponding parity check matrix is H = [P T I]. We denote the information message that we want to transmit as u = [u 1 · · · u K ]. These bits, together with the coded bits generated as c = uP, with c = [c 1 · · · c M ], are transmitted through a noisy channel. The corrupted sequence at the decoder is denoted as [u c ], where c m = c m + e 1 m and u k = u k + e 2 k , with e 1 m and e 2 k being the noise introduced by the channel. Notice that the proposed code is systematic with rate K/N, where N = K + M. In this paper, we will denote as regular (X, Y ) LDGM codes those irregular systematic LDPC codes in which all the N − K check nodes have degree Y + 1, all the K systematic bit nodes have degree X, and each of the N − K coded bit nodes has degree 1 and is associated to its corresponding check node. In other words, the parity matrix P of an (X, Y ) LDGM code has exactly X nonzero entries per row and Y nonzero entries per column. We will also abuse this notation to include irregular LDGM codes whose average degree distributions are also (X, Y ) (i.e., the average number of nonzero entries per row and per column in their matrices P is X and Y , resp.) In any case, it is obvious that the relationship between the code rate and the degree distributions is given by

Decoding algorithm
For the case of LDPC codes, the decoder goal is to iteratively estimate the most probable solution of the equation s = eH T , where s represents the syndrome calculated from the received sequence, H is the parity check matrix of the code, and e is the error pattern that we are interested in calculating (the sparsest one satisfying the equation above). Analogously, we can see LDGM decoding as a method to find the most probable solution for the equation c = uP, where c is the vector of coded bits generated at the encoder (i.e., before they are corrupted by the channel noise), G = [I P] is the generator matrix of the code, and u is the information message that we want to calculate. Figure 1 shows the graph associated with the proposed code. The decoding algorithm for LDGM codes can be derived by applying belief propagation [27] (or factor graph decoding [28]) over the graph described in Figure 1. The application of belief propagation for the proposed codes presents an important difference with respect to standard LDPC codes: in standard LDPC codes, the syndrome nodes are fixed to a deterministic value (i.e., for each position they are either 0 or 1). In the proposed codes, the coded bit nodes c are random variables, with distribution calculated depending on the received corrupted coded bits c and the distribution of the noise vector e 1 (i.e., c m = c m + e 1 m ). Although this decoding procedure is only exact for networks without cycles, we will see in our simulations that the proposed codes achieve good performance. In order to facilitate the algorithm implementation, we present the decoding method indicating only the modifications with respect to the case of standard LDPC codes. Using the notation commonly utilized in the LDPC literature, we will denote by r x mk , x ∈ {0, 1}, the message propagated from coded bit node c m to systematic bit node u k (and by q x mk , x ∈ {0, 1} the message propagated from systematic bit node u k to coded bit node c m ), if the standard LDPC decoding algorithm were to be directly applied over the graph shown in Figure 1. We now indicate the modifications necessary to deal with LDGM codes. The messages passed from coded bit nodes to systematic bit nodes will be denoted by R x mk . Q x mk indicate the messages exchanged from systematic bit nodes to coded bit nodes.
(1) Initialization. Fix the probability of the systematic bit nodes to its a priori value. For every (m, k) such that u k and c m are connected, let 1}. p k denotes the probability that systematic bit u k is received in error.
(2) Message passing from the coded bit nodes to the systematic bit nodes.
(i) For every (m, k) pair such that u k and c m are connected, calculate r x mk as in standard LDPC decoding (but using the parameters Q x mk instead of q x mk ).
where γ m denotes the probability that coded bit c m is received in error. The intuitive idea behind the previous equations is that when the received coded bit is in error (which occurs with probability γ m ) the message to pass, R x mk , is not r x mk , but r 1−x mk . If the received coded bit is correct (which occurs with probability 1 − γ m ), R x mk = r x mk . (3) Message passing from the systematic bit nodes to the coded bit nodes. For every (m, k) pair such that u k and c m are connected, calculate Q x mk in the same way as q x mk is calculated in standard LDPC decoding. The only differences are that now parameters R x mk should be used instead of r x mk , and that the a priori probability for the systematic bit node u k is P x

CONCATENATED LDGM SCHEMES
As shown in [29], and first recognized by MacKay [5], LDGM codes are bad codes, since they present error floors that are independent of the block length. Specifically, LDGM codes with small degrees have high error floors but good convergence thresholds, while the contrary occurs if the degrees are high [29]. However, these error floors can be reduced and practically eliminated. The reason is that, as explained in [29] for BSCs, the number of errors for the blocks in error decays very fast, provided that the crossover probability  is small enough (a similar behavior can be observed in other channels when their quality improves). Moreover, the LDGM decoder produces a good indication of where the residual errors are located. Most of the systematic bits in error will have a corresponding probability very close to 0.5, compared with a probability close to 1 for the bits that have been successfully decoded. This means that the results obtained from the decoding of an LDGM code can be seen as produced by an equivalent channel introducing a small amount of "erasures" at specific locations. Notice however that this equivalent channel (consisting of the concatenation of the channel and the LDGM code) is not a standard erasure channel. The error locations are not known with certainty, but, on the other hand, there is a priori information about how the erasures should be filled. Specifically, the LDGM decoder provides this a priori probability for each of the systematic bits. This a priori probability can be easily exploited if those systematic bits for the LDGM code are in fact generated by another outer LDGM code. Then, the outer LDGM decoder would use the a priori probability to initialize its systematic and coded bit nodes in the decoding process, which would reduce the number of residual errors. The encoder diagram for the proposed concatenated scheme is shown in Figure 2. Figure 3 shows the graph representing the concatenated scheme. Decoding can be easily performed by applying the belief propagation algorithm [27] over the graph. Since the network has cycles (more than for the case of single codes), different activation schedules can lead to different results. In order to define different activation schemes, we assume that message passing is performed according to a global clock. At each clock cycle, a group of nodes is activated. The activation of a node is defined as the process in which the node reads all its incoming messages from all its neighboring nodes, performs computation, and outputs all its outgoing messages to its neighbors. We follow the convention that once a message is produced, it remains available to the corresponding nodes until the message gets updated.
We define two different scheduling schemes for the case of channel coding. The first one (decoding algorithm I) has already been defined in [29]. It first proceeds with the decoding of the inner code, and the last step is to reduce the residual errors by decoding the outer code. The second schedule (decoding algorithm II) iterates between the inner and outer code in each iteration. For notation purposes, we will assume that nodes are activated serially by order of appearance in the scheme definition, except when included in brackets (which means that activation for those nodes is performed in parallel within a clock cycle). 2 Then, the two different schedules can be expressed as follows.
(i) Decoding algorithm I [29]: It is possible to define other activation schedules by varying the number of times that the inner decoder is activated per iteration in the outer decoder. In general, we have observed that increasing the activation rate of the inner decoder leads to slightly higher error floors, while more frequent activations in the outer decoder slightly degrades the convergence threshold. This observation is consistent with the functioning of the concatenated scheme, since the inner code determines the convergence threshold and the outer code reduces the error floor by taking care of the small number of residual errors.
The concatenation of the two LDGM codes can be considered as a single irregular LDGM code of generator matrix G joint = G outer G inner . Therefore, decoding can be performed using the graph corresponding to G joint . Notice that this graph can be easily obtained from Figure 3 by eliminating the connections between the inner and outer coded bit nodes (connecting instead the systematic bit nodes directly to the corresponding inner coded bit nodes). In this case, a systematic bit node could be connected more than once to the same inner coded bit node. Then, if the number of connections is even, all connections cancel with each other and no connection appears in the final graph, while if the number of connections is odd, one of them is kept. However, as we illustrate in the next section, the performance of the "joint" scheme, G joint , is much worse than that of schedules I and II.

SIMULATION RESULTS OF LDGM CODES FOR CHANNEL CODING
In all our simulations, the matrices P inner and P outer are generated in a pseudorandom way without introducing cycles of length 4 or less. In all cases, at least 10 000 blocks are simulated, and, for the decoding of each block, the iterative process continues until 3 consecutive iterations produce the same result for the systematic bits or 100 iterations are run. We first encoded 9500 information bits with a regular (4, 76) outer LDGM code to produce a total of 10 000 bits. These bits were encoded again by a regular (6,6) inner LDGM code producing a total of 20 000 bits (i.e., overall rate R c = 0.475). For the joint scheme, the node-degree profiles (i.e., the percentage of nodes of a given degree) of the code resulting from the concatenation of the inner and outer codes (G joint = G outer G inner ) are λ(x) = 0.0004x 30 + 0.0277x 32 +0.9719x 34 for the systematic bit nodes and ρ(x) = 0.697810x 5 + 0.047619x 75 + 0.000095x 76 + 0.009429x 78 + 0.215524x 80 + 0.000667x 151 + 0.012190x 153 + 0.015048x 155 + 0.000190x 224 +0.000762x 226 +0.000571x 228 +0.000095x 230 for the coded bit nodes. Considering an AWGN channel, even for very high signal to noise ratios (4 dB above the Shannon limit), the residual BER for each block is always higher than 10 −2 , and presents oscillations with the iteration number. This behavior can be explained by the existence in G joint of a peculiar type of structure containing many short cycles of length 4, which are produced as described below. Figure 4a shows all the connections for a given outer coded bit node (triangle) in the concatenated LDGM scheme shown in Figure 3. Following the rules described in Section 3, it is obvious that in the graph corresponding to G joint , Figure 4a becomes the structure shown in Figure 4b. Figure 4b assumes that there are no other connections (either directly or through other outer coded bits) for the shaded systematic and inner coded bit nodes in Figure 4a. As described before, if an even number of connections between a given systematic bit and a given coded bit were to exist, G joint would present no connection between these two nodes. The important point is that G joint has as many of the structures shown in Figure 4b as outer coded bits (500). Even if some of the links in the structures are eliminated (in this example the probability of elimination of a link is less than 0.03), these structures are highly regular, and each of the 76 systematic bits presents many loops of length 4 (21, i.e., 7 choose 2, in the case of no link elimination), all of them involving the shaded bits in Figure 4b. The occurrence of this type of structure explains the poor performance of G joint and the oscillating behavior in the decoding process. Notice that when decoding is performed in the graph containing both the inner code and the outer code (Figure 3), this type of cycles is broken by the outer coded bit nodes (see Figure 4a).
The results presented in this section for the joint scheme assume a code G joint with the same node-degree distributions as G joint (in fact, with exactly the same number of nodes with a given degree), but with random connection assignments. In this way, the structures existing in G joint disappear and the performance is expected to improve. Figure 5 shows · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · 6 76 (a) · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·  the performance of the proposed decoding algorithms over a BSC as a function of the crossover probability, while Figure 6 illustrates the performance for AWGN channels as a function of E b /N 0 (where E b is the energy per information bit and N 0 denotes the one-side spectral density). For schedules I and II, no error floor appeared after simulating more than 10 000 blocks, and the resulting performance is very close to the theoretical limits (which correspond to a crossover parameter 0.118 for the BSC case and E b /N 0 = 0.08 dB for the AWGN channel assuming binary signaling [30]). These results are comparable to those obtained with turbo and irregular LDPC codes [31,32]. It is interesting to remark that both activation schedules lead to similar results for AWGN channels, while schedule II is slightly superior for BSCs. Notice that the performance of the "joint" scheme, G joint , is

BER
Decoding algorithm I Decoding algorithm II Joint scheme Figure 6: Performance of the joint scheme and of the decoding algorithms I and II when the proposed concatenated system is utilized over AWGN channels. The overall rate of the code is 0.475. The Shannon limit for this case (assuming binary signaling) is still worse than that of decoding algorithms I and II, but it presents no oscillating behavior. This worse performance can be explained by the existence of some short cycles in G joint , 3 and by noticing that random connection assignments lead to a graph structure different from the specific one resulting from the concatenation of the inner and outer codes (represented in Figure 3). This occurs because the graph in Figure 3 contains the outer coded bit nodes as additional variables that (due to the random connections) do not have equivalence in G joint . We also investigated the performance of LDGM codes over Rayleigh fading channels. We assume that the received sequence can be expressed as r k = c k y k +n k , where {y k } is the binary transmitted sequence, {n k } is a set of statistically independent Gaussian random variables with zero-mean, and {c k } is modeled as a Rayleigh process. We also assume an ideally interleaved channel, so that the sequence {c k } is uncorrelated in time k, and perfect channel side information (CSI) is available at the decoder (i.e., the value of c k is known). Figure 7 shows the performance of the concatenated LDGM code defined before in this Rayleigh fading environment. Notice that similar to the BSC and AWGN channels, no error floor appears here and decoding algorithms I and II outperform the "joint" scheme, G joint again. The theoretical limit in this case (assuming binary signaling) is E b /N 0 = 1.6 dB [30], and both schedules I and II achieve a performance within 1.3 dB from this limit. Figure 8 illustrates the system proposed in this paper for joint source-channel coding of correlated sources. For simplicity, we consider only two sources, but the approach can be easily extended to the case of more sources. The two sources are encoded independently from each other (i.e., for a given source neither the realization from the other source nor the correlation model are available at the encoder site) and transmitted through two different noisy channels to a common decoder. Since the correlation between the sources is exploited at the  common receiver, the value of E b /N 0 corresponding to the theoretical limit will be less than if the sources were independent. In this section, we review the theoretical limits for the case in which both channels are either noisy or noiseless (which corresponds to the case of compression of correlated sources).

Compression of correlated sources: Slepian-Wolf limit
It is well known [33,34] that two jointly ergodic sources (U 1 , U 2 ), defined over countably infinite alphabets, can be compressed at rates (R 1 , R 2 ), provided that As explained before, compression is performed independently for each source and the decoder jointly acts over the compressed versions of the sources to recover the original sequences.

Transmission of correlated sources over independent noisy channels
It has been recently shown [17,18] that the separation principle between source and channel coding applies to the case of transmission of correlated sources over separated noisy channels. In other words, the theoretical limit for the transmission of two sources generating i.i.d. random pairs can be obtained by performing first distributed data compression up to the Slepian-Wolf limit followed by channel coding. Therefore, assuming that both sources are encoded at the same rate (R 1 = R 2 = R/2), the theoretical limit in communications for a fixed transmission rate of R/2 information bits/channel use would then be achieved for each source when the two correlated sources are first compressed up to the joint entropy (H(U 1 , U 2 )) and then a capacity achieving channel code of rate R c = R/2 is used for each of them. By taking into account that the energy per generated source bit (E so ) can be related with the energy per information bit (E b ) by using the relation 2E so = H(U 1 , U 2 )E b , the theoretical limit for E so /N 0 (for the case of two independent channels with capacity C) can be obtained by solving the equation R/2 = C [18]. The previous separate source and channel coding approach would achieve the theoretical limit if two conditions are met. On the one hand, optimum source coding for correlated sources should be utilized. On the other, capacity achieving channel codes are necessary. The problem with this approach in practical systems is twofold. First, it is necessary to design good practical source codes for correlated sources. Moreover, in practical systems, errors introduced by the channel decoder could be catastrophic for the source decoder. Besides, in our approach, it does not seem reasonable to first use LDGM codes to compress the sources and then some other LDGM codes to add redundancy, since the use of one LDGM code (per source) can perform the combined operation. In order to avoid these problems, we propose a joint source-channel coding scheme which in practical situations achieves performance very close to the theoretical limits. In our approach, each of the correlated binary sources is not source encoded, but directly channel encoded with a channel code of rate R c . The information rate transmitted through the channel in this case is R 1 = R 2 = R/2 = H(U 1 , U 2 )R c /2 information bits/channel use. Notice that in order to keep the information rate per source (R/2), the code used in our joint source-channel coding approach (of rate R c ) has to be less powerful than in the separate source and channel coding scheme (code of rate R c = R/2).
Specifically, the relation between R c and R c to keep the same information rate through the channel, R/2 = R c , is given by R c = H(U 1 , U 2 )R c /2. The "weakness" of the code in the joint source-channel coding approach will be compensated by exploiting the correlation between sources in the decoder. Notice that the proposed joint source-channel coding approach allows a channel code of a single rate to be used in combination with sources having arbitrary joint entropy rates, with the modifications to maintain efficient coding involving only processing in the decoder.

TURBO-LIKE CODES FOR CORRELATED SOURCES: RATIONALE
As indicated in the introduction, compression (or joint source-channel coding) of multiterminal correlated sources can be seen as a problem of channel coding with side information. This is illustrated in Figure 9. To simplify the description, we assume that the information from source 2 is perfectly available at the decoder and that U 1 is the sequence generated by source 1 that we want to compress. In order to do so, U 1 is encoded by a systematic channel encoder, so that the nonsystematic coded bits, C 1 , constitute the compressed sequence for source 1 (i.e., the systematic bits are eliminated). The decoder utilizes the compressed sequence for source 1 (O 1 = C 1 ) plus the information proceeding from the other source (U 2 ). Notice that, because of the correlation between sources, U 2 can be used as a side information for source 1. In fact, U 2 can be thought as the corrupted version of the systematic bits in source 1 when U 1 is transmitted through a channel model defined by the correlation between sources. Therefore, recovery of U 1 can be interpreted as performing channel decoding over the corrupted version of U 1 (U 2 ) and the redundant/uncorrupted nonsystematic  Figure 9: Source coding as a problem of channel coding with side information. U 1 is encoded by a systematic channel encoder, so that the nonsystematic bits, C 1 , constitute its compressed version. U 2 can be thought of as the corrupted version of U 1 when U 1 is transmitted through a channel model defined by the correlation between sources, and is used by the decoder together with O 1 = C 1 to recover the original sequence U 1 .
bits O 1 = C 1 proceeding from U 1 . In the case in which U 2 were not perfectly available at the receiver, the decoder would consist of two blocks as the one shown in Figure 9 (one for source 1 and the other for source 2). Then, decoding for U 1 would use an estimate of U 2 as side information, and provide the resulting estimate of U 1 as side information for the decoding of U 2 , with this process continuing iteratively. The interpretation of joint source-channel coding of correlated sources as a problem of channel coding with side information is also straightforward. The only differences with respect to the case of pure source coding described above are: (i) sequence C 1 may include systematic bits, and (ii) sequence C 1 is corrupted by the channel noise, producing sequence O 1 , which is available at the decoder. Turbo-like codes are very well suited to be applied in the context described in Figure 9. The reason is two fold. First, they are pseudorandom codes, and therefore adequate to achieve the theoretical limits corresponding to random codes (by using one turbo-like code as encoder for each of the correlated sources). Second, turbo-like codes are very well prepared to exploit side information. In order to do so, the known probabilistic description of the different sequences available at the decoder can be easily incorporated in the decoding process, which will be (in general) represented by a graph. Moreover, even if the correlation model is not available at the decoder, it is still possible to estimate it jointly with the decoding process (in many occasions with little performance degradation). Sections 7 and 8 develop these ideas for the case of LDGM codes. Although the simulation results presented there focus on the case of joint source-channel coding, the particularization of the proposed schemes for the case of pure source coding results in performances very close to the Slepian-Wolf limit [16].

LDGM CODES FOR CORRELATED SOURCES: PROPOSED SYSTEM
As explained in Section 5, although the theoretical limits for transmission of correlated sources over noisy channels can be achieved by separation between source and channel coding, it may be advantageous in practical applications to use a joint source-channel coding approach, such as the one presented in this paper and shown in Figure 8 [35,36]. This approach can be particularized into some special cases such as source coding of a single source (by ignoring the other source and considering noiseless channels), distributed source coding (by considering noiseless channels) [37,38], and joint source-channel coding of single sources (by ignoring the other source).
For the development contained in this paper, we denote the two correlated binary information sequences as U 1 = u 1 1 u 1 2 . . . and U 2 = u 2 1 u 2 2 . . . with u j k ∈ {0, 1}. The correlation model is established by first generating the symmetric i.i.d. sequence U 1 (P(u 1 k = 0) = P(u 1 k = 1) = 1/2). Then, the sequence U 2 is defined as u 2 k = u 1 k ⊕ e k , where ⊕ indicates modulus 2 addition and e k is a random variable which takes value 1 with probability p and value 0 with probability 1 − p. Each source is independently encoded with a system composed of a serial concatenation of two LDGM codes. 4 For source j, the coded bits generated by the outer encoder, c j,out , and the information bits, u j , constitute the systematic bit nodes for the inner encoder, which further generates the inner coded bits c j,in . After encoding, the resulting bits are sent through the corresponding noisy channel, and decoded in the common receiver by applying the belief propagation algorithm over the graph representing both decoders, which is shown in Figure 10.
Several activation schedules can be utilized in the decoding process, and, since the graph presents cycles, they can lead to different performance. We consider the five different activation schedules shown below, where each repetition constitutes one iteration. Notation is consistent with the channel coding case explained in Section 3. 4 The use of a single LDGM code results in intolerable error floors.

SIMULATION RESULTS FOR JOINT SOURCE-CHANNEL CODING OF CORRELATED SOURCES
In this section, we first analyze the performance of the five activation schedules introduced in last section when each source is independently channel encoded by the serial concatenation of a (6.5,6.5) 5 inner LDGM code and a regular (4, 76) outer LDGM code (i.e., overall rate R c = 0.475) and transmitted through a noisy channel. The length of the information sequences is assumed to be L = 9500 and the correlation parameter is fixed to p = 0.1. Figure 11 shows the BER versus E so /N 0 when AWGN channels are considered. For schedules 1 to 4, no errors were observed at E so /N 0 = −0.7 dB after simulating more than 10 000 blocks. Since the theoretical limit in this case corresponds to E so /N 0 = −1.85 dB, the proposed system is within 1.15 dB from this limit. Similar results for the case of ideally interleaved Rayleigh fading channels with perfect CSI at the receiver are shown in Figure 12.
In this case, the gap with respect to the theoretical limit is around 1.5 dB. Notice that, for both figures, schedules 1 to 4 have very similar performances and their curves basically overlap. For schedule 5, which decodes first the inner code and then tries to eliminate the error floor with the outer code (without further exchange of information between the outer and the inner codes), the gap from the theoretical limit is larger. Table 1 shows the maximum, minimum, and average numbers of iterations required to achieve convergence at some values of E so /N 0 in Figures 11 and 12. Notice that each iteration in schedule 2 corresponds roughly to two iterations in schedules 1, 3, and 4 (most of the complexity is produced in the activation of the coded bit nodes). Taking this into account, we can observe that in average schedules 1 to 4 require approximately the same number of iterations. Notice that the number of iterations required for schedule 5 are obtained at different values of E so /N 0 than those of schedules 1 to 4, which means that the comparison between schedule 5 and the other schedules is not very significant.
In order to further assess the performance of the proposed system, we consider different values of the parameter p and study the system performance utilizing schedule 1. As before, the length of the information sequence is fixed to L = 9500. For different values of p, we use the same (4,76) outer LDGM code as before, but we consider different inner codes in order to optimize performance. Simulation results are presented in Table 2 for AWGN channels and in Table 3 for ideally interleaved Rayleigh fading channels with perfect CSI at the receiver. In both cases, the optimum degree of the inner code decreases with parameter p. For all different values of p, at a bit error rate of 10 −5 , the gap between the theoretical limit and the proposed system is within 1.8 dB for the AWGN channel and within 2.2 dB for the Rayleigh fading channel. Notice that this gap increases when p gets smaller, which was already pointed out in previous related work [21,25,26]. The gain of the proposed system is evident if we realize that, when the source correlation is not exploited in the decoding process, the achievable theoretical limits for E so /N 0 are 0.08 dB and 1.6 dB for the AWGN and Rayleigh fading channel, respectively. The proposed approach achieves a performance (in terms of convergence threshold) similar to the system proposed in [25,26] for joint source-channel coding of correlated sources over separated AWGN channels using turbo codes. Moreover, after simulating the same number of blocks as in [25,26], no error floor could be observed here. As shown in the appendix, the use of LDGM codes instead of turbo codes leads to a lower decoding complexity.

CONCLUSION
We proposed the use of LDGM codes for channel coding and joint source-channel coding of correlated sources over noisy channels. In order to avoid error floors, it is necessary to utilize concatenated schemes. However, they should not be decoded utilizing the equivalent LDGM code resulting from the concatenation, but by combining the graphs of the constituent codes. In terms of encoding/decoding complexity, the proposed scheme presents complexity advantages with respect to turbo and standard LDPC codes. For channel coding, the performance over BSCs, AWGN channels and ideally interleaved Rayleigh fading channels with perfect CSI at the receiver is comparable to that of turbo codes and standard irregular LDPC codes, and close to the theoretical limits even without much code design optimization. In the case of correlated sources, where previous work is almost nonexistent, the proposed system also achieves a performance close to the theoretical limits and similar to those of turbo codes.

APPENDIX CODING COMPLEXITY: LDGM VERSUS TURBO CODES
The encoding of a systematic LDGM code involves computation of the parity bits, each of which only depends on a finite number of systematic bits. Hence, similar to turbo codes, LDGM codes are encodable in linear time. From now on we will focus on the comparison between the two in terms of decoding complexity.

Complexity per decoding iteration
Reference [39] provides a detailed analysis on the decoding complexity of turbo codes. The main result is that for a turbo code with constituent encoders having rate k/n and S states, the total number of additions/subtractions (additions) and multiplications/divisions (multiplications) per information bit and per iteration are given by (i) additions [turbo (S, n)] = 4(3S + n − 4), (i) multiplications [turbo (S, n)] = 2(8S + 2n + 5). We now analyze the decoding complexity of an (X, Y ) LDGM code by following the development in [5] and our definitions of Q x mk and R x mk . Because of their lower complexity, we will disregard operations consisting of additions/multiplications by constants (notice that [39] disregards table look ups and maximum operations). We proceed in two steps. First, we calculate the number of operations required in the processing of a coded bit node. Second, we Table 3: For ideally interleaved Rayleigh fading channels with perfect CSI at the receiver and different correlation parameters p, theoretical limit for E so /N 0 in dB ([E so /N 0 ] l , taken in steps of 0.01 dB), value of E so /N 0 in dB for which the proposed system achieves a BER less than 10 −5 ([E so /N 0 ] s ) and gap (taken in steps of 0.05 dB) between the theoretical limit and the performance of the proposed system. look at the complexity in an information bit node. The total number of operations per information bit and per iteration will be the sum of the operations required in all the coded bit nodes plus the operations performed in all the information bit nodes divided by the total number of information bit nodes.
In order to calculate the number of operations in each coded bit node, notice that in (2) and (3) in this paper R 1 mk = 1−R 0 mk . Therefore, once (2) is calculated, (3) can be obtained without any additional complexity. Following the notation in [5], we define δQ mk = Q 0 mk − Q 1 mk = 1 − 2Q 1 mk . We also define D m = (−1) cm k∈L(m) δQ mk . Then, (49) in [5] can be calculated as δr mk = D m /(1 − 2Q 1 mk ). In this way, since as indicated in [5]  In order to calculate R 0 mk , we first calculate D m , which requires Y − 1 multiplications. Then, we utilize one more multiplication to obtain D m (1 − 2γ m ), and finally we calculate R 0 mk for k = 1 · · · Y , which requires Y more multiplications.
Therefore, we just need 2Y multiplications to perform all the processing required in a coded bit node. Since there are a total of N(1 − R c ) coded bit nodes and NR c information bits (where R c = Y/(X + Y ) as indicated in this paper), the total amount of processing in the coded bits divided by the number of information bits is 2Y (1 − R c )/R c = 2X multiplications.
In order to calculate the number of operations performed in an information bit node, we follow (50)-(53) in [5]. Notice that Q 0 mk = α mk Q 0 k /R 0 mk , m = 1 · · · X. By forcing Q 0 mk + Q 1 mk = 1, Q 0 mk can be calculated as Q 0 mk = 1/(1 + (Q 1 k /Q 0 k )(R 0 mk /(1 − R 0 mk ))). Therefore, after calculating α k Q 0 k and α k Q 1 k , which requires 2X multiplications, and Q 1 k /Q 0 k , which requires another multiplication, the calculation of Q 0 mk for a fixed m (counting an inversion as a multiplication) can be performed with 2 divisions. Therefore, the total number of operations to calculate all Q 0 mk , m = 1 · · · X, is 4X +1 multiplications/divisions. Since

Total number of decoding iterations
The total number of iterations required for convergence cannot be predicted through analysis. Table 1 in this paper shows the number of iterations required to achieve convergence for different activation schedules of the concatenated LDGM scheme in the case of joint source-channel coding. This number is greater than the one usually required in turbo coding schemes, but it is not enough to compensate the advantage of LDGM codes in each iteration. Compensation does not occur in the channel coding case either. For instance, for the concatenated scheme used over AWGN channels ([(6,6)(4,76)] with block size 20 000), the average number of iterations at an E b /N 0 of 0.8 dB above the Shannon limit is 21.7, which is about twice the number of iterations required in a comparable turbo code.

DISCLAIMER
The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the US Government.