EURASIP Journal on Applied Signal Processing 2005:6, 775–783 c ○ 2005 Hindawi Publishing Corporation Parallel and Serial Concatenated Single Parity Check Product Codes

The parallel and serial concatenation of codes is well established as a practical means of achieving excellent performance. In this paper, we introduce the parallel and serial concatenation of single parity check (SPC) product codes. The weight distribution of these codes is analyzed and the performance is bounded. Simulation results conﬁrm these bounds at high signal-to-noise ratios. The performance of these codes (and some variants) is shown to be quite good given the low decoding complexity and reasonably short blocklengths.


INTRODUCTION
The parallel and serial concatenation of codes is well established as a practical means of achieving excellent performance.Interest in code concatenation has been renewed with the introduction of turbo codes [1], otherwise known as parallel concatenated convolutional codes (PCCCs) [2], and the closely related serially concatenated convolutional codes (SCCCs) [3].In this paper, we introduce the parallel and serial concatenation of single parity check (SPC) product codes.These codes perform well and yet have a low overall decoding complexity.Similar work involving parallel concatenation of SPC codes (not SPC product codes) has been considered in [4], while serially concatenated SPC codes are investigated in [5].
It should be noted that the component codes are not recursive and therefore both the parallel concatenated code (PCC) and the serially concatenated code (SCC) should not exhibit any "interleaver gain" [2,3].However, in practice, the parallel and serial concatenation of nonrecursive codes can still perform very well, for example, the "turbo block code" [6].It will be shown that parallel and serially concatenated SPC product codes also perform well, especially considering the very low decoding complexity.The main reason for this good performance is the relatively small number of lowweight codewords.The weight distribution and performance bounds will be investigated in Section 5.

ENCODING THE PCC AND SCC
In general, a parallel concatenated code involves encoding a set of common data bits between multiple component codes, typically the data bits are interleaved between the component encoders.The component codes used throughout this paper are {n, d} SPC product codes, where d is the number of dimensions and n is the length of the SPC codes in every dimension [7,8].The encoder for a parallel concatenated SPC product code is shown in Figure 1a.In this case, the data bits are encoded using an {n, d} SPC product code in one branch of the code while the interleaved data bits are encoded using another {n, d} SPC product code in the second branch.Because the SPC product codes are systematic, it is not necessary to transmit the data bits from the second code as that information is already contained in the first codeword, and consequently the code rate is increased.This paper will only consider PCC SPC product codes with two branches or component codes, although the concept can easily be extended.Therefore, the length of each codeword is The encoder for a serially concatenated SPC product code is shown in Figure 1b.In this case, the data is encoded using an {n − 1, d} SPC product code.The resulting codeword is then interleaved and re-encoded using an {n, d} SPC product code.This can also be extended to more than a single serial concatenation; however, the decrease in code rate can be prohibitive.The blocklength for this simple two-stage SCC SPC product code is N = n d , while the number of data bits is K = (n − 2) d .Both the SCC and PCC have the same interleaver size, (n − 1) d , using these encoder definitions.
It is possible, although very unlikely, that both the PCC and SCC have a minimum distance given by the minimum distance of the SPC product code, d min = 2 d .This is possible because some of the minimum-distance codewords in the SPC product code have a zero-output parity weight.Consequently, the interleaver may map one of these minimumdistance codewords to another minimum-distance codeword in the second code.This event is very unlikely but will be investigated in Section 5 as part of the evaluation of the weight enumerator and associated performance bounds.

ITERATIVE DECODING OF CONCATENATED CODES
In order to iteratively decode the SCC or PCC, it is necessary to soft decode the component SPC product codes.This is described in [7,8] where the log-likelihood-based decoder will MAP decode the individual SPC codes, within each SPC product code, and pass the extrinsic information between each dimension.Specifically, the a posteriori probabilities (APPs) of the coded bits, in terms of a log-likelihood ratio (LLR), is given by [7] L where the extrinsic information of the kth bit in the SPC component code, L q (x k ) is given by and atanh is defined as the inverse hyperbolic tangent function.On the additive white Gaussian noise (AWGN) channel L c = 2/σ 2 , while L q (x k ) is the a priori information of the kth bit in the qth dimension.The a priori information is initially zero; however, in subsequent decodings, it is the sum of the extrinsic information from the other dimensions of the product code: A slightly modified version of this algorithm will be used here to decode the component SPC product codes.The only difference is that the a priori information, which is the sum of the extrinsic information in other dimensions (3), will include an extra term, L e (x k ).This extra extrinsic information comes from the other component code in either the PCC or SCC.The next two subsections consider iterative decoding of these codes in more detail.

Decoding the PCC
The iterative decoder for a two-branch PCC SPC product code is shown in Figure 2a.The aim of the decoder is to incorporate the extrinsic information from the other code in order to improve performance.Specifically, the extra extrinsic information, L e (x k ), is the average extrinsic information over all the dimensions of the other code for a particular bit, x k .Hence the a priori information for the decoding of the current SPC product code is given by  where L e (x k ) has been interleaved/deinterleaved so that x k and x k refer to the same bit.Note that because the PCC has only the data bits common to both codes, the extrinsic information that passed between the decoders can only apply to those bits.This approach implicitly assumes that the extra extrinsic information from the other SPC product code is independent.Due to the interleaving, this is a good assumption for the first decoding iteration.However, as more iterations are completed, the extrinsic information becomes more correlated (hence the improvement in performance with each decoding iteration will decrease).
A single decoding iteration of the PCC decoder (shown in Figure 2a) consists of one decoding cycle for both component SPC product codes using the modified version of the decoding algorithm.In each decoding cycle, the appropriately interleaved average extrinsic information on the data bits from the other code (generated in the previous decoding iteration) is used to adjust the a priori LLRs, as defined by (4).Note that the decoding cycle is defined by calculating the extrinsic information for all bits in the SPC product code after decoding in only one dimension.Initially all extrinsic information is zero.Typically the PCC decoder performs at least 2d decoding iterations, or 4d decoding cycles, where d is the number of dimensions in the component SPC product codes.It was found that these many iterations were required for the decoder to converge.

Decoding the SCC
The iterative decoder for a serially concatenated SPC product code is shown in Figure 2b.Initially the inner SPC product code (SPC product code 2 in Figure 1b) is soft decoded for a single decoding cycle.Once again the extrinsic information is calculated for all bits in every dimension using a modified version of the LLR decoding algorithm.Note that the codeword from the outer code (SPC product code 1 in Figure 1b) is completely contained within the inner codeword, although in an interleaved form.Hence the average extrinsic information from the inner code is available for all bits in the outer code after deinterleaving.The second half of the decoding iteration calculates the extrinsic information on all bits in every dimension for the outer code (again using a modified version of the decoding algorithm).The average extrinsic information is then interleaved and passed back to the inner code for use in the next decoding iteration.
In both PCC and SCC, the binary decision on the data bits, dk , is determined by the soft output, L out , where ( Specifically, Typically the decision is made on the data bits from the outer code in the SCC, or the last code to be decoded in the PCC.Note that the extrinsic information needs to be interleaved/deinterleaved so that x k and x k correspond to the same bit.

PERFORMANCE RESULTS
In all simulations, randomly generated interleavers were employed.No attempt was made to optimize them, so further gains may be possible in specific applications where an appropriate interleaver can be specially designed.

PCC performance
The performance of an (8, 7) three-dimensional PCC SPC product code is shown in Figure 3.This code has rate R = 0.5037 and blocklength N = 681, and can achieve a BER of 10 −5 at E b /N 0 = 3.37 dB.This is 3.25 dB away from the binary input AWGN capacity of the system.The performance of a number of PCC SPC product codes is shown in Figure 4, with the code rate plotted against the E b /N 0 required to achieve a BER of 10 −5 .Note that the binary input AWGN capacity is defined by the signal-to-noise ratio such that the probability of error can be driven to zero as the blocklength tends to infinity.These PCC SPC product codes have quite short blocklengths (especially the two-and threedimensional examples), and consequently they cannot force the probability of error, P e , to zero at such a low signal-tonoise ratio.Therefore these codes will be compared to the sphere-packing bound [9,10,11] which lower bounds the best possible probability of codeword error for any code of a given blocklength and code rate.The three-dimensional (8, 7) PCC SPC product code can achieve P e = 10 −4 at E b /N 0 = 4.02 dB (see Figure 3).The sphere-packing bound requires E b /N 0 ≥ 1.33 dB to achieve this probability of codeword error, hence the PCC is only 2.69 dB away from the best possible code.Furthermore, it should be noted that a 2-3 dB performance improvement is obtained by using a threedimensional SPC PC component code instead of a two dimensional SPC PC component code, with a relatively small change in code rate.
Finally, note that the four-dimensional PCC SPC product codes have a larger minimum distance, and so have a lower error floor and a steeper roll-off than the three-dimensional codes.Therefore, even though the performance of the fourdimensional codes seems only slightly better than the threedimensional codes in Figure 4, at a lower BER, the difference is greater.

SCC performance
A performance comparison between various serially concatenated SPC product codes is given in Figure 5, with the code rate plotted against the signal-to-noise ratio (SNR) required to achieve a BER of 10 −5 .The performance of the SCC codes is very similar to that of the PCC codes, but the SCC codes have a slightly lower code rate and shorter blocklength (for the same size interleaver).For example, the three-dimensional, n = 8, SCC SPC product code has R = 0.4219 and N = 512.This SCC code achieves a BER of 10 −5 at E b /N 0 = 3.67 dB, which is somewhat worse than the corresponding PCC code.However, as the size of the component codes increases, the performance converges to that of the PCC codes.Also note that the SCCs with threedimensional SPC PC component codes also give a 2-3 dB advantage in performance over the two-dimensional SPC PC component codes, as with the corresponding PCC codes.Although the performance of the four-dimensional SCC codes appears quite poor in comparison to the three-dimensional codes, the larger minimum distance will result in better performance, comparatively, at a lower BER.

BOUNDS ON PERFORMANCE
The results given in the previous section show that PCC and SCC codes have quite good performance given their decoding simplicity and short blocklengths.The reason for the performance improvement over the component SPC product codes [7] is the reduction in the number of low-weight codewords.This will be investigated by considering the inputoutput weight enumerator function (IOWEF) of the concatenated code (both serial and parallel), under the uniform interleaver assumption [2].
In the case of a PCC, it is well known [2] that where A Cp (X, Y ) is the IOWEF of the parallel concatenated code while A C1 (x, Y ) and A C2 (x, Y ) are the conditional IOWEFs (CIOWEF) 1 of the component codes.In this case, A C1 (x, Y ) = A C2 (x, Y ) since the component SPC product codes are identical.In a similar way, the IOWEF of the SCC, A Cs (X, Y ), can be written as [3] where A Co (X, k) and A Ci (k, Y ) are the CIOWEFs of the outer and inner codes, respectively, and N 1 is the length of the outer code.
Unfortunately, it is very difficult to directly calculate the IOWEF of an SPC product code with more than two dimensions.Consequently, we will introduce a lower bound on the IOWEF of the SPC product code with three or more dimensions.This bound will underestimate the weight of all but the minimum distance codewords (which are known exactly), hence we can upper bound the probability of codeword error.This lower bound on the IOWEF is given in the following theorem.Theorem 1.A lower bound on the IOWEF for an {n, d} SPC product code, A d (X, Y ), is given by Proof.By construction, a d-dimensional SPC product code consists of n − 1 independent (d − 1)-dimensional SPC product codes which are encoded in the dth dimension using SPC component codes.The parity checks in the dth dimension also form a (d − 1)-dimensional codeword due to the structure of the product code.Let 0 ≤ i ≤ n − 1 be the number of (d − 1)-dimensional product codewords with a nonzero weight.If i = 1, then the encoding of the last dimension will result in a copy of the codeword in the parity check bits, hence the codeword weight doubles (resulting in the term (n − 1)(A d−1 (X, Y 2 ) − 1)).For i = 2, the weight of the two nonzero (d − 1)-dimensional product codes is the product of two IOWEFs, resulting in (A d−1 (X, Y ) − 1) 2 .However if the two codewords are the same (which occurs exactly once for every codeword), then the checks in the dth dimension will have zero weight.Otherwise, the resulting checks form a (d − 1)-dimensional codeword with weight at least 2 d−1 .Thus in the case i = 2, we will multiply the IOWEF by Y 2 d−1 to increase the codeword weight.However we must also take into account the codewords, for i = 2, which have zero parity weight in the dth dimension.Since these patterns only occur when both (d − 1)-dimensional codewords are the same, we know that the combined IOWEF of these codewords must be While it is possible to do better than the proposed bound, (A d−1 (X, Y ) − 1) i will always lower bound the codeword weight since it does not take into account any extra weight due to the encoding of the last dimension.
This bound consistently underestimates the true weight of every codeword except those at minimum distance.This is because all minimum-distance codewords are determined by the cases i = 1 and i = 2 (since i ≥ 3 combines three or more (d − 1)-dimensional SPC product codes, each of which has minimum weight 2 d−1 ).Furthermore, the case i = 1 produces the exact IOWEF while i = 2 generates the correct number of weight 2 d codewords.In the case of a three-dimensional SPC product code, the bound is reasonably good since the IOWEF of the two-dimensional SPC product code is known exactly (see the appendix).Note that the bound becomes less accurate as the number of dimensions increases since the IOWEF from the previous dimension, A d−1 (X, Y ), must also be bounded.
The bound on the IOWEF of the SPC product codes can be used to calculate the average IOWEF of the PCC and SCC codes.This average IOWEF (over the ensemble of possible interleavers) can then be used to bound the performance of the concatenated code.Specifically, the union bound may be used: where A i j is the weight distribution.The main disadvantage of the union bound is that it diverges, that is, the bound on probability of error becomes greater than one at the cutoff rate.This is the motivation for employing the improved bounds of Duman and Salehi [12].These bounds are based upon the parameterized bounds of Gallager [13] which, for random codes, are useful at all rates up to capacity.The improved bound is shown in Figure 6 together with the union bound and the simulation results for an (8,7) three-dimensional PCC.The "combined bound" in Figure 6 is the improved bound from [12] which is always better than both the original bound of Duman and Salehi and the union bound.Clearly the simulation results converge to the bounds at high SNR, indicating that the number of minimum-distance codewords is correct, and that the suboptimal iterative decoding approaches that of maximum-likelihood (ML) decoding.

VARIANTS OF PCC AND SCC CODES
A number of variations on the standard SCC and PCC codes have been investigated.One variant involves using a randomly interleaved SPC product code [7,8] as the component code in both PCC and SCC codes.This "better" component code will improve the overall performance of the PCC or SCC at higher SNRs.Another variant involves not transmitting the checks on checks for the inner code of an SCC SPC product code.The motivation for considering this code is the good performance of an SPC product code without checks on checks at very low SNR [7].The poor minimum distance of these codes can be alleviated by serial concatenation with a standard SPC product code.

RI SPC product code component codes
Performance comparisons between the standard and randomly interleaved (RI) SPC product codes [7] show that RI SPC product codes are significantly better at higher SNRs.Specifically a four-dimensional (8, 7) RI SPC product code outperforms the equivalent SPC product code by 1.25 dB at a BER of 10 −5 .This would indicate that the performance of the PCC (or SCC) can be improved by using a randomly interleaved SPC product code.This is confirmed by simulation results at high SNR; however, at low SNR the performance is somewhat worse than the noninterleaved SPC product code PCC (see Figure 7).This behavior can be attributed to the reduction in the overall code rate of the con-   catenated code and the consequent increase in the noise variance.For instance, the four-dimensional (8, 7) PCC has code rate R = 0.4146, hence at E b /N 0 = 2 dB, the noise variance is σ 2 = 0.76.However, both the standard and randomly interleaved 4D (8, 7) SPC product codes have rate R = 0.5862, so a noise variance of σ 2 = 0.76 corresponds to E b /N 0 = 0.5 dB.The results in [7] indicate that at this signal-to-noise ratio, the performance of the standard and randomly interleaved SPC product codes is almost identical (in fact, the standard SPC product code performs marginally better).Therefore, it can be expected that at E b /N 0 = 2 dB, the performance of the PCC using either component code will be very similar.However, this does not take into account the availability of the extrinsic information to the bits in each component code.
The parity bits in the RI SPC product code are not decoded in all dimensions of the product code [7], hence extrinsic information from all dimensions is not available to the RI SPC product code (unlike the standard SPC product code).Therefore, the extra extrinsic information available on the data bits, due to the other code in the PCC, can be used indirectly by all bits in the standard SPC product code, but not by all bits in every dimension for the RI SPC product code.This is a disadvantage of the RI SPC product code at low SNR.However, at a slightly higher SNR, E b /N 0 = 2.3 dB, the inherently better performance of the RI SPC product code allows the PCC to perform better than the original code (see Figure 7).

SCCs with modified component codes
Another simple variation on the original SCC SPC product code involves not transmitting the checks on checks for the inner code.The motivation for this construction is to use the improved performance of the SPC product codes  without checks on checks at very low SNR [7] in the inner code, while maintaining a relatively large minimum distance (and hence good asymptotic performance) by using the standard SPC product code as the outer code.Figure 8 compares the performance of this modified SCC to the regular SCC SPC product code for twelve decoding iterations.As expected, the performance at low SNR is somewhat better, but the minimum distance of the code is less than the original SCC SPC product code, hence the performance at high SNR is slightly worse.The blocklength is slightly less than that of the original SCC SPC code, N = (n − 1) d + d(n − 1) d−1 , and the code rate is slightly higher at R = K/N, where K = (n − 2) d .

CONCLUSIONS
Parallel and serial concatenated SPC product codes are very simple to encode and decode.Furthermore, their performance is reasonably good when compared to the spherepacking bound.Three-dimensional PCCs, in particular, have an advantageous tradeoff between blocklength, code rate, and performance.An interesting characteristic of the PCC and SCC codes is the reduction in the number of low-weight codewords, compared to the original product codes.This is confirmed by bounds on the average IOWEF of the PCC and SCC.The performance results converge to the union bound for an (8, 7) three-dimensional PCC, indicating that iterative decoding approaches ML decoding at high SNR.Variants of PCC and SCC codes show that performance can be improved at low SNR by not transmitting the checks on checks for the inner code.Furthermore, improved performance at high SNR can be achieved by using randomly interleaved SPC product codes as the component codes.

APPENDIX IOWEF OF A 2D SPC PRODUCT CODE
The IOWEF of a two-dimensional SPC product code can be calculated directly by enumerating all the possible codewords if n ≤ 6, which corresponds to M ≤ 2 25 codewords.Unfortunately, as the number of information bits, K = (n − 1) 2 , increases, this calculation becomes impractical.The solution to this problem is to calculate the IOWEF of the dual of the SPC code, and then apply a dual identity to obtain the IOWEF of the code.
The WEF of the two-dimensional dual SPC product code is given in [14] by The summation in (A.1) is equivalent to generating the WEF from a nonsystematic construction of the dual code.Specifically, the nonsystematic parity check matrix, H, is defined by Note that the outer sum of (A.1) is equivalent to generating all possible combinations of the top n rows of (A.2), while the inner sum considers all combinations of the remaining n − 1 rows at the bottom of the matrix.Unfortunately, the IOWEF cannot be generated from this form of the parity check matrix, but a minor modification gives a systematic matrix, H sys , which can be used to find the IOWEF.The matrices H and H sys are related by where   Recall that the dual codewords are generated by c ⊥ = mH = mPH sys , so we can assert that (A.1) produces the codeword weight corresponding to an input weight of i + j provided the nth bit of the message vector is not used (since P is an identity matrix except for the nth row).Furthermore, if the nth bit is used, the input weight of the last n−1 bits is inverted (to produce the same codeword weight as given by (A.1)).Therefore, the two-dimensional dual SPC product code has an IOWEF (in codeword form) given by n−1 j X i+n−1− j Y in+ jn−2i j . (A.5) Writing the IOWEF (in parity form) as a homogeneous function [15] gives × Y in+ jn−2i j−i−n+1+ j Z N−K−in− jn+2i j+i+n−1− j , (A.6) where K = n 2 − (n − 1) 2 , N − K = (n − 1) 2 (for the dual code), and the exponents of X and Y represent the data and parity weights, respectively.Now we need to find a MacWilliams-type identity relating the IOWEF of the dual to that of the code.Note that the IOWEF of a code C is defined, in a homogeneous parity form, as where the vectors a and b represent the data and parity bits, respectively, of the codeword c ∈ C. Now using the coordinate partition which splits the data and parity bits of the code and applying the result in [16], we obtain, after some algebraic manipulation, the following dual identity: In many cases, the coefficients A i j are desired, so it may be more convenient to expand (A.6) to find the dual coefficients, A ⊥ lm , and use the method A ⊥ lm P i (l; N − K)P j (m; K), (A.9) where P b (a; c) are the Krawtchouk polynomials:

Figure 1 :
Figure 1: PCC and SCC SPC product encoders.(a) Two-branch PCC SPC product encoder.(b) Encoder for a serially concatenated two-stage SPC product code.

Figure 2 :
Figure 2: Iterative decoders for both the PCC and SCC.(a) PCC SPC product decoder.(b) SCC SPC product decoder.

Figure 3 :
Figure3: BER performance and P e , the probability of codeword error over the entire concatenated codeword, for a 3D (8, 7) PCC SPC product code with 1 to 8 decoding iterations.

Figure 7 :
Figure 7: Performance of a PCC with component 4D (8, 7) RI SPC product codes compared to the PCC simulation results with component 4D (8, 7) SPC product codes, for 1 to 12 decoding iterations.

Figure 8 :
Figure 8: Performance of a 4D (8, 7) SCC without checks on checks for the inner code compared to a standard 4D (8, 7) SCC SPC product code, for 1 to 12 decoding iterations.