New Technique of Near Maximum Likelihood Detection Processes

This paper introduces new detector named newly adaptively designed near maximum likelihood detector (NADD). This detector combines adaptively three types of near maximum likelihood detectors, pseudobinary, pseudoquaternary, and pseudooctonary. The performances of NADD, pseudobinary, pseudoquaternary, and pseudooctonary detectors are measured using data transmission at 9.6 kb/s over telephone channel. Simulation results show that the performance of NADD is better than the performances of pseudobinary, and pseudoquaternary detectors, but little bit worse than the performance of pseudooctonary detector.


Introduction
The Maximum Likelihood Sequence Detector(MLSD) is a procedure for estimating a sequence of bits from a sequence of channel output observables, given a model of the communication system. In the presence of Intersymbol Interference(ISI), the Viterbi algorithm(VA) provides an efficient way of computing the MLSD [1,2]. However, the VA still becomes impractical when the time spread of the ISI is large because of the exponential relation between ISI time spread and VA complexity.
Another way which is considered in this paper is to use detectors called Near Maximum Likelihood Detectors [19][20][21][22][23][24][25][26]. These detectors operate similarly to Viterbi algorithm, but using different selection process for the stored sequences of possible data symbol values, and only a very few of these sequences are stored with the corresponding costs. 1 Data transmission system Fig. 1 shows the model of data transmission system. The first part in this model is random data generator which generates binary data, and each 4-bit is mapped into one of 16-point QAM constellation. Thus, the output of random data generator is data symbols { }, and the possible values of are given by all combination of ±1, Then, the data symbols { } enter the Quadrature Amplitude Modulation(QAM) transmitter which consists of transmitter filter and QAM modulator. The transmitter filter is a low-pass filter performs the function of limiting the signal spectrum before modulation process. The resulting output of the QAM transmitter is QAM signal with carrier frequency of 1800 Hz and symbol rate of 2400 baud giving an information rate of 2400 × 4 bits = 9600 b/s. The output of the QAM transmitter passes through telephone channel, and Additive White Gaussian Noise (AWGN) added to the signal before entering the QAM receiver. The QAM receiver consists of QAM demodulator and receiver filter. The receiver filter is a low-pass filter used in combination with the transmitter filter to produce realistic levels of intersymbol interference. The output of QAM receiver is data symbols { } used by the Least Mean Square (LMS) estimator to estimate the sampled impulse response (SIR) of baseband telephone channel. Finally, the data symbols { } and SIR are used by the detector to obtain the detected symbols { ′ }. The pseudobinary detector (BD), pseudoquaternary detector (QD) [20], and pseudooctonary detector (OD) [24] are described as follows.
Just prior to the receipt of the signal sample at time = , the detector holds in store ( -component) vectors −1 given below ( = 2 for pseudobinary, = 4 for pseudoquaternary and = 8 for pseudooctonary), where is possible value of . Each stored vector is associated with a cost −1 given by where { ℎ } is the sampled impulse response of baseband telephone channel (estimated by LMS estimator) having length of ( + 1) (where < ), and −1 is the corresponding estimate of the noise component in the received sample −1 .
On the receipt of the signal sample , the detector expands every vector −1 into four ( + 1) components vectors { }, given below, having smallest cost. The selection of is achieved through the use of simple threshold-level comparison and does not involve the computation of any costs.
The first -components of are as shown in the original vector −1 and the last component takes on any one of the four different values of its 16 possible values. In pseudobinary, the number of expanded vectors is 8 (see Fig. 2), in pseudoquaternary is 16 (see Fig. 3), while, in pseudooctonary is 32 (see Fig. 4). Then the detector evaluates for each expanded vector its cost given by The detector then selects the vector with the smallest cost and takes its first component − as the detected value ′ − of the data symbol − . All vectors { } for which ′ − ̸ = − are discarded, and the first components of all remaining vectors are omitted to give the corresponding -component vectors

Fig. 2. Configuration of pseudobinary detector
After each detection process, and to prevent overflow due to the increase in costs over any transmission, the smallest cost is subtracted from the cost of each vector, so that the value of the smallest cost is always reduced to zero.
The starting up procedure for the detector begins with k stored vectors { −1 } that are all the same and correct. A zero cost is allocated to one of the k vectors and a very high cost to each of the remaining vectors. After a few received samples, the detector holds vectors which are all different and are all derived from the original vector with zero cost. This newly adaptively designed detector (NADD) combines adaptively BD, QD, and OD. The value of changes between 2, 4, and 8 which is assigned every time a signal sample is received (see Fig. 5). One criteria of assigning the value of is given below, The above criterion assigns the value of depending on comparison between the current minimum value of and the average or half of the average of previous three minimum values of . When the value of ( ) increases, more vectors are needed to be evaluated, so increases too, and vice versa.  before subtracting the cost of any vector from the smallest cost.
The starting up procedure begins with = 8 vectors that are all the same and correct. A zero cost is allocated to one of the eight vectors and a very high cost to each of the remaining seven. The initial values of ( −3 ) , , and ( −1 ) are zero.

Simulation results
A series of computer simulation tests have been carried out on the system in Fig. 1 with four types of detectors, BD, QD, OD, and NADD to determine their relative tolerance to AWGN when operating over telephone channel.
The performance of the whole system is measured by drawing symbol error rate (SER) versus signal-to-noise ratio (SNR). The SER is given by where NEDS is the number of erroneous detected samples & NTS is the number of total transmitted samples. Fig. 6 shows comparison among the four detectors. It seems that at error rate of 10 −5 ,the performance of NADD is better than the performance of BD by approximately 0.6 dB, and better than the performance of QD by approximately 0.2 dB , but worse than the performance of OD by approximately 0.1 dB. Fig. 6. Error rate perfomance Conclusion A new detector was developed to mitigate the ISI introduced by the communication channel. This detector which is named NADD combines adaptively three detectors BD, QD, and OD. So, the three detectors can be replaced by one detector which leads to reduce the complexity of whole detector model. Simulation results show that the performance of NADD is better than that for BD and QD but little bit worse than that for OD.