Residual Compensation-Based Extreme Learning Machine for MIMO-NOMA Receiver

Extreme learning machine (ELM) uses a simple machine learning (ML) architecture that allows its implementation for smart IoT network operations. However, some inaccuracy lies between the predicted and the actual data during the ELM training, which can be caused due to the limitation of the modeling representation. This paper thus investigates a residual compensation-based ELM (RC-ELM) for its application in designing a receiver for MIMO-NOMA aided IoT systems. In RC-ELM, the base ELM layers determine the relationship between the transmitted and received data and the additional layers of the RC-ELM attempt to compensate the error introduced during the training mechanism. The analysis of the appropriate number of compensation layers for training error minimization is conducted on the basis of the bit error rate (BER) and the error vector magnitude (EVM) performances of the RC-ELM training. A minimum BER improvement of 5% for user 1 and 18% for user 2 is shown with with the aid of RC-ELM for a two-IoT user instance. The EVM is marginally increased by 0.0008% for user 1 and 1.61% for user 2 in training stage. Besides, the RC-ELM receiver is also compared to the minimum mean square error (MMSE), the classic ELM and the trained multilayer perceptron (MLP) receivers in terms of BER and EVM. Both of the ELM receivers show improved performances with respect to the other receivers.


I. INTRODUCTION
The deployment of 5G/B5G has created the opportunity to realize Internet of Things (IoT) into reality. Since the popularity of the internet is ever increasing, the growth of the number of communication devices is escalating exponentially. Thus, multiple access technology is being brought into attention for massive access provision to IoT devices. However, accommodating large number of smart IoT devices within a given bandwidth and ensuring QoS parameters such as low latency and high throughput at the same time can be a challenging task.
The associate editor coordinating the review of this manuscript and approving it for publication was Bilal Khawaja .
Non-orthogonal multiple access (NOMA) is seen a promising enabling technology to address this issue [1]. In orthogonal multiple access (OMA), the users are allocated with orthogonal time/frequency resources. On the contrary, in NOMA, the users are allocated with same frequency resources at the same time. Thus, massive accessibility with increased throughput and lower latency is ensured. By combining NOMA with multiple-input multiple-output (MIMO), enhanced spectral efficiency is also achieved. MIMO-NOMA has thus received much interest in research domain and industrial applications to enhance the IoT network performance.
Generally, power domain NOMA is adopted for multiple access provision. Multiple users with different power allocations are accommodated in same time and frequency resources. Later, the signals from multiple sources are superimposed and transmitted to the desired destination. However, this leads to the inter-user interference problem since the time and frequency resources utilized by the users are non-orthogonal, and consequently complicates the extraction of the required information from the received signal. Thus, several studies are being conducted on MIMO-NOMA signal processing, which includes channel estimation, signal detection and interference cancellation operations. In [2], a zero-forcing (ZF) beamforming is studied for multi-user downlink MIMO-NOMA system where each cluster contains two single-antenna user devices. Channel state information (CSI) of each cluster-head is extracted for the ZF beamforming design. Maximum-likelihood based receiver is utilized in [3] for uplink MIMO-NOMA to detect quadrature phase shift keying (QPSK) modulated symbols. A precoding scheme, which focuses on minimization of transmission power consumption, is suggested in [4] where the transmitted signal is recovered with the help of the precoder and the minimum mean square error (MMSE) equalizer. An iterative linear MMSE (LMMSE) is designed in [5] multiuser detection in MIMO-NOMA system. The mismatch between LMMSE detection and decoding is attempted to remove in the study for better signal recovery and achievable rate is then analyzed. The performance of these approaches depends on the accuracy of the channel estimation. Besides, the inter-user interference mitigation approaches required in MIMO-NOMA also relies on channel information-based techniques [6]. However, parameters such as auto-covariance of the channel and noise variance are challenging to obtain in practical communication systems [7]. One possible approach can be the adoption of machine learning (ML) techniques to conduct such signal processing mechanisms [8].
Several studies on ML algorithm such as deep neural network (DNN)-based approaches are conducted for MIMO-NOMA systems to detect signals [9], [10], [11]. DNN based precoding and decoding approaches are proposed in [9] for MIMO-NOMA system with successive interference cancellation (SIC) scheme. Uplink signal detection for MIMO-NOMA is performed by means of convolutional neural networks (CNN) in [10]. The iterative soft-thresholding technique is the foundation of the adaptive DL-based receiver architecture known as MMNet [11]. The performance of the MMNet receiver is evaluated for massive MIMO (mMIMO) channel with spatial correlation. However, the tuning of DNN parameters such as weights and biases are performed by means of iterative mechanisms, which may complicate the implementation of such DNN algorithms. Contrary to the tuning mechanism-based ML algorithms, extreme learning machine (ELM) uses randomly assigned input weights and biases (that connect between the input layer and the hidden layer) and perform supervised learning with the help of training datasets by determining the output weights (connecting the hidden layer and the output layer), which is responsible for determining the desired output. In [12] and [13], the ELM-based receiver systems are proposed which do not require any channel information. However, due to the limitation of modeling representation, some training inaccuracy may be introduced during the learning stage as ML approaches, including ELM, might not always reach the global minima with the given ML parameters. Thus regularization technique is applied to ensure that the ML algorithms perform well when generalized. A residual compensation-based ELM (RC-ELM) is proposed for building the MIMO-NOMA receiver in order to reduce training error. To our knowledge, this is the first time an RC-ELM has been researched in relation to creating a MIMO-NOMA receiver. The following highlights the paper's contribution.
• An RC-ELM-based receiver is presented for a multiuser MIMO-NOMA aided IoT system. The base ELM layers are used to map the transmitted and received data, whereas the additional layers are used to compensate the error introduced during training. A bit error rate (BER) analysis of choosing the effective number of compensation layers to mitigate training error by means of the RC-ELM receiver is conducted.
• The performance of the proposed ELM receiver is contrasted with that of the MMSE, multilayer perceptron (MLP) and classic ELM receivers. It is demonstrated that both of the ELM receivers provides better BER than the MMSE and the MLP receivers.
The paper is organized as follows. Section II discusses the MIMO-NOMA system model, which includes data transmission and reception, channel estimation and signal detection with linear receiver. Section III discusses the basics of ELM-based receiver and section IV discusses the RC-ELM receiver. Section V presents the simulation parameters and discussions on the results. Section VI concludes the paper.
Notations: Matrix is represented by the bold uppercase letters, vector is represented by bold lowercase letters and scalar variables are represented by small letters. The inverse, transpose, conjugate transpose and Moore-Penrose inverse operations are denoted by (.) −1 , (.) T , (.) * and (.) † respectively. The matrix inverse operation with regularization parameter is represented by( .). Activation function is represented by G(.). The Frobenius norm of a matrix is denoted by ||.|| F .

II. SYSTEM MODEL A. MIMO-NOMA DATA TRANSMISSION AND RECEPTION
A visual representation of the downlink MIMO-NOMA system is presented in Fig. 1. The source data bits for the IoT user k are digitally modulated. Afterwards, inverse discrete Fourier transformation (IDFT) of the data symbols of each IoT user is carried out, followed by cyclic prefix (CP) addition. Each user is allocated with power p k and multiplied with the respective signals. Later, the respective signals are added. After that, the superposed data transferred to the receiver through wireless channel [14]. After receiving the superposed data by each IoT user, CP is removed and discrete Fourier transformation (DFT) is carried out. The received DFT symbols can be mathematically expressed as shown in (1).
Here, Y k ∈ C N R ×N is the received symbol matrix by the user k, N R is the number of receiving antennas and N is the number of subcarriers.
H k , H k follows Rayleigh distribution and d k is the distance between the BS and the user k. X ∈ C N T ×N is the superposed signal transmitted by the BS with N T transmitting antennas and X = K k=1 √ p k X k , where p k is the power allocated to the user k. Z k is the added noise with power σ 2 Z k .

B. CHANNEL ESTIMATION AND EQUALIZATION WITH LINEAR RECEIVER
Channel estimation is performed by means of transmitted and received reference symbols. If the transmitted and received reference symbols are represented by X γ and Y γ ,k , then according to (1), Generally, channel is estimated in practical communication network by means of LS estimation method due to its computational simplicity [7]. The LS estimation of the channelF k is determined as follows [15].
A Bayesian-based MMSE estimation method can be used to reduce the average mean square error (MSE) of the LS estimation of the channel [16], [17].
Here, σ 2 is the estimated channel variance. Since, MMSE approach provides better estimation of the channel, F k,MMSE is chosen for performing the equalization operation. The MMSE equalization is expressed as follows. The user with highest power can directly decode its own signal and the user with low power needs to perform SIC operation. Each user receives their desired data bits after executing digital demodulation.

III. CLASSIC EXTREME LEARNING MACHINE (ELM) FOR MIMO-NOMA SYSTEM
The classic ELM-based receiver for the MIMO-NOMA system is presented in this section. Instead of tuning the input weights by back-propagation algorithms, the input weights as well as the biases, that connect the input layer and the hidden layer, are randomly chosen. The output weights, that connect the hidden layer and the output layer, are determined in the training stage of ELM. Keeping the input weights and biases unchanged in testing stage, the received data symbols are processed to obtain the transmitted data.
The working principle of the ELM algorithm is demonstrated in Fig. 2. Training of ELM algorithm is required prior to its implementation, which is performed with the help of transmitted-received reference symbol pairs. At first, the received reference symbols Y γ ,k are fed into the input layer, with input neurons is equal to N R . The hidden layer output M γ ,k ∈ C L×N is determined with the help of the activation function G(.). The output layer neuron weight parameter β 0,k ∈ C L×N T is then determined from the transmitted reference symbols X γ and M γ ,k . The following mathematical equations state the training mechanism of ELM.
In the equation (6) and (7), subscript (.) γ refers to terms associated with reference signals. W k ∈ C L×N R and b k ∈ C L×1 are input weight and bias parameters respectively which are randomly chosen and kept constant once assigned. The symbol α, in (7), is the regularization parameter and used to solve the least square problem β 0,k = argmin β 0,k ||M γ ,0,k β 0,k − X γ || [18].
Symbol detection is executed in testing stage. The hidden layer output M 0,k is computed with the received data symbols Y k as shown in (8). The detected symbols are obtained  from (9).
Here,X k,ELM denotes the ELM detected symbols. It is to be noted that detection by means of ELM algorithm takes place without determining the channel matrix directly. It directly removes the channel effect from the received data by performing online training with the help of the transmitted and the received reference symbols.

IV. RESIDUAL COMPENSATION-BASED ELM (RC-ELM) RECEIVER FOR MIMO-NOMA
Though the classic ELM is an effective ML algorithm for regression or classification, some error may exist in the training of the algorithm. Suppose, X is the actual data andX k,ELM is the predicted symbols with the help of the ELM receiver. The estimation error can then be expressed as follows.  (6) Determination of output weight β 0,k from (7) Computation ofÊ γ ,s from (15) and τ s from (17) (8) Determination ofX k,ELM from (9) Determination ofX k,RC−ELM from (18) The first compensation layer operation will always be performed according to (11) -(12).
In order to reduce this error occurred in the training stage, the RC-ELM receiver is presented in this paper. In RC-ELM [19], additional components (which are known as compensation layers) are added to the classic ELM layers, as shown in Fig. 3. In the input of each compensation layer, Y γ ,k is provided as input. The reference symbols Y γ ,k , if were used in (8), then the expression in (9) would provide an estimated outputX γ . According to (10), the error between the actual reference symbols and the ELM equalized symbols could be written as E γ ,1 represents the training error. The first compensation layer performs the estimation of E γ ,1 according to the VOLUME 11, 2023 following equation.
Here, E γ ,1 is the target output in the first compensation layer. The first layer prediction inaccuracy E γ ,2 is then determined by In the second compensation layer, considering E γ ,2 as the desired output, the RC-ELM training is executed as follows.
In the testing stage, the estimated errors and the compensation layer weights are used. After performing equalization VOLUME 11, 2023 according to (9), the final output by means of RC-ELM for the IoT user k can be represented as follows.
The mechanism of the RC-ELM receiver is summarized in Algorithm 1.

V. SIMULATION RESULTS
This section presents the simulation results for MIMO-NOMA communication system operating at 30 GHz carrier frequency. The user 1 and user 2 are respectively at the distances 20 m and 10 m apart from the BS. The transmission power is considered to be 30 dBm and the values of p 1 and p 2 for user 1 and user 2 are respectively 0.92 and 0.08. Normalized 16-quadrature amplitude modulation (16-QAM) is chosen in our study. Transmitted data are organized in physical resource block (PRB) arrangement [12]. The transmitted and received reference symbols at one OFDM symbol time are considered for channel estimation as well as the online training of both of the ELM receivers and the MLP receiver. The parameters of simulation are stated in Table 1.
To design both the ELM receivers, activation function tanh is used [12], [13]. The input weights W and biases b follow uniform distribution within the interval [-0.01,0.01]. The input layer in both the ELM receivers consists of 2 neurons (each neuron takes the symbols received at each antenna) and the output layer consists of 4 neurons (each neuron takes the symbols transmitted from each antenna). Fig. 4 demonstrates the BER obtained by adopting the classic ELM-based receiver with different hidden layer neurons, which are L 1 = 2, L 2 = 4 and L 3 = 8. It is observed that the BER for L 2 and L 3 is superior to that for L 1 . Besides, the difference in BER values for L 2 and L 3 is almost negligible in all SNR values. For simplicity, L 2 = 4 is chosen in this study for both of the classic ELM and the RC-ELM receivers.  Fig. 5 demonstrates the BER obtained by the classic ELM receiver and the RC-ELM receiver in training stage with 2, 3 and 4 compensation layers. As the regularization parameter α varies from 10 −2 to 10 −5 , the BER performance improves with the increase of SNR. The RC-ELM receiver is able to provide lower BER than the classic ELM. The RC-ELM with 2 compensation layers shows lower BER with respect to the RC-ELM with 3 and 4 compensation layers. In Fig. 5d, the BER values of the classic ELM receiver is very similar to that of the RC-based receiver, although marginal improvement is observed in high SNR values. For instance, at SNR = 35 dB, compared to the classic ELM receiver, the RC-ELM provides about 5% lower BER for user 1 and about 18% lower BER for user 2. Table 2 presents the EVM performance (determined on the basis of the Euclidean distance between the equalized symbols and the reference constellation symbols) of the classic ELM and the RC-ELM receivers with training symbols. Overall, RC-ELM with 2 compensation layers shows descent performance with respect to the classic ELM receiver as well as the RC-ELM receivers with 3 and 4 compensation layers. For user 1, compared to the classic ELM, the EVM performance improvement with RC-ELM with 2 compensation layers, within 25 − 30 dB SNR, for α = 10 2 , 10 3 , 10 4 and 10 5 are observed to be 4.7%, 0.34%, 0.01% and 0.0008% respectively. For user 2, the EVM performance, for α = 10 2 , 10 3 , 10 4 and 10 5 , is improved by 9.2%, 13.5%, 0.28% and 1.61% within the same SNR values. Based on the BER and the EVM performances observed with the reference symbols, the number of compensation layers in RC-ELM in testing stage is chosen to be 2.
An MLP-based receiver is also implemented in this study for performance comparison [20]. A three hidden layer incorporated-MLP algorithm is trained online in order to detect the real and imaginary part of the received symbols. For each part, the number of neurons in each layer is 4, 8 and 6 respectively. Similar to the ELM receivers, the number of input layer and output layer is respectively chosen to be equal to 2 and 4. Activation function tanh is also considered for the MLP receiver at each hidden layer and the weight parameters are updated with the learning rate of 0.005 and momentum of 0.8. Fig. 6 presents the BER obtained by the MMSE, the trained classic and the RC-ELMs as well as the MLP receivers. For all the values of α, both of the ELM receivers provide better performance than the MLP receivers. For α = 10 −5 , the BER performances by both of the ELM receivers are almost identical. Even the ELM receivers show improved performance against the MMSE receiver. At the BER value of about 10 −2 , about 2.5 dB and 5 dB gains are observed by the ELM receivers (with respect to the MMSE receiver) for user 1 and user 2 respectively. Table 3 presents the EVM performance of the MMSE and the ML algorithms (in testing stage). The ELM receivers shows better performance for α = 10 −5 .
The accuracy of the channel estimation determines the inter-user interference mitigation and as a result, the recovery of the signal in NOMA-based communication networks. Thus, any error in the channel estimation will affect the overall signal recovery process by means of the linear receiver, since such receiver utilize the estimated channel information. On the other hand, ELM receiver does not require to directly estimate channel. It rather uses the reference symbols to perform signal detection and minimize the interference. Contrary to the complicated structure of the MLP algorithm-based approach, ELM does not require tuning of input weights and biases, which simplifies its application. The addition of the compensation layers in ELM for RC-ELM receiver may increase complications. However, it is able to minimize the error caused in the training of the ELM algorithm, which is reflected by the BER and the EVM analyses in training stage.

VI. CONCLUSION
An RC-ELM based receiver is studied in this paper for designing a receiver for the MIMO-NOMA aided IoT system. The proposed receiver uses the classic ELM algorithm for equalizing the symbols and the additional compensation layers are used to minimize the errors introduced in the classic ELM training. The BER and the EVM analyses of both of the ELM receivers in training stage is conducted. The performance of the RC-ELM is also compared to the MMSE, classic ELM and MLP receivers. In terms of BER and EVM, It is shown that both of the ELM receivers have shown improved performance against the MMSE and the MLP receivers. Furthermore, it is shown that the RC-ELM receiver training is more reliable than the conventional ELM training. The automatic estimation of the RC-compensation ELM's layer count in order to enhance the performance of its input-output mapping could be a future research target using RC-ELM.  His research interests include the broad area of digital communications systems and signal processing, with emphasis on wireless communications, optical wireless communications, wireless power transfer and applications and communications, and signal processing for biomedical engineering. He has been involved as the general chair, the technical program chair, and a member of technical program committees in several IEEE and non-IEEE conferences. In the past, he was an editor of several IEEE journals. From 2012 to 2015, he was the Editor-in-Chief of IEEE COMMUNICATIONS LETTERS. Currently, he serves as the Associate Editor-in-Chief for IEEE OPEN JOURNAL OF THE COMMUNICATIONS SOCIETY. He is one of the highly-cited authors across all areas of electrical engineering, recognized from Clarivate Analytics as a Web of Science Highly-Cited Researcher in the seven consecutive years (2015-2021). VOLUME 11, 2023