Forecasting the Exchange Rate for USD to RMB using RNN and SVM

One of the most important mechanisms supporting world trade is the exchange rate. Depreciation or appreciation of any currency, especially those main currencies such as US dollar, Pound sterling, Renminbi, could greatly affect international trade leading to greater impact to businesses and people’s wellbeing. Recently, researchers have been exploring the use of machine learning techniques to forecast time series data in the financial area. This paper will use machine learning techniques, namely Recurrent Neural Network (RNN), Support Vector Machine (SVM), and a traditional model, namely the ARIMA model which is selected as a benchmark. The result shows that RNN has the best performance compared with both SVM and ARIMA. This paper aims to forecast the exchange rate for USD to RMB, which could give the involved country, institution or people the foresight of the situation and prepare for risk.


Introduction
As a monetary instrument to reinforce foreign markets, the exchange rate plays an important role in international trade, the balance of international payment, monetary policy, economic growth and so on. In 2018, China's total trade value reached 4.62 trillion US dollars, therefore, it cannot be neglected that almost all transactions between world currencies are executed in dollars. In an era of information explosion, big news has the potential to break a relatively stable foreign exchange market, however, the emerging media indicators to explain the exchange rate forecasting is rare.
Traditional technical analysis in exchange rates is using a linear method which forecasts the future trends of exchange rates by analyzing historical data, however, these methods show unpredictable results in the real economic environment. It is worth to consider how to promote the rise of nonparametric exchange rate forecasting methods to reflect the nonlinear characteristics of exchange rate changes; further, how to investigate the hidden relationship within the historical data without imposing any fixed relationships. Simon et al. [1] who triggered new thinking to solve the above problems by finding that economics and Artificial Intelligence are similar in "decision making and problem solving". With the rapid development of artificial intelligence technology and computer science, many economists have joined in the research of artificial intelligence. Hinton et al. [2] put forward the concept of deep learning to solve the training problem in the neural network.
After solving many hidden layers in the training problem, the development and application of the improved neural network model are booming, with the characteristics of self-learning, random simulation and association recognition; it is a non-linear dynamic system. Laptev [3] suggests that the  [5], which is based on the principle of neuron propagation in biology, and completes the construction of the network through a certain amount of data, and finally completes the prediction of dependent variables by inputting independent variables. Guresen [6] found that Artificial Neural Network (ANN) has a good forecasting ability to time series data by a panel data of the exchange rates. Researchers will not be satisfied with only one model, more and more machine learning techniques are employed to financial series forecasting. For example, the original formulation of Support Vector Machines (SVM) is for binary classification problems, SVM have been extensions to financial series forecasting [7] [8], Ho et al. [9] compared the traditional econometric model and new technic, by comparing RNN and ARIMA model, and found that RNN beats ARIMA model in time series prediction. All of the above researchers found that compared with the traditional linear prediction model, the neural network has higher accuracy [10], and it can determine the influence of independent variables on the network through training error and generalization error [11].
The main contributions of this paper are as follows; firstly, to compare the performance of RNN, SVM, and ARIMA model to forecast the exchange rate. Secondly, we use the Baidu index as a new media index, which shows the people timely attention. The structure of this paper is divided into four parts. Section 1 is background and reviews previous research. Section 2 focuses on the construction of RNN and SVM. Section 3 data selection. Section 4 compares and evaluates the empirical results. Section 5 summarizes.

Methodology
The paper will employ Recurrent Neural Networks (RNN) and Support Vector Machine (SVM), to improve the Renminbi to US dollar exchange rate forecasting, and ARIMA was employed as a benchmark.

machine learning
Machine learning (ML) can improve the economic problem of prediction and classification performance by learning and training historical data. A neural network (ANN) is a machine learning algorithm based on the model of a human neuron, which can be generalized and does not make any assumption about the underlying distribution of data.

Recurrent neural networks
Recurrent neural networks (RNN) are a type of artificial neural network designed to recognize patterns in sequences of data, it can condition its predictions on multivariate input series together with scalar inputs which makes it flexible enough to incorporate multiple data sources. Figure 1 gives the construction of the RNN. The inputs and outputs are one fixed vector in NN, while RNNs are bidirectional; the subsequent input can learn from the feedback of the output in any process which means it detects patterns in the input sequence and learns when they will probably reoccur, the results are improved by the recurrent that output may become a new input that enables the neural network to train and learn in the forecast process.
The recurrent neural networks can forecast the future values of exchange rate depending on the values generated at the preceding, the nonlinear autoregressive external (Exogenous) inputs express as follows, where, y(t) and x(t) are the exchange rate outputs and inputs at the t times instant and d are the number of time steps for which the current output is regressed on the output and input respectively. To find the best architecture of the RNN, the forecasts are also completed with differing combinations of layers and neurons per layers. Figure2 SVM to find maximum the margin.

Support vector machine
Support Vector Machines (SVMs) are a set of supervised learning methods to find a separate plane, which maximuise the margin, aims to classify, regress and predict in financial research. Figure 2 shows the imagination of Maximum-margin hyperplane and margins for an SVM trained with samples from two classes. Samples on the margin are called the support vectors.
The high-dimensional version maximum margin hyperplane can be represented as the following equation in terms of the support vectors: [ where is a constant, are weights that determine the hyperplane weight of are the attribute values for any 1,2, … , . In order to find the optimally hyperplane, we minimized the distance (error function) as follows:

Forecasting performance
We employ three algorithms to calculate the MSE.

Valuate the training performance.
The performance function used for the neural identifier under consideration is the mean square error (MSE), given by where n is the sample size.
Where is the sample size.

Training of the neural network.
There are three methods to the algorithms, namely, Levenberg-Marquardt algorithm (LM), Scaled Conjugate Gradient algorithm (SCG), and Bayesian Regularization (BR). The Levenberg-Marquardt algorithm is designed to find the lowest error value of loss function taking the form of squared errors. SCG is a supervised learning algorithm for feedforward neural networks, and is a member of the class of conjugate gradient methods. SCG has a faster convergence than the other algorithms which use standard backpropagation. The Bayesian framework for neural networks is based on the probabilistic interpretation of network parameters. The process minimizes a combination of squared errors, and weights, and then determines the combination so as to produce a network that generalizes well which is called Bayesian regularization. In this study, underfitting is avoided by performing several numbers of runs on the model and selecting the one with the best performance metric, i.e. lowest MSE. Overfitting is avoided by using regularization techniques like Bayesian regularization technique. Selected factors are given as inputs to RNN to forecast exchange rate the USD to RMB. Data consists of 156 observations, which can be divided into three groups. The first group consists of 70% of observations for training. The second group of 15% is for validation. The rest 15% is in the last group which is used for testing. In order to run the RNN process, "NNSTART" package in MATLAB is employed.

Training Process
To find the best artificial RNN, a process of comparing three different algorithms, and different numbers of hidden neurons and layers, is employed. Table 1 relatively shows the comparison of the three algorithms as discussed in the methodology for the next 23-month exchange rate prediction while setting different numbers of hidden neurons and layers. The optimum number of neurons is obtained by varying the numbers and selecting the one with the lowest MSE. The result of MSE in different algorithms shows that the BR algorithm with 6 hidden neurons and 4 layers in RNN has the lowest value of 0.0011 which is considered as the best performance. Meanwhile, the lowest value of the MSE that belongs to SCG is 0.00357 and the highest one of 0.0665 followed by 0.00523 belong to LM and BR, respectively. The MSE of SVM is 0.01, which is higher than the lowest result in RNN. We also compared RNN, SVM with the traditional econometrics model, namely, ARIMA, the result shows that ARIMA (1,2,1) is best fitted to our data resulting from the lowest AIC (-315.48). The MSE of ARIMA (1,2,1) is 0.211 which is higher than all of RNN with different algorithms, different numbers of hidden neurons and delays. The lower MSE indicates a better performance in forecasting. Therefore, the BR algorithm with 6 hidden neurons and 4 layers in RNN has the best performance. Figure 3 shows the actual and predicted values of the exchange rate for USD to RMB by using an algorithm for RNN, which canobserve that the performance of output value is good. Figure 4 shows the performance plots of BR algorithm for recurrent networks, the training performance for RNN is 0.0011 at epoch 198, also provide the minimum MSE for the RNN. Figure 5 is the performance and regression plots of the BR algorithm for RNN. Figure 6 shows the training state of Recurrent network using BR algorithm, the validation checks is 0 at epoch 198. Figure 8 shows the error histogram plots of the RNN using BR algorithm. The blue color in the histogram plots indicates training, orange color indicates testing and the thin orange line represents zero error. The error is measured by the difference  Figure 7 that the error for RNN by the BR algorithm is around 0.00517. Figure 8 gives a comparison between actual and SVM predicted values of the exchange rate for USD to RMB. ARIMA (1,2,1) is the best fit for the data. The MSE of ARIMA is 0.211 which is higher than the MSE of RNN models.

Conclusion
Traditional theoretical models not only are not a good description of the randomness and volatility of market changes, but they also do not consider the impact of emerging network factors such as online public opinion and media orientation on exchange rate fluctuations. Nowadays, the widely used deep learning method has a good effect on the learning and prediction of time series data. This work analyses RNN in RMB to USD exchange rate forecasting by comparing three algorithms, namely, Levenberg-Marquardt algorithm (LM), Scaled Conjugate Gradient algorithm (SCG), and Bayesian