A Survey of Forex and Stock Price Prediction Using Deep Learning

The prediction of stock and foreign exchange (Forex) had always been a hot and profitable area of study. Deep learning application had proven to yields better accuracy and return in the field of financial prediction and forecasting. In this survey we selected papers from the DBLP database for comparison and analysis. We classified papers according to different deep learning methods, which included: Convolutional neural network (CNN), Long Short-Term Memory (LSTM), Deep neural network (DNN), Recurrent Neural Network (RNN), Reinforcement Learning, and other deep learning methods such as HAN, NLP, and Wavenet. Furthermore, this paper reviewed the dataset, variable, model, and results of each article. The survey presented the results through the most used performance metrics: RMSE, MAPE, MAE, MSE, accuracy, Sharpe ratio, and return rate. We identified that recent models that combined LSTM with other methods, for example, DNN, are widely researched. Reinforcement learning and other deep learning method yielded great returns and performances. We conclude that in recent years the trend of using deep-learning based method for financial modeling is exponentially rising.


Introduction
The share market is a snapshot of future growth expectation of the companies as well as the economy. Many factors have attributed to the stock's price fluctuation, which includes but not limited to macro-economic factors, the market anticipation and confidence in the company's management and operation. The advancement of technology allows the public to access a larger quantity of information in a timelier manner. It means that stock analysis has become more and more difficult as a considerable amount of data has to be processed in a relatively short time. People hope that the progress made in big data, especially in the Deep Learning field, can help them analyses stock information [1].
Forex is one of the largest financial markets in the world. The prediction of the exchange rate can provide investors with useful decision-making references to increase return and reduce risk. However, the exchange rate is always under the influence of many factors, such as countries' economy, politics, society, international situation, etc., so the complexity of the matter has made Forex prediction and forecasting a challenging research topic. [2] Nowadays, Forex Forecasting tasks apply many different deep learning models as the computer, and artificial intelligence technology matures.
Forex and stock are similar in many aspects. For example, they both have comparable technical indicators, both have similar charts (candle chart), and they would both be affected by its country's market sentiment. Therefore, this paper will discuss the application of deep learning to both Forex and the stock market and explore the impact of different deep learning methods on their price trend prediction accuracy.
The continuous development in the AI field leads to the wide use of deep learning techniques in many research fields and practical scenarios. Applications include natural language processing, image recognition, medical predictions, and more. The neural networks used in these applications have also developed and improved due to the rise of deep learning. For example, reinforcement learning has gained popularity since AlphaGo defeated the best chess player at the time using it, and reinforcement learning has been implemented in the financial prediction field since then [76]. These technological breakthroughs have given the stock and Forex prediction models a solid foundation to start and a greater room to improve.
The highly complex nonlinear relationship of deep learning can fully describe the complex characteristics of the influencing factors. Besides, many other fields have verified the accuracy of a deep learning model for prediction accuracy, such as image classification, gene analysis. Research results are also obtained for Time-series data analysis and prediction with a deep learning algorithm; for example, deep learning is used to predict the offline store traffic [3]. Overall, deep learning models have excellent performance in other research fields. Therefore, it is feasible to predict stock and Forex trends with deep learning.
Financial researchers around the world have been studying and analysing the changes in the stock and Forex market. The broadening application of artificial intelligence has led to an increasing number of investors using deep learning model to predict and study the stock and Forex price. It has been proven that the fluctuation of stock and Forex price could be predicted [4]. Different from the traditional statistical and econometric models, deep learning can describe complex influencing factors. Therefore, this paper will investigate the different effects of different deep learning methods on stock and Forex forecasting according to the existing published papers. This survey will analyse each paper from the following aspects: 1. What is the dataset of this paper; 2.What is the variable of this paper;3. What kind of deep learning model had been adopted; 4. What is the result of the prediction model?
The structure of this paper will be as follows: firstly, the introduction of Forex and stock combined with deep learning; secondly, the criteria and research methods of the article selected by the survey, thirdly, the impact and analysis of different deep learning methods on stocks and Forex prediction; fourthly, the discussion and analysis of the above methods; finally, the conclusion of the whole paper.

Convolutional neural network (CNN)
CNN was widely used in the field of image recognition because of its powerful pattern recognition ability; its use was also extended to the field of economic prediction. Similar to the traditional neural network, CNN was composed of multiple neurons connected by a hierarchical structure, and the weights and bias between layers can be trained. CNN was different from the network structure of Fully Connected network such as DBN/SAE/BP, as the CNN could share the weight among the neurons in each layer of the network. Hence the model significantly reduced the weight of the network and avoided falling into dimensional disaster and local minimisation [5].
If the characteristics of the stock market at a specific time point were regarded as a feature graph, CNN had the potential to extract the characteristics of the stock market at the corresponding period from these feature graphs. Therefore, CNN could be used to build a timing-selection model and ultimately used to complete the construction of the timing-selection strategy.

Recurrent Neural Network (RNN)
RNN belonged to the neural network, and it was good at modelling and processing sequential data. The specific expression was that the RNN was able to memorise the previous state, and the previous state could be used in the current state calculation. The different hidden layers were non-independent, and the input of the current hidden layer included not only the output of the input layer but also the output of the previously hidden layer. For this reason, RNN would have a good performance in dealing with the sequential data.
The advantage of RNN was that it would consider the context of data in the process of training, which is very suitable for the scenario of stocks and Forex because the fluctuation at a particular time often contains some connection with the previous trend.

Long Short-Term Memory (LSTM)
LSTM model is one of the variants of the RNN. Its core contribution was to introduce the design of self-loop to generate the path of the gradient, which could continuously flow for an extended period. The weight of the self-loop was also updated in each iteration, which solved the gradient vanishing problem that was easily generated when the RNN model updated the weights [6].
The modelling of time series was essentially a process of nonlinear parameter fitting. The LSTM model could perform well to reveal the correlation of nonlinear time series in the delay state space, and to realise the purpose of stock prediction [7]. In the stock or Forex trend prediction model based on LSTM, it obtained the corresponding data characteristics from the stock or Forex history data.

Deep neural network (DNN)
DNN was a neural network with at least one hidden layer. It could provide modelling for complicated nonlinear functions and own a high-level abstraction ability which meant the fitting power of the model would be significantly improved. Meanwhile, it was a kind of discriminant model, which could be trained through the backpropagation algorithm.
Since the DNN was good at dealing with prediction problems with sizable data and complicated nonlinear mapping relations, an intelligent stock and Forex prediction system can be designed based on a DNN to predict stock and Forex trend. Hopefully, the model was able to achieve far higher accuracy than human beings.

Reinforcement learning
Reinforcement learning was one of the deep learning methods that focus on how to act according to the current situation to profit maximisation. In reinforcement learning, there were two basic elements: state and action. A strategy was defined as performing a particular action in a specific state. All the learner had to do was to learn a good strategy by continually exploring and learning.
If the state is regarded as the attribute and the action as the label, it was easy to know that both supervised learning and reinforcement learning was trying to find a map and inferred the label/action from the known attribute/state. In this way, the strategy in reinforcement learning was equivalent to the classification/regression in supervised learning. However, in practical problems, reinforcement learning did not have such labelling information as supervised learning, and the results were often obtained after the attempt of the action. Therefore, reinforcement learning would continuously adjust the previous strategy through the feedback of the result information, for this reason, the algorithm would learn: in which state to take which step would have the most beneficial result.
Therefore, reinforcement learning could learn the best timing trading action (selecting the best price, trading duration, and order size) according to the market response. It can view the contextual information (price, news, public opinion, interest rate, fees, trading action, returns, losses, etc.) of the transaction as an environment of reinforcement learning. Gains or losses could be thought as the reward for learning, trading actions as actions, and factors as states or observations to realise the prediction of the stock and Forex trend.

Other deep learning methods
In this paper, we will discuss the prediction of stock and Forex trend by other deep learning methods, for example, Hybrid Attention Networks (HAN) and self-paced learning mechanism (NLP); multi-filters neural network (MFNN), and Wavenet. The frequency of these methods in the selected articles is not high so that they will be discussed in section 4.7.

Paper selection methods
In the past few years, there had been many deep learning model papers on stock and Forex forecasting. In this paper, the articles analysed in this paper were all from the DBLP computer science bibliography and Microsoft Academic.
Firstly, the keywords were searched in DBLP are:' CNN stock/Forex';' LSTM stock/Forex';' Deep learning stock/ Forex';' RNN stock/Forex' and' Reinforcement learning stock/Forex'. The keywords were searched in Microsoft Academic, then the filter of '2015-2021' and 'Deep learning' were applied.
Secondly, The quality of the selected articles from DBLP was ensured by excluding all the journals and conferences that were informally published. The quality of the selected articles from Microsoft Academic were controlled by excluding all the journals and conferences that were informally published as well as a minimum of 5 citations. Filtering on citations as the searching method was designed to detect under-study area with in this field, the paper with 5 more citations can be an indication that the area could be potentially explored. Furthermore, it should be noted that we only consider papers with a novel model, implementing existing model would not be analysised in this review.
At the same time, the timeliness of the survey was provided by focusing on publications after 2015. Among them, there were 4 papers in 2015,1 papers in 2016, 15 papers in 2017, 30 papers in 2018, 28 papers in 2019, and 10 papers in 2020, in total 88 existing papers would be reviewed. Figure 1 The systematic literature review in this paper

Selected paper statistics
In this paper, the goal was to study the use of six different deep learning methods in Forex/stock (which are CNN, LSTM, DNN, RNN, Reinforcement learning, and other deep learning methods) with different datasets, input variables, and model type. All the results were compared together for discussion and for concluding after individually analysed. Figure 1 showed the annual distribution of papers collected and reviewed. Figure 2 showed the distribution of the different methods in this paper. Figure 3 showed the systematic literature review in this paper. Table 1 showed the distribution of different types of articles in this paper.    17 articles that used CNN technology for the stock and Forex prediction were briefly  described below, and as shown in the table. Table 2 shows the author, variables, dataset, and model of all the papers mentioned. Maqsood, H. proposed a CNN model that made use of the price and sentiment analysis as input and compared the proposed model with the Linear Regression and SVM. He concluded that not all the significant events have a serious impact on stock exchange prediction. However, more important local events could affect the performance of prediction algorithms [8].

Papers Descriptions
Patil, P. proposed a new network using graph theory and CNN, which leveraged Spatio-temporal relationship information between different stocks by modelling the stock market as a complex network. Meanwhile, the model used both stock indicators and financial news as input [9].
Sim, HS proposed a CNN network that using the 9 technical indicators (Close Price, SMA, EMA, ROC, MACD, Fast%K, Slow%D, Upper Band, Lower Band) to verify the applicability of the CNN method in the stock market. And he concluded that the use of technical indicators in stock price forecasting by CNN has no positive effect [10].
Hoseinzade, E. proposed two models: 2D-CNN and 3D-CNN using 82 different technical indicators. And These two structures could improve the predictive performance of the baseline algorithm by about 3% to 11% [11].
Eapen, J.proposed a model that had multiple pipelines of CNN and bi-directional LSTM units. And it could improve prediction performance by 9% using a single pipeline deep learning model and by over a factor of six using support vector machine regressor model on S&P 500 grand challenge dataset [12].
Yang, H.m, proposed a multi-indicator feature selection for stock index prediction based on a multichannel CNN structure without sub-sampling, named MI-CNN framework. In this method, candidate indicators were selected by the maximal information coefficient feature selection (MICFS) approach, to ensure the correlation with stock movements whilst reducing redundancy between different indicators [13].
Cai, S. proposed the CNN and LSTM forecasting system with financial news and historical data of the stock market. It had generated seven prediction models. According to the ensemble learning method, the seven models are constructed into one ensemble model to obtain an aggregated model. Unfortunately, all models had a lower prediction accuracy [14].
Oncharoen, P. proposed a new framework to train a DNN for stock market prediction. A new loss function was developed by adding a risk-reward function, which was derived from the trading simulation results [15].
Liu, Y. proposed to incorporate a joint model using the TransE model for representation learning and a CNN, which extracted features from financial news articles. And then he combined the model with LSTM to predict the price movement. The model could improve the accuracy of text feature extraction while reducing the sparseness of news headlines [16].
Selvin, S. used CNN, LSTM, and RNN architectures to forecast the price of NSE listed companies and compares their performance. The final results showed that CNN is the best architecture to predict the price of the stock because it could identify the trend of the directional change [17].
Liu, S.proposed a CNN-LSTM model, and the model performed basic Momentum strategy and Benchmark model whose return rates were 0.882 and 1.136 respectively. The CNN part could extract useful features even from low signal-to-noise time-series data, and the LSTM part could predict future stock prices with high accuracy, then the predicting outcomes were used as timing signals [18].
Gudelek, M.U. proposed a 2D-CNN model. This model adopted a sliding window approach then generated images by taking snapshots that are bounded by the window over a daily period. The model could predict the next day's prices with 72% accuracy and end up with 5:1 of our initial capital [19].
Ding, X. proposed a deep learning method for the event-driven stock market prediction. Firstly, events were extracted from news text and represented as dense vectors, trained using a novel neural tensor Network(NTN). Secondly, CNN was used to model both short-term and long-term influences of events on stock price movements [20].
Zhao, Y. proposed a Wavelet Denoised-ResNet CNN for Forex exchange rate prediction. The technical indicators were treated as an image matrix, the image matrix was first denoised using Wavelet method, then processed by ResNet CNN, and lastly, LightGBM was used to replace the softmax layer to output a prediction. [71] Chen, S. and He, H. proposed a CNN model with a novel architecture, where it generated better results than benchmark RNN model. [76] Ugur Gudelek, M proposed a CNN model where financial data is taken as a image, then using CNN to predict the desired trading action, the model yielded reasonable results. [77] Chen, Jou-Fan proposed a novel CNN model which utilized the power of Gramian Angular Field, the results produced were average, but it is a interesting research direction. [78] Wen, M proposed a CNN model which relied on the reconstructed of time series, it turned the time series into segment, then CNN was used to classify each segment. The model generated good results. [90] 4.1.2 RNN 5 articles that used RNN technology for the stock and Forex prediction were briefly described below, and as shown in the table. Table 3 showed the author, variables, dataset, and model of all the papers mentioned above. Attention-based RNN-ARIMA Ni, L. proposed a CRNN model to predict the price of 9 kinds of forex pair. The results showed that the proposed model performed much better than the LSTM model and the CNN model [53].
Li, C. proposed a Multi-task RNN model with MRFs. The MMPL is proposed to automatically extract diversified and complementary features from individual stock price sequences which means there is no need for the technical indicators. Features learned by MMPL were passed to a binary MRF with a weighted lower linear envelope energy function to utilise intra-clique higherorder consistency between stocks [54].
Chen, W proposed an RNN-Boost model that made use of the technical indicators and sentiment features, and Latent Dirichlet allocation (LDA) features to predict the price of stocks. Its results showed that the proposed model outperformed the single-RNN model [55].
Zhang, R. proposed a Deep and Wide Neural Networks (DWNN) model where CNN's convolution layer was added to the RNN's hidden state transfer process. The results showed that the DWNN model could reduce the prediction mean squared error by 30% compared with the general RNN model [56].
Zeng. Z proposed a novel Attention-based RNN (ARNN) where wavelet denoised input was passed to ARNN. The forecast was calculated using the ARIMA and the output of ARNN model. [73] 4.1.3 LSTM 27 articles that used LSTM technology for the stock and Forex prediction were briefly described below, and as shown in the table. Table 4 showed the author, variables, dataset, and model of all the papers mentioned. Nikou, M. proposed an LSTM model and compared it with the ANN model, SVR model, and RF model. The results showed that the LSTM model performed better in the prediction of the close price of iShares MSCI United Kingdom than the other models mentioned in the paper [21].
Fazeli, A. proposed an LSTM model; his result showed an improve through using LeakyReLU. MSE came below 0.1. Whilst compared with the ReLU function; the MSE went well above 0.2. He examined the effect of RSI, Williams %R, and volatility on the loss of the model. It was shown that the model's loss was reduced by using only RSI, which contributed to the performance of the model [22].
Xu, Y. proposed an attention-based LSTM model that performs better than the regular LSTM. It was found that finance tweets that were posted from market closure to market open the next day had more predictive power on the next day's stock movement. The weighted sentiment on the max follower on StockTwits also performed much better than other methods [23].
Lakshminarayanan, S.K. proposed an LSTM model combined with Crud oil price, gold price, and moving average, which performed much better than the LSTM model without them and the SVM model. It showed that the Crud oil and gold price had some impact on the stock price prediction [24].
Rana, M. proposed an LSTM model that outperformed the LR and SVR model. He also compared the different activation functions with different optimisers and concluded that the tanh activation with the adam algorithm performs best with the accuracy of 98.49% [25]. Naik, N. proposed an RNN with the LSTM model. The model had the ability to keep the memory of historical stock returns in order to forecast future stock return output. It was worth noting that the recent stock information rather than old related stock information was stored and used. The network also outperformed the Feed Forward ANN model [26].
Lai, C.Y. proposed a Dynamic LSTM model The results showed that stock prediction accuracy based on MAE, MAPE, RMSE, and MSE obtained by the dynamic LSTM model was much better than that by the static LSTM model. The dynamic model also consistently outperformed the linear models SA-5 and EMA-0.5 when predicting four stocks [27].
Lai, C.Y. proposed an LSTM model which used average previous five days stock market information (open, high, low, Volume, close) as the input value. The initial prediction was calculated using the value. The prediction was then used as part of the average of the stock price information for the next five days through the ARIMA method. Moreover, he utilised Technical Analysis Indicators to consider whether to buy stocks or continue to hold stocks or sell stocks [28].
Hossain, M.A. proposed an LSTM model followed by GRU. Both of the LSTM and GRU were powerful recurrent networks that could perform better and faster in terms of accuracy in regression-based prediction. And the proposed model outperformed the LSTM only, GRU only, and GRU followed by the LSTM model [29].
Baek, Y. proposed a novel data augmentation approach for stock market index forecasting through ModAugNet framework, which consisted of two modules: an overfitting prevention LSTM module and a prediction LSTM module. The overfitting problems mainly caused by the limited availability of data points for training [30].
Kim, H.Y. proposed some LSTM model to forecast stock price volatility that combined the LSTM model with various generalised autoregressive conditional heteroscedasticity (GARCH) -type models. He found that the GEW-LSTM model, which combined all three models, GARCH, EGARCH, and EWMA, with LSTM, performed best [31].
Li, H. proposed an improved MI-LSTM based on LSTM and attention mechanism, which achieved better performance in extracting potential information and filtering noise. The model could assign different weights to different input series to keep the dominant status of the mainstream while absorbing information from leaky input gates [32].
Cheng, L.-C. proposed an Attention-based LSTM model that could solve the problem of exploding and vanishing gradients and thus did not effectively capture long-term dependencies [33]. Shah, D. proposed an LSTM model which was compared with the DNN model. The proposed model was able to predict volatile movements in the true data. In general, it was able to recognise the directionality of the changes in the true data more accurately than the DNN [34].
Lin, B.-S. proposed an LSTM model to predict the price of the top 10 industries included in Taiwan Stock Exchange Corporation (TWSE).In the experimental results, most of the results were reasonable except for the MTK stock [35].
Skehin, T. proposed a linear Autoregressive Integrated Moving Average (ARIMA) model and LSTM network for each series to produce next-day predictions. Wavelet methods decomposed a series into approximation and detail components to better explain behaviour over time. He combined these techniques in a novel ensemble model in an attempt to increase forecast accuracy [36].
Zhang, X. proposed a simple but efficient method to predict future stock return ranking without handcrafted features [37].
Achkar, R. proposed an approach to predict stock market ratios using artificial neural networks. It considered two different techniques BPA-MLP and LSTM-RNN their potential, and their limitations. And the LSTM-RNN model outperformed the other one slightly [38].
Zeng, Y. proposed an LSTM model which is based on the dataset SSE50, and the results showed that the accuracy is above 65% [39].
Shao, X. proposed a Kmeans-LSTM model that used the time window to divide the stock price sequential data into several equal sub-sequences. And the K-means algorithm is used to cluster the stock price sub-sequence (He did not give the specific result data.) [40].
Zhao, Z. proposed a time-weighted LSTM. Unlike other models which treat data indiscriminately, the proposed model could carefully assign weights to data according to their temporal nearness towards the data that was used for prediction. And the results showed that the proposed model outperformed the SVM, RNN, and Random Forest, model [41].
Nelson, D.M. proposed an LSTM model which was based on the price history and technical analysis indicators. And the results showed that the proposed model was better than the MLP and Random Forest model [42].
Dos Santos Pinheiro proposed a character-based neural language model for an eventbased trading model that was combined with NLP and LSTM. The results showed that the proposed model performed better than some model proposed in other papers such as WI-RCNN and SI-RCNN [43].
Buczkowski, P. tackled the problem of stock market predicting feasibility, especially when predictions were based only on a subset of available information, namely: financial experts' recommendations. The analysis was based on data and results from the ISMIS 2017 Data Mining Competition [44].
Akita, R. proposed an approach that converted newspaper articles into their distributed representations via Paragraph Vector and modelled the temporal effects of past events on open prices about multiple companies with LSTM [3].
Chen, K proposed an LSTM model to predict the price of the Chinese stock. But the results showed that the accuracy is only 27.2% [45].
Qi, L. proposed an LSTM model to predict the price of the forex exchange rate. The model used technical indicators to calculate events then LSTM is used to make the prediction based on the events. [75] Pang, Xiong Wen propsed two improved deep LSTM model with embedded layers. The results showed its improvement over the benchmark. [80] Feng, Fuli, et al. proposed a noval LSTM model that combined with stock relational model, it had a great performance compared with its benchmarks. [81] Chung, Hyejung, and Kyung-shik Shin proposed a LSTM model which is optimized by genetic algorithm (GA). The novel hybrid model had outperformed the benchmark. [82] Li, Jiahong, et al. proposed a LSTM model that was combined with a Naïve Bayne's model. This hybrid model had outperformed benchmarks, and delivered promising results. [83] Zhang, Kang, et al. propsed a novel generative adversarial network based on MLP and LSTM. The model is able to better results than some of the vanilla models. [84] Jin, Zhigang, et al. propsed to incorporate sentiment analysis model into LSTM, they successfully created a novel model which deliver a reasonable result. [91] Long, Jiawei, et al. propsed a hybrid model, which utilised CNN to extract transactional information, then it was passed through a LSTM to predict the stock price. The model showed improvement over some of the vanilla benchmarks. [92] Chen,M proposed a LSTM model which uses the sentiment data collected. It combined both sentimental model as well as LSTM to produce a good result. [93] Qian,Fei. Proposed a LSTM model which used ARIMA to improve its prediction power. The model yielded a reasonable result. [94] Li.Z and Tam, V proposed a LSTM model that used wavelet denoising technique before passing through the LSTM model, the model had produced good result. [95] 4.1.4 DNN 13 articles that used DNN technology for the stock and Forex prediction were briefly described below, and as shown in the table. Table 5 shows the author, variables, dataset, and model of all the papers mentioned. Chatzis, S.P. proposed a DNN model in which it used Boosted approaches to predict stock market crisis episodes. According to his research, it was meaningful to know the stock market crisis to predict the price, even though his research was not specific to certain prediction methods [48].
Abe, M. proposed a DNN network and his results showed that DNNs generally outperform shallow neural networks, and the best networks also outperformed representative machine learning models [49]. Nakagawa, K. proposed a deep factor model and a shallow model with DNN. The deep factor model outperformed the linear model. This implied that the relationship between the stock returns in the financial market and the factors is nonlinear, rather than linear. The deep factor model also outperformed other machine learning methods including SVR and random forest. The shallow model was superior in accuracy, while the deep model was more profitable [50].
Chong, E. proposed DNN networks and examined the effects of three unsupervised feature extraction methods including principal component analysis, auto-encoder, and the restricted Boltzmann machine, on the network's overall ability to predict future market behaviour. Empirical results suggested that DNNs could extract additional information from the residuals of the auto-regressive model and improve prediction performance; the same could not be said when the auto-regressive model is applied to the residuals of the network [51].
Singh, R. proposed a 2-Directional 2-Dimensional Principal Component Analysis (2D²PCA)+DNN which outperformed than the RNN and (2D²PCA) +RBFNN. The paper found that the best results were generated from a window size of 20 and a dimension of 10 × 10. The deep learning method for higher dimensions and large window sizes was giving a limited performance [52].
Yu, Pengfei, and Xuesong Yan proposed a novel DNN model which incorporated LSTM as well as phase-space recognition. The model had produce promising results. [87] Yong, Bang Xiang, et al. propsed a DNN model with 40 nodes which showed a reasonable results and appear to be a highly profitable model. [89] 4.1.5 Reinforcement learning 8 articles that used reinforcement learning technology for the stock and Forex prediction were briefly described below, and as shown in the table. Table 6 showed the author, variables, dataset, and model of all the papers mentioned. to the LSTM layer. The reinforcement learning defines agents' policy neural network structure, reward, and action, and provides buying, selling, and holding probabilities as final output [58].
Jia, W. proposed a Reinforcement Learning with an LSTM-based agent who could automatically sense the dynamics of the stock market and alleviate the difficulty of manually designing indicators from massive data. The paper had compared a wide range of different input sets [59].
Carapuço, J. proposed a reinforcement learning-Q network model, Three hidden layers of ReLU neutrons were trained as RL agents through the Q-learning algorithm under a novel simulated market environment framework. The framework was able to consistently induce stable learning that generalised to out-of-sample data [60].
Kang, Q. proposed to apply the state-of-art Asynchronous Advantage Actor-Critic algorithm(A3C algorithm) to solve the portfolio management problem and designed a standalone deep reinforcement learning model [61].
Zhu, Y. proposed an adaptive box-normalisation (ABN) stock trading strategy based on reinforcement learning, which improved the original box theory. In his ABN strategy, the stock market data was independently normalised inside each oscillation box [62].
Si, W. proposed a reinforcement learning model which had multi-objective and LSTM agent. It could be found that feature learning could contribute to better performances. The LSTM network made continuous decisions and could change positions at the right time, which reduced the transaction cost, and the multi-objective structure made good profits within the acceptable risk [63].
Pastore, A. analysed data for 46 players extracted from a financial market online game and tested whether Reinforcement Learning could capture these players' behaviour using a riskiness measure based on financial modelling [64].

Other Deep Learning Methods
8 articles that used other deep learning technology for the stock and Forex prediction were briefly described below, and as shown in the table. Table 7 showed the author, variables, dataset, and model of all the papers mentioned. Long, W. proposed a novel end-to-end model named multi-filters neural network (MFNN) specifically for feature extraction on financial time series samples and price movement prediction task. Both convolutional and recurrent neurons are integrated to build the multi-filters structure. And the results showed that the proposed model is better than the RNN, CNN, and traditional machine learning methods [65].
Wu, J.-L. proposed a keyword-based attention network into Hierarchical Attention Networks (HAN), namely the HKAN model, to learn the relationships between dimensional sentiments (trend and trading) and stock messages which outperformed the HAN network [66].
Cho, C.-H.proposed three different models to predict the price of the stock, they were: LSTM，Seq2seq and Wavenet. According to his experiments, Wavenet outperformed the other two models [67].
Minh, D.L proposed a network to predict the directions of stock prices by using both financial news and sentiment dictionary. His results showed that TGRU achieved better performance than GRU and LSTM and Stock2Vec is more efficient in dealing with financial datasets [68].
Hu, G. proposed a Convolutional AutoEncoder model to learn a stock representation and converted a 4-channel stock time series (lowest, highest, open, and close price for the day) to candlestick charts by using synthesis technique to present price history as images. This method had successfully avoided expensive annotation. And the proposed model outperformed the FTSE 100 index and many well-known funds in terms of total return [69].
Hu, G. proposed a Hybrid Attention Networks (HAN) to predict the stock trend based on the sequence of recent related news. Moreover, he applied the self-paced learning(SPL) mechanism to achieve effective and efficient learning. The results showed the proposed model outperforms than RNN, Temporal-Attention-RNN, News-Attention-RNN, and HAN [70].
Kim, T. proposed a 2D relative-attentional gated transformer which was used to optimise portfolio return. It used general reinforcement learning with the agent incorporating 2D Relative-attentional Gated Transformer. [72] Zhang, Z. proposed a genetic algorithm using a crossover of technique indicators as input. It had successfully outperformed some of the traditional trading strategies. [74] Hu, Ziniu proposed hybrid attention networks which is a novel way of combining financial time-series information and news nature language information. [79] Shi, Lei, et al. propsed a system where it perform factor analysis then utilise multiple depp learning method to design a model which outperformed benmark. [86] 4.2 Papers Results grouped by the method used

CNN
Since the performance metrics used in different articles were different, our survey analysed them based on various metrics. Results used were the average performance of the best performing models in the mentioned papers. Table 8 showed the metrics and results for the papers which used the CNN model. 0.2631 0.156*10^-6 [16] 0.7133 AE [8] 0.029+/-.005 [76] 0.73 [90] 0.6367

RNN
Since the performance metrics used in different articles were different, our survey analysed them based on different metrics. Results used were the average performance of the best performing models in the mentioned papers. Table 9 shows the metrics and results for the papers which used the RNN model. 68.95% 66.54% F-measure [54] 0.7658 Recall [54] 0.7471 Precision [54] 0.7855 MSE [56] 0.057443

Reinforcement learning
Since the performance metrics used were different in different articles, our survey would analyse them based on different metrics. Results used were the average performance of the best performing models in the mentioned papers. Table 12 showed the metrics and results for the papers which used the Reinforcement learning model.

Other Deep Learning Methods
Since the performance metrics used in different articles were different, our survey analysed them based on different metrics that were used. Results used were the average performance of the best performing models in the mentioned papers. Table 13 showed the metrics and results for the papers which used other deep learning methods.

CNN
Some findings can be drawn from reviewing the CNN models: 1. According to the datasets that all the paper used, 6 papers used the combination of technical analysis and sentiment and news analysis to predict the stock. The rest of them used the method of technical analysis only.
2. As for the variables, the close price was the choice of all CNN models, and five papers were using close price only.
3. It could be found that 12 of the papers changed the traditional CNN model to pursue higher performance of prediction. And the combination of CNN and LSTM was the most common model. 4. The metrics used in each paper were different; there were 11 metrics used for measuring the performance of the CNN model. 5.Multiple articles selected RMSE, Return rate, F-measure, Sharpe ratio, and accuracy. We could find that the paper [19] had the highest accuracy than any other papers, paper [13] had the highest return rate followed by the paper [18], [11] and [15]. Paper [71] had the lowest RMSE. Paper [16] had a higher F-measure than paper [11], and [15], but paper [15] achieved a higher Sharpe ratio than paper [11].

RNN
In the section of the RNN based model, the following conclusions could be drawn: 1. According to the datasets that all the paper used, there was 1 paper that used the combination of technical analysis and sentiment and news analysis to predict the stock. And the rest of them used technical analysis only.
2. For the variables, all the RNN based models used the multivariable input and open price, close price, highest price, and the lowest price are used in all models as an input.
3. It could be found that all the papers changed the traditional RNN model to pursue higher performance of prediction. Two of the papers chose the C-RNN based model. 4. The metrics used in each paper were different; in total, there were 8 metrics used for measuring the performance of the RNN based model. 5.RMSE and accuracy were selected by multiple articles. We could note that paper [55] has a much lower RMSE than paper [53], and paper [54] had higher accuracy than paper [55].

LSTM
Following points were worth discussing from reviewing the LSTM papers: 1. According to the datasets that all the paper used, 3 papers used a combination of technical analysis and sentiment and news analysis to predict the stock. The rest of them used technical analysis only with the exception of one paper which used expert recommendations.
2. As for the variables, the close price was the choice of 23 LSTM based models, there were 8 papers using close price only, and 12 papers selected to include close price, open price, high price, and low price in their input.
3. It could be found that 15 of the papers changed the traditional LSTM model to pursue higher performance of prediction. Attention-based LSTM and LSTM with RNN were the most frequent models that showed up in three different papers. Two papers chose the method of LSTM with GRU to improve the model. 4.The metrics used in each paper were different; there were 17 metrics used for measuring the performance of the LSTM based model.
Multiple articles selected 5.RMSE, MAPE, MAE, accuracy, and MSE, we could find that Paper [25], paper [94] and paper [27] was in the lowest order of magnitude with paper [25] achieving the lowest RMSE; paper [21] was in the second-lowest order of magnitude; paper [28], [35], [36] and paper [34] were in the third lowest order of magnitude. Paper [26] was in the fourth lowest order of magnitude and paper [24] had the highest order of magnitude.
7. As for MAE, paper [75], [31], [27] and [29] had the lowest order of magnitude MAE with paper [75] having the best performance; paper [26] and [21] were in the second-lowest order of magnitude while paper [30] was in the third lowest order of magnitude. Paper [24] had the highest order of magnitude. 8. As for MSE, paper [81]had the lowest MSE and was in the lowest order of magnitude; paper [29] , [31] and [22] were in the second-lowest order of magnitude while paper [28] and [21] were in the third lowest order of magnitude. Paper [32] and [30] were in the fourth lowest order of magnitude, paper [27] was in the fifth-lowest order of magnitude, and paper [24] had the highest order of magnitude of MAE.

DNN
In the section of DNN-based model, the following conclusions could be drawn: 1. According to the datasets that all the paper used, no paper used the combination of technical analysis and sentiment and news analysis to predict the stock. All of them used the method of technical analysis only.
2. As for the variables, six of the seven DNN-based models used multivariable input, and only one paper used close price as its sole input.
3. It could be found that 3 of the papers changed the traditional DNN model to pursue higher performance of prediction. All of the improved models were not duplicated.
4. Because the metrics used in each paper were different; there were 11 different metrics used for measuring the performance of the DNN-based model. 5. Multiple articles selected 5.RMSE, MAE, accuracy, return rate, and MSE. We could find that the paper [52] had the lowest RMSE followed by the paper [50] and [51], with the latter two in different orders of magnitude.
6. As for MAE, paper [50] had a lower MAE than paper [51], and the accuracy of the paper [47] was higher than paper [46] and [87]. Furthermore, paper [50] had a higher return rate than paper [49]. And paper [49] had a lower MSE than paper [51].

Reinforcement learning
In the section of Reinforcement learning-based model, the following conclusions could be drawn: 1. According to the datasets that all the paper used, no paper used the combination of technical analysis and sentiment and news analysis to predict the stock. All of them used technical analysis only.
2. As for the variables, 7 of the reinforcement learning-based models used the multivariable input, and only one paper solely used close price as input.
3. It could be found that three of 6 papers changed the traditional Reinforcement learning model to pursue higher performance of prediction. 3 of the improved models were combined with LSTM.
4. The metrics used in each paper were different; in total, there were 3 metrics used in the measurement performance of the Reinforcement learning-based model. 5. Multiple articles selected 5. Sharpe ratio and return rate, we could find that paper [58] had a much higher Sharpe ratio than paper [63]; paper [62] had a higher return rate than paper [59] then followed by the paper [60].

Other Deep Learning Methods
In the section of other deep learning methods based model, the following conclusions could be drawn: 1. According to the datasets that all the paper used, 4 papers made use of sentiment and news analysis to predict the stock. The rest of them used the method of technical analysis only.
2. As for the variables, five of the other deep learning methods models used the multivariable input and only one paper used candlestick charts alone as input.
3. It could be found that there were 5 different models in the other deep learning methods section. The only model that appeared three times was HAN, which consist of an ordinary HAN model and two modified HAN models. The rest of the models were not duplicated.
4. Because the metrics used in each paper were different, there were 9 metrics used in the measurement performance of this section.
5. Multiple articles selected 5. Accuracy, Sharpe ratio, and Return rate, we could find that the paper [86] had the highest accuracy.Paper [65] had a much higher Sharpe ratio than paper [69]. 6. As for the return rate, paper [70] had the highest return rate followed by the paper [65], [69]and paper [68] had the lowest return rate.
Among all the papers, 33% of the CNN papers had an RMSE below 0.001, 33% of the CNN papers were in the range of 0.01-0.1, and 33% of the CNN papers were in the range of 0.1-1. 36% of the LSTM papers were in the range of 0.01-0.1 and 1-10 respectively, , the rest LSTM papers were distributed equally in range of <0.001, 10-100 and above 100 with 9%in each range. 33.3% of the DNN papers were in the range of 0.001-0.01, 0.01-0.1 and 0.1-1 respectively. The RNN papers were distributed equally in the range of 0.001-0.01, 0.01-0.1 and above 100, i.e. 33% in each range. The only paper that used other deep learning method had an RMSE in the range of 0.01-0.1.
Among all the papers, the only CNN paper was in the range of 2-10. 37.5% of the LSTM papers were in the range of 1-1.5, 12.5% of the LSTM papers were in the range of 0.5-1, 1.5-2, and 2-10 respectively, 25% of LSTM papers in a range of 0-0.5. The two DNN papers were in the range 0-0.5, and 1-1.5. Lastly, the only one RNN paper was in the range of 0-0.5.
Among all the papers that used MAE as measurement, 50% of CNN paper was in the range of 1-10; the other 50% had an MAE small than 0.01. 42.9% of the LSTM papers were in the range of 0.01-0.1, 28.6% of the LSTM papers were in the range of 0.1-1, 14.2% of the LSTM papers were in the range of 1-10 and above 100 respectively, 50% of the DNN papers were in the range of 0.01-0.1 and 0.1-1 respectively. The two RNN papers were in the range of 0.01-0.1 and 0.1-1.
Among all the papers that used MSE as performance measurement, the only CNN paper was in the range of 0.1-1, 40% of the LSTM papers were in the range of 0-0.01, 20%of the LSTM papers were in the range of 0.01-0.1, 10% of the LSTM papers were in the range of 0.1-1, 1-10, 10-100 and above 100 respectively, 50% of the DNN papers were in the range of 0.01-0.1 and 0.1-1 respectively. The only RNN based model and reinforcement learningbased model was in the range of 0-0.01 and 0.01-0.1 respectively.
Among all the paper that used the Sharpe ratio, all CNN papers were in the range of 0.1-1. The two DNN papers were in the range of 1-2 and 5-10. 50% of the Reinforcement learning paper was in the range of 0.1-1, and 50% of the Reinforcement learning paper was in the range of 2-5. 50% of the other deep learning method paper was in the range of 0.1-1, and 50% of the other deep learning method paper was in the range of 5-10.
Among all the papers that used return rate as a performance measurement, all the CNN papers were in the range of 1.2-1.4, and the only LSTM paper was in the range of 1.0-1.2, all the DNN papers were in the range of 1.0-1.2. 50% of the Reinforcement learning papers were in the range of 1.0-1.2, and 50% of the Reinforcement learning papers were in the range of 1.8-2. The other deep learning method papers performed well with 14% paper in the range of 1.0-1.2, 42% paper in the range of 1.4-1.6 , and 42% paper in the range of 1.6-1.8.

Conclusions
This paper provided a detailed review of 88 papers from 2015 to the present on predicting stock /Forex price movements through deep learning methods. The existing stock/Forex models were evaluated through analysing data sets, variables, the use of different models, and the metrics of evaluation. The research review included a wide range of techniques: CNN, LSTM, DNN, RNN, Reinforcement learning, and other deep learning methods such as HAN, NLP, and Wavenet. Furthermore, the data sets, variables, models, and their different results were analysed and compared within each technique. Then our paper discusses the main performance metrics of all models. They are RMSE, MAPE, MAE, MSE, Accuracy, Sharpe ratio and Return rate.
This paper aimed to contribute to the research of stock/ Forex market prediction through the analysis of the above different deep learning prediction models. Through the review, It can be identified that there is a lack of studies on the combination of multiple deep learning methods, espically in respect to other deep learning methods. The hybrid networks are showing promising signs for future research. In the future, we would design a specific hybrid model based on the above analysis, incorporating latest technology such as advanced genetic algorithms, self attention neural networks to predict the stock/Forex market.
Author Contributions: First two authors contributed equally, MK conceived & designed the study supervised the work and revised the manuscript.
Funding: This research received no external funding.

Conflicts of Interest:
The authors declare no conflict of interest.