Abstract
Machine learning has emerged as a powerful tool in studying the behavior of stock movement. However, it has yet to be highly accurate due to market randomness. This article aims to improve stock movement classification accuracy by addressing macroeconomic factors, which have been neglected in previous machine learning stock prediction studies. Hence, we propose a Risk Adapting Stock Trading System (RAST) using both technical and macroeconomic indicators. The simulated trading result of the system presented here proves that a combination of these two types of indicators is more effective than only using technical indicators when associated with machine learning.
1. Introduction
The prediction of stock prices has received much attention from investors and researchers, while there has been disagreement on market efficiency. The effective market hypothesis (EMH) was raised in 1970, which stated that a market is said to be efficient if the price fully reflects available information [1]. The EMH asserts that financial markets are efficient. Some research supported it from various angles. One pointed out that investors are rewarded for risk baring instead of knowing better than the market because of EMH [2]. Another found that the market is efficient except for some indication of postevent price drift [3]. However, many attacks on the EMH also occurred in the 1980s and 1990s. An article rejected market efficiency by showing “excess volatility” in stock markets [4]. Another study rejected through finding reversals in weekly security returns [5]. Moreover, some strong evidence of the predictable nature of security was discovered in 1990 [6]. In the literature tracing the history of the EMH, just under half of the papers reviewed support market efficiency [7]. Therefore, this paper employs multiple indicators to predict stock movement.
Traditional indicators, technical and fundamental, are commonly used in stock predictions. Technical indicators have been used the most all features, which include historical closing prices, opening prices, and trading volume. Fundamental indicators are also instructive information, involving earnings-per-share (EPS), price-to-earnings (P/E), turnover rate, macroeconomic factors, etc. Recently, text data, such as financial news, is emerging in studies as well.
The latest developments in machine learning have inspired a new way of predicting stock movement combined with indicators. Nevertheless, much of the literature in the machine learning field seems to neglect macroeconomic data, while its determination of stock prices has been widely discussed [8, 9]. The reason for the problem may be the data quality requirement of machine learning models. Most of the macroeconomic time series do not change for a rather long period or only have minor changes, which may limit the impact on stock trend, so do not trade off the calculation cost.
Despite the existence of abundant research on stock prediction, we still face the challenge of applying them in industry because of heterogeneous sources of stock information, such as news from many different presses, and the integration of models. On the one hand, investors might not have access to specific information or could not obtain enough features in a timely manner required for input. On the other hand, much research concentrates only on the prediction of stock prices and may be too complex to apply directly to the real market.
To solve the above issues, this paper selects several financial factors, including ten-year treasury constant maturity rates, Federal Reserve Funds (FED) rate, inflation rate, Standard and Poor’s (S&P) 500, unemployment rate, and EUR/USD foreign exchange rate. We proposed hybrid models combining macroeconomic and technical data with LSTM and support vector machine (SVM), respectively. A comparison of technical indicators only and technical macroeconomic combination in the two models on directional movement proves that these factors have important effects on the trend. Then we constructed an efficient trading system based on momentum signal, RSI signal, moving average signal, Bollinger signal, LSTM signal, and SVM signal to meet various risk preferences. This trading system shows profitability by an empirical study using S&P 500 stocks. The highest return among all signals is 579.98% for around two years, and the highest return among all machine learning signals is 443.73%, which is LSTM with macroeconomic factors.
The rest of this paper is organized as follows. “Data” section presents the data used in the simulated trading. “Related Work” section contains literature reviews. “Preliminaries” section demonstrates SVM and LSTM. “Method” section provides information about signals, models, and trading system structure. “Result” section shows asset changes in the back testing and compares different signals’ performances. “Discussion” section explores experimental results and future research directions. “Conclusion” section summaries findings and outcome.
2. Related Work
A considerable amount of work has been published on stock price prediction by different indicators. Some researchers used technical ones. These indicators effectively show historical stock behaviors and have been popular in much stock research [10–12]. With an increasing number of phenomena implying that social interaction may have an impact on stock returns, researchers have turned to examining the effectiveness of financial news and social media on stock price prediction. A study applied financial news, announcements, and financial statements to predict stock prices [13]. A recent work used the data of users and stocks they followed from a trader social media platform as a supplement to traditional variables [14].
With these indicators, some articles have predicted stock movement by machine learning method. A neural network based on Levenberg-Marquardt, scaled conjugate gradient, and Bayesian regularization was conducted for predicting stock [15]. Statistical arbitrage strategies were constructed by Elman neural networks and a multicriteria decision method [16, 17]. Different deep learning methods, such as deep neural networks, gradient-boosted trees, and random forests, were compared for their performance on stock returns [18]. Long short-term memory (LSTM) and random forests were used to predict directional movements based on intraday returns, closing prices, and opening prices [19]. The empirical evidence proves the profitability of applying machine learning to stock prediction.
3. Data
To investigate the effect of macroeconomic factors on the stock price, we used the factors described in Table 1, which were retrieved from Federal Reserve Data (FED) [20]. Monthly unemployment rates were repeated for all days of the corresponding month to fill the fields in our daily records.
The S&P 500 stocks data, including open, high, low, and close prices and trading volume, from the period September 2011 to August 2021 were obtained from Yahoo Finance [21]. Some stocks were filtered out for an initial public offering time later than September 2011. Stock prices are all adjusted by deducting dividends from last closing sale price and by dividing splitting ratio. We finally select 460 stocks to construct the portfolio. This 10-year period of each stock contains 2,514 data points in which the markets were open. Table 1 presents explanations for each field in the data set.
4. Preliminaries
4.1. SVM
SVM is a supervised learning model that tries to classify data into corresponding categories and separate them as best as possible. The model attempts to find a hyperplane, denoted as that best separates the data, and the closest data on two sides lie on and , where is the weight vector, is the input feature vector, and is the bias parameter.
4.2. LSTM
LSTM is a special type of recurrent neural network (RNN) architecture used in deep learning. It can process sequence data and keep track of the long-term dependencies between the intervals. Traditional RNNs could also keep track of short-term dependencies between data but does not work well for long-term dependencies because the long-term gradients tend to vanish over time, which is also called as the gradient vanishing problem. The LSTM structure, as shown in Figure 1, solves this by containing a forget gate in its structure, in order to decide whether a piece of information should be forgotten, kept, or combined with a new piece of information [22]. In Figure 1, and are long-term information, and are short-term information, and is current input vector.
4.3. Markowitz Portfolio Optimization
Markowitz Portfolio Optimization is a finance theory aiming to find the optimal weights for a portfolio based on the idea of the efficient frontier. An efficient frontier contains the most efficient portfolios that take the smallest level of risk with a given level of expected return. Markowitz Portfolio Optimization takes a portfolio on the frontier and tries to find the optimal weights of each security to maximize the portfolio return and minimize portfolio risk. To compute the weights of the portfolio for any return , we shall solve the constrained minimum problem below: where the minimum is taken over all vectors such that where is one-row matrix of weights, is covariance matrix between returns, is one-row matrix of expected returns, is any value of expected portfolio return, and is one-row matrix with all entries equal to 1 [23].
5. Methodology
5.1. Moving Average Signal
Moving average (MA) is an indicator that takes prices in a period of time and takes the average of them to capture the trend in that period. MA can help filter out noises and determine potential support and resistance levels. Our trading system would first calculate the MA of 5 days (short period) and 30 days (long period) for each day. Then, if a short period MA of a day is larger than the long period MA of that day, and at the same time the short period MA of the day before that day is smaller than the long period MA of the day before that day, the system will generate a “buy” signal since this situation usually occurs before the price begins to rise. In contrast, when a day’s short period MA is smaller than that day’s long period MA, and the day before that day’s short period MA is larger than the day before that day’s long period MA, the system will generate a “sell” signal since this usually occurs before the price begins to fall.
5.2. RSI Signal
The Relative Strength Index (RSI) is an indicator that measures the magnitude of recent price changes to evaluate overbought and oversold conditions. It oscillates between 0 and 100 [24]. Our trading system calculates the 6-day RSI (short period) and 24-day RSI (long period) of each day. For each day, if the short period RSI result is less than 20, the system will generate a “buy” signal. If the result is larger than 80, the system will generate a “sell” signal. In addition, if a day’s short period RSI is larger than that day’s long period RSI and the day before that day’s short period RSI is smaller than the day before that day’s long period RSI, the system will generate a “buy” signal since this situation shows a strong possibility for the price to rise. In contrast, if a day’s short period RSI is smaller than that day’s long period RSI, and the day before that day’s short period RSI is larger than the day before that day’s long period RSI, the system will generate a “sell” signal since this shows a strong possibility for the price to fall.
5.3. Momentum Signal
Momentum is an indicator that measures the price change in a period. The momentum effect indicates the tendency of stocks to show a persistent performance. If the price of a stock has been rising, it is more likely to rise continuously for a short period [25]. Our trading system calculates the 35-day momentum for each day, and if the result is positive, the system will generate a “buy” signal. In contrast, if the result is negative, the system will generate a “sell” signal.
5.4. Bollinger Signal
Bollinger Bands is an indicator that measures the volatility of prices. It calculates three “Bands,” where the middle band measures the moving average of a period, and the upper and lower bands are calculated by the standard deviation of the price [26]. Our trading system calculates the Bollinger Bands of each day. If a day’s close price is smaller than the day’s lower band and the day before that day’s close price is larger than the day before that day’s upper band, the system will generate a “buy” signal since that means the stock might just had an unusual fall and is likely to rise back. In contrast, if a day’s close price is higher than that day’s upper band, and the day before that day’s close price is lower than the day before that day’s lower band, the system will generate a “sell” signal since this means the stock might just had an unusual rise and is likely to fall back.
5.5. Machine Learning Signals and Models
Using SVM and LSTM, we constructed two hybrid models using both macroeconomic and technical indicators to forecast directional movement in the stocks. The crucial difference between them is that SVM treats all features as vectors without considering the sequence order. LSTM, on the contrary, reserves the autocorrelation of price series. Specifically, we apply 5-day lagged close prices on the recurrent units.
The main steps of the hybrid models, as shown in Figure 2, can be summarized as follows: (1)Calculate technical indicators(2)Preprocess data(3)Combine data and train SVM/LSTM model(4)Input data to the model and output predicted signal of directional movement
5.6. Trading System Structure
As shown in Figure 3, we propose a framework including data collection, data processing, portfolio optimization, funding decision, signals calculation, and trading to address complex stock prediction problem, namely, “Risk Adapting Stock Trading System” (RAST).
In the first part “data input,” we selected stock historical data and macroeconomic data as input. These data include daily price of open, low, high, close, volume, ten-year treasury constant maturity rates, FED rate, inflation rate, S&P 500, unemployment rate, and EUR/USD foreign exchange rate. The combination feasibility of historical data and macroeconomic data is proven in next section by empirical experiment as Table 2.
The second part “preprocessing” calculates technical indicators of 5-day MA and Bollinger Upper/Lower Bands. Then log-returns derived from prices are used to produce covariance matrix. The annualized log-return and covariance are calculated by mean value times 252 trading days in a year.
The third part “portfolio optimization” utilizes Markowitz Portfolio Optimization. It returns an efficient frontier graph to check on the correlation between expected return and volatility to meet different risk preference. With input of selected expected return, optimized asset weights will be used in the next module.
After “portfolio optimization,” the “funding decision” part decides how much money to invest on every asset in the portfolio. It requires a user input of total fund amount. The money will be distributed for each individual stock bearing on their weights calculated in the last module. It is also an upper limit for each stock’s cash availability. In the trading system, we treat each stock as an independent investment at the aspect of funding usage. It efficiently avoids the collision when many “buy” signals show up at the same time.
The fifth part “signals” produces different trading signals with back-testing result. The signals include “MA signal,” “RSI signal,” “momentum signal,” “Bollinger signal,” “SVM signal,” and “LSTM signal.” These signals are binary “buy” or “sell.” Back-testing result includes maximum drawdowns and returns of each signal. This result provides a reference for user to meet various risk preferences and profit expectation. The selected signal at this stage will be the only criterion in the next “trading” module to decide whether to buy or sell.
The last part “trading” is in charge of trading decision, trading amount, cash availability, and positions. Here we trade 1 unit (100 shares) per time. The opening position decision will be made only when the stock goes up for two days continuously and no “sell” signal. After long 1 unit, the system will do transaction based on coming signals. When a “buy” signal shows up, the system will long 1 unit if cash is more than cost. Other than depending on cash availability, the “sell” signal is based on positions. If current position is greater or equal to 100 shares, we sell 1 unit otherwise no trading.
6. Result
In the empirical study, we used S&P 500 stocks to construct the portfolio by Markowitz Portfolio Optimization, while some stocks were filtered out for the initial public offering time later than September 2011. The final amount of stocks included in the portfolio is 460. Then we conducted simulated trading on 4 portfolios corresponding to 20%, 30%, 40%, and 45% excepted return ratios. Buying or selling is based on momentum, RSI, MA, Bollinger, SVM, and LSTM signals separately. The simulated trading provides $200,000 for each run at start and conducts from 2019-09-03 to 2021-08-30. This part summarizes the empirical findings, including efficient frontier graph, table of portfolio returns based on different signals/indictors, and portfolio asset changes graph against different expected returns.
In Figure 4, the efficient frontier shows optimal portfolios’ annual returns from -2.5% to 40% against annual volatility from 3% to 20%.
Table 2 shows the max drawdown and return statistics for 20%, 30%, 40%, and 45% expected returns using momentum, RSI, MA, Bollinger, SVM, and LSTM signals, respectively, from 2019-9-03 to 2021-08-30. Macroeconomic data is applied on SVM and LSTM as an addition to verify the usefulness of the hybrid machine learning model combining technical and macroeconomic indicators.
Figures 5–12 show portfolios’ asset changes graphs with respect to different expected returns level and momentum, RSI, MA, Bollinger, SVM, and LSTM signals. The initial cash is $200,000 in each graph.
7. Discussion
Table 2 shows the results of different expected returns using different signals. The four different expected returns are four different points taken from the efficient frontier in Figure 4. As Table 2 shows, generally, the larger the expected return was, the larger the actual return would be for all signals. The max drawdown also increased for all signals when the expected return increased. But if we see the extent of the increase of returns and max-drawdowns, momentum benefits the most when increasing expected return. As expected return increased from 20% to 45%, its returns increased from 1.94% to 579.98%, while max drawdown also increased from 0.0294 to 0.55. This shows when the expected return is high, momentum’s nature of keep buying the good performing stocks might give extremely high returns but also can cause extremely high drawdowns. The signal RSI, on the other hand, does not benefit as much from increasing expected returns. Its returns only increased by 14% while max-drawdown increased by 18%.
Comparing the results from different signals, we can see that the signals that gave us the highest returns are momentum, LSTM (with and without macro) when expected return is 45%. Their returns for this expected return are 580%, 444%, and 124%, but when comparing returns in all levels of expected returns, LSTM (with macro) and SVM (with macro) are consistently performing better than the majority of other signals.
Comparing the signals LSTM and SVM with or without the use of macroeconomics, we can see the returns are much better for both signals when using macroeconomics when expected returns are high, while max-drawdowns also increase a little. It shows that macroeconomics is highly useful to get higher returns.
Figures 5–12 shows the movement graphs of these different signals with different expected returns. We started with $200,000 of assets at the beginning of each movement, and all of them ended up higher at the end after 2 years. From these graphs, we can see that all the signals experienced a large drawdown in the period of early 2020, which should be expected since that period was largely affected by the COVID-19 pandemic. What is interesting is we can observe that even in such a difficult period, signals Bollinger and RSI only had very little drawdowns, indicating that even though their returns at the end are lower than other signals, they might be less risky to survive in difficult periods when the market is not performing well. MA is also a signal that performed very differently than others that it keeps at almost the same level through the whole period of 2 years, showing that its performance is not much related to how the market is behaving.
Overall, the evidence from this study suggests that the combination of technical and macroeconomic data is feasible to apply on machine learning and help trading in stock market. Another important practical implication is that the strategy in the trading system is profitable. Finally, a number of potential improvements can be considered. The principal one is to add short selling function. The current selling decision will be made only when having enough long positions. Long-only trading will pay some opportunities cost when the market allows short. For the sake of different risk preferences and market regulations, short selling can be switched on/off at users’ discretion. The second consideration is transaction cost. This system does not include transaction cost into calculation, which may produce nonnegligible negative cash flow. Further research should be undertaken to explore setting limit on transactions to reduce unnecessary buying and selling actions.
8. Conclusion
In this research, we implemented a Risk Adapting Stock Trading System (RAST) that can automatically trade in the stock market to generate returns for users according to their risk preference. Even though most investors see macroeconomics and technical indicators with machine learning models as two very different styles in investing, the result of this research shows that it is useful to combine macroeconomics with technical indicators using machine learning methods to generate higher returns than using just one of them when the expected returns are high. The LSTM method combined with macroeconomics has relatively high returns for expected returns ranging from 20% to 45%, therefore outperforming other signals overall. Another advantage of this trading system is that it only requires data that are available and easy to find, which makes it much easier to be used in the real world.
Data Availability
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Conflicts of Interest
The author declares no conflicts of interest.