Abstract
Stock/futures price forecasting is an important financial topic for individual investors, stock fund managers, and financial analysts and is currently receiving considerable attention from both researchers and practitioners. However, the inherent characteristics of stock/futures prices, namely, high volatility, complexity, and turbulence, make forecasting a challenging endeavor. In the past, various approaches have been proposed to deal with the problems of stock/futures price forecasting that are difficult to resolve by using only a single soft computing technique. In this study, a hybrid procedure based on a backpropagation (BP) neural network, a feature selection technique, and genetic programming (GP) is proposed to tackle stock/futures price forecasting problems with the use of technical indicators. The feasibility and effectiveness of this procedure are evaluated through a case study on forecasting the closing prices of Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) futures of the spot month. Experimental results show that the proposed forecasting procedure is a feasible and effective tool for improving the performance of stock/futures price forecasting. Furthermore, the most important technical indicators can be determined by applying a feature selection method based on the proposed simulation technique, or solely on the preliminary GP forecast model.
Similar content being viewed by others
References
Zarandi MHF, Rezaee B, Turksen IB, Neshat E (2009) A type-2 fuzzy rule-based expert system model for stock price analysis. Expert Syst Appl 36(1):139–154. doi:10.1016/j.eswa.2007.09.034
Thomsett MC (1998) Mastering fundamental analysis. Dearborn Finanical Publishing, Chicago
Thomsett MC (1999) Mastering technical analysis. Dearborn Finanical Publishing, Chicago
Box GEP, Jenkins GM (1970) Time series analysis, forecasting and control. Holden-Day, San Francisco
Bollerslev T (2001) Generalized autoregressive conditional heteroskedasticity. J Econ 100(1):A307–A327
Pai PF, Lin CS (2005) A hybrid ARIMA and support vector machines model in stock price forecasting. Omega-Int J Manag Sci 33(6):497–505. doi:10.1016/j.omega.2004.07.024
Chang PC, Liu CH (2008) A TSK type fuzzy rule based system for stock price prediction. Expert Syst Appl 34(1):135–144. doi:10.1016/j.eswa.2006.08.020
Ince H, Trafalis TB (2008) Short term forecasting with support vector machines and application to stock price prediction. Int J Gen Syst 37(6):677–687. doi:10.1080/03081070601068595
Huang CL, Tsai CY (2009) A hybrid SOFM-SVR with a filter-based feature selection for stock market forecasting. Expert Syst Appl 36(2):1529–1539. doi:10.1016/j.eswa.2007.11.062
Liang X, Zhang HS, Mao JG, Chen Y (2009) Improving option price forecasts with neural networks and support vector regressions. Neurocomputing 72(13–15):3055–3065. doi:10.1016/j.neucom.2009.03.015
Hadavandi E, Shavandi H, Ghanbari A (2010) Integration of genetic fuzzy systems and artificial neural networks for stock price forecasting. Knowl Based Syst 23(8):800–808. doi:10.1016/j.knosys.2010.05.004
Lu CJ (2010) Integrating independent component analysis-based denoising scheme with neural network for stock price prediction. Expert Syst Appl 37(10):7056–7064. doi:10.1016/j.eswa.2010.03.012
Cheng CH, Chen TL, Wei LY (2010) A hybrid model based on rough sets theory and genetic algorithms for stock price forecasting. Inf Sci 180(9):1610–1629. doi:10.1016/j.ins.2010.01.014
Fausett L (1994) Fundamentals of neural networks: architectures, algorithms, and applications. Prentice-Hall, Englewood Cliffs
Patterson DW (1996) Artificial neural networks: theory and applications. Prentice Hall, Singapore
Lin CT, Yeh HY (2009) Empirical of the Taiwan stock index option price forecasting model—applied artificial neural network. Appl Econ 41(15):1965–1972. doi:10.1080/00036840601131672
Yazici B, Memmedli M, Aslanargun A, Asma S (2010) Analysis of international debt problem using artificial neural networks and statistical methods. Neural Comput Appl 19(8):1207–1216. doi:10.1007/s00521-010-0422-4
Bilgili M, Sahin B (2010) Comparative analysis of regression and artificial neural network models for wind speed prediction. Meteorol Atmos Phys 109(1–2):61–72. doi:10.1007/s00703-010-0093-9
Hashemi MR, Ghadampour Z, Neill SP (2010) Using an artificial neural network to model seasonal changes in beach profiles. Ocean Eng 37(14–15):1345–1356. doi:10.1016/j.oceaneng.2010.07.004
Liu H, Motoda H (1998) Feature selection for knowledge discovery and data mining. Kluwer, Boston
Piramuthu S (2004) Evaluating feature selection methods for learning in data mining applications. Eur J Oper Res 156(2):483–494. doi:10.1016/s0377-2217(02)00911-6
Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97(1–2):273–324. doi:10.1016/s0004-3702(97)00043-x
Battiti R (1994) Using mutual information for selecting features in supervised neural-net learning. IEEE Trans Neural Netw 5(4):537–550
Jensen R, Shen Q (2005) Fuzzy-rough data reduction with ant colony optimization. Fuzzy Sets Syst 149(1):5–20. doi:10.1016/j.fss.2004.07.014
Song XP, Ding YS, Huang JW, Ge Y (2010) Feature selection for support vector machine in financial crisis prediction: a case study in China. Expert Syst 27(4):299–310. doi:10.1111/j.1468-0394.2010.00546.x
Bradley PS, Mangasarian OL, Street WN (1998) Feature selection via mathematical programming. INFORMS J Comput 10(2):209–217
Mladenic D (2006) Feature selection for dimensionality reduction. In: Saunders G, Grobelnik M, Gunn S, ShaweTaylor J (eds) Subspace, latent structure and feature selection, vol 3940. Lecture notes in computer science, pp 84–102
Sivanandam SN, Deepa SN (2008) Introduction to genetic algorithms. Springer, Berlin
Koza JR, Keane MA, Streeter MJ, Mydlowec W, Yu J, Lanza G (2005) Genetic programming IV: routine human-competitive machine intelligence
Koza JR, Streeter MJ, Keane MA (2008) Routine high-return human-competitive automated problem-solving by means of genetic programming. Inf Sci 178(23):4434–4452. doi:10.1016/j.ins.2008.07.028
Ciglaric I, Kidric A (2006) Computer-aided derivation of the optimal mathematical models to study gear-pair dynamic by using genetic programming. Struct Multidiscip Optim 32(2):153–160. doi:10.1007/s00158-006-0004-3
Etemadi H, Rostamy AAA, Dehkordi HF (2009) A genetic programming model for bankruptcy prediction: empirical evidence from Iran. Expert Syst Appl 36(2):3199–3207. doi:10.1016/j.eswa.2008.01.012
Hwang TM, Oh H, Choung YK, Oh S, Jeon M, Kim JH, Nam SH, Lee S (2009) Prediction of membrane fouling in the pilot-scale microfiltration system using genetic programming. Desalination 247(1–3):285–294. doi:10.1016/j.desal.2008.12.031
Bae H, Jeon TR, Kim S, Kim HS, Kim D, Han SS, May GS (2010) Optimization of silicon solar cell fabrication based on neural network and genetic programming modeling. Soft Comput 14(2):161–169. doi:10.1007/s00500-009-0438-9
Langdon WB, Poli R (2002) Foundations of genetic programming. Springer, New York
Kim KJ, Han I (2000) Genetic algorithms approach to feature discretization in artificial neural networks for the prediction of stock price index. Expert Syst Appl 19(2):125–132
Kim KJ, Lee WB (2004) Stock market prediction using artificial neural networks with optimal feature transformation. Neural Comput Appl 13(3):255–260. doi:10.1007/s00521-004-0428-x
Tsang PM, Kwok P, Choy SO, Kwan R, Ng SC, Mak J, Tsang J, Koong K, Wong TL (2007) Design and implementation of NN5 for Hong Kong stock price forecasting. Eng Appl Artif Intell 20(4):453–461. doi:10.1016/j.engappai.2006.10.002
Lai RK, Fan CY, Huang WH, Chang PC (2009) Evolving and clustering fuzzy decision tree for financial time series data forecasting. Expert Syst Appl 36(2):3761–3773. doi:10.1016/j.eswa.2008.02.025
Author information
Authors and Affiliations
Corresponding author
Appendix: Descriptions and definitions of technical indicators used in this study
Appendix: Descriptions and definitions of technical indicators used in this study
1.1 Notations
- i :
-
The day i
- \( {\text{HP}}_{i} \) :
-
The highest price of day i
- \( {\text{LP}}_{i} \) :
-
The lowest price of day i
- \( {\text{OP}}_{i} \) :
-
The opening price of day i
- \( {\text{CP}}_{i} \) :
-
The closing price of day i
- \( {\text{TV}}_{i} \) :
-
The trade volume of day i
-
1.
10-day moving average
The 10-day moving average is the mean price of the futures over the most recent 10 days and is calculated by:
$$ {\text{MA}}\_10_{i} = \frac{{\sum\nolimits_{j = i - 9}^{i} {{\text{CP}}_{j} } }}{10}. $$(18) -
2.
20-day bias
The 20-day bias is the deviation between the closing price and the 20-day moving average (MA_20) and is calculated by:
$$ {\text{BIAS}}\_20_{i} = \frac{{{\text{CP}}_{i} - {\text{MA}}\_20_{i} }}{{{\text{MA}}\_20_{i} }} . $$(19) -
3.
Moving average convergence/divergence
The moving average convergence/divergence is a momentum indicator that shows the relationship between two moving averages. First, define the demand index (DI) as:
$$ {\text{DI}}_{i} = {{\left( {{\text{HP}}_{i} + {\text{LP}}_{i} + 2 \times {\text{CP}}_{i} } \right)} \mathord{\left/ {\vphantom {{\left( {{\text{HP}}_{i} + {\text{LP}}_{i} + 2 \times {\text{CP}}_{i} } \right)} 4}} \right. \kern-\nulldelimiterspace} 4} . $$(20)Next, define the 12-day exponential moving average (EMA_12) and 26-day exponential moving average (EMA_26) as:
$$ {\text{EMA}}\_12_{i} = \frac{11}{13} \times {\text{EMA}}\_12_{i - 1} + \frac{2}{13} \times {\text{DI}}_{i} $$(21)and
$$ {\text{EMA}}\_26_{i} = \frac{25}{27} \times {\text{EMA}}\_26_{i - 1} + \frac{2}{27} \times {\text{DI}}_{i}, $$(22)respectively. Then, the difference between EMA_12 and EMA_26 can be calculated by:
$$ {\text{DIF}}_{i} = {\text{EMA}}\_12_{i} - {\text{EMA}}\_26_{i} . $$(23)Hence, the moving average convergence/divergence can be defined by:
$$ {\text{MACD}}_{i} = \frac{8}{10} \times {\text{MACD}}_{i - 1} + \frac{2}{10} \times {\text{DIF}}_{i} . $$(24) -
4.
9-day stochastic indicator K
The 9-day stochastic indicator K is defined as:
$$ \text{K}\_9_{i} = \frac{2}{3} \times \text{K}\_9_{i - 1} + \frac{1}{3} \times \frac{{{\text{CP}}_{i} - {\text{LP}}\_9_{i} }}{{{\text{HP}}\_9_{i} - {\text{LP}}\_9_{i} }} \times 100. $$(25)where \( {\text{LP}}\_9_{i} \) and \( {\text{HP}}\_9_{i} \) are the lowest and highest prices of the previous 9 days, i.e., days i, i − 1, …, i − 7, and i − 8, respectively.
-
5.
9-day stochastic indicator D
The 9-day stochastic indicator D is defined as:
$$ \text{D}\_9_{i} = \frac{2}{3} \times \text{D}\_9_{i - 1} + \frac{1}{3} \times {\text K}\_9_{i} . $$(26)where \( \text{K}\_9_{i} \) is the 9-day stochastic indicator K of day i, as previously defined.
-
6.
9-day Williams overbought/oversold index
The 9-day Williams overbought/oversold index is a momentum indicator that measures overbought and oversold levels and is calculated by:
$$ {\text{WMS}}\% \text{R}\_9_{i} = \frac{{{\text{HP}}\_9_{i} - {\text{CP}}_{i} }}{{{\text{HP}}\_9_{i} - {\text{LP}}\_9_{i} }} $$(27)where \( {\text{LP}}\_9_{i} \) and \( {\text{HP}}\_9_{i} \) are the lowest and highest prices of the previous 9 days, i.e., days i, i − 1, …, i − 7, and i − 8, respectively.
-
7.
10-day rate of change
The 10-day rate of change measures the percent changes of the current price relative to the price of 10 days ago and is calculated by:
$$ {\text{ROC}}\_10_{i} = \frac{{{\text{CP}}_{i} - {\text{CP}}_{i - 10} }}{{{\text{CP}}_{i - 10} }} \times 100. $$(28) -
8.
5-day relative strength index
The relative strength index is a momentum oscillator that compares the magnitude of recent gains to the magnitude of recent losses. First, define the gain of day i as:
$$ \text{G}_{i} = \left\{ {\begin{array}{*{20}c} {{\text{CP}}_{i} - {\text{CP}}_{i - 1} } \hfill & {{\text{if}}\quad {\text{CP}}_{i} > {\text{CP}}_{i - 1} } \hfill \\ 0 \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right..$$(29)Similarly, the loss of day i is calculated by:
$$ \text{L}_{i} = \left\{ {\begin{array}{*{20}c} {{\text{CP}}_{i - 1} - {\text{CP}}_{i} } \hfill & {{\text{if}}\quad {\text{CP}}_{i} < {\text{CP}}_{i - 1} } \hfill \\ 0 \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right..$$(30)Next, the 5-day average gain (AG_5) and 5-day average loss (AL_5) can be calculated by:
$$ {\text{AG}}\_5_{i} = \frac{4}{5} \times {\text{AG}}\_5_{i - 1} + \frac{1}{5} \times \text{G}_{i} $$(31)and
$$ {\text{AL}}\_5_{i} = \frac{4}{5} \times {\text{AL}}\_5_{i - 1} + \frac{1}{5} \times \text{L}_{i} , $$(32)respectively. Hence, the 5-day relative strength index can be defined by:
$$ {\text{RSI}}\_5_{i} = \frac{{{\text{AG}}\_5_{i} }}{{{\text{AG}}\_5_{i} + {\text{AL}}\_5_{i} }} \times 100. $$(33) -
9.
24-day commodity channel index
The commodity channel index is used to identify cyclical turns in commodities. First, define the typical price (TP) as:
$$ {\text{TP}}_{i} = \frac{{{\text{HP}}_{i} + {\text{LP}}_{i} + {\text{CP}}_{i} }}{3}. $$(34)Next, calculate the 24-day simple moving average of the typical price (SMATP_24) by:
$$ {\text{SMATP}}\_24_{i} = \frac{{\sum\nolimits_{j = i - 23}^{i} {{\text{TP}}_{j} } }}{24}. $$(35)Then, the 24-day mean deviation (MD_24) can be calculated by:
$$ {\text{MD}}\_24_{i} = \frac{{\sum\nolimits_{j = i - 23}^{i} { | {\text{TP}}_{j} - {\text{SMATP}}\_24_{i} |} }}{24}. $$(36)Hence, the 24-day commodity channel index can be defined as:
$$ {\text{CCI}}\_24_{i} = \frac{{{\text{TP}}_{i} - {\text{SMATP}}\_24_{i} }}{{0.015 \times {\text{MD}}\_24_{i} }}. $$(37) -
10.
26-day volume ratio
The 26-day volume ratio is defined by:
$$ {\text{VR}}\_26_{i} = \frac{{{{{\text{TVU}}\_26_{i} - {\text{TVF}}\_26_{i} } \mathord{\left/ {\vphantom {{{\text{TVU}}\_26_{i} - {\text{TVF}}\_26_{i} } 2}} \right. \kern-\nulldelimiterspace} 2}}}{{{{{\text{TVD}}\_26_{i} - {\text{TVF}}\_26_{i} } \mathord{\left/ {\vphantom {{{\text{TVD}}\_26_{i} - {\text{TVF}}\_26_{i} } 2}} \right. \kern-\nulldelimiterspace} 2}}} \times 100\% . $$(38)where \( {\text{TVU}}\_26_{i} \), \( {\text{TVD}}\_26_{i} \), and \( {\text{TVF}}\_26_{i} \) represent the total trade volumes of stock prices rising, falling, and holding, respectively, from the previous 26 days, i.e., days i, i − 1, …, i – 24, and i − 25.
-
11.
13-day psychological line
The psychological line is a volatility indicator based on the number of time intervals that the market was up during the preceding period. The 13-day psychological line is defined by:
$$ {\text{PSY}}\_13_{i} = \frac{{{\text{TDU}}\_13_{i} }}{13} \times 100\% . $$(39)where \( {\text{TDU}}\_13_{i} \) is the total number of days with regard to stock price rises of the previous 13 days, i.e., days i, i − 1, …, i – 11, and i − 12.
-
12.
14-day plus directional indicator
First, define plus directional movement (+DM) and minus directional movement (−DM) as:
$$ + {\text{DM}}_{i} = {\text{HP}}_{i} - {\text{HP}}_{i - 1} $$(40)and
$$ - {\text{DM}}_{i} = {\text{LP}}_{i - 1} - {\text{LP}}_{i}, $$(41)respectively. The plus true directional movement (+TDM) can be calculated by:
$$ + {\text{TDM}}_{i} = \left\{ {\begin{array}{*{20}c} { + {\text{DM}}_{i} } \hfill & {{\text{if}}\quad + {\text{DM}}_{i} > - {\text{DM}}_{i} \quad {\text{and}}\quad + {\text{DM}}_{i} > 0} \hfill \\ 0 \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right.. $$(42)Similarly, the minus true directional movement (−TDM) can be calculated by:
$$ - {\text{TDM}}_{i} = \left\{ {\begin{array}{*{20}c} { - {\text{DM}}_{i} } \hfill & {{\text{if}}\quad + {\text{DM}}_{i} < - {\text{DM}}_{i} \quad {\text{and}}\quad - {\text{DM}}_{i} > 0} \hfill \\ 0 \hfill & {\text{otherwise}} \hfill \\ \end{array} } \right.. $$(43)Hence, the 14-day plus directional movement (+DM_14) can be calculated by:
$$ + {\text{DM}}\_14_{i} = \frac{13}{14} \times \left( { + {\text{DM}}\_1 4_{i - 1} } \right) + \frac{1}{14} \times \left( { + {\text{TDM}}_{i} } \right). $$(44)Similarly, the 14-day minus directional movement (−DM_14) can be calculated by:
$$ - {\text{DM}}\_14_{i} = \frac{13}{14} \times \left( { - {\text{DM}}\_14_{i - 1} } \right) + \frac{1}{14} \times \left( { - {\text{TDM}}_{i} } \right). $$(45)Next, define the true range (TR) as:
$$ {\text{TR}}_{i} = {\text{Max}}\{ {\text{HP}}_{i} - {\text{LP}}_{i} ,|{\text{HP}}_{i} - {\text{CP}}_{i - 1} |,|{\text{LP}}_{i} - {\text{CP}}_{i - 1} |\} . $$(46)The 14-day true range (TR_14) can be calculated by:
$$ {\text{TR}}\_14_{i} = \frac{13}{14} \times {\text{TR}}\_14_{i - 1} + \frac{1}{14} \times {\text{TR}}_{i} . $$(47)Therefore, the 14-day plus directional indicator can be defined as:
$$ + {\text{DI}}\_14_{i} = \frac{{ + {\text{DM}}\_14_{i} }}{{{\text{TR}}\_14_{i} }}. $$(48) -
13.
14-day minus directional indicator
The 14-day minus directional indicator is defined as:
$$ - {\text{DI}}\_14_{i} = \frac{{ - {\text{DM}}\_14_{i} }}{{{\text{TR}}\_14_{i} }}. $$(49)where \( - {\text{DM}}\_14_{i} \) and \( {\text{TR}}\_14_{i} \) are, respectively, the 14-day minus directional movement and 14-day true range of day i, as previously defined in (45) and (47).
-
14.
26-day buying/selling momentum indicator
The 26-day buying/selling momentum indicator is defined as:
$$ {\text{AR}}\_26_{i} = \frac{{\sum\nolimits_{j = i - 25}^{i} {\left( {{\text{HP}}_{j} - {\text{OP}}_{j} } \right)} }}{{\sum\nolimits_{j = i - 25}^{i} {\left( {{\text{OP}}_{j} - {\text{LP}}_{j} } \right)} }}. $$(50) -
15.
26-day buying/selling willingness indicator
The 26-day buying/selling willingness indicator is defined as:
$$ {\text{BR}}\_26_{i} = \frac{{\sum\nolimits_{j = i - 25}^{i} {\left( {{\text{HP}}_{j} - {\text{CP}}_{j - 1} } \right)} }}{{\sum\nolimits_{j = i - 25}^{i} {\left( {{\text{CP}}_{j - 1} - {\text{LP}}_{j} } \right)} }}. $$(51)
Rights and permissions
About this article
Cite this article
Hsu, CM. A hybrid procedure with feature selection for resolving stock/futures price forecasting problems. Neural Comput & Applic 22, 651–671 (2013). https://doi.org/10.1007/s00521-011-0721-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-011-0721-4