Skip to main content
SpringerLink
Log in

On the Computation of Optimized Trading Policies Using Deep Reinforcement Learning

  • Conference paper
  • First Online:
Telematics and Computing (WITCOM 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1280))

Included in the following conference series:

  • 733 Accesses

Abstract

In this paper we present a deep reinforcement learning-based methodology for computing optimized trading policies. During the first stage of the methodology, we employ Gated Recurrent Units (GRUs) to predict the immediate future behaviour of the time series that describe the temporal dynamics of the value of a set of assets. Then, we employ a Deep Q-Learning Architecture to compute optimized trading policies that describe, at every point in time, which assets have to be bought and which have to be sold in order to maximize profit. Our experimental results, which are based on trading cryptocurrencies, show that the proposed algorithm effectively computes trading policies that achieve incremental profits from an initial budget.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. More than 1,000 cryptocurrencies have already failed - here’s what will affect successes in the future. https://finance.yahoo.com/news/more-1-000-cryptocurrencies-already-144541916.html. Accessed 03 Jun 2020

  2. Crypto payments processor coinpayments partners with shopify. https://ibsintelligence.com/ibs-journal/ibs-news/crypto-payments-processor-coinpayments-partners-with-shopify/. Accessed 03 Jun 2020

  3. Lee, J.W., Kim, S.D., Lee, J.: An intelligent stock trading system based on reinforcement learning. IEICE Trans. Inf. Syst. 86(20), 296–305 (2003)

    Google Scholar 

  4. Lee, J.W., Park, J., Jangmin, O., Lee, J., Hong, E.: A multiagent approach to \(q\)-learning for daily stock trading. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 37(6), 864–877 (2007)

    Article  Google Scholar 

  5. Dempster, M.A.H., Payne, T.W., Romahi, Y., Thompson, G.W.P.: Computational learning techniques for intraday FX trading using popular technical indicators. IEEE Trans. Neural Netw. 12(4), 744–754 (2001)

    Article  Google Scholar 

  6. Chen, Y., Mabu, S., Hirasawa, K., Hu, J.: Trading rules on stock markets using genetic network programming with sarsa learning. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO 2007, New York, NY, USA, p. 1503. Association for Computing Machinery (2007)

    Google Scholar 

  7. Moody, J., Saffell, M.: Learning to trade via direct reinforcement. IEEE Trans. Neural Netw. 12(4), 875–889 (2001)

    Article  Google Scholar 

  8. Moody, J., Wu, L.: Optimization of trading systems and portfolios. In: Proceedings of the IEEE/IAFE 1997 Computational Intelligence for Financial Engineering (CIFEr), pp. 300–307 (1997)

    Google Scholar 

  9. Dempster, M.A.H., Leemans, V.: An automated FX trading system using adaptive reinforcement learning. Expert Syst. Appl. 30(3), 543–552 (2006). Intelligent Information Systems for Financial Engineering

    Article  Google Scholar 

  10. Gorse, D.: Application of stochastic recurrent reinforcement learning to index trading. In: ESANN 2011: European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (2011)

    Google Scholar 

  11. Zhang, J., Maringer, D.: Using a genetic algorithm to improve recurrent reinforcement learning for equity trading. Comput. Econ. 47(4), 551–567 (2016). https://doi.org/10.1007/s10614-015-9490-y

    Article  Google Scholar 

  12. Jangmin, O., Lee, J., Lee, J.W., Zhang, B.T.: Adaptive stock trading with dynamic asset allocation using reinforcement learning. Inf. Sci. 176(15), 2121–2147 (2006)

    Article  Google Scholar 

  13. Deng, Y., Bao, F., Kong, Y., Ren, Z., Dai, Q.: Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 653–664 (2017)

    Article  Google Scholar 

  14. David, W.L.: Agent inspired trading using recurrent reinforcement learning and LSTM neural networks (2017)

    Google Scholar 

  15. Mnih, V.: Playing atari with deep reinforcement learning. 2013. cite arxiv:1312.5602Comment. NIPS Deep Learning Workshop (2013)

  16. Get started with bitcoin. https://bitcoin.org/en/. Accessed: 2020–06-04

  17. What is litecoin? https://litecoin.org/. Accessed 04 Jun 2020

  18. What is ethereum? https://ethereum.org/what-is-ethereum/. Accessed 04 Jun 2020

  19. Lin, L.J.: Reinforcement learning for robots using neural networks. Ph.D. thesis, USA (1992)

    Google Scholar 

  20. Liu, R., Zou, J: The effects of memory replay in reinforcement learning, October 2017

    Google Scholar 

  21. Bertsekas, D.P.: Reinforcement learning and optimal control. In: Bertsekas, D., Nedić, A., Ozdaglar, A.E. (eds.) Athena Scientific Optimization and Computation Series. Athena Scientific, Belmont (2019)

    Google Scholar 

  22. Top 100 cryptocurrencies by market capitalization. https://coinmarketcap.com/. Accessed 04 Jun 2020

Download references

Author information

Authors and Affiliations

  1. Centro de Investigación en Computación, Av. Juan de Dios Bátiz S/N, Nueva Industrial Vallejo, Gustavo A. Madero, 07738, Ciudad de México, CDMX, Mexico

    Uriel Corona-Bermudez, Rolando Menchaca-Mendez & Ricardo Menchaca-Mendez

Authors
  1. Uriel Corona-Bermudez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rolando Menchaca-Mendez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ricardo Menchaca-Mendez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Uriel Corona-Bermudez .

Editor information

Editors and Affiliations

  1. Instituto Politécnico Nacional, México, Mexico

    Miguel Félix Mata-Rivera

  2. Instituto Politécnico Nacional, México, Mexico

    Roberto Zagal-Flores

  3. Universidad Mayor, Santiago de Chile, Chile

    Cristian Barria-Huidobro

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Corona-Bermudez, U., Menchaca-Mendez, R., Menchaca-Mendez, R. (2020). On the Computation of Optimized Trading Policies Using Deep Reinforcement Learning. In: Mata-Rivera, M.F., Zagal-Flores, R., Barria-Huidobro, C. (eds) Telematics and Computing. WITCOM 2020. Communications in Computer and Information Science, vol 1280. Springer, Cham. https://doi.org/10.1007/978-3-030-62554-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-62554-2_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-62553-5

  • Online ISBN: 978-3-030-62554-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation