Abstract
The integration of historical data has become one of the most pressing issues for the financial services industry: trading floors rely on real-time analytics of ticker data with very strong emphasis on speed, not scale, yet, a large number of critical tasks, including daily reporting and backtesting of models, put emphasis on scale. As a result, implementers continuously face the challenge of having to meet contradicting requirements and either scale real-time analytics technology at considerable cost, or deploy separate stacks for different tasks and keep them synchronized—a solution that is no less costly.
In this paper, we propose Adaptive Data Virtualization, as an alternative approach, to overcome this problem. ADV lets applications use different data management technologies without the need for database migrations or re-configuration of applications. We review the incumbent technology and compare it with the recent crop of MPP databases and draw up a strategy that, using ADV, lets enterprises use the right tool for the right job flexibly. We conclude the paper summarizing our initial experience working with customers in the field and outline an agenda for future research.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Garland, S.: Big Data Analytics: Tackling the Historical Data Challenge. Wired Magazine, Innovation Insights, October 2014
SAP HANA, May 2015. http://hana.sap.com/abouthana.html
Kx Systems, May 2015. http://kx.com
MemSQL, May 2015. http://www.memsql.com
New York Stock Exchange: Market Data, Data Products-Daily TAQ, May 2015. http://www.nyxdata.com/Data-Products/Daily-TAQ
Security Technology Analysis Center: STAC Benchmark Council-Various Benchmark Results, May 2015. http://stacresearch.com/research
Shasha, D.: KDB+ Database and Language Primer. Kx Systems, May 2005. http://kx.com/q/d/primer.htm
Soliman, M., et al.: Orca: a modular query optimizer architecture for big data. In: ACM SIGMOD Conference, May 2014
Informatica, May 2015. http://www.informatica.com
Amazon Redshift, May 2015. http://aws.amazon.com/redshift
IBM Netezza, May 2015. www.ibm.com/software/data/netezza
PostgreSQL, May 2015. http://www.postgresql.org
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Antova, L., Baldwin, R., Gu, Z., Waas, F.M. (2019). An Integrated Architecture for Real-Time and Historical Analytics in Financial Services. In: Castellanos, M., Chrysanthis, P., Pelechrinis, K. (eds) Real-Time Business Intelligence and Analytics. BIRTE BIRTE BIRTE 2015 2016 2017. Lecture Notes in Business Information Processing, vol 337. Springer, Cham. https://doi.org/10.1007/978-3-030-24124-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-24124-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-24123-0
Online ISBN: 978-3-030-24124-7
eBook Packages: Computer ScienceComputer Science (R0)