Abstract
Developing tools for monitoring the correlations among thousands of financial data streams in an online fashion can be interesting and useful work. We aimed to find highly correlative financial data streams in local patterns. A novel distance metric function slope duration distance (SDD) is proposed, which is compatible with the characteristics of actual financial data streams. Moreover, a model monitoring correlations among local patterns (MCALP) is presented, which dramatically decreases the computational cost using an algorithm quickly online segmenting and pruning (QONSP) with O(1) time cost at each time tick t, and our proposed new grid structure. Experimental results showed that MCALP provides an improvement of several orders of magnitude in performance relative to traditional naive linear scan techniques and maintains high precision. Furthermore, the model is incremental, parallelizable, and has a quick response time.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agrawal, R., Faloutsos, C., Swami, A., 1993. Efficient Similarity Search in Sequence Databases. Proc. Int. Conf. on Foundations of Data Organization and Algorithms, Chicago, Illinois. Springer-Verlag, Germany, p.69–74.
Bentley, J.L., Weide, B.W., Yao, A.C., 1980. Optimal expected-time algorithms for closest point problems. ACM Trans. Mathem. Software (TOMS), 6(4):563–580. [doi:10.1145/355921.355927]
Berndt, D.J., Clifford, J., 1996. Finding Patterns in Time Series: A Dynamic Programming Approach. Proc. Advances in Knowledge Discovery and Data Mining, AAAI/MIT Press, Menlo Park, CA, USA, p.229–248.
Chen, Q., Chen, L., Lian, X., Liu, Y., Jeffrey, X.Y., 2007. Indexable PLA for Efficient Similarity Search. Proc. VLDB Conf., Vienna, Austria. VLDB Endowment, USA, p.435–446.
Chen, Y.G., Nascimento, M.A., Ooi, B.C., Tung, A.K.H., 2007. Spade: On Shape-based Pattern Detection in Streaming Time Series. Proc. IEEE ICDE, Istanbul, Turkey. IEEE, USA, p.786–795. [doi:10.1109/ICDE.2007.367924]
Guha, S., Gunopulos, D., Koudas, N., 2003. Correlating Synchronous and Asynchronous Data Streams. Proc. ACM SIGKDD, Washington, D.C., USA. ACM, USA, p.529–534. [doi:10.1145/956750.956814]
Keogh, E., 2002. Exact Indexing of Dynamic Time Warping. Proc. VLDB Conf., Hong Kong, China. Morgan Kaufmann, USA, p.406–417.
Korn, F., Jagadish, H.V., Faloutsos, C., 1997. Efficiently Supporting Ad Hoc Queries in Large Datasets of Time Sequences. Proc. SIGMOD Conf., Birmingham, UK, p.289–300. [doi:10.1145/253260.253332]
Lian, X., Chen, L., Yu, J.X., Wang, G.R., Yu, G., 2007. Similarity Match over High Speed Time Series Streams. Proc. IEEE ICDE Conf., Istanbul, Turkey. IEEE, USA, p.1086–1095. [doi:10.1109/ICDE.2007.368967]
Papadimitriou, S., Yu, P.S., 2006. Optimal Multi-scale Patterns in Time Series Streams. Proc. ACM SIGMOD, Chicago, Illinois. ACM, USA, p.647–658. [doi:10.1145/1142473.1142545]
Papadimitriou, S., Sun, J., Faloutsos, C., 2005. Streaming Pattern Discovery in Multiple Time-series. Proc. VLDB Conf., Trondheim, Norway. ACM, USA, p.697–708.
Papadimitriou, S., Sun, J., Yu, P.S., 2006. Local Correlation Tracking in Time Series. Proc. IEEE ICDM, Hong Kong, China. IEEE, USA, p.456–465. [doi:10.1109/ICDM.2006.99]
Sakurai, Y., Papadimitriou, S., Faloutsos, C., 2005. Braid: Stream Mining through Group Lag Correlations. Proc. ACM SIGMOD, Baltimore, Maryland. ACM, USA, p.599–610. [doi:10.1145/1066157.1066226]
Sakurai, Y., Faloutsos, C., Yamamuro, M., 2007. Stream Monitoring under the Time Warping Distance. Proc. IEEE ICDE, Istanbul, Turkey. IEEE, USA, p.1046–1055. [doi:10.1109/ICDE.2007.368963]
Wu, H., Salzberg, B., Zhang, D., 2004. Online Event-driven Subsequence Matching over Financial Data Streams. Proc. ACM SIGMOD, Paris, France. ACM, USA, p.23–34. [doi:10.1145/1007568.1007574]
Zhang, T.C., Yue, D.J., Gu, Y., Yu, G., 2007. Boolean Representation Based Data-adaptive Correlation Analysis over Time Series Streams. Proc. ACM CIKM Conf., Lisboa, Portugal. ACM, USA, p.203–212. [doi:10.1145/1321440.1321471]
Zhu, Y., Shasha, D., 2002. Statstream: Statistical Monitoring of Thousands of Data Streams in Real Time. Proc. VLDB Conf., Hong Kong, China. Morgan Kaufmann, USA, p.358–369.
Author information
Authors and Affiliations
Corresponding author
Additional information
Project (Nos. 2006AA01Z430 and 2007AA01Z309) supported by the National Hi-Tech Research and Development Program (863) of China
Rights and permissions
About this article
Cite this article
Jiang, T., Feng, Yc., Zhang, B. et al. Monitoring correlative financial data streams by local pattern similarity. J. Zhejiang Univ. Sci. A 10, 937–951 (2009). https://doi.org/10.1631/jzus.A0820445
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.A0820445