Abstract
This paper will demonstrate a novel method for consolidating data in an engineered hypercube network for the purpose of optimizing query processing. Query processing typically calls for merging data collected from a small subset of server nodes in a network. This poses the problem of managing efficiently the exchange of data between processing nodes to complete some relational data operation. The method developed here is designed to minimize data transfer, measured as the product of data quantity and network distance, by delegating the processing to a node that is relatively central to the subset. A hypercube not only supports simple computation of network distance between nodes, but also allows for identifying a node to serve as the center for any data consolidation operations. We will show how the consolidation process can be performed by selecting a subgraph of a complex network to simplify the selection of a central node and thus facilitate the computations required. We will also show a prototype implementation of a hypercube using Software-Defined Networking to support query optimization in a distributed heterogeneous database system, making use of network distance information and data quantity.
This material is based upon work supported by the National Science Foundation under Grant No. 1818884.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arabi, K.: Mobile computing opportunities, challenges and technology drivers. In: IEEE DAC 2014 Keynote (2014)
Arabi, K.l Trends, opportunities and challenges driving architecture and design of next generation mobile computing and iot devices. In: MIT MTL Seminar (2015)
Bent, G., Dantressangle, P., Stone, P., Vyvyan, D., Mowshowitz, A.: Experimental evaluation of the performance and scalability of a dynamic distributed federated database. In: Proceedings of the Second Annual Conference of ITA (2009)
Bent, G., Dantressangle, P., Vyvyan, D., Mowshowitz, A., Mitsou, V.: A dynamic distributed federated database. In: Proceedings 2nd Annual Conference International Technology Alliance (2008)
Bent, G.A., Dantressangle, P., Stone, P.D.: Optimising data transmission in a hypercube network. Technical report, IBM-UK (2019)
Bernstein, P.A., Chiu, D.-M.W.: Using semi-joins to solve relational queries. J. ACM 28(1), 25–40 (1981)
Bernstein, P.A., Goodman, N., Wong, E., Reeve, C.L., Rothnie, J.B., Jr.: Query processing in a system for distributed databases (sdd-1). ACM Trans. Database Syst. 6(4), 602–625 (1981)
Bouganim, L., Fabret, F., Mohan, C., Valduriez, P.: Dynamic query scheduling in data integration systems. In: 16th International Conference on Data Engineering, pp. 425–434 (2000)
Chaudhuri, S.: An overview of query optimization in relational systems. In: Proceedings of the Seventeenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, pp. 34–43. ACM (1998)
Chourabi, H., et al.: Understanding smart cities: an integrative framework. In: 2012 45th Hawaii International Conference on System Sciences, pp. 2289–2297 (2012)
City of New York. https://opendata.cityofnewyork.us. NYC OpenData
Cormen, T.: Introduction to Algorithms, vol. 15, 2nd edn. The MIT press, Cambridge (2001)
Floyd, R.W.: Algorithm 97: shortest path. Communi. ACM 5(6), 345 (1962)
Garcia-Molina, H., Ullman, J.D., Widom, J.: Database Systems - The Complete Book, 2nd edn. Pearson Education (2009)
Jiang, Y., Taniar, D., Leung, C.: High performance distributed parallel query processing. Comput. Syst. Sci. Eng. 16, 277–289 (2001)
Kawaguchi, A., et al.: A model of query performance in dynamic distributed federated databases taking account of network topology. In: Annual Conference of International Technology Alliance in Network and Information Science (ACITA2012) (2012)
Kawaguchi, A., Mowshowitz, A., Shibata, M.: Semi-operational data reductions for query processing in highly distributed data environments (extended abstract). In: US-Japan Workshop on Programmable Networking, Kyoto, Japan (2020)
Miorandi, D., Sicari, S., De Pellegrini, F., Chlamtac, I.: Internet of things: Vision, applications and research challenges. Ad Hoc Netw. 10(7), 1497–1516 (2012)
Mowshowitz, A., Bent, G.: Formal properties of distributed database networks. In: Annual Conference of the International Technology Alliance, University of Maryland (2007)
Mowshowitz, A., et al.: Query optimization in a distributed hypercube database. In: Proceedings of the Fourth Annual Conference of ITA (2010)
Mowshowitz, A., Kawaguchi, A., Tsuru, M.: Topology as a factor in overlay networks designed to support dynamic systems modeling. In: 13th International Conference on Intelligent Networking and Collaborative Systems (INCoS-2021) (2021). in press
Open Networking Foundation. https://mininet.org. Mininet
Open Networking Foundation. https://opennetworking.org. OpenFlow
Organization, N.: Networkx - Network Analysis in Python. https://networkx.org
Penrose, M.: Random Geometric graphs. Oxford University Press, Oxford (2003)
Rothnie, J., Jr., Bernstein, P., Fox, S., Goodman, N., Hammer, M., Landers, T., Reeve, C., Shipman, D., Wong, E.: Introduction to a system for distributed databases (sdd-1). ACM Trans. Database Syst. (TODS) 5(1), 1–17 (1980)
Ryu SDN Framework Community. https://ryu-sdn.org. Ryu
Saadawi, T., Kawaguchi, A., Lee, M.J., Mowshowitz, A.: Secure resilient edge cloud designed network (invited). IEICE Trans. Commun. E103-B(4) (2020)
Taniar, D., Leung, C.H.C., Rahayu, J.W., Goel, S.: High Performance Parallel Database Processing and Grid Databases. John Wiley & Son, Hoboken (2008)
Toce, A., Mowshowitz, A., Kawaguchi, A., Stone, P., Dantressangle, P., Bent, G.: Hyperd: Analysis and performance evaluation of a distributed hypercube database databases. In: Proceedings of the Sixth Annual Conference of ITA (2012)
Toce, A., Mowshowitz, A., Kawaguchi, A., Stone, P., Dantressangle, P., Bent, G.: An efficient hypercube labeling schema for dynamic peer-to-peer networks. J. Parallel Distrib. Comput. 102, 186–198 (2017)
Yi, S., Li, C., Li, Q.: A survey of fog computing: concepts, applications and issues. In: Proceedings of the 2015 Workshop on Mobile Big Data (2015)
Yildirim, I.: Query operations in highly distributed environment. Master’s thesis, City College of New York (2014)
Acknowledgement
Authors of this paper are grateful to students in the Senior Design courses offered during the academic year 2020–2021 at the City College of New York. Each of the six teams produced a distributed database application which runs on the hypercube of \(2^4\)-nodes and successfully completed performance experiments to exhibit the result in this paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kawaguchi, A., Ha, N.V., Tsuru, M., Mowshowitz, A., Shibata, M. (2022). Query Processing in Highly Distributed Environments. In: Barolli, L., Chen, HC., Miwa, H. (eds) Advances in Intelligent Networking and Collaborative Systems. INCoS 2021. Lecture Notes in Networks and Systems, vol 312. Springer, Cham. https://doi.org/10.1007/978-3-030-84910-8_30
Download citation
DOI: https://doi.org/10.1007/978-3-030-84910-8_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-84909-2
Online ISBN: 978-3-030-84910-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)