Skip to main content
SpringerLink
Log in

(A)kNN Query Processing on the Cloud: A Survey

  • Conference paper
  • First Online:
Book cover Algorithmic Aspects of Cloud Computing (ALGOCLOUD 2016)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10230))

Included in the following conference series:

Abstract

A k-nearest neighbor (kNN) query determines the k nearest points, using distance metrics, from a given location. An all k-nearest neighbor (AkNN) query constitutes a variation of a kNN query and retrieves the k nearest points for each point inside a database. Their main usage resonates in spatial databases and they consist the backbone of many location-based applications and not only. Although (A)kNN is a fundamental query type, it is computationally very expensive. During the last years a multiplicity of research papers has focused around the distributed (A)kNN query processing on the cloud. This work constitutes a survey of research efforts towards this direction. The main contribution of this work is an up-to-date review of the latest (A)kNN query processing approaches. Finally, we discuss various research challenges and directions of further research around this domain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abbasifard, M.R., Ghahremani, B., Naderi, H.: A survey on nearest neighbor search methods. Int. J. Comput. Appl. 95, 39–52 (2014)

    Google Scholar 

  2. Abdelsadek, A., Hefeeda, M.: DIMO: distributed index for matching multimedia objects using MapReduce. In: Proceedings of the 5th ACM Multimedia Systems Conference, pp. 115–126. ACM, New York (2014)

    Google Scholar 

  3. Aji, A., Wang, F.: High performance spatial query processing for large scale scientific data. In: Proceedings of the on SIGMOD/PODS 2012 PhD Symposium, pp. 9–14. ACM, New York (2012)

    Google Scholar 

  4. Aji, A., Wang, F., Saltz, J.H.: Towards building a high performance spatial query system for large scale medical imaging data. In: Proceedings of the 20th International Conference on Advances in Geographic Information Systems, pp. 309–318. ACM, New York (2012)

    Google Scholar 

  5. Aji, A., Wang, F., Vo, H., Lee, R., Liu, Q., Zhang, X., Saltz, J.: Hadoop GIS: a high performance spatial data warehousing system over MapReduce. Proc. VLDB Endow. 6, 1009–1020 (2013)

    Article  Google Scholar 

  6. Akdogan, A., Demiryurek, U., Kashani, F.B., Shahabi, C.: Voronoi-based geospatial query processing with MapReduce. In: Proceedings of the IEEE 2nd International Conference on Cloud Computing Technology and Science, pp. 9–16. IEEE Computer Society, Washington, DC (2010)

    Google Scholar 

  7. Aly, M., Munich, M., Perona, P.: Distributed Kd-trees for retrieval from very large image collections. In: Proceedings of the British Machine Vision Conference (BMVC) (2011)

    Google Scholar 

  8. Andreica, M.I., Tapus, N.: Sequential and MapReduce-based algorithms for constructing an in-place multidimensional quad-tree index for answering fixed-radius nearest neighbor queries. Acta Universitatis Apulensis - Mathematics-Informatics, pp. 131–151 (2012)

    Google Scholar 

  9. Baig, F., Mehrotra, M., Vo, H., Wang, F., Saltz, J., Kurc, T.: SparkGIS: efficient comparison and evaluation of algorithm results in tissue image analysis studies. In: Wang, F., Luo, G., Weng, C., Khan, A., Mitra, P., Yu, C. (eds.) Big-O(Q)/DMAH -2015. LNCS, vol. 9579, pp. 134–146. Springer, Cham (2016). doi:10.1007/978-3-319-41576-5_10

    Google Scholar 

  10. Bhatia, N.: Vandana: Survey of Nearest Neighbor Techniques. CoRR abs/1007.0085 (2010)

    Google Scholar 

  11. Böhm, C., Krebs, F.: The k-nearest neighbour join: turbo charging the KDD process. Knowl. Inf. Syst. 6, 728–749 (2004)

    Article  Google Scholar 

  12. Candan, K.S., Nagarkar, P., Nagendra, M., Yu, R.: RanKloud: a scalable ranked query processing framework on hadoop. In: Proceedings of the 14th International Conference on Extending Database Technology, pp. 574–577. ACM, New York (2011)

    Google Scholar 

  13. Cary, A., Sun, Z., Hristidis, V., Rishe, N.: Experiences on processing spatial data with MapReduce. In: Winslett, M. (ed.) SSDBM 2009. LNCS, vol. 5566, pp. 302–319. Springer, Heidelberg (2009). doi:10.1007/978-3-642-02279-1_24

    Chapter  Google Scholar 

  14. Cech, P., Kohout, J., Lokoc, J., Komárek, T., Marousek, J., Pevný, T.: Feature extraction and malware detection on large HTTPS data using MapReduce. In: Amsaleg, L., Houle, M.E., Schubert, E. (eds.) SISAP 2016. LNCS, vol. 9939, pp. 311–324. Springer, Cham (2016). doi:10.1007/978-3-319-46759-7_24

  15. Chatzimilioudis, G., Costa, C., Zeinalipour-Yazti, D., Lee, W.-C., Pitoura, E.: Distributed in-memory processing of all k nearest neighbor queries. IEEE Trans. Knowl. Data Eng. 28, 925–938 (2016)

    Article  Google Scholar 

  16. Chen, Y., Patel, J.M.: Efficient evaluation of all-nearest-neighbor queries. In: Proceedings of the 23rd IEEE International Conference on Data Engineering, pp. 1056–1065. IEEE Computer Society, Washington, DC (2007)

    Google Scholar 

  17. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: Proceedings of the 6th Symposium on Operating Systems Design and Implementation, pp. 137–150. USENIX Association, Berkeley (2004)

    Google Scholar 

  18. Deng, Z., Zhu, X., Cheng, D., Zong, M., Zhang, S.: Efficient kNN classification algorithm for big data. Neurocomputing 195, 143–148 (2016)

    Article  Google Scholar 

  19. Dhanabal, S., Chandramathi, S.: A review of various k-nearest neighbor query processing techniques. Int. J. Comput. Appl. 31, 14–22 (2011)

    Google Scholar 

  20. Dong, X., Feifei, L., Bin, Y., Gefei, L., Liang, Z., Minyi, G.: Simba: efficient in-memory spatial analytics. In: Proceedings of the 2016 International Conference on Management of Data, pp. 1071–1085. ACM, New York (2016)

    Google Scholar 

  21. Du, Q., Li, X.: A novel KNN join algorithms based on Hilbert R-tree in MapReduce. In: Proceedings of the 3rd International Conference on Computer Science and Network Technology, pp. 417–420. IEEE (2013)

    Google Scholar 

  22. Eldawy, A., Mokbel, M.F.: SpatialHadoop: a MapReduce framework for spatial data. In: Proceedings of the 31st IEEE International Conference on Data Engineering, pp. 1352–1363. IEEE Computer Society, Washington, DC (2015)

    Google Scholar 

  23. Emrich, T., Graf, F., Kriegel, H.-P., Schubert, M., Thoma, M.: Optimizing all-nearest-neighbor queries with trigonometric pruning. In: Gertz, M., Ludäscher, B. (eds.) SSDBM 2010. LNCS, vol. 6187, pp. 501–518. Springer, Heidelberg (2010). doi:10.1007/978-3-642-13818-8_35

    Chapter  Google Scholar 

  24. Gkoulalas-Divanis, A., Verykios, V.S., Bozanis, P.: A network aware privacy model for online requests in trajectory data. Data Knowl. Eng. 68, 431–452 (2009)

    Article  Google Scholar 

  25. Ioup, E., Shaw, K., Sample, J., Abdelguerfi, M.: Efficient AKNN spatial network queries using the M-Tree. In: Proceedings of the 15th Annual ACM International Symposium on Advances in Geographic Information Systems, pp. 46:1–46:4. ACM, New York (2007)

    Google Scholar 

  26. Ji, C., Dong, T., Li, Y., Shen, Y., Li, K., Qiu, W., Qu, W., Guo, M.: Inverted grid-based kNN query processing with MapReduce. In: Proceedings of the 7th ChinaGrid Annual Conference, pp. 25–32 (2012)

    Google Scholar 

  27. Ji, C., Li, Z., Qu, W., Xu, Y., Li, Y.: Scalable nearest neighbor query processing based on Inverted Grid Index. J. Network Comput. Appl. 44, 172–182 (2014)

    Article  Google Scholar 

  28. Kokotinis, I., Kendea, M., Nodarakis, N., Rapti, A., Sioutas, S., Tsakalidis, A.K., Tsolis, D., Panagis, Y.: NSM-Tree: efficient indexing on top of NoSQL databases. In: Post-proceedings of the 2nd International Workshop on Algorithmic Aspects of Cloud Computing (2016)

    Google Scholar 

  29. Liao, H., Jizhong, H., Jinyun, F.: Multi-dimensional index on hadoop distributed file system. In: Proceedings of the 2010 IEEE Fifth International Conference on Networking, Architecture, and Storage, pp. 240–249. IEEE Computer Society, Washington, DC, USA (2010)

    Google Scholar 

  30. Liu, T., Rosenberg, C., Rowley, H.A.: Clustering billions of images with large scale nearest neighbor search. In: Proceedings of the 8th IEEE Workshop on Applications of Computer Vision, p. 28. IEEE Computer Society (2007)

    Google Scholar 

  31. Lu, P., Chen, G., Ooi, B.C., Vo, H.T., Wu, S.: ScalaGiST: scalable generalized search trees for mapreduce systems [Innovative Systems Paper]. Proc. VLDB Endow. 7, 1797–1808 (2014)

    Article  Google Scholar 

  32. Lu, W., Shen, Y., Chen, S., Ooi, B.C.: Efficient processing of k nearest neighbor joins using mapreduce. Proc. VLDB Endow. 5, 1016–1027 (2012)

    Article  Google Scholar 

  33. Mahapatra, R.P., Chakraborty, P.S.: Comparative analysis of nearest neighbor query processing techniques. Procedia Comput. Sci. 57, 1289–1298 (2015)

    Article  Google Scholar 

  34. Maillo, J., Ramireza, S., Triguero, I., Herrera, F.: kNN-IS: an iterative spark-based design of the k-nearest neighbors classifier for big data. Knowledge-Based Systems (2016, in press)

    Google Scholar 

  35. Maillo, J., Triguero, I., Herrera, F.: A MapReduce-based k-nearest neighbor approach for big data classification. In: IEEE TrustCom/BigDataSE/ISPA, pp. 167–172. IEEE Computer Society, Washington, DC (2015)

    Google Scholar 

  36. Maleki, E.F., Azadani, M.N., Ghadiri, N.: Performance evaluation of spatialhadoop for big web mapping data. In: Proceedings of the 2016 Second International Conference on Web Research. IEEE Computer Society, Washington, DC (2016, to be published)

    Google Scholar 

  37. Muja, M., Lowe, D.G.: Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans. Pattern Anal. Mach. Intell. 36, 2227–2240 (2014)

    Article  Google Scholar 

  38. Naami, K.M.A., Seker, S., Khan, L.: GISQF: an efficient spatial query processing system. In: Proceedings of the 2014 IEEE International Conference on Cloud Computing, pp. 681–688. IEEE Computer Society, Washington, DC (2014)

    Google Scholar 

  39. Nishimura, S., Das, S., Agrawal, D., Abbadi, A.E.: MD-HBase: a scalable multi-dimensional data infrastructure for location aware services. In: Proceedings of the 2011 IEEE 12th International Conference on Mobile Data Management, vol. 01, pp. 7–16. IEEE Computer Society, Washington, DC (2011)

    Google Scholar 

  40. Nodarakis, N., Pitoura, E., Sioutas, S., Tsakalidis, A., Tsoumakos, D., Tzimas, G.: Efficient multidimensional AkNN query processing in the cloud. In: Decker, H., Lhotská, L., Link, S., Spies, M., Wagner, R.R. (eds.) DEXA 2014. LNCS, vol. 8644, pp. 477–491. Springer, Cham (2014). doi:10.1007/978-3-319-10073-9_41

    Google Scholar 

  41. Nodarakis, N., Pitoura, E., Sioutas, S., Tsakalidis, A., Tsoumakos, D., Tzimas, G.: kdANN+: a rapid AkNN classifier for big data. Trans. Large-Scale Data Knowl. Centered Syst. 24, 139–168 (2016)

    Google Scholar 

  42. Nodarakis, N., Sioutas, S., Tsakalidis, A., Tzimas, G.: Large scale sentiment analysis on Twitter with spark. In: Proceedings of the Workshops of the EDBT/ICDT 2016 Joint Conference, CEUR Workshop Proceedings, vol. 1558 (2016). CEUR-WS.org

  43. Nodarakis, N., Sioutas, S., Tsakalidis, A., Tzimas, G.: MR-SAT: a MapReduce algorithm for big data sentiment analysis on Twitter. In: Proceedings of the 12th International Conference on Web Information Systems and Technologies, vol. 1, pp. 140–147. SciTePress (2016)

    Google Scholar 

  44. Plaku, E., Kavraki, L.E.: Distributed computation of the knn graph for large high-dimensional point sets. J. Parallel Distrib. Comput. 67, 346–359 (2007)

    Article  MATH  Google Scholar 

  45. Reyes-Ortiz, J.L., Oneto, L., Anguita, D.: Big data analytics in the cloud: spark on Hadoop vs MPI/OpenMP on Beowulf. Procedia Comput. Sci. 53, 121–130 (2015)

    Article  Google Scholar 

  46. Roussopoulos, N., Kelley, S., Vincent, F.: Nearest neighbor queries. In: Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, pp. 71–79. ACM, New York (1995)

    Google Scholar 

  47. Song, G., Rochas, J., Huet, F., Magoulès, F.: Solutions for processing K nearest neighbor joins for massive data on MapReduce. In: Proceedings of the 23rd Euromicro International Conference on Parallel, Distributed and Network-based Processing, March 2015, Turku, Finland (2015)

    Google Scholar 

  48. Song, G., Rochas, J., Huet, F., Magoulès, F.: K nearest neighbour joins for big data on MapReduce: a theoretical and experimental analysis. IEEE Trans. Knowl. Data Eng. 28, 2376–2392 (2016)

    Article  Google Scholar 

  49. Stupar, A., Michel, S., Schenkel, R.: RankReduce - processing K-nearest neighbor queries on top of MapReduce. In: Proceedings of the 8th Workshop on Large-Scale Distributed Systems for Information Retrieval, pp. 13–18. ACM, New York (2010)

    Google Scholar 

  50. Sun, K., Kang, H., Park, H.-H.: Tagging and classifying facial images in cloud environments based on KNN using MapReduce. Optik - Int. J. Light Electron Optics 126, 3227–3233 (2015)

    Article  Google Scholar 

  51. Sun, Z., Zhang, H., Liu, Z., Xu, C., Wang, L.: Migrating GIS big data computing from Hadoop to Spark: an exemplary study Using Twitter. In: Proceedings of the IEEE 9th International Conference on Cloud Computing, pp. 351–358. IEEE Computer Society, Washington, DC (2016)

    Google Scholar 

  52. Talavera-Llames, R.L., Pérez-Chacón, R., Martínez-Ballesteros, M., Troncoso, A., Martínez-Álvarez, F.: A nearest neighbours-based algorithm for big time series data forecasting. In: Martínez-Álvarez, F., Troncoso, A., Quintián, H., Corchado, E. (eds.) HAIS 2016. LNCS (LNAI), vol. 9648, pp. 174–185. Springer, Cham (2016). doi:10.1007/978-3-319-32034-2_15

    Chapter  Google Scholar 

  53. Tang, M., Yu, Y., Malluhi, Q.M., Ouzzani, M., Aref, W.G.: LocationSpark: a distributed in-memory data management system for big spatial data. PVLDB 9, 1565–1568 (2016)

    Google Scholar 

  54. Triguero, I., Peralta, D., Bacardit, J., García, S., Herrera, F.: MRPR: a MapReduce solution for prototype reduction in big data classification. Neurocomputing 150(Part A), 331–345 (2015)

    Article  Google Scholar 

  55. Wang, C., Karimi, S.: Parallel duplicate detection in adverse drug reaction databases with spark. In: Proceedings of the 19th International Conference on Extending Database Technology, pp. 551–562. ACM, New York (2016)

    Google Scholar 

  56. Wang, F., Aji, A., Vo, H.: High performance spatial queries for spatial big data: from medical imaging to GIS. SIGSPATIAL Special 6, 11–18 (2014)

    Article  Google Scholar 

  57. Wang, J., Wu, S., Gao, H., Li, J., Ooi, B.C.: Indexing multi-dimensional data in a cloud system. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 591–602. ACM, New York (2010)

    Google Scholar 

  58. Wang, K., Han, J., Tu, B., Dai, J., Zhou, W., Song, X.: accelerating spatial data processing with MapReduce. In: Proceedings of the IEEE 16th International Conference on Parallel and Distributed Systems, pp. 229–236, IEEE Computer Society, Washington, DC (2010)

    Google Scholar 

  59. Xavier, P., Francis, F.S.: Improvisation to the R\(^*\)-Tree kNN join principles in distributed environment. Int. J. Comput. Appl. 101, 20–24 (2014)

    Google Scholar 

  60. Yang, M., Zheng, L., Lu, Y., Guo, M., Li, J.: Cloud-assisted spatio-textual k nearest neighbor joins in sensor networks. In: Proceedings of the 1st International Conference on Industrial Networks and Intelligent Systems, pp. 12–17. ICST, Gent, Belgium (2015)

    Google Scholar 

  61. Yokoyama, T., Ishikawa, Y., Suzuki, Y.: Processing all k-nearest neighbor queries in hadoop. In: Gao, H., Lim, L., Wang, W., Li, C., Chen, L. (eds.) WAIM 2012. LNCS, vol. 7418, pp. 346–351. Springer, Heidelberg (2012). doi:10.1007/978-3-642-32281-5_34

    Chapter  Google Scholar 

  62. Yu, J., Wu, J., Sarwat, M.: GeoSpark: A cluster computing framework for processing large-scale spatial data. In: Proceedings of the 23rd International Conference on Advances in Geographic Information Systems, 03–06 November 2015. Association for Computing Machinery (2015)

    Google Scholar 

  63. Zhang, C., Li, F., Jestes, J.: Efficient parallel kNN joins for large data in MapReduce. In: Proceedings of the 15th International Conference on Extending Database Technology, pp. 38–49. ACM, New York (2012)

    Google Scholar 

  64. Zhang, F., Zheng, Y., Xu, D., Du, Z., Wang, Y., Liu, R., Ye, X.: Real-time spatial queries for moving objects using storm topology. ISPRS Int. J. Geo-Inf. 5, 178 (2016)

    Article  Google Scholar 

  65. Zhang, H., Sun, Z., Liu, Z., Xu, C., Wang, L.: Dart: a geographic information system on hadoop. In: Proceedings of the IEEE 8th International Conference on Cloud Computing, pp. 90–97. IEEE (2015)

    Google Scholar 

  66. Zhang, J., Mamoulis, N., Papadias, D., Tao, Y.: All-nearest-neighbors queries in spatial databases. In: Proceedings of the 16th International Conference on Scientific and Statistical Database Management, pp. 297–306. IEEE Computer Society, Washington, DC (2004)

    Google Scholar 

  67. Zhang, S., Han, J., Liu, Z., Wang, K., Feng, S.: Spatial queries evaluation with MapReduce. In: Proceedings of the 8th International Conference on Grid and Cooperative Computing, pp. 287–292. IEEE Computer Society, Washington, DC (2009)

    Google Scholar 

  68. Zhong, Y., Han, J., Zhang, T., Li, Z., Fang, J., Chen, G.: Towards parallel spatial query processing for big spatial data. In: Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum, pp. 2085–2094. IEEE Computer Society, Washington, DC (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering and Informatics Department, University of Patras, 26504, Patras, Greece

    Nikolaos Nodarakis, Angeliki Rapti & Athanasios K. Tsakalidis

  2. Department of Informatics, Ionian University, 49100, Corfu, Greece

    Spyros Sioutas

  3. Department of Cultural Heritage, Management and New Technologies, University of Patras, 26504, Patras, Greece

    Dimitrios Tsolis

  4. Computer and Informatics Engineering Department, Technological Educational, Institute of Western Greece, 26334, Patras, Greece

    Giannis Tzimas

  5. Centre of Excellence for International Courts, University of Copenhagen, 1455, Copenhagen, Denmark

    Yannis Panagis

Authors
  1. Nikolaos Nodarakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angeliki Rapti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Spyros Sioutas
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Athanasios K. Tsakalidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dimitrios Tsolis
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Giannis Tzimas
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yannis Panagis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikolaos Nodarakis .

Editor information

Editors and Affiliations

  1. Computer Science and Software Engineering, Swinburne University of Technology, Hawthorn, Victoria, Australia

    Timos Sellis

  2. Informatics, Ionian University, Kerkyra, Greece

    Konstantinos Oikonomou

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Nodarakis, N. et al. (2017). (A)kNN Query Processing on the Cloud: A Survey. In: Sellis, T., Oikonomou, K. (eds) Algorithmic Aspects of Cloud Computing. ALGOCLOUD 2016. Lecture Notes in Computer Science(), vol 10230. Springer, Cham. https://doi.org/10.1007/978-3-319-57045-7_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-57045-7_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-57044-0

  • Online ISBN: 978-3-319-57045-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation