Skip to main content
SpringerLink
Log in

Semantic-Based Image Retrieval Using RS-Tree and Knowledge Graph

  • Conference paper
  • First Online:
Intelligent Information and Database Systems (ACIIDS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13757))

Included in the following conference series:

  • 840 Accesses

Abstract

High-level semantic image retrieval is the problem which has applied in many different fields. In this paper, we approached a method of similarity image retrieval and image semantic extraction based on the combination of RS-Tree and knowledge graph. The main tasks include: (1) building a knowledge graph to store objects, attributes, and relationships of objects on images; (2) searching a set of similar images based on the low-level features stored in RS-Tree; (3) extracting a scene graph of image using Visual Genome dataset; (4) generating SPARQL query based on the scene graph to extract high-level semantic of the image from the built knowledge graph. In order to evaluate the efficiency as well as compare the accuracy of RS-Tree, the COREL and Wang image datasets are used. On the basis of the proposed method, a knowledge graph is built based on the Visual Genome dataset. The experimental results are compared with related works to demonstrate the effectiveness of the proposed method. Therefore, our proposed method is feasible in semantic-based image retrieval systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Li, X., Yang, J., Ma, J.: Recent developments of content-based image retrieval (CBIR). Neurocomputing 452, 675–689 (2021)

    Article  Google Scholar 

  2. Hameed, I.M., Abdulhussain, S.H., Mahmmod, B.M.: Content-based image retrieval: a review of recent trends. Cogent Eng. 8(1), 1927469 (2021)

    Article  Google Scholar 

  3. Bai, C., Chen, J.N., Huang, L., Kpalma, K., Chen, S.: Saliency-based multi-feature modeling for semantic image retrieval. J. Vis. Commun. Image Represent. 50, 199–204 (2018)

    Article  Google Scholar 

  4. Cui, P., Liu, S., Zhu, W.: General knowledge embedded image representation learning. IEEE Trans. Multim. 20(1), 198–207 (2017)

    Article  Google Scholar 

  5. Zareian, A., Karaman, S., Chang, S.-F.: Bridging knowledge graphs to generate scene graphs. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12368, pp. 606–623. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58592-1_36

    Chapter  Google Scholar 

  6. Johnson, J., et al.: Image retrieval using scene graphs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3668–3678 (2015)

    Google Scholar 

  7. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst. 28 (2015)

    Google Scholar 

  8. Dai, B., Zhang, Y., Lin, D.: Detecting visual relationships with deep relational networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3076–3086 (2015)

    Google Scholar 

  9. Krishna, R., et al.: Visual genome: connecting language and vision using crowdsourced dense image annotations. Int. J. Comput. Vis. 123(1), 32–73 (2017)

    Article  MathSciNet  Google Scholar 

  10. Wang, S., Wang, R., Yao, Z., Shan, S., Chen, X.: Cross-modal scene graph matching for relationship-aware image-text retrieval. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1508–1517 (2020)

    Google Scholar 

  11. Schroeder, B., Tripathi, S.: Structured query-based image retrieval using scene graphs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 178–179 (2020)

    Google Scholar 

  12. Yoon, S., et al.: Image-to-image retrieval by learning similarity between scene graphs. arXiv preprint arXiv:2012.1470.4322 (2020)

  13. Qi, M., Wang, Y., Li, A.: Online cross-modal scene retrieval by binary representation and semantic graph. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 744–752, October 2017

    Google Scholar 

  14. Ramnath, S., Saha, A., Chakrabarti, S., Khapra, M.M.: Scene graph based image retrieval--a case study on the CLEVR dataset. arXiv preprint arXiv:1911.00850 (2019)

  15. Quinn, M.H., Conser, E., Witte, J.M., Mitchell, M.: Semantic image retrieval via active grounding of visual situations. In: 2018 IEEE 12th International Conference on Semantic Computing (ICSC), pp. 172–179. IEEE, January 2018

    Google Scholar 

  16. Ahmed, K.T., Ummesafi, S., Iqbal, A.: Content based image retrieval using image features information fusion. Inf. Fus. 51, 76–99 (2019)

    Article  Google Scholar 

  17. Chhabra, P., Garg, N.K., Kumar, M.: Content-based image retrieval system using ORB and SIFT features. Neural Comput. Appl. 32(7), 2725–2733 (2018). https://doi.org/10.1007/s00521-018-3677-9

    Article  Google Scholar 

  18. Lande, M.V., Bhanodiya, P., Jain, P.: An effective content-based image retrieval using color, texture and shape feature. In: Mohapatra, D.P., Patnaik, S. (eds.) Intelligent Computing, Networking, and Informatics. AISC, vol. 243, pp. 1163–1170. Springer, New Delhi (2014). https://doi.org/10.1007/978-81-322-1665-0_119

    Chapter  Google Scholar 

  19. Haldurai, L., Vinodhini, V.: Parallel indexing on color and texture feature extraction using r-tree for content based image retrieval. Int. J. Comput. Sci. Eng. 3, 11–15 (2015)

    Google Scholar 

  20. Nhi, N.T.U., Le, T.M., Van, T.T.: A model of semantic-based image retrieval using C-Tree and neighbor graph. Int. J. Seman. Web Inf. Syst. 18(1), 1–23 (2022)

    Article  Google Scholar 

  21. Li, J., Wang, J.Z.: Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1075–1088 (2003)

    Article  Google Scholar 

  22. Wang, J.Z., Li, J., Wiederhold, G.: SIMPLIcity: semantics-sensitive integrated matching for picture libraries. IEEE Trans. Pattern Anal. Mach. Intell. 23(9), 947–963 (2001)

    Article  Google Scholar 

  23. Thanh, L.T.V., Thanh, L.M., Thanh, V.T.: Semantic-based image retrieval using RS-tree and neighbor graph. In: Rocha, A., Adeli, H., Dzemyda, G., Moreira, F. (eds.) Information Systems and Technologies. WorldCIST 2022. LNNS, vol. 469. Springer, Cham, pp. 165–176 (2022). https://doi.org/10.1007/978-3-031-04819-7_18

Download references

Acknowledgment

The authors would like to thank the Faculty of Information Technology, University of Sciences - Hue University for their professional advice for this study. We would also like to thank Ba Ria - Vung Tau University, University of Education HCMC, and research group SBIR HCM, which are sponsors of this research. We also would like to express our sincere thanks to reviewers for their helpful comments on this article.

Author information

Authors and Affiliations

  1. University of Science - Hue University, Hue, Vietnam

    Le Thi Vinh Thanh & Thanh Manh Le

  2. Ba Bia – Vung Tau University, Vũng Tàu, Vietnam

    Le Thi Vinh Thanh

  3. HCMC University of Education, Ho Chi Minh City, Vietnam

    Thanh The Van

Authors
  1. Le Thi Vinh Thanh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thanh The Van
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thanh Manh Le
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thanh Manh Le .

Editor information

Editors and Affiliations

  1. Wrocław University of Science and Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

  2. Vietnam National University, Ho Chi Minh City, Ho Chi Minh City, Vietnam

    Tien Khoa Tran

  3. Al-Farabi Kazakh National University, Almaty, Kazakhstan

    Ualsher Tukayev

  4. National University of Kaohsiung, Kaohsiung, Taiwan

    Tzung-Pei Hong

  5. Wrocław University of Science and Technology, Wrocław, Poland

    Bogdan Trawiński

  6. University of Newcastle, Newcastle, NSW, Australia

    Edward Szczerbicki

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Thanh, L.T.V., Van, T.T., Le, T.M. (2022). Semantic-Based Image Retrieval Using RS-Tree and Knowledge Graph. In: Nguyen, N.T., Tran, T.K., Tukayev, U., Hong, TP., Trawiński, B., Szczerbicki, E. (eds) Intelligent Information and Database Systems. ACIIDS 2022. Lecture Notes in Computer Science(), vol 13757. Springer, Cham. https://doi.org/10.1007/978-3-031-21743-2_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21743-2_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21742-5

  • Online ISBN: 978-3-031-21743-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation