Skip to main content
SpringerLink
Log in

Bit-wise attention deep complementary supervised hashing for image retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Deep hashing is effective and efficient for large-scale image retrieval. Most of existing deep hashing methods train a single hash table by utilizing the output of the penultimate fully-connected layer of a convolutional neural network as the deep feature of images. They concentrate on the semantic information but neglect the fine-grain image structure. To address this issue, this paper proposes an advanced image hashing method, Bit-wise Attention Deep Complementary Supervised Hashing (BADCSH). It is an end-to-end system that trains a sequence of hash tables in a boosting manner, each of which is trained by correcting errors caused by all previous ones. Features from different levels of the network are used to train different hash tables. The hash table trained with features at one level reveals a level of semantic content of the image, while the hash table trained with features at a lower level contains structural information of the image that makes up the semantic content. Moreover, the hash layer is used as an embedded layer of the network to generate hash codes. A dense attention layer is added to the hash layer to treat various hash bits differently, in order to reduce hash code redundancy and maximize overall similarity preservation. Finally, the hash tables trained on different levels of features are fused by weights computed based on their respective performance. Experiments on three real-world image databases demonstrate that the proposed method achieves the best performance among state-of-the-art comparative hashing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Anastasios D, Nikolaos D, Stefanos K (2000) A fuzzy video content representation for video summarization and content-based retrieval. Signal Process 80 (6):1049–1067

    Article  Google Scholar 

  2. Babenko A, Lempitsky V (2015) Aggregating deep convolutional features for image retrieval. In: Proc. of the IEEE Conference on Computer Vision, pp 1269–1277

  3. Babenko A, Slesarev A, Chigorin A, Lempitsky VS (2014) Neural codes for image retrieval. In: European Conference on Computer Vision, pp 584–599

  4. Bell S, Zitnick CL, Bala K, Girshick R (2016) Inside-Outside Net: Detecting objects in context with skip pooling and recurrent neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2874–2883

  5. Cordelia S, Mohr R (1997) Local grayvalue invariants for image retrieval. IEEE Trans Pattern Anal Machine Intell 19(5):530–535

    Article  Google Scholar 

  6. Datar M, Immorlica N, Indyk P, Mirrokni VS (2004) Locality-sensitive hashing scheme based on p-stable distributions. In: Proc. the Twentieth Annual Symposium on Computational Geometry, pp 253–262

  7. Dniz G, Bueno J, Salido, La Torre FD (2011) Face recognition using histograms of oriented gradients. in Proc Pattern Recognit Lett 32(12):1598–1603

    Article  Google Scholar 

  8. Gordo A, Almazn J, Revaud J, Larlus D (2016) Deep image retrieval: Learning global representations for image search. European conference on computer vision 241–257

  9. Gong Y, Lazebnik S, Gordo A, Perronnin F (2013) Iterative quantization: a procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans Pattern Anal Mach Intell 35(12):2916–2929

    Article  Google Scholar 

  10. Gong Y, Wang L, Guo R, Lazebnik S (2014) Multi-scale orderless pooling of deep convolutional activation features. In: 13th European Conference on Computer Vision, pp 392–407

  11. Hu D, Nie F, Li X (2019) Deep binary reconstruction for cross-modal hashing. IEEE Trans Multimed 21(4):973–985

    Article  Google Scholar 

  12. Huang D, Shan C, Ardabilian M, Wang Y, Chen L (2011) Local binary patterns and its application to facial image analysis: A survey. IEEE Trans on Syst Man Cybern 41(6):765–781

    Article  Google Scholar 

  13. Jing C, Dong Z, Pei M, Jia Y (2019) Heterogeneous hashing network for face retrieval across image and video domains. IEEE Trans Multimed 21 (3):782–794

    Article  Google Scholar 

  14. Kafai M, Eshghi K, Bhanu B (2014) Discrete cosine transform locality-sensitive hashes for face retrieval. IEEE Trans Multimed 16(4):1090–1103

    Article  Google Scholar 

  15. Kang WC, Li WJ, Zhou ZH (2016) Column sampling based discrete supervised hashing. Thirtieth AAAI conference on artificial intelligence

  16. Kang C, Zhu L, Qian X, Han J, Wang M, Tang YY (2019) Geometry and topology preserving hashing for sift feature. IEEE Trans Multimed 21 (6):1563–1576

    Article  Google Scholar 

  17. Kim S, Choi S (2013) Multi-view anchor graph hashing. In: Proc IEEE Int Conf Acoust Speech Signal Process, pp 3123–3127

  18. Kim S, Kang Y, Choi S (2012) Sequential spectral learning to hash with multiple representations. In: Proc European conference on computer vision, pp 538–551

  19. Kong W, Li WJ, Guo M (2012) Manhattan hashing for large-scale image retrieval. In: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval, pp 45–54

  20. Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. Neural Information Processing Systems

  21. Kulis B, Darrell T (2009) Learning to hash with binary reconstructive embeddings. In: Proc Neural Information Processing Systems, pp 1042–1050

  22. Kulis B, Grauman K (2009) Kernelized locality-sensitive hashing for scalable image search. In: Proc IEEE International Conference on Computer Vision, pp 2130–2137

  23. Lai H, Pan Y, Liu Y, Yan S (2015) Simultaneous feature learning and hash coding with deep neural networks. In: Proc IEEE conference on computer vision and pattern recognition, pp 3270–3278

  24. Li J, Ng WWY, Xing T, Kwong S, Wang H (2019) Weighted multi-deep ranking supervised hashing for efficient image retrieval 1st Nov.

  25. Li P, Cheng J, Lu H (2013) Hashing with dual complementary projection learning for fast image retrieval, in Proc. Neurocomputing 120(10):83–89

    Article  Google Scholar 

  26. Li P, Wang M, Cheng J, Xu C, Lu H (2013) Spectral hashing with semantically consistent graph for image indexing. IEEE Trans Multimed 15(1):141–152

    Article  Google Scholar 

  27. Li X, Lin G, Shen C, et al. (2013) And learning hash functions using column generation[J]. Computer ence 142–150

  28. Lin J, Li Z, Tang J (2017) Discriminative deep hashing for scalable face image retrieval. In: Proc International Joint Conference on Artificial Intelligence, pp 2266–2272

  29. Liong VE, Jiwen Lu, Gang W, Moulin P, Jie Z (2015) Deep hashing for compact binary codes learning. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston MA, pp 2475–2483

  30. Liu H, Wang R, Shan S, Chen X (2016) Deep supervised hashing for fast image retrieval. In: Proc IEEE Conference on Computer Vision and Pattern Recognition, pp 2064–2072

  31. Liu W, Mu C, Kumar S, Chang S-F (2014) Discrete graph hashing. Advances in Neural Information Processing Systems 27:3419–3427

    Google Scholar 

  32. Liu W, Wang J, Ji R, Jiang YG, Chang SF (2012) Supervised hashing with kernels. In: Proc Computer Vision and Pattern Recognition, pp 2074–C2081

  33. Lv Y, Ng WW, Zeng Z, Yeung DS, Chan PP (2015) Asymmetric cyclical hashing for large scale image retrieval. IEEE Trans Multimedia 17(8):1225–1235

    Article  Google Scholar 

  34. Ma L, Li H, Meng F, Wu Q, Ngan KN (2017) Learning efficient binary codes from high-level feature representations for multilabel image retrieval. IEEE Trans Multimed 19(11):2545–2560

    Article  Google Scholar 

  35. Ng WWY, Li J, Tian T, Wang H, Kwong S, Wallace J (2020) Multi-level supervised hashing with deep features for efficient image retrieval. Neurocomputing

  36. Ng YH, Yang F, Davis LS (2015) Exploiting local features from deep networks for image retrieval. In: Proc. of the IEEE conference on computer vision and pattern recognition, pp 53–61

  37. Norouzi M, Fleet DJ (2011) Minimal loss hashing for compact binary codes. In: Proc. International Conference on Machine Learning, pp 353–360

  38. Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. International Journal of Computer Vision 42(3):145–175

    Article  Google Scholar 

  39. Radenovic F, Tolias G, Chum O (2019) Fine-Tuning CNN image retrieval with no human annotation. IEEE Trans Pattern Anal Machine Intell 41 (7):1655–1668

    Article  Google Scholar 

  40. Raginsky M, Lazebnik S (2009) Locality-sensitive binary codes from shift-invariant kernels. In: Proc Conf. and Workshop on Neural Information Processing Systems, pp 1509–1517

  41. Shen F, Shen C, Liu W, Shen HT (2015) Supervised discrete hashing. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition, pp 37–45

  42. Shen F, Yang Y, Liu L, Liu W, Tao D, Shen HT (2017) Asymmetric binary coding for image search. IEEE Trans Multimed 19(9):2022–2032

    Article  Google Scholar 

  43. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Computer Science

  44. Song D, Liu W, Ji R, Meyer DA, Smith JR (2015) Top rank supervised binary coding for visual search. In: 2015 IEEE international conference on computer vision (ICCV) Santiago, pp 1922–1930

  45. Tiakas E, Rafailidis D, Dimou A, Daras P (2013) MSIDX: multi-sort indexing for efficient content-based image search and retrieval. IEEE Trans Multimed 15(6):1415–1430

    Article  Google Scholar 

  46. Tiana X, Zhoua X, Ng WWY, Li J, Wang H (2019) Bootstrap dual complementary hashing with semi-supervised re-ranking for image retrieval, Neurocomputing, 31st Oct.

  47. Tieu K, Paul V (2004) Boosting image retrieval. Int J Comput Vis 56(1-2):17–36

    Article  Google Scholar 

  48. Tolias G, Sicre R, Jgou H (2015) Particular object retrieval with integral max-pooling of CNN activations. Computer Science

  49. Tzelepi M, Tefas A (2017) Deep convolutional learning for content based image retrieval. Neurocomputing

  50. Venters C, Cooper M (2000) A review of content-based image retrieval systems, JISC Technology Applications Programme, http://www.jtap.ac.uk/reports/htm/jtap-054.html

  51. Wang D, Cui P, Ou M, Zhu W (2015) Deep multimodal hashing with orthogonal regularization. In: Proc 24th conference on artificial intelligence, pp 2291–2297

  52. Wang J, Kumar S, Chang S-F (2010) Semi-supervised hashing for scalable image retrieval. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition

  53. Wang X, Shi Y, Kitani K (2016) Deep supervised hashing with triplet labels. In: Proc Asian Conference on Computer Vision, pp 70–84

  54. Wang Y, Zhang L, Nie F, Li X, Chen Z, Wang F (2019) WeGAN: deep image hashing with weighted generative adversarial networks. In: IEEE Transactions on Multimedia

  55. Weiss Y, Torralba A, Fergus R (2009) Spectral hashing. In: Proc advances in neural information processing systems, pp 1753–1760

  56. Wu C, Zhu J, Cai D, Chen C, Bu J (2013) Semi-supervised nonlinear hashing using bootstrap sequential projection learning. IEEE Transaction Knowledge and Data Engneering 25(6):1380–1393

    Article  Google Scholar 

  57. Xia R, Pan Y, Lai H, Liu C, Yan S (2014) Supervised hashing for image retrieval via image representation learning. In: Proc Conference on Artificial Intelligence, pp 2156–2162

  58. Xu H, Wang J, Li Z, Zeng G, Li S, Yu N (2011) Complementary hashing for approximate nearest neighbor search. In: Proc International Conference on Computer Vision

  59. Yandex AB, Lempitsky V (2015) Aggregating local deep features for image retrieval. IEEE International Conference on Computer Vision 1269–1277

  60. Yao T, Long F, Mei T, Rui Y (2016) Deep semantic-preserving and ranking-based hashing for image retrieval. In: Proc International Joint Conference on Artificial Intelligence, pp 3931–3937

  61. Yong R, Huang TS, Chang S-F (1999) Image retrieval: Past, present, and future. J Vis Commun Image Represent 10(1):1–23

    Article  MathSciNet  Google Scholar 

  62. Zhang D, Wang F, Si L (2011) Composite hashing with multiple information sources. In: Proc 34th Int ACM SIGIR Conf Res Develop Inf Retr, pp 225–234

  63. Zhang J, Peng Y (2018) Query-Adaptive image retrieval by deep-weighted hashing. IEEE Trans Multimed 20(9):2400–2414

    Article  Google Scholar 

  64. Zhang R, Lin L, Zhang R, Zuo W, Zhang L (2015) Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Trans Image Processing 24(12):4766–4779

    Article  MathSciNet  Google Scholar 

  65. Zhang Z, Zou Q, Lin Y, Chen L, Wang S (2019) Improved deep hashing with soft pairwise similarity for multi-label image retrieval. In: IEEE Transactions on Multimedia

  66. Zheng L, Wang SJ, Wang JD, Tian Q (2016) Accurate image search with multi-scale contextual evidences. Int J Comput Vis 1:1–3

    Article  MathSciNet  Google Scholar 

  67. Zheng L, Zhao YL, Wang SJ, Wang JD, Tian Q (2016) Good practice in CNN feature transfer

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China under Grant 61876066, the 67th Chinese Postdoctoral Science Foundation (2020M672631), Guangdong Province Science and Technology Plan Project (Collaborative Innovation and Platform Environment Construction) 2019A050510006, Guangdong Science and Technology Plan Project 2018B050502006, and the EU Horizon 2020 Programme (700381, ASGARD).

Author information

Authors and Affiliations

  1. Guangdong Provincial Key Laboratory of Computational Intelligence and Cyberspace Information, Computer Science and Engineering, South China University of Technology, Guangzhou, China

    Wing W. Y. Ng, Jiayong Li & Xing Tian

  2. School of Computing, Ulster University, Jordanstown, United Kingdom

    Hui Wang

Authors
  1. Wing W. Y. Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiayong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xing Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xing Tian.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ng, W.W.Y., Li, J., Tian, X. et al. Bit-wise attention deep complementary supervised hashing for image retrieval. Multimed Tools Appl 81, 927–951 (2022). https://doi.org/10.1007/s11042-021-11494-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11494-8

Keywords

Search

Navigation