High-Order Correlation Embedding for Large-Scale Multi-modal Hashing

Abstract

Benefitting from the superb storage and computational efficiency, hashing has received considerable research attention on large-scale multi-modal retrieval. However, most existing methods are mainly built based upon matrix optimization without high-order correlation and equally treat the training instances, which fail to fuse heterogeneous sources and ignore the heuristic information contained by the sampling order. To this end, we, for the first time, propose a novel tensor-based supervised discrete learning framework named Discrete Multi-modal Correlation Hashing (DMCH) to perform a high-order correlation preserved semantic hash learning. Specifically, DMCH stacks all the modality-private matrices into a third-order tensor to simultaneously exploit the high-order intrinsic correlations across heterogeneous sources, which explicitly enforces the consistent and private properties of different modalities. Moreover, DMCH selects the training samples from reliable to unreliable ones to extract heuristic information contained by the instance learning order, which increases the robustness of the model. Furthermore, the specific semantic labels are utilized as specific prior knowledge to preserve full-scale supervision instead of the widely-used pair-wise similarity. Finally, the jointly learning objective is formulated to concurrently preserve the modality-common information and modality-private semantics in the learned hash codes. Extensive experiments on four public datasets demonstrate the state-of-the-art performance of our proposed method.