doiSerbia

Multi-scale image semantic recognition with hierarchical visual vocabulary

Jiang Xinghao (School of Information Security Engineering Shanghai Jiao Tong University, Shanghai, China + Key Lab. of Shanghai Information Security Management and Technology Research, Shanghai, China)
Sun Tanfeng (School of Information Security Engineering Shanghai Jiao Tong University, Shanghai, China + Key Lab. of Shanghai Information Security Management and Technology Research, Shanghai, China)
Guanglei Fu (School of Information Security Engineering Shanghai Jiao Tong University, Shanghai, China)

Local features have been proved to be effective in image/video semantic analysis. The BOVW (bag of visual words) scheme can cluster local features to form the visual vocabulary which includes an amount of words, where each word is the center of one clustering feature. The vocabulary is used to recognize the image semantic. In this paper, a new scheme to construct semantic-binding hierarchical visual vocabulary is proposed. Some attributes and relationship of the semantic nodes in the model are discussed. The hierarchical semantic model is used to organize the multi-scale semantic into a level-by-level structure. Experiments are performed based on the LabelMe dataset, the performance of our scheme is evaluated and compared with the traditional BOVW scheme, experimental results demonstrate the efficiency and flexibility of our scheme.

Keywords: local feature, bag of visual words, image semantic analysis, visual vocabulary