Ziqi Shi
ABSTRACT
With the rapid development of computer vision, 3D data is increasing rapidly. How to retrieve similar model from a large number of models has become a hot research topic. However, in order to meet people's demand, the retrieval accuracy need to be further improved. In terms of multi-view 3D model retrieval, how to effectively learn the information between views is the key to improving performance. In this paper, we propose a novel 3D model retrieval algorithm based on attention and multi-view fusion. Specifically, we mainly constructed two modules. First, dynamic attentive graph learning module is used to learn the intrinsic relationship between view blocks; Then we propose the Attention-NetVlad algorithm, which combines the channel attention algorithm and the NetVlad algorithm. It learns the information between feature channels to enhance the feature expression ability firstly, then uses the NetVlad algorithm to fuse multiple view features into a global feature according to the clustering information. Finally the global feature is used as the only feature of the model to retrieve according to Euclidean distance. In comparison with other state-of-the-art methods by utilizing ModelNet10 and ModelNet40 the proposed method has demonstrated significant improvement for retrieval mAP. Our experiments also demonstrate the effectiveness of the modules in the algorithm.
- Charles Ruizhongtai Qi, Hao Su, Kaichun Mo and Leonidas J. Guibas. 2017. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’17).IEEE, Honolulu, HI, USA. 77-85.https://doi.org/10.1109/CVPR.2017.16Google Scholar
Cross Ref
- Ali Cheraghian and Lars Petersson.2019. 3DCapsule: Extending the Capsule Architecture to Classify 3D Point Clouds. In Proceedings of the Winter Conference on Applications of Computer (WACV’19). Waikoloa Village, HI, USA,1194-1202. https://doi.org/10.1109/WACV.2019.00132Google ScholarCross Ref
- Hongsen Liu, Yang Cong, Chenguang Yang and Yandong Tang. 2019. Efficient 3D object recognition via geometric information preservation.Pattern Recognit 92, (2019),135-145. https://doi.org/10.1016/j.patcog.2019.03.025Google ScholarDigital Library
- Jianwen Jiang , Di Bao ,Ziqiang Chen , Xibin Zhao and Yue Gao. 2019. MLVCNN: Multi-loop-view convolutional neural network for 3D shape retrieval. In Proceedings of the Conference on Artificial Intelligence (AAAI2019). AAAI, Honolulu,Hawaii, USA,8513-8520. https://doi.org/10.1609/aaai.v33i01.33018513Google ScholarDigital Library
- Heyu Zhou,An-An Liu, Weizhi Nie and Jie Nie. 2020. Multi-view saliency guided deep neural network for 3-d object retrieval and classification. IEEE Trans Multim 22,6 (2020) 1496–1506.https://doi.org/10.1109/TMM.2019.2943740Google ScholarCross Ref
- Zhirong Wu , Shuran Song , Aditya Khosla , Fisher Yu , Linguang Zhang, Xiaoou Tang and Jianxiong Xiao. 2015. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the Conference on Computer Vision and Pattern Recognition.(CVPR’15) IEEE.Boston, MA, USA ,1912–1920. https://doi.org/10.1109/CVPR.2015.7298801Google ScholarCross Ref
- Hang Su, Subhransu Maji, Evangelos Kalogerakis and Erik G. Learned-Miller. 2015 Multi-view convolutional neural networks for 3d shape recognition. In Proceedings of the International Conference on Computer Vision(ICCV’15). IEEE. Santiago, Chile, 945-953 https://doi.org/10.1109/ICCV.2015.114Google ScholarDigital Library
- Cheng Wang, Ming Cheng, Ferdous Sohel, Mohammed Bennamoun andJonathan Li. 2019. Normalnet: A voxel-based cnn for 3d object classification and retrieval. Neurocomputing 323,JAN.5 (2019),139–147. https://doi.org/10.1016/j.neucom.2018.09.075Google ScholarCross Ref
- Takahiko Furuya, Ryutarou Ohbuchi. 2016. Deep aggregation of local 3d geometric features for 3d model retrieval.In Proceedings of the British Machine Vision Conference(BMVC’16).York, UKGoogle Scholar
- Sudhakar Kumawat and Shanmuganathan Raman. 2019. Lp-3dcnn: Unveiling local phase in 3d convolutional neural networks.In Proceedings of the Conference on Computer Vision and Pattern Recognition(CVPR’19). IEEE, Long Beach, CA, USA, 4898–4907Google Scholar
- Yifan Feng ,Zizhao Zhang ,,Xibin Zhao , Rongrong Ji and Yue Gao. 2018. Gvcnn: Group view convolutional neural networks for 3d shape recognition. In Proceedings of the Conference on Computer Vision and Pattern Recognition(CVPR’18).IEEE, Salt Lake City, UT, USA, 264–272Google Scholar
- Weizhi Nie, Shu Xiang and An-an Liu. 2018. Multi-scale cnns for 3d model retrieval. Multim Tools Appl 77,17 (2018), 22953–22963. https://doi.org/10.1007/s11042-018-5641-1Google ScholarDigital Library
- Kai Sun , Jiangshe Zhang, Junmin Liu, Ruixuan Yu and Zengjie Song. 2021. DRCNN: Dynamic routing convolutional neural network for multi-view 3D object recognition. IEEE Trans. Image Process. 30 (2021), 868-877. https://doi.org/10.1109/TIP.2020.3039378Google ScholarDigital Library
- Weizhi Nie, Yue Zhao, Dan Song and Yue Gao. 2021 Dan: deep-attention network for 3d shape recognition. IEEE Trans. Image Process, 30 (2021), 4371-4383. https://doi.org/10.1109/TIP.2021.3071687Google ScholarCross Ref
- Yue Zhao, Weizhi Nie, An-An Liu, Zan Gao and Yuting Su. 2021. Svhan: Sequential view based hierarchical attention network for 3d shape recognition. In Proceedings of the Conference on Multimedia (MM’2021). ACM, China, 2130-2138. https://doi.org/10.1145/3474085.3475371Google ScholarDigital Library
- Petar Velickovic ,Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio and Yoshua Bengio. 2017. Graph Attention Networks.CoRR. abs/1710.10903. (2017) http://arxiv.org/abs/1710.10903Google Scholar
- Joan Bruna, Wojciech Zaremba , Arthur Szlam and Yann LeCun. 2014. Spectral Networks and Locally Connected Networks on Graphs. In Proceedings of the International Conference on Learning Representations Computer Science (ICLR’14). Banff, AB, Canada. http://arxiv.org/abs/1312.6203Google Scholar
- Chong Mou, Jian Zhang and Zhuoyuan Wu. 2021. Dynamic Attentive Graph Learning for Image Restoration. In Proceedings of the International Conference on Computer Vision (ICCV’21). IEEE, Montreal, QC, Canada, 4308—4317. https://doi.org/10.1109/ICCV48922.2021.00429Google ScholarCross Ref
- Kaiming He , Xiangyu Zhang , Shaoqing Ren and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’16) IEEE. Las Vegas, NV, USA,770-778 https://doi.org/10.1109/CVPR.2016.90Google ScholarCross Ref
- Relja Arandjelovic , Petr Gronat , Akihiko Torii, Tomas Pajdla and Josef Sivic. 2016. NetVLAD: CNN Architecture for Weakly Supervised Place Recognition. In Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’16). IEEE Las Vegas, NV, USA, 5297—5307. https://doi.org/10.1109/CVPR.2016.572Google ScholarCross Ref
- Qilong Wang, Banggu Wu , Pengfei Zhu , Peihua Li , Wangmeng Zuo and Qinghua. 2020. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. In Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’20) IEEE Seattle, WA, USA 11531—11539Google ScholarCross Ref
- P.University. 2015. ModelNet40 Retrieved November 20, 2020 from http://modelnet.cs.princeton. edu/Google Scholar
- Michael M. Kazhdan ,Thomas A. Funkhouser and Szymon Rusinkiewicz.2003. Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors. In Proceedings of the Eurographics Symposium on Geometry Processing.ACM, Aachen, Germany,156-164. https://doi.org/10.2312/SGP/SGP03/156-165Google ScholarCross Ref
- Ding-Yun Chen, Xiao-Pei Tian , Yu-Te Shen and Ming Ouhyoung. 2003. On visual similarity based 3d model retrieval. Comput Graph Forum 22, 3 (2003), 223-232. https://doi.org/10.1111/1467-8659.00669Google ScholarCross Ref
- Haoxuan You, Yifan Feng, Rongrong Ji and Yue Gao. 2018. PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition. In Proceedings of the Multimedia Conference (MM’18). ACM, Seoul, Republic of Korea, 1310-1318 https://doi.org/10.1145/3240508.3240702Google ScholarDigital Library
- Haoxuan You, Yifan Feng, Xibin Zhao, Changqing Zou , Rongrong Ji and Yue Gao. 2019 PVRNet: Point-View Relation Neural Network for 3D Shape Recognition . In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’19). Honolulu, Hawaii,USA.9119-9126. https://doi.org/10.1609/aaai.v33i01.33019119Google ScholarDigital Library
- Weizhi Nie, Qi Liang, Yixin Wang, dXing Wei and Yuting Su. 2021. MMFN: Multimodal Information Fusion Networks for 3D Model Classification and Retrieval. ACM Trans. Multim. Comput. Commun. Appl.16,4 (2021) 131:1-131:22. https://doi.org/10.1145/3410439Google ScholarDigital Library
- Chao Ma, Yulan Guo , Jungang Yang and Wei An. 2019. Learning Multi-View Representation With LSTM for 3-D Shape Recognition and Retrieval. IEEE Trans. Multim.21,5 (2019), 1169—1182. https://doi.org/10.1109/TMM.2018.2875512Google ScholarDigital Library
- Xinwei He, Yang Zhou, Zhichao Zhou, Song Bai and Xiang Bai. 2018. Triplet-Center Loss for Multi-View 3D Object Retrieval. In Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR’18). IEEE, Salt Lake City, UT, USA,1945-1954Google ScholarCross Ref
- Zhizhong Han, Mingyang Shang , Zhenbao Liu, Chi-Man Vong, Yu-Shen Liu, Matthias Zwicke, Junwei Han and C. L. Philip Chen. 2019. SeqViews2SeqLabels: Learning 3D Global Features via Aggregating Sequential Views by RNN With Attention. IEEE Trans. Image Process, 28,2(2019) 658-672. https://doi.org/10.1109/TIP.2018.2868426Google ScholarDigital Library
Index Terms
- 3D Model Retrieval Algorithm Based on Attention and Multi-view Fusion
Index terms have been assigned to the content through auto-classification.
Recommendations
Exploring Deep Learning for View-Based 3D Model Retrieval
In recent years, view-based 3D model retrieval has become one of the research focuses in the field of computer vision and machine learning. In fact, the 3D model retrieval algorithm consists of feature extraction and similarity measurement, and the ...
Multi-View Graph Matching for 3D Model Retrieval
3D model retrieval has been widely utilized in numerous domains, such as computer-aided design, digital entertainment, and virtual reality. Recently, many graph-based methods have been proposed to address this task by using multi-view information of 3D ...
A 3D model retrieval method based on multi-feature fusion
3D model retrieval is a hot topic in information retrieval, and it is of great importance to fuse multi-feature of 3D models to achieve high quality retrieval. Therefore, in this paper, we propose a novel 3D model retrieval method based on the multi-...
Comments