计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (4): 307-313.doi: 10.11896/jsjkx.230900087

• 人工智能 • 上一篇    下一篇

改进的跨模态关联歧义学习的虚假信息检测方法研究

段钰潇, 胡艳丽, 郭浩, 谭真, 肖卫东   

  1. 国防科技大学信息系统工程重点实验室 长沙410073
  • 收稿日期:2023-09-15 修回日期:2023-11-06 出版日期:2024-04-15 发布日期:2024-04-10
  • 通讯作者: 谭真(tanzhen08a@nudt.edu.cn)
  • 作者简介:(duanyuxiao19@nudt.edu.cn)
  • 基金资助:
    国家自然科学基金(62272469,72301284);国家重点研发计划(2022YFB3102600);湖南省科技创新计划(2023RC1007)

Study on Improved Fake Information Detection Method Based on Cross-modal CorrelationAmbiguity Learning

DUAN Yuxiao, HU Yanli, GUO Hao, TAN Zhen, XIAO Weidong   

  1. Science and Technology on Information Systems Engineering Laboratory,National University of Defense Technology,Changsha 410073,China
  • Received:2023-09-15 Revised:2023-11-06 Online:2024-04-15 Published:2024-04-10
  • Supported by:
    National Natural Science Foundation of China(62272469,72301284),National Key R & D Program of China(2022YFB3102600) and Science and Technology Innovation Program of Hunan Province(2023RC1007).

PDF (PC)

39

摘要： 近年来,随着互联网及多媒体技术的迅猛发展,人们获取信息更加方便快捷,然而虚假信息在网络上的传播也日益严重,负面影响不断扩大。为了增强信息的可信度和欺骗性,虚假信息呈现多模态发展趋势,使得检测工作面临更大挑战。现有的多模态虚假信息检测方法大多关注多模态特征的形成,对于跨模态歧义和不同模态特征在检测中的贡献率的研究尚不完善,忽略了不同模态特征间固有差异性对虚假信息检测的影响。为解决该问题,提出了构建改进的跨模态关联歧义学习的虚假信息检测模型,通过对文本和图像特征进行跨模态歧义学习,利用歧义得分更新单模态与融合特征的权重,自适应地拼接单模态与融合特征;同时采用网格搜索动态分配文本、图像特征权重,提高检测准确率。在Twitter数据集上对该模型的有效性进行验证,其相比基线模型准确率提高了6%,相比未进行动态权重分配的检测方法性能提升了1.6%。

关键词: 虚假信息检测, 多模态, 跨模态关联, 歧义学习, 融合特征

Abstract: In recent years,with the rapid development of the Internet and multimedia technology,it is more convenient for people to obtain information,but the spread of fake information on the Internet is also increasingly serious,and the negative impact is constantly expanding.In order to enhance the credibility and deception,fake information presents a multi-modal development trend,which makes the detection work face greater challenges.The existing multi-modal fake information detection methods pay more attention to the formation of multi-modal features.The research on the contribution rate of cross-modal ambiguity and different modal features in detection is not perfect,ignoring the impact of inherent differences among different modal features on fake information detection.To solve the problem,this paper proposes to construct an improved fake information detection model based on cross-modal correlation ambiguity learning.Through cross-modal ambiguity learning of text and image features,the weights of unimodal features and fused features are updated by the ambiguity score.The unimodal features and fused features are combined adaptively,and the weights of text and image features are dynamically assigned by grid search to improve the detection accuracy.The effectiveness of the model is verified by experiments on the Twitter dataset.The accuracy is improved by 6% compared with the baseline model and 1.6% compared with the detection without dynamic weight assignment.

Key words: Fake news detection, Multimodal, Cross-modal correlation, Ambiguity learning, Fusion features

中图分类号: 

  • TP391
[1]JIN Z,CAO J,WANG B,et al.Research on social multimedia rumor detection technology integrating multi-modal features [J].Journal of Nanjing University of Information Science and Technology(Natural Science Edition),2017,9(6):583-592.
[2]ISLAM S,SARKAR T,KHAN S H,et al.COVID-19-related infodemic and its impact on public health:A global social media analysis[J].Am.J.Trop.Med.Hyg,2020,103(4),1621-1629.
[3]CAO J,SHENG Q,QI P.Progress and prospect of Internet false information detection [J].Communication of China Computer Society,2020,16(3):52-57.
[4]KHATTAR D,GOND J,GUPTA M,et al.MVAE:Multimodal Variational Autoencoder for Fake News Detection[C]//The World Wide Web Conference.2019:2915-2921.
[5]WANG Y Q,MA F L,JIN Z W,et al.EANN:Event Adversa-rial Neural Networks for Multi-Modal Fake News Detection[C]//ACM SIGKDD International Conference on Knowledge Discovery Data Mining(KDD).2018:849-857.
[6]POPAT K,MUKHERJEE S,YATES A,et al.DeClarE:Debunking Fake News and False Claims Using Evidence-Aware Deep Learning[C]//Proceeding of the 2018 Conference on Empirical Methods in Natural Language Processing(EMNLP),Brussels,Belgium.USA:ACL.2018:22-32.
[7]SONG C,SHU K,WU B.Temporally evolving graph neural network for fake news detection[J].Information Processing & Management,2021,58:102712.
[8]LIU Y,WU Y F B.Early detection of fake news on social media through propagation path classifica-tion with recurrent and convolutional networks[C]//The 32th AAAI Conference on Artificial Intelligence(AAAI-18).2018:354-361.
[9]XUE J,WANG Y,TIAN Y,et al.Detecting fake news by exploring the consistency of multimodal data[J].Information Processing & Management.2021,58(5):102610.
[10]ZHOU X,WU J,ZAFARANI R.Safe:Similarity-aware multi-modal fake news detection[J].arXiv:2003.04981,2020.
[11]ZHANG G,LI J.Detecting Social Media Fake News with Semantic Consistency Between Multi-model Contents[J].Data Analysis and Knowledge Discovery,2021,5(5):21-29.
[12]SHIVANGI S,MUDIT D,RAJIV R S,et al.Inter-Modality Discordance for Multimodal Fake News Detection[C]//ACM Multimedia Asia(MMAsia'21).2021:1-7.
[13]QI P,CAO J,LI X R,et al.Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multi-modal Clues[C]//ACM MM21.2021:1212-1220.
[14]QI P,CAO J,SHENG Q.Semantics-Enhanced Multi-ModalFake News Detection[J].Journal of Computer Research and Development,2021,58(7):1456-1465.
[15]CHEN Y,LI D,ZHANG P,et al.Cross-modal ambiguity lear-ning for multimodal fake news detection[C]//Proceedings of the ACM Web Conference.2022:2897-2905.
[16]HUA J,CUI X D,LI X H,et al.Multimodal fake news detection through data augmentation-based contrastive learning[J].Applied Soft Computing,2023,136(C):1568-4946.
[17]YING Q,HU X,ZHOU Y,et al.Bootstrapping multi-view representations for fake news detection[C]//AAAI.2023.
[18]BOIDIDOU C,PAPADOPOULOS S,ZAM-POGLOU M,et al.Detection and visualization of misleading content on Twitter[J].International Journal of Multimedia Information Retrieval,2018,7(1):71-86.
[19]JIN Z,CAO J,GUO H,et al.Multimodal fusion with recurrent neural networks for rumor detection on microblogs[C]//Proceedings of the 25th ACM International Conference on Multimedia.2017:795-816.
[20]WU Y,ZHAN P,ZHANG Y,et al.Multimodal Fusion with Co-Attention Networks for Fake News Detection[OL].https://aclanthology.org/2021.findings-acl.226/.
[21]CHEN Y X,LI D S,ZHANG P,et al.Cross-modal Ambiguity Learning for Multimodal Fake News Detection[C]//Procee-dings of the ACM Web Conference 2022(WWW'22).Virtual Event,Lyon,France.ACM,New York,NY,USA,2022:2897-2905.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发