在社交媒體的快速發展下，伴隨著假新聞氾濫問題日益嚴重，進而加劇了社會的極化 現象。在這樣的情況下使得檢測假新聞的議題愈發重要。然而假新聞的迅速傳播和演 變，以及缺乏全面的數據，帶給假新聞檢測很大的難關。以前使用不同方法檢測假新 聞的嘗試，在準確率方面遇到了限制。為了應對這些挑戰，我們提出了一種集成文本 和社交網絡信息的多模式假新聞檢測 (MFND) 方法。 MFND 結合了新聞的語義表示、 社交媒體上的傳播模式和超圖技術來豐富數據集。 在兩個真實世界數據集上的實驗結 果表明，MFND 優於最先進的模型，實現了更高的準確性。

With the rapid development of social media, the proliferation of fake news has become a pressing issue, further exacerbating societal polarization. Detecting fake news has become increasingly important in such circumstances. However, the rapid spread and evolution of fake news, along with the lack of comprehensive data, pose significant challenges to fake news detection. Previous attempts to detect fake news using different methods encountered limitations in terms of accuracy. To address these challenges, we propose a Multimodal Fake News Detection (MFND) approach that integrates text and social network information. MFND combines semantic representation of news, propagation patterns on social media, and hypergraph techniques to enrich the dataset. Experimental results on two real-world datasets demonstrate that MFND outperforms state-of-the-art models, achieving the highest accuracy on fake news detection.

[1] BBC. Territorial disputes in the South China Sea. https://www.bbc.com/ zhongwen/trad/world-53398946.
[Accessed: June 19, 2023].
[2] Tian Bian, Xi Xiao, Tingyang Xu, Peilin Zhao, Wen bing Huang, Yu Rong, and Junzhou Huang. Rumor detection on social media with bi-directional graph convolutional networks. In AAAI Conference on Artificial Intelligence, 2020.
[3] Lu Cheng, Ruocheng Guo, Kai Shu, and Huan Liu. Causal understanding of fake news dissemination on social media. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2020.
[4] Mingxi Cheng, Shahin Nazarian, and Paul Bogdan. Vroc: Variational autoencoderaided multi-task rumor classifier based on text. Proceedings of The Web Conference 2020, 2020.
[5] Matteo Cinelli, Gianmarco De Francisci Morales, Alessandro Galeazzi, Walter Quattrociocchi, and Michele Starnini. The echo chamber effect on social media. Proceedings of the National Academy of Sciences, 118, 2021.
[6] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pretraining of deep bidirectional transformers for language understanding. ArXiv, abs/1810.04805, 2019.
[7] Mudit Dhawan, Shakshi Sharma, Aditya Kadam, Rajesh Sharma, and Ponnurangam Kumaraguru. Game-on: Graph attention network based multimodal fusion for fake news detection. ArXiv, abs/2202.12478, 2022. 37
[8] Kaize Ding, Jianling Wang, Jundong Li, Kai Shu, Chenghao Liu, and Huan Liu. Graph prototypical networks for few-shot learning on attributed networks. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020.
[9] Yingtong Dou, Kai Shu, Congyin Xia, Philip S. Yu, and Lichao Sun. User preference aware fake news detection. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021.
[10] Yifan Feng, Haoxuan You, Zizhao Zhang, R. Ji, and Yue Gao. Hypergraph neural networks. In AAAI Conference on Artificial Intelligence, 2018.
[11] Matthias Fey and Jan E. Lenssen. Fast graph representation learning with pytorch geometric. ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
[12] Li Gao, Lingyun Song, Jie Liu, Bolin Chen, and Xuequn Shang. Topology imbalance and relation inauthenticity aware hierarchical graph attention networks for fake news detection. In International Conference on Computational Linguistics, 2022.
[13] Amir Globerson, Gal Chechik, Fernando C Pereira, and Naftali Tishby. Euclidean embedding of co-occurrence data. J. Mach. Learn. Res., 8:2265–2295, 2004.
[14] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NIPS, 2017.
[15] Yi Han, Shanika Karunasekera, and Christopher Leckie. Graph neural networks with continual learning for fake news detection from social media. ArXiv, abs/2007.03316, 2020. 38
[16] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2015.
[17] Zhenyu He, Ce Li, Fan Zhou, and Yi Yang. Rumor detection on social media with event augmentations. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021.
[18] Matthew Honnibal and Ines Montani. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear, 2017.
[19] Linmei Hu, Tianchi Yang, Luhao Zhang, Wanjun Zhong, Duyu Tang, Chuan Shi, Nan Duan, and Ming Zhou. Compare to the knowledge: Graph neural fake news detection with external knowledge. In Annual Meeting of the Association for Computational Linguistics, 2021.
[20] Ching-Chuan Huang. Ca-wav2lip: Coordinate attention-based speech to lip synthesis in the wild. 2022.
[21] Emrah Inan. Zoka: a fake news detection method using edge-weighted graph attention network with transfer models. Neural Computing and Applications, 34:11669 – 11677, 2022.
[22] Ujun Jeong, Kaize Ding, Lu Cheng, Ruocheng Guo, Kai Shu, and Huan Liu. Nothing stands alone: Relational fake news detection with hypergraph neural networks. 2022 IEEE International Conference on Big Data (Big Data), pages 596–605, 2022. 39
[23] Peng Jin, Yue Zhang, Xingyuan Chen, and Yunqing Xia. Bag-of-embeddings for text classification. In International Joint Conference on Artificial Intelligence, 2016.
[24] Zhiwei Jin, Juan Cao, Han Guo, Yongdong Zhang, and Jiebo Luo. Multimodal fusion with recurrent neural networks for rumor detection on microblogs. Proceedings of the 25th ACM international conference on Multimedia, 2017.
[25] Renuka Joshi. Accuracy, precision, recall f1 score: Interpretation of performance measures, Sep 2016.
[26] Zhezhou Kang, Yanan Cao, Yanmin Shang, Tao Liang, Hengzhu Tang, and Lingling Tong. Fake news detection with heterogenous deep graph convolutional network. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2021.
[27] Ling Min Serena Khoo, Hai Leong Chieu, Zhong Qian, and Jing Jiang. Interpretable rumor detection in microblogs by attending to user interactions. ArXiv, abs/2001.10667, 2020.
[28] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. CoRR, abs/1312.6114, 2013.
[29] Thomas Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. ArXiv, abs/1609.02907, 2016.
[30] Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. ArXiv, abs/1506.06726, 2015.
[31] R´emi Lebret and Ronan Collobert. Word embeddings through hellinger pca. In Conference of the European Chapter of the Association for Computational Linguistics, 2013.
[32] Omer Levy and Yoav Goldberg. Linguistic regularities in sparse and explicit word representations. In Conference on Computational Natural Language Learning, 2014.
[33] Omer Levy and Yoav Goldberg. Neural word embedding as implicit matrix factorization. In NIPS, 2014.
[34] Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI Conference on Artificial Intelligence, 2018.
[35] Yitan Li, Linli Xu, Fei Tian, Liang Jiang, Xiaowei Zhong, and Enhong Chen. Word embedding revisited: A new representation learning and explicit matrix factorization perspective. In International Joint Conference on Artificial Intelligence, 2015.
[36] Huan Liu, Jinhui Li, and Wenzhaoting Hu. Multi-image fusion multi-modal rumor detection. 2022 IEEE Conference on Telecommunications, Optics and Computer Science (TOCS), pages 313–318, 2022.
[37] Yang Liu and Yi fang Brook Wu. Early detection of fake news on social media through propagation path classification with recurrent and convolutional networks. In AAAI Conference on Artificial Intelligence, 2018.
[38] Yi-Ju Lu and Cheng te Li. Gcan: Graph-aware co-attention networks for explainable fake news detection on social media. ArXiv, abs/2004.11648, 2020.
[39] Jing Ma, Wei Gao, and Kam-Fai Wong. Rumor detection on twitter with tree41 structured recursive neural networks. In Annual Meeting of the Association for Computational Linguistics, 2018.
[40] Hayato Matsumoto, Soh Yoshida, and Mitsuji Muneyasu. Propagation-based fake news detection using graph neural networks with transformer. 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), pages 19–20, 2021.
[41] Tomas Mikolov, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. In International Conference on Learning Representations, 2013.
[42] Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. Distributed representations of words and phrases and their compositionality. ArXiv, abs/1310.4546, 2013.
[43] Federico Monti, Fabrizio Frasca, Davide Eynard, Damon Mannion, and Michael M. Bronstein. Fake news detection on social media using geometric deep learning. ArXiv, abs/1902.06673, 2019.
[44] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. https://pytorch.org, 2019.
[45] Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In North American Chapter of the Association for Computational Linguistics, 2018. 42
[46] Martin Potthast, Johannes Kiesel, Kevin Reinartz, Janek Bevendorff, and Benno Stein. A stylometric inquiry into hyperpartisan and fake news. ArXiv, abs/1702.05638, 2017.
[47] Carey E. Priebe, Cencheng Shen, Ningyuan Teresa Huang, and Tianyi Chen. A simple spectral failure mode for graph convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44:8689–8693, 2020.
[48] Piotr Przyby la. Capturing the style of fake news. In AAAI Conference on Artificial Intelligence, 2020.
[49] Muhammad Atif Qureshi and Derek Greene. Eve: explainable vector based embedding technique using wikipedia. Journal of Intelligent Information Systems, pages 1–29, 2017.
[50] Alec Radford and Karthik Narasimhan. Improving language understanding by generative pre-training. 2018.
[51] Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. Truth of varying shades: Analyzing language in fake news and political fact-checking. In Conference on Empirical Methods in Natural Language Processing, 2017.
[52] Yuxiang Ren and Jiawei Zhang. Fake news detection on news-oriented heterogeneous information networks through hierarchical graph attention. 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2020.
[53] Victoria L. Rubin, Niall Conroy, Yimin Chen, and Sarah Cornwell. Fake news or truth? using satirical cues to detect potentially misleading news. 2016. 43
[54] Natali Ruchansky, Sungyong Seo, and Yan Liu. Csi: A hybrid deep model for fake news detection. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017.
[55] Kai Shu. Beyond news contents : The role of social context for fake news detection. 2018.
[56] Kai Shu, Limeng Cui, Suhang Wang, Dongwon Lee, and Huan Liu. defend: Explainable fake news detection. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019.
[57] Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big data, 8 3:171–188, 2018.
[58] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large scale image recognition. CoRR, abs/1409.1556, 2014.
[59] Shivangi Singhal, Rajiv Ratn Shah, Tanmoy Chakraborty, Ponnurangam Ku maraguru, and Shin’ichi Satoh. Spotfake: A multi-modal framework for fake news detection. 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM), pages 39–47, 2019.
[60] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, A. Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Conference on Empirical Methods in Natural Language Processing, 2013. 44
[61] Chenguang Song, Kai Shu, and Bin Wu. Temporally evolving graph neural network for fake news detection. Inf. Process. Manag., 58:102712, 2021.
[62] Twitter Developer. Twitter api, 2021.
[63] Vaibhav Vaibhav, Raghuram Mandyam Annasamy, and Eduard H. Hovy. Do sentence interactions matter? leveraging sentence level representations for fake news classification. In Conference on Empirical Methods in Natural Language Processing, 2019.
[64] Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. ArXiv, abs/1706.03762, 2017.
[65] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio’, and Yoshua Bengio. Graph attention networks. ArXiv, abs/1710.10903, 2017.
[66] Michela Del Vicario, Alessandro Bessi, Fabiana Zollo, Fabio Petroni, Antonio Scala, Guido Caldarelli, Harry Eugene Stanley, and Walter Quattrociocchi. The spreading of misinformation online. Proceedings of the National Academy of Sciences, 113:554 – 559, 2016.
[67] Soroush Vosoughi, Deb K. Roy, and Sinan Aral. The spread of true and false news online. Science, 359:1146 – 1151, 2018.
[68] Yaqing Wang, Fenglong Ma, Zhiwei Jin, Ye Yuan, Guangxu Xun, Kishlay Jha, Lu Su, and Jing Gao. Eann: Event adversarial neural networks for multi-modal fake news detection. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018. 45
[69] Wikipedia. Disinformation in the Russian invasion of Ukraine. https://en. wikipedia.org/wiki/Disinformation_in_the_Russian_invasion_of_Ukraine.
[Accessed: June 19, 2023].
[70] Wikipedia. Pizzagate conspiracy theory. https://en.wikipedia.org/wiki/ Pizzagate_conspiracy_theory.
[Accessed: June 19, 2023].
[71] Wikipedia. Russian interference in the 2016 United States elections. https://en.wikipedia.org/wiki/Russian_interference_in_the_2016_ United_States_elections.
[Accessed: June 19, 2023].
[72] Yang Wu, Pengwei Zhan, Yunjian Zhang, Liming Wang, and Zhen Xu. Multimodal fusion with co-attention networks for fake news detection. In Findings, 2021.
[73] Han Xiao. bert-as-service. https://github.com/hanxiao/bert-as-service, 2018.
[74] Han Xiao. Bert-as-a-service, 2019.
[75] Junxiao Xue, Yabo Wang, Yichen Tian, Yafei Li, Lei Shi, and Lin Wei. Detecting fake news by exploring the consistency of multimodal data. Information Processing & Management, 58:102610 – 102610, 2021.
[76] Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard H. Hovy. Hierarchical attention networks for document classification. In North American Chapter of the Association for Computational Linguistics, 2016.
[77] Liang Yao, Chengsheng Mao, and Yuan Luo. Graph convolutional networks for text classification. In AAAI Conference on Artificial Intelligence, 2018.
[78] Sheng yi Jiang, Xiaoting Chen, Liming Zhang, Sutong Chen, and Haonan Liu. User46 characteristic enhanced model for fake news detection in social media. In Natural Language Processing and Chinese Computing, 2019.
[79] Feng Yu, Q. Liu, Shu Wu, Liang Wang, and Tieniu Tan. A convolutional approach for misinformation identification. In International Joint Conference on Artificial Intelligence, 2017.
[80] Huaiwen Zhang, Quan Fang, Shengsheng Qian, and Changsheng Xu. Multi-modal knowledge-aware event memory network for social media rumor detection. Proceedings of the 27th ACM International Conference on Multimedia, 2019.
[81] Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Graph neural networks: A review of methods and applications. ArXiv, abs/1812.08434, 2018.
[82] Xinyi Zhou, Jindi Wu, and Reza Zafarani. Safe: Similarity-aware multi-modal fake news detection. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2020.