Multi-modal Fake News Detection using Multimodal Approach of BERT and ResNet110

Abstract

Globally, the usage of social media has significantly increased and has become the most common way for people to deplete news. The easy sharing of multimedia content on social media has caused the fake news dimension, which threatens the stability as well as security of the society. Fake news detection (FND) in social media becomes challenging, because of which various tools are developed to detect them. Multi-modal FND aims to determine fake data by text as well as images. Most commonly, researchers identify fake news only as text, but not as images. This research proposes a multimodal approach for detecting fake news in the formats of both text and image, and for classifying news as real or fake. The proposed multimodal-based convolutional neural network (CNN) combines the designs of both text and image of fake news. This method utilizes two classification methods named bidirectional encoder representations from transformers (BERT) for text, and ResNet110 for images. This method uses the Fakeddit dataset to estimate and evaluate the performance. The experimental results of the proposed ResNet110+BERT model achieves respective accuracy, precision, recall and F1-score values of about 0.931, 0.944, 0.942, and 0.946, which is superior when compared to the existing methods, recurrent CNN (RCNN) and fine-grained multimodal fusion network (FMFN). From the analysis, it is proven that the proposed method ResNet110-BERT achieves an accuracy of 0.931, and hence shows better results for overall metrics when compared to the existing methods of RCNN and FMFN.