Abstract
In recent years, misinformation on the Web has become increasingly rampant. The research community has responded by proposing systems and challenges, which are beginning to be useful for (various subtasks of) detecting misinformation. However, most proposed systems are based on deep learning techniques which are fine-tuned to specific domains, are difficult to interpret and produce results which are not machine readable. This limits their applicability and adoption as they can only be used by a select expert audience in very specific settings. In this paper we propose an architecture based on a core concept of Credibility Reviews (CRs) that can be used to build networks of distributed bots that collaborate for misinformation detection. The CRs serve as building blocks to compose graphs of (i) web content, (ii) existing credibility signals –fact-checked claims and reputation reviews of websites–, and (iii) automatically computed reviews. We implement this architecture on top of lightweight extensions to Schema.org and services providing generic NLP tasks for semantic similarity and stance detection. Evaluations on existing datasets of social-media posts, fake news and political speeches demonstrates several advantages over existing systems: extensibility, domain-independence, composability, explainability and transparency via provenance. Furthermore, we obtain competitive results without requiring finetuning and establish a new state of the art on the Clef’18 CheckThat! Factuality task.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
In our opinion, current AI systems cannot truly assess veracity since this requires human skills to access and interpret new information and relate them to the world.
- 7.
- 8.
Note that ClaimReview is not suitable since it is overly restrictive: it can only review Claims (and it assumes the review aspect is, implicitly, accuracy).
- 9.
The source code is available at https://github.com/rdenaux/acred.
- 10.
- 11.
- 12.
Note that usability evaluation of the generated explanations is not in the scope of this paper.
- 13.
Our implementation has support for machine translation of sentences, however this adds a confounding factor hence we leave this as future work.
- 14.
https://github.com/KaiDMML/FakeNewsNet, although we note that text for many of the articles could no longer be retrieved, making a fair comparison difficult.
- 15.
- 16.
acred’s data collector is used to build the ClaimReview database described in Sect. 4; it does not store the itemReviewed URL values; only the claimReviewed strings.
- 17.
As stated above, we used the results of this analysis to inform the changes implemented in \(\mathtt {acred}^{\mathtt {+}}\).
References
Babakar, M., Moy, W.: The state of automated factchecking. Technical report (2016)
Boland, K., Fafalios, P., Tchechmedjiev, A.: Modeling and contextualizing claims. In: 2nd International Workshop on Contextualised Knowledge Graphs (2019)
Cazalens, S., Lamarre, P., Leblay, J., Manolescu, I., Tannier, X.: A content management perspective on fact-checking. In: The Web Conference (2018)
Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., Specia, L.: SemEval-2017 task 1: semantic textual similarity multilingual and cross-lingual focused evaluation. In: Proceedings of the 10th International Workshop on Semantic Evaluation, pp. 1–14 (2018)
Guha, R.V., Brickley, D., Macbeth, S.: Schema.org: evolution of structured data on the web. Commun. ACM 59(2), 44–51 (2016)
Hassan, N., et al.: ClaimBuster: the first-ever end-to-end fact-checking system. In: Proceedings of the VLDB Endowment, vol. 10, pp. 1945–1948 (2017)
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. Technical report (2019)
Marwick, A., Lewis, R.: Media Manipulation and Disinformation Online. Data & Society Research Institute, New York (2017)
Mensio, M., Alani, H.: MisinfoMe: who’s interacting with misinformation? In: 18th International Semantic Web Conference: Posters & Demonstrations (2019)
Mensio, M., Alani, H.: News source credibility in the eyes of different assessors. In: Conference for Truth and Trust Online (2019, in press)
Nakov, P., et al.: Overview of the CLEF-2018 CheckThat! Lab on automatic identification and verification of political claims. In: Bellot, P., et al. (eds.) CLEF 2018. LNCS, vol. 11018, pp. 372–387. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98932-7_32
Papadopoulos, S., Bontcheva, K., Jaho, E., Lupu, M., Castillo, C.: Overview of the special issue on trust and veracity of information in social media. ACM Trans. Inf. Syst. (TOIS) 34(3), 1–5 (2016)
Pérez-Rosas, V., Kleinberg, B., Lefevre, A., Mihalcea, R.: Automatic detection of fake news. In: COLING (2018)
Pomerleau, D., Rao, D.: The fake news challenge: exploring how artificial intelligence technologies could be leveraged to combat fake news (2017)
Schiller, B., Daxenberger, J., Gurevych, I.: Stance detection benchmark: how robust is your stance detection? (2020)
Shu, K., Mahudeswaran, D., Wang, S., Lee, D., Liu, H.: FakeNewsNet: a data repository with news content, social context and spatialtemporal information for studying fake news on social media. Technical report (2018)
Tchechmedjiev, A., et al.: ClaimsKG: a knowledge graph of fact-checked claims. In: Ghidini, C., et al. (eds.) ISWC 2019. LNCS, vol. 11779, pp. 309–324. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30796-7_20
Thorne, J., Vlachos, A., Cocarascu, O., Christodoulopoulos, C., Mittal, A.: The FEVER 2.0 shared task. In: Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER), pp. 1–6 (2019)
Wang, D., Simonsen, J.G., Larsen, B., Lioma, C.: The Copenhagen team participation in the factuality task of the competition of automatic identification and verification of claims in political debates of the CLEF-2018 fact checking lab. CLEF (Working Notes) 2125 (2018)
Zubiaga, A., Aker, A., Bontcheva, K., Liakata, M., Procter, R.: Detection and resolution of rumours in social media: a survey. ACM Comput. Surv. 51(2), 1–36 (2018)
Acknowledgements
Work supported by the European Commission under grant 770302 – Co-Inform – as part of the Horizon 2020 research and innovation programme.
Thanks to Co-inform members for discussions which helped shape this research and in particular to Martino Mensio for his work on MisInfoMe. Also thanks to Flavio Merenda and Olga Salas for their help implementing parts of the pipeline.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Denaux, R., Gomez-Perez, J.M. (2020). Linked Credibility Reviews for Explainable Misinformation Detection. In: Pan, J.Z., et al. The Semantic Web – ISWC 2020. ISWC 2020. Lecture Notes in Computer Science(), vol 12506. Springer, Cham. https://doi.org/10.1007/978-3-030-62419-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-62419-4_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-62418-7
Online ISBN: 978-3-030-62419-4
eBook Packages: Computer ScienceComputer Science (R0)
