“The article linked below was recently published by the Journal of the Association for Information Science and Technology (JASIST).

Title

Studying Effectiveness of Web Search For Fact Checking

Authors

Maram Hasanain
Qatar University

Tamer Elsayed
Qatar University

Source

Journal of the Association for Information Science and Technology (JASIST)
First Published: October 14, 2021
DOI: 10.1002/asi.24577

Abstract

Web search is commonly used by fact checking systems as a source of evidence for claim verification. In this work, we demonstrate that the task of retrieving pages useful for fact checking, called evidential pages, is indeed different from the task of retrieving topically relevant pages that are typically optimized by search engines; thus, it should be handled differently. We conduct a comprehensive study on the performance of retrieving evidential pages over a test collection we developed for the task of re-ranking Web pages by usefulness for fact-checking. Results show that pages (retrieved by a commercial search engine) that are topically relevant to a claim are not always useful for verifying it, and that the engine’s performance in retrieving evidential pages is weakly correlated with retrieval of topically relevant pages. Additionally, we identify types of evidence in evidential pages and some linguistic cues that can help predict page usefulness. Moreover, preliminary experiments show that a retrieval model leveraging those cues has a higher performance compared to the search engine. Finally, we show that existing systems have a long way to go to support effective fact checking. To that end, our work provides insights to guide design of better future systems for the task.

Direct to Full Text Article”