Giving Voice to Digital Democracies is part of the Centre for the Humanities and Social Change at CRASSH, funded by the Humanities and Social Change International Foundation. The Fact-checking Hackathon took place from 10 – 12 January 2020 at the Cambridge University Engineering Department.
Project manager Marcus Tomalin welcomed attendees to the event, before Mevan Babkar, Head of automated fact-checking at FullFact, gave an insightful talk about human-based fact-checking. She discussed the various ways in which information can be used and abused, and she explained FullFact’s fact-checking processes. It was particularly fascinating to hear about their work during the recent general election.
James Thorne, a PhD student at the Department of Computer Science and Technology, talked about fact extraction and verification, and how approaches from Natural Language Processing can help. He also discussed Fact Extraction and VERification (FEVER) shared-tasks.
Jonty Page, a current 4th-year engineering student, gave an overview of an open-source fact-checking system the participants could develop during the Hackathon, and he highlighted some potential challenges and topics they could explore. Given a claim to be fact-checked, the baseline system (i) retrieves Wikipedia pages relevant to the claim, (ii) selects particular sentences from those pages which relate to the claim, and (iii) classifies those sentences either as supporting or refuting the original claim or else as providing too little information to either support or refute it.
Creating an interdisciplinary environment
The task of dealing with false claims automatically is necessarily an interdisciplinary task. TheHackathon created a collaborative environment for researchers from a variety of backgrounds. The weekend brought together people with expertise in areas including linguistics, psychology, sociology, education, criminology, mathematics, philosophy, critical thinking, natural language processing, computer science, and software engineering. Therefore, it was a profoundly interdisciplinary event. On the second day of the Hackathon, Dr Shauna Concannon ran some introductory sessions on Python for participants who wanted to learn more about coding and especially using Python to analyse natural language.
Ideas & projects
The teams worked on different aspects of the fact-checking task, including developing new methods for retrieving relevant sentences and documents by integrating information contained in hyperlinks, identifying claims that required multiple pieces of evidence in order to be correctly classified; identifying problematical linguistic patterns (such as claims that required comparisons or which included temporal assertions or quotations), and developing new methods for evaluating conflicting evidence using a confidence scoring metric. The interdisciplinary interest that this event generated confirms the urgent need for inclusive and collaborative events that bridge the divide between technology, the humanities, and the social sciences.
Thoughts from some of the participants
It was a great opportunity to come together with people from different backgrounds, people who are doing mathematics, engineering, computer science, linguistics, criminology.
I came to the fact-checking hackathon because I think it is a very important problem to work on. I learnt that automated fact-checking is a very hard task that involves a number of different components.
I feel that some new perspectives, notably what some of the [participants] have been doing with linguistics work can probably yield a smarter understanding of the way truth can be found in sentences. Seeing the different approaches of people trying to solve this problem – really various degrees of expertise and that’s all come together to produce what I feel are very informed contributions and definitely people who are very different to me – so, good to see that.
Click to play a video