Skip to main content
Open Book Publishers

8. Algorithms Against Antisemitism? Towards The Automated Detection of Antisemitic Content Online

Chapter of: Antisemitism in Online Communication: Transdisciplinary Approaches to Hate Speech in the Twenty-First Century(pp. 205–236)

Export Metadata

  • ONIX 3.0
    • Thoth
    • Project MUSE
      Cannot generate record: No BIC or BISAC subject code
    • OAPEN
    • JSTOR
      Cannot generate record: No BISAC subject code
    • Google Books
      Cannot generate record: No BIC, BISAC or LCC subject code
    • OverDrive
      Cannot generate record: Missing Long Abstract
  • ONIX 2.1
  • CSV
  • JSON
  • OCLC KBART
  • BibTeX
    Cannot generate record: Missing Author/Editor Details
  • CrossRef DOI deposit
    Cannot generate record: This work does not have any ISBNs
  • MARC 21 Record
    Cannot generate record: MARC records are not available for chapters
  • MARC 21 Markup
    Cannot generate record: MARC records are not available for chapters
  • MARC 21 XML
    Cannot generate record: MARC records are not available for chapters
Metadata
Title8. Algorithms Against Antisemitism?
SubtitleTowards The Automated Detection of Antisemitic Content Online
ContributorElisabeth Steffen(author)
Milena Pustet(author)
Helena Mihaljević(author)
DOIhttps://doi.org/10.11647/obp.0406.08
Landing pagehttps://www.openbookpublishers.com/books/10.11647/obp.0406/chapters/10.11647/obp.0406.08
Licensehttps://creativecommons.org/licenses/by/4.0/
CopyrightElisabeth Steffen, Milena Pustet, Helena Mihaljević
PublisherOpen Book Publishers
Published on2024-06-21
Long abstractThe proliferation of hateful and violent speech in online media underscores the need for technological support to combat such discourse, create safer and more inclusive online environments, support content moderation and study political-discourse dynamics online. Automated detection of antisemitic content has been little explored compared to other forms of hate-speech. This chapter examines the automated detection of antisemitic speech in online and social media using a corpus of online comments sourced from various online and social media platforms. The corpus spans a three-year period and encompasses diverse discourse events that were deemed likely to provoke antisemitic reactions. We adopt two approaches. First, we explore the efficacy of Perspective API, a popular content- moderation tool that rates texts in terms of, e.g., toxicity or identity-related attacks, in scoring antisemitic content as toxic. We find that the tool rates a high proportion of antisemitic texts with very low toxicity scores, indicating a potential blind spot for such content. Additionally, Perspective API demonstrates a keyword bias towards words related to Jewish identities, which could result in texts being falsely flagged and removed from platforms. Second, we fine-tune deep learning models to detect antisemitic texts. We show that OpenAI’s GPT-3.5 can be fine-tuned to effectively detect antisemitic speech in our corpus and beyond, with F1 scores above 0.7. We discuss current achievements in this area and point out directions for future work, such as the utilisation of prompt-based models.
Page rangepp. 205–236
Print length236 pages
LanguageEnglish (Original)
Contributors

Elisabeth Steffen

(author)
https://orcid.org/0000-0003-0170-968X

Elisabeth Steffen is a researcher at the Berlin University of Applied Sciences (HTW Berlin). Her research interests include the intersections between natural language processing, supervised machine learning algorithms and cultural, social and political studies. This interdisciplinary focus is also reflected in her academic education as both a computer scientist (BA in Computer Science and Business Administration, HTW– University of Applied Sciences Berlin) and anthropologist of contemporary European societies (MA in European Ethnology, Humboldt University Berlin). Currently, she works as a research assistant in the project Digitaler Hass on conspiracy theories in the context of the COVID-19 pandemic.

Milena Pustet

(author)
https://orcid.org/0000-0003-3825-6530

Milena Pustet (BSc, University of Potsdam) is a research assistant at the Berlin University of Applied Sciences (HTW Berlin), specialising in data science, machine learning and natural language processing. As a member of the research projects Digitaler Hass and Decoding Antisemitism, which aim to better understand and combat online hate speech and antisemitism, her research focuses on understanding social phenomena online, particularly antisemitic and conspiracy-related discourses on social media.

Helena Mihaljević

(author)
https://orcid.org/0000-0003-0782-5382

Prof. Helena Mihaljević holds a chair in Data Science at the Berlin University of Applied Sciences (HTW Berlin). Her expertise lies in analysing data and technology, utilising cutting-edge methods in data science, machine learning and natural language processing. She is an accomplished researcher with a passion for interdisciplinary projects. She has contributed to a variety of projects, including the algorithmic detection of conspiracy theories and antisemitic hate speech. In her current role as the principal investigator for the research project Digital Hate, she focuses on studying conspiracy narratives related to the COVID-19 pandemic. She is also a Co-Investigator on the international research project Decoding Antisemitism, coordinated by the Technische Universität Berlin. Prior to her role at HTW Berlin, Prof. Mihaljević served as a Senior Data Scientist and gained substantial experience in scientific information infrastructure. She earned her PhD in Mathematics with a focus on the topological dynamics of entire transcendental functions.

References
  1. Aluru, Sai Saketh, Binny Mathew, Punyajoy Saha, and Animesh Mukherjee, 2020. “Deep Learning Models for Multilingual Hate Speech Detection”. Preprint, https://arxiv.org/abs/2004.06465
  2. Basile, Valerio, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso and Manuela Sanguinetti, 2019. “SemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter”. In: Proceedings of the 13th International Workshop on Semantic Evaluation. Minneapolis, MN, USA: Association for Computational Linguistics, 54–63, https://doi.org/10.18653/v1/S19-2007
  3. Chandra, Mohit, Dheeraj Pailla, Himanshu Bhatia, Aadilmehdi Sanchawala, Manish Gupta, Manish Shrivastava and Ponnurangam Kumaraguru, 2021. “’Subverting the Jewtocracy’: Online Antisemitism Detection Using Multimodal Deep Learning”. In: Proceedings of the 13th ACM Web Science Conference 2021 (WebSci ’21), Virtual Event, United Kingdom, 148–157, https://doi.org/10.1145/3447535.3462502
  4. Chiu, Ke-Li, Annie Collins and Rohan Alexander, 2022. “Detecting Hate Speech with GPT-3”. Preprint, http://arxiv.org/abs/2103.12407
  5. Dixon, Lucas, John Li, Jeffrey Sorensen, Nithum Thain and Lucy Vasserman, 2018. “Measuring and Mitigating Unintended Bias in Text Classification”. In: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. New Orleans LA USA: ACM, 67–73, https://doi.org/10.1145/3278721.3278729
  6. Elroy, Or and Abraham Yosipof, 2022. “Analysis of COVID-19 5G Conspiracy Theory Tweets Using SentenceBERT Embedding”. In: Artificial Neural Networks and Machine Learning – ICANN 2022: 31st International Conference on Artificial Neural Networks, Bristol, UK, September 6–9, 2022, Proceedings, Part II. Berlin: Springer-Verlag, 186–196, https://link.springer.com/chapter/10.1007/978-3-031-15931-2_16
  7. Falkenberg, Mark and Andrea Baronchelli, 2023. “How Can We Better Understand the Role of Social Media in Spreading Climate Misinformation?” Grantham Research Institute on Climate Change and the Environment. January 2023, https://www.lse.ac.uk/granthaminstitute/news/how-can-we-better-understand-the-role-of-social-media-in-spreading-climate-misinformation
  8. Gilardi, Fabrizio, Meysam Alizadeh, and Maël Kubli, 2023. “ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks”. In: Proceedings of the National Academy of Sciences 120, No. 30, e2305016120, https://doi.org/10.1073/pnas.2305016120
  9. González-Pizarro, Felipe and Savvas Zannettou, 2022. “Understanding and Detecting Hateful Content Using Contrastive Learning”. In: Proceedings of the Seventeenth International AAAI Conference on Web and Social Media (ICWSM 2023). June 5–8, 2023, Limassol, Cyprus. Palo Alto, CA: AAAI Press, 257-268, https://doi.org/10.1609/icwsm.v17i1.22143
  10. Horta Ribeiro, Manoel, Shagun Jhaver, Savvas Zannettou, Jeremy Blackburn, Gianluca Stringhini, Emiliano De Cristofaro and Robert West, 2021. “Do Platform Migrations Compromise Content Moderation? Evidence from r/The_Donald and r/Incels”. In: Proceedings of the ACM on Human-Computer Interaction 5 (CSCW2), 1–24, https://doi.org/10.1145/3476057
  11. Hoseini, Mohamad, Philipe Melo, Fabricio Benevenuto, Anja Feldmann and Savvas Zannettou, 2023. “On the Globalization of the QAnon Conspiracy Theory Through Telegram”. In: Proceedings of the 15th ACM Web Science Conference 2023 (WebSci ‘23). Association for Computing Machinery, New York, USA, 75–85, https://doi.org/10.1145/3578503.3583603
  12. Huang, Fan, Haewoon Kwak and Jisun An, 2023. “Is ChatGPT Better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech”. In: Companion Proceedings of the ACM Web Conference 2023, 294–97. Austin, TX: ACM, https://doi.org/10.1145/3543873.3587368
  13. Hutchinson, Ben, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong and Stephen Denuyl, 2020. “Social Biases in NLP Models as Barriers for Persons with Disabilities”. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5491– 5501, https://doi.org/10.18653/v1/2020.acl-main.487
  14. Jikeli, Günther, Sameer Karali, Daniel Miehling and Katharina Soemer, 2023. “Antisemitic Messages? A Guide to High-Quality Annotation and a Labeled Dataset of Tweets”. Preprint, http://arxiv.org/abs/2304.14599
  15. Jikeli, Günther, Damir Cavar, Weejeong Jeong, Daniel Miehling, Pauravi Wagh and Denizhan Pak, 2022. “Toward an AI Definition of Antisemitism?” In: Monika Hübscher and Sabine von Mering (eds). Antisemitism on Social Media. Abingdon: Routledge, 193–212
  16. Lauer, Stefan and Nicholas Potter (eds.), 2023. Judenhass Underground. Antisemitismus in emanzipatorischen Subkulturen und Bewegungen. Berlin / Leipzig: Hentrich & Hentrich Verlag
  17. Lees, Alyssa, Vinh Q. Tran, Yi Tay, Jeffrey Sorensen, Jai Gupta, Donald Metzler and Lucy Vasserman,, 2022. “A New Generation of Perspective API: Efficient Multilingual Character-level Transformers”. In: KDD ‘22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining 2022. 3197–3207, https://doi.org/10.1145/3534678.3539147
  18. Li, Lingyao, Lizhou Fan, Shubham Atreja and Libby Hemphill, 2023. ”‘HOT’ ChatGPT: The Promise of ChatGPT in Detecting and Discriminating Hateful, Offensive, and Toxic Comments on Social Media”. ACM Transactions on the Web 18 (2), Article No. 30, 1–36, https://doi.org/10.1145/3643829
  19. Liu, Pengfei, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi and Neubig, Graham, 2023. “Pre-Train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing”. ACM Computing Surveys 55, No. 9, Article No. 195, 1–35, https://doi.org/10.1145/3560815
  20. Mandl, Thomas, Sandip Modha, Gautam Kishore Shahi, Hiren Madhu, Shrey Satapara, Prasenjit Majumder, Schaefer, Johannes, Tharindu Ranasinghe, Marcos Zampieri, Durgesh Nandini and Amit Kumar Jaiswal, 2021. “Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages”, http://arxiv.org/abs/ https://doi.org/10.48550/arXiv.2112.09301
  21. Mathew, Binny, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal and Animesh Mukherjee, 2022. “HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection”. In: Proceedings of the AAAI Conference on Artificial Intelligence, 35 (17), 14867–14875
  22. Mendelsohn, Julia, Ronan Le Bras, Yejin Choi and Maarten Sap, 2023. “From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models“. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, Canada, 15162–15180, https://doi.org/10.18653/v1/2023.acl-long.845
  23. Meta, 2022. Community Standards Enforcement | Transparency Center, https://transparency.fb.com/data/community-standards-enforcement
  24. Mihaljević, Helena and Elisabeth Steffen, 2022. “How Toxic Is Antisemitism? Potentials and Limitations of Automated Toxicity Scoring for Antisemitic Online Content”. In: Proceedings of the 2nd Workshop on Computational Linguistics for Political Text Analysis (CPSS-2022), KONVENS 2022, 1–12. 01 January 2022. Potsdam, Germany
  25. Moffitt, J. D., Catherine King and Kathleen M. Carley, 2021. “Hunting Conspiracy Theories During the COVID-19 Pandemic“. Social Media + Society, 7 (3), https://doi.org/10.1177/20563051211043212
  26. Phillips, Samantha C., Lynnette Hui Xian Ng, Kathleen M. Carley, 2022. “Hoaxes and Hidden Agendas: A Twitter Conspiracy Theory Dataset: Data Paper“. In: Companion Proceedings of the Web Conference 2022. WWW ’22. New York: Association for Computing Machinery, 876–880, https://doi.org/10.1145/3487553.3524665
  27. Pogorelov, Konstantin, Daniel Thilo Schroder, Luk Burchard, Johannes Moe, Stefan Brenner, Petra Filkukova and Johannes Langguth, 2020. “FakeNews: Corona Virus and 5G Conspiracy Task at MediaEval 2020“. In: Working Notes Proceedings of the MediaEval 2020 Workshop, http://ceur-ws.org/Vol-2882/paper64.pdf
  28. Poletto, Fabio, Valerio Basile, Manuela Sanguinetti, Cristina Bosco and Viviana Patti, 2021. “Resources and Benchmark Corpora for Hate Speech Detection: A Systematic Review“. Language Resources and Evaluation, 55 (2), 477–523, https://doi.org/10.1007/s10579-020-09502-8
  29. Röttger, Paul, Bertram Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts and Janet B. Pierrehumbert, 2021. “HateCheck: Functional Tests for Hate Speech Detection Models“. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 41–58, https://doi.org/10.18653/v1/2021.acl-long.4
  30. Steffen, Elisabeth, Helena Mihaljević, Milena Pustet, Nyco Bischoff, María do Mar Castro Varela, Yener Bayramoğlu and Bahar Oghalai, 2022. “Codes, Patterns and Shapes of Contemporary Online Antisemitism and Conspiracy Narratives — an Annotation Guide and Labeled German-Language Dataset in the Context of COVID-19“. In: Proceedings of the Seventeenth International AAAI Conference on Web and Social Media (ICWSM 2023). June 5–8, 2023, Limassol, Cyprus. Palo Alto, CA: AAAI Press, https://doi.org/10.1609/icwsm.v17i1.22216
  31. Wang, Yau-Shian and Yingshan Chang, 2022. “Toxicity Detection with Generative Prompt-Based Inference”. Preprint, http://arxiv.org/abs/2205.12390
  32. White, Jules, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith and Douglas C. Schmidt, 2023. “A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT“. Preprint, http://arxiv.org/abs/2302.11382
  33. Wiegand, Michael, Melanie Siegel and Josef Ruppenhofer, 2018. “Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language“. In: Proceedings of GermEval 2018, 14th Conference on Natural Language Processing (KONVENS 2018), https://epub.oeaw.ac.at/0xc1aa5576_0x003a10d2.pdf
  34. Zampieri, Marcos, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra and Ritesh Kumar, 2019. “SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)”. In: Proceedings of the 13th International Workshop on Semantic Evaluation. Minneapolis, MN, USA: Association for Computational Linguistics, 75–86, https://doi.org/10.18653/v1/S19-2010
  35. Zampieri, Marcos, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis and Çağrı Çöltekin, 2020. “SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)”. In: Proceedings of the Fourteenth Workshop on Semantic Evaluation, 2020, https://doi.org/10.18653/v1/2020.semeval-1.188