Source-linked AI summary
The Surge of Anti-Semitism in German Social Media following the October 7 Attacks
Gregor Wiedemann, Daniel Wehrend
TL;DR
The paper examines how October 7 affected anti-Semitism in German social media and whether LLMs can detect its forms at scale. It evaluates open-weight models with and without user context on Facebook and Telegram posts, finding a significant surge on both platforms and post-attack convergence in discourse patterns.
Problem
The paper asks how well German-language social media posts can be automatically classified for anti-Semitism and its specific manifestations, and how October 7 affected category prevalence and salience across platforms.
Source: §1 · p. 1Method
The study applies open-weight LLMs to German-language Facebook and Telegram posts, testing user information as additional context and validating model selection against expert-annotated posts.
Source: §1 · p. 2Results
Takeaways & Limitations
User context reduces false positives in cases such as criticism of anti-Semitic statements and neutral reporting, while the best-performing models can support further quantitative analysis.
Source: §1 · p. 2The keyword-based collection may miss anti-Semitic content that does not refer to Israeli or Jewish affairs.
Source: §3.1 · p. 4, Limitations · p. 9
Abstract
from arXiv · showhide
We investigate the extent to which the Hamas attacks on Israel of October 7, 2023, have affected German social media debates about Judaism and Israel. For this, we develop an approach to detect 26 anti-Semitic categories in user postings via large language models (LLMs). The approach is applied to Facebook and Telegram posts (N=125,718) from three months before and after the event. Methodically, we test different open-weight models in two setups---with and without user information as additional context to the post text. The best setup achieves up to 83 % F1-score for binary anti-Semitism detection on our manually coded validation set. User context provides valuable information for most LLMs and drastically reduces false positives, for example, when (critically) reporting on anti-Semitic incidents. Concerning our topic, we find that anti-Semitism is surging significantly on both platforms, while being about ten times more prevalent on Telegram compared to Facebook. Facebook users express anti-Semitic views most likely in posts about an alleged genocide in Gaza carried out by the Israeli army, whereas classic anti-Semitic stereotypes related to power and conspiracy theories are dominant on Telegram. After the attack, the discourse patterns on both platforms show signs of convergence, as classic anti-Semitism increases on Facebook, whereas Israel-related categories surge on Telegram.
1 Introduction
The paper asks whether the October 7 attacks changed anti-Semitism in German social media debates and how well LLMs can detect its forms. It applies automated analysis to Facebook and Telegram, finding a post-attack convergence in language patterns despite significant platform differences.
1 Introduction
- The paper asks whether German-language posts reveal anti-Semitism, including Holocaust denial, conspiracy theories, and denial of Israel’s right to exist, and whether user information improves LLM classification.Source: §1 · p. 1
- The study examines how October 7 affected the prevalence and salience of anti-Semitism categories in public Facebook and Telegram posts about Judaism or Israel, sampled three months before and after the attack.Source: §1 · p. 1
- The study evaluates open-weight LLMs for detecting 26 anti-Semitic categories in German-language posts and reports satisfactory performance for further quantitative analysis.A domain expert helped manually annotate a validation sample of about 800 posts for model selection and prompt optimization.Source: §1 · p. 2
- Adding information about a post’s source reduces false positives, including when users criticize anti-Semitic statements or newspapers report neutrally on such events.Source: §1 · p. 2
- After October 7, anti-Semitic language patterns converged across Facebook and Telegram, despite significant differences in prevalence.The study treats the platforms as representing mainstream and fringe voices in German public discourse.Source: §1 · p. 2
2 Related work
The related work frames anti-Semitism detection as a context-sensitive task with contested definitions and limited language resources. This study uses the IHRA definition as a baseline and combines LLM classification with a fine-grained codebook for German social media.
2.1 Definitions of Anti-Semitism
- The study adopts the IHRA definition as its baseline and operationalizes it through 26 sub-categories with diagnostic indicators and positive and negative examples.The JDA distinguishes classic and Israel-related anti-Semitism from criticism that is not inherently anti-Semitic, emphasizing context-sensitive boundaries that are difficult to classify computationally.Source: §2.1 · p. 2
2 Related work
- Prior research identifies ambiguity, annotation subjectivity, and domain shift as recurring hate-speech detection challenges, while antisemitism-specific work emphasizes careful annotation and borderline cases.Post-October 7 monitoring has often used qualitative analyses of small social-media samples.Source: §2.2 · p. 3
2.2 Automatic Anti-Semitism detection
- Public academic language resources for anti-Semitism remain limited, and earlier LLM work focused mainly on binary classification, leaving fine-grained typologies unresolved.The study addresses this gap with scalable LLM classification and a project-specific codebook for German-language social media, distinguishing forms of anti-Semitism from related but non-antisemitic discourse.Source: §2.2 · p. 3
3 Data and Methodology
The study builds a keyword-based dataset of public German-language posts, develops a 26-category codebook, and evaluates open-weight LLMs for binary and fine-grained antisemitism detection.
3.1 Data Collection
- The dataset covers public German-language Facebook and Telegram posts matching Jewish- or Israel-related keywords from July 7, 2023, to January 7, 2024.The collection is bounded by public content available through CrowdTangle and the German-language channels covered in the SMO collection.Source: §3.1 · p. 4
3.2 Data Annotation
- The 26-category codebook informed iterative zero-shot and few-shot sampling, producing an expert-labeled validation set of 803 posts.The sampling combined random and category-stratified examples to balance coverage of dataset characteristics with attention to rare categories.Source: §3.2 · p. 5
- Only 16 of 26 subcategories achieved moderate or substantial coder agreement, so results for small categories require particular caution.The authors relied on expert labels for ground truth; some categories had few examples, and five cases had precision below 20 %, risking prevalence overestimation.Source: §3.2 · p. 5, §3.3 · p. 6
3.3 Model selection
- Providing user context generally improved older LLMs’ precision and recall, while the effect disappeared for the newest Qwen and Gemma models.The context included author names and platform usernames, which can help distinguish neutral reporting or criticism from antisemitic expression.Source: §3.3 · pp. 5–6
- Gemma 4 31b achieved 83 % F1-score (88 % accuracy) and Qwen 3 32b achieved 81 % F1-score (85 % accuracy) for binary detection.OpenAI’s gpt-oss 120b reached ca. 79 % F1-score and 84 % accuracy, while classifying the validation set about five times faster than the newer top-scoring models.Source: §3.3 · p. 6, Table 2 · p. 6
- About half of the subcategory classes reached around 60 % F1-score or higher, while most smaller classes fell below 50 % F1-score.Accuracy remained high even for imbalanced classes, and smaller models sometimes hallucinated rationales unsupported by post text.Source: §3.3 · p. 6
4 Results
The study finds that anti-Semitism increased across categories on both platforms, with Telegram showing far higher prevalence, while category profiles shifted toward greater similarity after October 7.
4 Results
- On Facebook, genocide accusations against Palestinians by the Israeli army (K17) dominated after the attacks, while Telegram remained dominated by Jewish-conspiracy clichés (K07).K17 had played a negligible role on Facebook before the attacks; Telegram's classic cliché remained dominant before and after.Source: §4 · pp. 6, 8
- Telegram had 10,726 anti-Semitic posts (15.9%), versus 1,028 (1.7%) on Facebook, and counts rose across all categories on both platforms.The full dataset was labeled with gpt-oss 120b; Figure 2 presents category frequencies by platform.Source: §4 · p. 6, Figure 2 · p. 7
- Telegram's category-share increases centered on anti-Israel antisemitism, while classic tropes declined relatively; Facebook showed increases in some classic tropes and less articulation of boycott calls, insults, and apartheid analogies.The genocide accusation had the strongest increase in category share on Telegram; Figure 3 presents category shares by platform.Source: Figure 3 · p. 7, §4 · p. 8
- The platforms' previously distinct category patterns became more similar, as anti-Israel antisemitism rose on Facebook and was added to Telegram's already strong traditional antisemitism.The platforms therefore showed closer alignment between mainstream and fringe discourse patterns.Source: §4 · p. 8
5 Concluding remarks
The study applies a 26-category codebook and expert-annotated validation to German Facebook and Telegram posts. It finds a post-October 7 surge on both platforms and shifts in category composition, while noting the study's scope and methodological limits.
5 Concluding remarks
- The study operationalizes antisemitism through a tailored 26-category codebook and an expert-annotated validation set for German-language Facebook and Telegram posts.The codebook draws on an adaptation of the IHRA definition, as described in the conclusion.Source: §5 · p. 8
- The authors argue that genocide characteristics probably had not yet been met in Gaza during the study period.This qualifies the discussion of genocide allegations in the paper.Source: §5 · p. 8
- Anti-Semitism surged after October 7 on both platforms, with Telegram's prevalence roughly an order of magnitude higher than Facebook's.The analysis applies a well-performing model to a comprehensive sample of Judaism- and Israel-related posts.Source: §5 · p. 8
- After October 7, genocide accusations became most salient on Facebook, while Telegram combined classic tropes with rising Israel-focused accusations, bringing the platforms' discourse patterns closer.The authors say the profiles converged against different discursive baselines.Source: §5 · p. 8
Limitations
The study's keyword-based collection and context-sensitive classification constrain coverage and require continued expert oversight.
Limitations
- Keyword-based collection of Israeli- and Jewish-affairs posts does not guarantee coverage of all anti-Semitic content on the platforms.The authors frame this as a limitation of their data-collection approach.Source: Limitations · p. 9
- Performance is uneven across categories, with systematic confusion between semantically adjacent labels and meaning that depends on context.The authors call for continued human oversight and iterative refinement.Source: Limitations · p. 9
- Definitions of antisemitism remain debated, while the changing Middle East crisis further polarizes the debate; LLMs do not replace expert judgment.The paper presents LLMs as scalability instruments for expert-defined analyses at a particular point in time.Source: Limitations · p. 9
A.1 Subcategories
Table A.1 summarizes the English translation of the codebook and annotated-example counts by category.
A.1 Subcategories
- Table A.1 summarizes the translated codebook alongside the number of annotated examples for each category.Source: §A.1 · p. 10
A.2 Label co-occurrence
Figure 4 uses true-versus-predicted label co-occurrence counts to show which categories are semantically close and frequently confused.
A.2 Label co-occurrence
- The heatmap displays co-occurrence counts between true labels and labels predicted by the best-performing model on the validation dataset.These counts indicate which true labels tend to be predicted together.Source: §A.2 · p. 10
- The displayed confusions identify categories that are semantically close and frequently mixed up during automatic coding.Source: §A.2 · p. 10
- Evil (K02) is frequently confused with both secret power and conspiracies (K07) and accusations of genocide in Gaza (K17).Source: §A.2 · p. 10
A.3 Increase/decrease of category shares
Figure 5 shows differing pre/post-attack developments in category shares that lead to an annealing of discourse patterns across the platforms.
A.3 Increase/decrease of category shares
- Figure 5 shows different developments in anti-Semitic category shares that lead to an annealing of discourse patterns on both platforms.Source: §A.3 · p. 10
B Online Resources
The online resources include the final LLM classification prompt, an expert-annotated validation set, and a codebook with category definitions and agreement information.
B Online Resources
- The online resources include the final system prompt used to classify anti-Semitism categories with LLMs and the expert-annotated validation set.Source: B Online Resources · p. 10
- Table 4 presents category definitions, example counts, and Cohen’s Kappa inter-coder agreement for the expert validation set.The names and descriptions are translated from the German original; the full original codebook and corresponding LLM prompts are available in the online resources.Source: Table 4
- Figure 4 is a heatmap of co-occurrence counts, with true labels as rows and predicted labels as columns.Source: Figure 4 · p. 12
- Figure 5 presents pre/post-October 7 differences in category shares relative to the overall number of platform posts.Source: Figure 5 · p. 12