Move Fast and Mend Things: Keeping Up with Evolving AI Harms Using Social Media Commentary

1 Introduction

• Novel AI capabilities and downstream contexts generate diverse harms that ex-ante measures and top-down frameworks may not anticipate or capture.The paper distinguishes AI risk—the ex-ante probability of a hazard—from AI harm: materialised setbacks, including subjective fears and anticipatory anxieties.Source: §1 · p. 1
• Participatory and sociotechnical methods provide nuanced local insights but are resource-intensive, narrow in scope, and slow relative to AI deployment.The paper therefore calls for macro-scale approaches that dynamically identify evolving harms.Source: §1 · p. 1
• The study analyses 5.7 million Reddit posts from 1,000 subreddits over 18 months, filtering them to 575,000 AI-harm posts for taxonomy construction and temporal analysis.The authors describe social-media disclosures as partial, distributed, and noisy traces rather than direct ground truth.Source: §1 · pp. 1–2
• The work contributes a reusable LLM-assisted pipeline, a curated dataset and bottom-up taxonomy, and analyses of users’ perceptions of existing and emerging harms.The authors present social-media discussion as a source of insights for AI regulators, researchers, and developers.Source: §1 · p. 2

2.1 Mapping AI Risks and Harms

• Risk assessments, benchmarks, red-teaming, consultations, incident tracking, and audits map AI risks, while established taxonomies organise harms across technical and societal domains.Expert-informed taxonomies can require resource-intensive updates and may lag changing capabilities and social contexts.Source: §2.1 · p. 2

2.2 LLM-Facilitated Analysis of AI Harms Discourse on Social Media

• Social-media analysis can capture unprompted themes and perspectives on harms that surveys, consultations, product logs, or system telemetry may miss.Existing analyses often focus on specific product categories, whereas the authors include perspectives across diverse AI-tool ecosystems.Source: §2.2 · pp. 2–3
• Online disclosures are biased and may not reflect offline populations, actual perspectives, or behaviour, so social-media analysis is not a substitute for participatory research.The authors frame posts as one lens that may provide early-warning signals for risks requiring complementary investigation.Source: §2.2 · p. 3
• LLMs may offer scalable thematic analysis, but concerns about bias, inconsistency, and transparency motivate reflexive, iterative codebook development and manual evaluation.The authors describe multiple rounds of data exploration and manual analysis to build codebooks and evaluation data at each processing stage.Source: §2.2 · p. 3

3 Method

• The pipeline applies an LLM-developed thematic codebook to Reddit posts, generates concise summaries, and constructs a taxonomy from those summaries; the annotated, paraphrased dataset is released as a research tool.The authors note that Reddit’s user base is skewed and that their findings are not presented as representative of broader public concern.Source: §3 · p. 3

3.1 Collecting AI harms posts from Reddit

• The team collected posts from a policy-compliant Reddit crawl, refined its filtering prompt using manually labelled samples, and identified 575,000 AI-harm posts among 5.7 million posts.On a balanced sample, filtering accuracy was 0.84 and F1 was 0.81; recall was 0.97, which the authors prioritised to avoid discarding non-standard harm expressions.Source: §3.1 · pp. 3–4

3.2 LLM-facilitated Thematic Analysis and Text synthesis

• The thematic codebook captures affected and AI-using entities, author perspective, perceived severity, referenced data types, and connected AI capabilities; summaries were factual in 93% and complete in 96% of sampled cases.Coding precision was 80%–100% across five features, while the Entity Using AI tool feature reached 67% precision and was retained for exploratory utility.Source: §3.2 · p. 4

3.3 Taxonomy Generation and Assignment

• The bottom-up taxonomy pipeline extracts concepts from post summaries, merges them in stages, and organises the most prevalent concepts into a two-level hierarchy with defined mechanisms, impacts, and scope boundaries.The taxonomy was compared with the MIT AI Risk taxonomy: it fully covered 13 nodes, partially covered 8, and did not cover 3; it also surfaced granular user-described harms absent from that framework.Source: §3.3 · pp. 4–5
• Posts receive one or more taxonomy labels when a category directly describes a core aspect of the text, with assignments restricted to confidence of at least 70%.Parent categories are considered when no subnode applies.Source: §3.3 · p. 5
• The final taxonomy-assignment evaluation found 89.2% precision across 59 possible labels on a diverse sample, compared with a 1.69% uniform-random baseline.All 60 evaluated posts had some or all labels correct; human agreement was Po = 71.7% and Gwet’s AC1 = 0.645.Source: §3.3 · p. 5

4.1 RQ1: What human-AI interaction harms can we detect from a bottom-up analysis of social media posts about AI, and how are these different from those in existing AI harm taxonomies?

• The bottom-up taxonomy contains 12 broad harm categories and 47 subnodes, surfacing granular, user-centred harms that traditional frameworks miss.Its categories span familiar technology-facilitated and platform harms, harms tied to novel AI deployments, and operational friction.Source: §4.1 · p. 5, Table 1 · p. 6
• User frustrations account for 45% of posts, followed by hallucinations at 16% and psychological distress at 14%.Multi-node assignment contributes to the frustration share; the broad filtering prompt prioritised recall over precision.Source: §4.1 · p. 6

4.1.1 Algorithmic Bias, Discrimination and Unfairness.

• Gig workers describe perceived pay disparities from opaque, unpredictable management algorithms, while job candidates report opaque AI hiring and evaluation with inadequate disability accommodations.These subnodes locate fairness concerns in users’ day-to-day lives and economic stability.Source: §4.1.1 · p. 6

4.1.2 False and misleading content.

• The taxonomy separates accidental errors, including factual, instruction-following, code, and calculation failures, from deliberate manipulation such as deepfake abuse.Reported consequences range from lost productivity and mistrust to psychological, reputational, and bodily harms.Source: §4.1.2 · pp. 6–7

4.1.3 Privacy.

• Privacy discussions span four subnodes, including fears of exposed personal inputs, leaked business data, uncontrolled training use, and AI-enabled surveillance.Posts also describe consent concerns, difficult opt-outs, and workarounds such as sandboxed browsing.Source: §4.1.3 · p. 7

4.2 RQ2: How are social media narratives around prominent human-AI interaction harms changing over time, and what does this indicate about changing contexts of AI use?

• Across the 18-month period, posts increased overall, while three case studies show disproportionate growth in agentic privacy and security, workplace adoption concerns, and companion discontinuation grief.The cases illustrate rapid acceleration, steady growth, and event-driven spikes, respectively.Source: §4.2 · p. 7

4.2.1 The rise of agentic privacy and security concerns.

• Agentic privacy and security concerns rose steeply in early 2026 as users questioned whether they could see or stop consequential actions by autonomous systems.Posts describe unsanctioned actions, prompt-injection risks, and agents confidently reporting success despite execution failures.Source: §4.2.1 · pp. 7–8, Figure 1 · p. 8

4.2.2 The operational burden of AI adoption in the workplace.

• Workplace posts show a growing share of discussion about vendor hype, while employees describe unsupported AI mandates adding operational work and reducing productivity.Accounts connect leadership’s uncritical adoption push with frontline workers correcting low-quality outputs without adequate support.Source: §4.2.2 · pp. 8–9, Figure 2 · p. 9

5.1 Move Where It Matters and Mend Things: Prioritising Lived Experience

• The bottom-up taxonomy complements expert frameworks by surfacing everyday harms they aggregate or overlook, including four privacy concerns and companion discontinuation grief.Online discourse also captures third-party effects, data-practice anxieties, and defensive workarounds that product logs may miss.Source: §5.1 · p. 11
• Reddit discourse is individualistic, with over 70% of posts tagged as affecting the author, while systemic societal harms are less well covered.The authors argue that combining bottom-up lived-experience signals with top-down taxonomies is essential for identifying risks left unmitigated in practice.Source: §5.1 · p. 11

5.2 Move and Mend Things Fast: Addressing Risks and Harms as they Emerge

• The pipeline can surface publicly shared harm signals without waiting for formal incident reporting or periodic taxonomy revisions that may take months or years.Its detection latency is bounded by when affected people choose to disclose online; interviews, co-design, and community audits remain essential for contextual understanding.Source: §5.2 · p. 11
• Temporal findings point to actionable design priorities: agent action logs and rollback, consultative workplace rollouts, and safer companion-service transitions.The discussion links these priorities to user concerns identified in the three case studies, including advance notice and exportable chat histories.Source: §5.2 · p. 11

5.3 Move Fast And Mend Things Again: Ensuring Pipeline Flexibility and Usability

• The modular pipeline can run on varied organic discourse and be adapted to different research agendas; its findings can localise concerns for targeted review.Examples include product-specific monitoring and using growing harm categories to inform red-teaming priorities.Source: §5.3 · pp. 11–12

6 Limitations and Future Work

• Reddit’s demographic skew and the researchers’ analytical commitments shape which harms are visible, so the taxonomy is a situated lens rather than a comprehensive map of public concern.The authors note likely underrepresentation of harms experienced by older adults, women, non-English speakers, and marginalised communities.Source: §6 · p. 12
• LLM analysis creates tensions between interpretive depth and scalability, stochastic outputs challenge reproducibility, and the taxonomy cannot be compared with human researchers’ alternative conceptualisations.The authors do not systematically evaluate performance in languages other than English and describe the taxonomy as a scalable scouting tool, not a ground-truth ontology.Source: §6 · p. 12
• Applying the pipeline to non-English forums, regional or mobile-first platforms, and specific online communities could surface harms underrepresented in the Reddit dataset.The authors also propose comparing taxonomies generated with different models, prompts, and parameters to assess robustness versus model artefact.Source: §6 · p. 12
• Future work could map taxonomy nodes to regulatory instruments and incident databases, and test whether user anxieties signal harms materialising in practice.An interactive tool for product and policy teams is another proposed direction, not an approach explored in this work.Source: §6 · p. 12

7 Conclusion

• Organic social-media discourse can rapidly surface granular, vernacular AI harms that complement traditional top-down risk frameworks.The conclusion situates these harms in the complex interplay between AI systems, user practices, and diverse deployment contexts.Source: §7 · pp. 12–13
• The authors do not endorse ever-faster AI deployment; they emphasize thoughtful, human-centred design, robust safeguards, and harm-response infrastructure that keeps pace with model development.They note that no single research group or governance body can unilaterally control deployment pace.Source: §7 · p. 13
• The pipeline compresses harm-detection timelines from months to days and can be rerun inexpensively across new products, platforms, and data sources.The authors say its outputs can direct resource-intensive qualitative research and ground governance decisions in empirical evidence of user experience.Source: §7 · p. 13

A Model Parameters

• All LLM inference used Gemini 3.5 Flash Lite through the Google Cloud Vertex AI Batch Prediction API, using an August 2026 snapshot.Source: A Model Parameters · p. 17
• Sampling temperature was T=0.0, and constrained decoding used a strict JSON schema to support deterministic, reproducible outputs across runs.The schema included at least one enumerated identifier set, integer certainty score, or natural-language justification string.Source: A Model Parameters · p. 17
• Prompts used a zero-shot, single-turn format, and Vertex AI batch inputs and outputs were isolated to the user’s project and not used by Google for model training or fine-tuning.Source: A Model Parameters · p. 17

B Inference Scale, Token Consumption, and Financial Cost

• $3,514.92 USD was the total computational cost across all experimental stages, equivalent to $0.32 USD per 1,000 evaluated posts.Source: B Inference Scale, Token Consumption,… · p. 17
• 10.68 million instances were processed across the end-to-end analysis pipeline.Source: B Inference Scale, Token Consumption,… · p. 17
• The pipeline used 7.46 billion prompt tokens and 1.91 billion generated tokens.Source: B Inference Scale, Token Consumption,… · p. 17

• The listed sources include AI-focused communities such as r/ai_agents, r/AIethics, and r/ArtificialInteligence.Source: C Seed subreddits · p. 17
• Technology, cybersecurity, and privacy forums include r/technology, r/cybersecurity, r/DigitalPrivacy, and r/privacyguides.Source: C Seed subreddits · p. 17
• Advice and support sources include r/advice, r/anxiety, r/offmychest, and r/therapyGPT.Source: C Seed subreddits · p. 17

D Inter-Annotator Agreement

• Observed agreement and Gwet’s AC1 show consistently strong annotator reliability across pipeline stages, while Cohen’s κ falls to 0.000 in some subsets.High positive prevalence and a dominant class without annotator variation deflate κ.Source: D Inter-Annotator Agreement · p. 18

D.1 Comparison with MIT AI Domain Risk Taxonomy

• Table 4 maps the MIT AI Risk Taxonomy to the paper’s taxonomy.Source: Table 4 · p. 19
• Table 5 reports consensus-resolved labels of MIT taxonomy subnodes as covered, partially covered, or fully covered by the Reddit-derived taxonomy.Source: Table 5 · p. 19

• AI System Safety, Failure & Limitations covers technical failures in which models produce confident falsehoods, incorrect code, faulty calculations, or distorted information presented as fact.The category concerns informational unreliability rather than malicious deception.Source: E Taxonomy of AI Risks and Harms… · p. 20
• Calculation and quantitative-processing errors include incorrect financial, statistical, or scientific computations, while code-generation errors include insecure, broken, or buggy code.Source: E Taxonomy of AI Risks and Harms… · p. 20
• Factual hallucinations involve fabricated facts or sources, whereas instruction-following failures involve ignoring explicit user constraints or formatting rules.Source: E Taxonomy of AI Risks and Harms… · p. 20

• Algorithmic discrimination includes demographic disparities and biased hiring or recruitment decisions that can exclude protected groups.Source: Algorithmic Bias, Discrimination, and… · p. 20
• Autonomous-agent risks include runaway actions and tool-execution errors, while cybersecurity risks include malware generation, credential leaks, prompt injection, and supply-chain vulnerabilities.Source: Algorithmic Bias, Discrimination, and… · pp. 20–21
• Financial harms include AI-enabled fraud, predatory lending, unexpected charges, and broader livelihood losses from workforce automation.Source: Algorithmic Bias, Discrimination, and… · p. 21
• Intellectual-property harms include unauthorized creative reproduction, ingestion of copyrighted code, and plagiarism or attribution failures.Source: Algorithmic Bias, Discrimination, and… · p. 21
• Misleading-media harms include deepfakes, political disinformation, and misleading vendor claims; platform-governance harms include moderation errors and limited recourse.Source: Algorithmic Bias, Discrimination, and… · pp. 21–22
• Privacy harms cover unauthorized collection and exposure, including leakage of commercial code, PII, and data gathered through surveillance or tracking.Source: Algorithmic Bias, Discrimination, and… · p. 22
• Psychological harms include dependency, anxiety, grief after companion discontinuation or persona drift, and severe crises or self-harm facilitation.Source: Algorithmic Bias, Discrimination, and… · pp. 22–23
• Technical failures include context loss, outages, bugs, and update breakage, while user-frustration categories cover abandonment, minor annoyances, paywall friction, and workflow disruption.Source: Algorithmic Bias, Discrimination, and… · p. 23

• Posts describe algorithmic hiring rejections, perceived unfair gig pay or timing penalties, and workplace monitoring that one employee says prompted them to quit.Source: G Example Posts · pp. 27–28
• Posts describe AI-edited images used in revenge-porn campaigns and users' concerns about their personal and relationship effects.Source: G Example Posts · p. 27
• Privacy examples range from fear that deleted chatbot conversations could leak to concerns about AI scribing medical visits and sharing card details with shopping agents.Source: G Example Posts · p. 27
• Users report commercial data exposure through a coding agent revealing another project's information and employees pasting customer records into a chatbot.Source: G Example Posts · p. 27
• Users describe model-training data use as nonconsensual, report opt-out barriers and reset preferences, and seek filters to block ingestion of personal materials.Source: G Example Posts · p. 28
• Smart-home users describe AI features that infer emotions, recall past comments, or learn habits, raising concerns about invasive ambient monitoring.Source: G Example Posts · p. 28