Personalizing Content Moderation on Social Media: User Perspectives on Moderation Choices, Interface Design, and Labor

1 INTRODUCTION

• The Intel Bleep demo prompted conflicting reactions: some saw personal filtering as useful, while others objected to tuning abuse categories or shifting filtering responsibility to users.The introduction notes that users hold different opinions and preferences about personal moderation tools.Source: §1 · p. 2
• Personal content moderation lets users configure content-based choices for their own accounts, unlike account-based blocking, amid diverse cultural and community norms.The introduction notes that platforms and third-party tools have begun offering personal controls.Source: §1 · p. 2
• The study addresses scarce empirical evidence by asking how users choose settings, understand and control interfaces, and perceive configuration labor and stakeholder responsibility.The introduction identifies these as three research questions.Source: §1 · pp. 3–4
• Researchers interviewed 24 social media users using a simulated feed with word filters, toxicity toggles, intensity sliders, and proportion sliders, alongside industry examples.Participants adjusted controls, inspected feed changes, and discussed tool use, labor, and platform responsibility.Source: §1 · pp. 4–5
• Users described varied responses to offensive content, resisted restrictive filters over missed relevant posts, and called for clearer, context-aware controls and shared moderation responsibility.The findings also identify granularity and example content as design considerations.Source: §1 · p. 5
• The paper discusses context-aware tools, clearer interface meanings, exceptions for user groups or cultural contexts, iterative settings, and feedback-informed preferences as design directions.It also considers the roles of platforms, third-party developers, and lawmakers.Source: §1 · p. 5

2.1 Content Moderation

• Platform moderation combines automated filters, paid or volunteer moderators, and feed algorithms, while personal tools let users shape their own view through account- or content-based controls.The paper focuses on content-based tools and distinguishes them from account blocking or muting; it identifies this as a distinct category for studying general users’ preferences.Source: §2.1 · pp. 6–7

2.2 User Control and Transparency in Interactive Recommender Systems

• Interactive recommender systems offer a design analogue: user controls can support transparency and controllability, while examples may explain why a particular output appears.The paper examines whether examples can aid understanding and acceptance of personal moderation tools.Source: §2.2 · pp. 7–8
• Recommendation and moderation differ in purpose: moderation removes content likely to harm users, so users may want more precise controls than recommendation settings provide.The paper also notes that platforms often separate the policies and teams for these functions.Source: §2.2 · pp. 7–8

2.3 Responsibilities of Platforms and Lawmakers

• Personal moderation raises empirical questions about user labor, free expression, and which speech platforms should remove for everyone versus leave users to filter individually.The study uses participants’ examination of sample interfaces to elicit views on labor and stakeholder responsibilities.Source: §2.3 · p. 9

3 METHOD

• The study used a technology probe—a simulated social media feed with four personal moderation interfaces—in 24 semi-structured interviews.Source: §3 · p. 9

3.1 Simulating Personal Content Moderation Interfaces

• The four interfaces were a toxicity toggle, word filter, intensity slider, and proportion slider, chosen to explore differences in granularity, transparency, and control.Source: §3.1 · p. 11

3.1.1 Comment Curation.

• Researchers curated a 30-comment feed from toxicity-rated Twitter comments, selecting comprehensible, topically diverse examples across toxicity buckets.Source: §3.1.1 · p. 11, §3.1.2 · p. 11

3.1.2 Interface Implementation.

• Each interface altered the feed differently, and switching interfaces reset other settings to avoid interaction effects.Source: §3.1.2 · p. 12

3.2 Participant Recruitment

• Recruitment prioritized users familiar with or likely to benefit from personal moderation, oversampled marginalized groups, and sought demographic diversity.Source: §3.2 · p. 12

3.3 Data Collection

• Participants tried the interfaces while thinking aloud, reviewed existing tools, and discussed configuration labor and platform and policymaker responsibilities.Source: §3 · p. 9, §3.3 · pp. 13–14

3.4 Analysis

• The researchers used iterative qualitative coding and memo writing, discussed emerging concepts, and contacted some participants to clarify responses.Source: §3.4 · pp. 14–15

3.5 Positionality Statement

• The authors’ concern about online harm, including its disproportionate effects on marginalized people, coexists with concern about threats to free speech from overly restrictive moderation.Source: §3.5 · p. 15

• Section 4.1 examines participants’ varied responses to online harms and their concerns about missing valuable content.Source: §4 · p. 15
• Section 4.2 reports challenges participants identified while engaging with personal moderation tools and interview probes.Source: §4 · p. 15
• Sections 4.3 and 4.4 address participants’ views of configuration labor and how platforms and lawmakers could better serve them.Source: §4 · p. 15

4.1 Reacting to Online Harms and Concerns about Missing Out on Valuable Content

• Participants encountered harms across feeds, comments, direct messages, and profile pictures, and responded by moderating, ignoring, or quitting social media.Source: §4.1 · p. 15, §4.1.1 · p. 15

4.1.1 Responding to Online Harm.

• Some participants used platform tools to protect their mental health, but others ignored harmful content or left platforms after negative experiences.Source: §4.1.1 · p. 16

4.1.2 Concerns around Personally Moderated Content.

• Participants feared filters could create echo chambers or hide important posts, including warning signs of offline harm and content they wanted to respond to.Source: §4.1.2 · p. 16

4.1.3 Inclination to Remain Open-minded.

• Participants wanted to remain open-minded, and some distinguished hiding a post for themselves from platform removal that prevents everyone from seeing it.Source: §4.1.3 · p. 17

4.2.1 Defining the Terms and Criteria.

• Participants found categories such as toxicity and name-calling ambiguous, with meanings varying across people and communities.They wanted clearer criteria, including how systems classify content and how category definitions account for evolving concepts.Source: Ambiguous Definitions · pp. 17–18
• Participants struggled to understand what slider levels meant and how overlapping categories would affect which posts they saw.They wanted concrete explanations of level distinctions and how systems resolve competing filters.Source: Ambiguous Definitions · p. 18, Lack of Transparency about Relevant… · p. 18

4.2.2 Failure to Account for Environmental Context.

• Participants said moderation outcomes depend on context, which many tools could not incorporate or let users specify.Examples included cultural uses of words, reclaimed slurs, and language that could be harmful in one context but affirming in another.Source: §4.2.2 · p. 19, Accommodating Use of Reclaimed Slurs · p. 19

4.2.3 Granularity of Control.

• Participants valued sliders for finer control but said extra levels were useful only when their differences were understandable.Some wanted continuous sliders, while others found neighboring levels difficult to distinguish.Source: Slider-based Controls · pp. 19–20
• Word filters’ fixed word-by-word choices could demand extensive setup, miss variants, and expose users to uncomfortable terms.Participants also worried that bad actors could evade filters by changing spellings.Source: Word Filters · p. 20

4.2.4 Leveraging Examples to Instill Transparency and Control.

• Examples helped participants understand what settings would filter, but examples of offensive content could require seeing unwanted material.Some participants also wanted criteria instead of examples that might expose them to content they hoped to avoid.Source: Using Examples to Align Mental Models · pp. 20–21, Using Examples as a Form of Control · p. 21
• Participants wanted examples to serve as controls or feedback, and some tested settings by switching between controls and their feed.This iterative testing helped users form more accurate mental models but could be labor intensive.Source: Using Examples as Feedback for Control · p. 21

4.3 Labor Involved in Configuring Moderation Systems

• Participants viewed configuring moderation as a trade-off between greater feed-curation agency and the labor required.Some were frustrated when inappropriate comments remained visible despite effort spent changing settings.Source: §4.3 · p. 21
• Some participants considered configuration an added burden, with word filters requiring users to identify keywords and category filters demanding attention.One participant preferred category filters to word filters because they lacked time to assemble a list of words to block; another felt configurations might rarely need updating once set up.Source: §4.3 · p. 22
• Participants proposed reducing setup work with curated keyword groups, friends’ filter lists, and feed-change previews.Some also argued that users should be compensated because their configurations provide valuable training data for platforms’ algorithmic systems.Source: §4.3 · p. 22

4.4 Distribution of Responsibility

• Most participants said platforms should remove egregious content and regulate some harmful material by default, regardless of personal settings.They cited users who may be unable to configure tools and platforms’ knowledge of misinformation and hate-speech trends.Source: §4.4 · pp. 22–23
• Participants described platforms as having a responsibility to maintain safe content delivery, beyond offering users personal controls.Some argued that platform governance should reflect platforms’ wider social role and corporate incentives.Source: §4.4 · pp. 22–24
• Participants also wanted personal moderation tools because they offer users control over individualized preferences that site-wide policies may not capture.Some saw these preferences as culturally specific and noted that platform policies can reflect Western norms.Source: §4.4 · p. 23
• Participants supported limits on personal curation, including keeping socially beneficial information such as missing-child alerts and health information visible.The passage also reports a consensus that content violating local laws or calls to violence should be removed for everyone.Source: §4.4 · p. 23
• Many participants distrusted platform motives, and some turned to third-party tools developed by universities or nonprofits.Participants also considered lawmakers’ roles, while expressing concerns about bureaucracy and politically motivated censorship.Source: §4.4 · p. 23

5.1 Addressing Harms while Still Enabling Freedom of Speech

• Users differed in how they experienced offensive speech, supporting individualized moderation tools and user education about their utility.The paper connects these differences to prior research on varied responses to extremist content.Source: §5.1.1 · p. 24

5.1.1 Effectively Mitigating Potential Harms.

• The authors propose searchable, culture-aware categories and moderation controls that extend beyond news feeds.They also note that some users may want to tune search results rather than their news feeds.Source: §5.1.1 · p. 24
• Fear of missing relevant content can make users wary of over-moderating, so the authors discuss context-aware tools and labels as possible middle grounds.Examples include fact-checks, interstitial warnings, and content-sensitivity alerts.Source: §5.1.1 · p. 24
• Personal moderation changes what users see without restricting others’ speech, but users may need education about its limits.The authors suggest platforms may be more willing to leave borderline content up if users can set personal preferences.Source: §5.1.2 · p. 24

5.1.2 Impact on Freedom of Speech.

• Some participants treated others’ ability to reduce their view of content as contrary to free speech, suggesting optional tools may prompt resistance.Source: §5.1.2 · p. 24
• Perceptions of sliders may depend on defaults and framing: Intel’s labels invited requests for more objectionable content, while Twitch’s emphasized filtering it out.The discussion notes that default placement and the wording of levels may affect reactions.Source: §5.1.2 · p. 25

5.2.1 Clearer Definitions of Harm Categories.

• Platforms should define abstract harm categories with examples and explain how category boundaries are drawn, despite disagreements about what should count as toxic.The authors favor imperfect but transparent tools over opaque systems that avoid public controversy.Source: §5.2.1 · p. 25

5.2.2 Transparency around the Algorithms Behind Controls.

• Users need information about moderation algorithms’ inputs and decision factors, while more interpretable outputs should align with the definitions platforms provide.Examples, criteria, and explanations of how settings interact can help users assess what to trust; inaccurate folk theories can cause confusion and disappointment.Source: §5.2.2 · pp. 25–26
• More slider levels do not ensure greater control: users need a clear mental model, and controls should account for community and linguistic context.Participants sometimes saw no difference between slider levels and wanted context-specific options, such as retaining racist comments from a particular user or group.Source: §5.2.3 · p. 26

5.2.3 Providing Desired Granularity and Continuous Adjustment.

• Personal moderation should support ongoing adjustment, including just-in-time controls and feedback that helps users evaluate whether settings work.Users may discover moderation needs only after encountering unwanted content, and their goals can change over time.Source: §5.2.3 · pp. 26–27

5.3 Managing Labor and Distribution of Responsibility

• Offer settings at different levels of specificity: quick setup can reduce cognitive labor while preserving the option to fine-tune individual preferences.Pre-built keyword categories can be imported and then adjusted phrase by phrase; shared preferences and feed improvement metrics may also encourage engagement.Source: §5.3 · p. 27
• Platforms should provide sensible defaults and maintain baseline review, since some users may rely on defaults or lack the digital literacy to configure personal tools.Personal settings do not remove platforms’ responsibility to address blatantly inappropriate posts or online harms.Source: §5.3 · p. 27
• Third-party developers can contribute moderation tools, while policymakers can incentivize platform investment and support for those developers.Participants were likelier to use tools built by academics and reliable nonprofits.Source: §5.3 · p. 27

5.4 Limitations and Future Work

• Three simulated controls addressed toxicity, leaving user preferences for other potentially undesirable categories, such as spam, sexual content, and violence, unexamined.The authors call for research on which other content categories users want to adjust separately.Source: §5.4 · p. 27
• The toy feed showed only 30 comments, lacked personalized content streams, and used text-only comments rather than audio, image, or video.Longitudinal study on real platforms and research on non-text moderation could reveal additional insights.Source: §5.4 · p. 28
• The toy platform’s abstract terms, such as “toxicity,” could have influenced findings about ambiguous definitions.Platform-specific studies are needed to clarify how far the issue persists in deployed tools.Source: §5.4 · p. 28
• The 24-participant interview sample was too small to establish the relative popularity of the tested interfaces.The authors plan representative large-scale surveys to address that question.Source: §5.4 · p. 28

6 CONCLUSION

• Personal content moderation tools should offer context awareness, clear interface meanings, and justifications, while platforms maintain effective baseline moderation.The paper also calls for policymakers to compel investment in innovative personal moderation tools.Source: §6 · p. 28