PaperlayerView exact revision on arXiv

Source-linked AI summary

Automated Comment Moderation Enhances Social Media Advertising Performance

Jiwoon Park, Julian De Freitas

arXiv:2609.16005v1cs.SIcs.CY

TL;DR

The paper addresses the open question of whether managing commentary surrounding social media advertisements improves advertising performance. Using field and lab evidence, it finds that automated comment management improves real advertising outcomes.

  • Problem

    Whether managing the commentary surrounding social media advertisements improves advertising performance remains an open question, with little empirical work testing this claim.

  • Method

    The paper uses converging field and lab evidence to examine automated comment management around advertisements.

  • Results

    Automated comment management improves real advertising outcomes, including website registrations and return on ad spend.

  • Takeaways & Limitations

    Managing the social context surrounding an advertisement is a targeted intervention for improving advertising performance.

  • Takeaways & Limitations

    The lab experiments relied on a fictitious hair dryer advertisement for female consumers, which may constrain generalizability across product categories.

Abstract

from arXiv · show

Social media advertising exposes brands not only to potential customers but also to unfiltered consumer discourse in the form of user comments. While comments can enhance authenticity and engagement, they also introduce reputational risks through spam, hate speech, and negative user-generated content. Despite the increasing prevalence of AI-powered comment moderation solutions, little causal evidence exists on whether moderation (i.e., hiding harmful comments) improves ad effectiveness. Across six empirical studies-including two large-scale field experiments and four online studies-we demonstrate that automated moderation of harmful comments causally improves ad performance, including conversion rates, return on ad spend, and purchase intentions. We also identify two important platform-governance boundary conditions: the gains from moderation depend on whether the platform is transparent about the brand's moderation behavior, and what types of comments are moderated. At the same time, the moderation effect persists when the brand is transparent about its own moderation practices. We advance theory on context effects in social media advertising, by uncovering the first targeted, preventative intervention for avoiding negative adjacencies. For managers, the results show that AI-assisted comment moderation impacts real ad performance but may be contingent upon platform-level transparency design.

Working Paper 27-011

Social media ads coexist with harmful user comments, yet prior research had not causally tested whether brand-controlled comment moderation improves real advertising performance. This paper addresses that gap using field and experimental evidence.

  • Motivation: Harmful comments create a controllable contextual environment beneath ads that may shape consumer evaluations and brand trust.Comments can include spam, misinformation, hate speech, and other harmful discourse, while contextual spillover research links toxic environments to less favorable brand attitudes.
  • Research Gap: Prior work had not measured how brand-initiated hiding of harmful ad comments affects advertising performance on real social media platforms.Existing studies emphasized attitudes or hypothetical incidents, while causal evidence from real campaigns remained limited.
  • Contribution: The paper provides the first causal evidence on whether automated management of harmful ad comments improves real social media advertising performance.The studies combine field evidence with laboratory evidence and examine both moderation and engagement.
  • Theoretical Focus: The authors focus on brand-controlled user-generated comments rather than external ad adjacencies, prior media exposure, or competing advertisements.This intervention is intended to be proactive and actionable, unlike contextual factors that brands cannot easily control or correct.
  • Outcome Focus: The paper examines advertising outcomes such as website registrations and return on ad spend rather than relying only on attitudes or purchase intentions.The authors identify financial advertising performance as the marketer-relevant outcome, while noting that prior work often used hypothetical incidents or descriptive data.

Automated Comment Moderation and Engagement

The paper distinguishes moderation, which hides harmful comments, from engagement, which responds to selected comments, and develops predictions about their effects and transparency conditions. It proposes that moderation may improve performance through brand trust, but that platform transparency and comment type can qualify the effect.

  • Concepts: Moderation hides harmful comments, whereas engagement responds to selected comments such as FAQs or purchase-intent comments.Harmful comments include universally harmful content and brand-directed attacks, while constructive complaints remain visible and engageable.
  • Intervention Scope: The intervention is targeted and less visible than disabling an entire comment section, because it hides specific harmful comments while retaining other discussion.The paper distinguishes this action from wholesale comment disabling and keeps constructive negative feedback visible.
  • Core Hypotheses: The paper predicts that automated comment management improves advertising performance chiefly through moderation rather than engagement.The proposed mechanism is that visible harmful comments signal poor oversight and reduce brand trust, whereas hiding them removes that negative context.
  • Brand Transparency: The positive effect of moderation is predicted to remain robust when the brand itself is transparent about the practice.This contrasts with the possibility that third-party platform disclosure may reduce trust by revealing hidden content or brand motives.
  • Platform Transparency: Platform transparency is predicted to reverse moderation’s positive effect when the hidden content targets the brand, but not when it is universally harmful.Consumers may infer self-interested manipulation when brands hide brand-directed hostility, whereas universally harmful content provides a less obvious self-interested motive.

Overview of Studies

The paper uses a two-step program combining field studies of automated comment management with online experiments testing its components, mechanisms, transparency conditions, and robustness. Across the program, the intervention is framed as a brand-controlled preventative response to harmful ad-comment context.

  • Online Studies: Three experiments and one follow-up replication test whether automated comment management affects purchase-related outcomes and clarify its mechanism and boundary conditions.Study 3 separates moderation from engagement, while Studies 4 and 5 examine platform and brand transparency.
  • Field Studies: Two field studies test the combined effect of automated comment moderation and engagement on advertising performance.Study 1 uses a before-after design on Trustpilot, while Study 2 uses a concurrent A/B test on Instagram and Facebook.
  • Theoretical Contribution: The research program targets user-generated comments as a brand-controllable contextual factor in social media advertising.It presents comment management as a preventative intervention for avoiding negative contextual effects rather than a response to external adjacencies.
  • Practical Contribution: The studies contribute to marketing practice by measuring an actionable intervention on real advertising performance and testing brand transparency as a practical condition.The contribution links comment management to outcomes on real social media platforms while examining whether transparency changes its effects.

Study 1: Large-Scale Comment Moderation and Engagement with Before vs. After Design

Study 1 evaluates a large-scale deployment of BrandBastion’s automated comment management using a before-after comparison in a financial-services advertising campaign. The intervention combined 24/7 hiding of harmful comments with brand responses to selected comments, and registration completion rate was the primary outcome.

  • Outcome Measure: Registration completion rate per link click was the primary ad-performance outcome, defined as whether consumers completed a registration form after clicking the ad.Impression-level metrics such as reach were unavailable, and the data were provided as aggregated period-level metrics.
  • Intervention: The intervention hid comments containing spam, scams, violence, inappropriate remarks, or personally identifiable information while leaving other comments visible.The system provided 24/7 moderation and engagement on the ad’s comment section.
  • Intervention: Brand engagement consisted of responses to pre-approved comment categories including FAQs and purchase-intent comments, while other comment types were manually engaged with in both conditions.This design combines automated moderation with responses on the brand’s behalf.
  • Campaign Scope: The campaign produced 62,033 completed registrations from 3,407,661 link clicks across a 10-month period.The dataset also contained 15,902 user-generated comments and cost $19,993.91 in total.

Results

Two field experiments found that automated comment management improved ad performance, with effects observed for registration completion and monetary return. The intervention reduced visible negative comments while increasing visible positive comments, although Study 1 cannot separate direct hiding effects from changes in subsequent commenting.

  • Study 1: Visible negative comments fell from 57.37% to 43.06%, while visible positive comments increased from 8.98% to 19.36% during treatment.Both sentiment shifts were statistically significant.
  • Study 1: Registration completion rate rose from 1.67% to 1.94% per link click under treatment, showing improved target ad performance.Treatment also produced 36,564 registrations versus 25,469 in control.
  • Study 1: Study 1’s automated comment management improved users’ likelihood of completing a web registration after clicking the ad link.The comparison controlled for other campaign aspects except time, because the design contrasted sequential periods.
  • Study 2: Study 2 used a concurrent A/B test on Facebook and Instagram to address Study 1’s temporal confound and test generalization across platforms.The study also targeted return on ad spend as a direct monetary outcome.

Methods

Study 2 compared otherwise matched concurrent campaigns with and without automated comment management on Instagram and Facebook. The treatment hid harmful comments and produced higher descriptive ROAS, while manipulation checks confirmed selective hiding of negative comments and comparable ad delivery across conditions.

  • Study 2: The concurrent A/B test held targeting, creative, setup, budget, and expenditure constant while varying only whether comment management was active.Control comments were unmanaged; treatment comments were moderated and automatically engaged according to predefined protocols.
  • Study 2: Similar impressions and reach across conditions suggest that comment management did not meaningfully alter algorithmic ad delivery.This supports the design’s focus on comment-management differences rather than materially different exposure.
  • Manipulation check: Negative comments were significantly more likely to be hidden in treatment than control, whereas positive-comment hiding rates did not differ significantly.The negative-comment hiding estimate was b = 5.44, with 95% CI = [2.24, 11.23].
  • Manipulation check: Treatment groups had fewer visible comments and slightly higher engagement per ad reach, confirming changes in the comment environment.Engagement was 74.7% in treatment versus 74.2% in control.
  • Study 2: Treatment campaigns generated higher ROAS than controls: 0.678 versus 0.459, with 22 versus 15 purchases and lower cost per purchase.The treatment and control groups had similar ad spend, making the comparison directly relevant to monetary return.
  • Overall result: Across two large-scale field experiments, automated comment management improved social media advertising performance and generalized beyond a single platform.Study 2 strengthened internal validity through a concurrent design in which automated comment management was the only condition difference.

Study 3: Does Moderation or Engagement Improve Ad Performance More?

Study 3 separated brand engagement from harmful-comment moderation in a controlled online experiment. Moderation increased purchase intentions, whereas engagement alone did not, and comparisons indicate that the effect reflected comment content rather than comment volume.

  • Method: The final sample included 398 participants after comprehension-check exclusions, and excluding nine non-female respondents did not change the results.The sample was 97.7% self-identified female, with mean age 40.5 years.
  • Method: The experiment used a 2 × 2 between-subjects design with a fictitious hair-dryer ad, positive comments, and harmful brand-attack comments.Participants indicated likelihood of clicking “Shop now” on a 100-point scale after viewing the sponsored post and comments.
  • Results: Brand engagement had no significant main effect, and moderation and engagement did not significantly interact.The engagement main effect was F(1, 394) = 1.56, p = .213; the interaction was F(1, 394) = 1.95, p = .164.
  • Results: Purchase intention was higher when harmful comments were moderated: M = 47.2 versus M = 36.9 without moderation.The moderation main effect was significant, F(1, 394) = 14.14, p < .001, η2 = 0.03.
  • Interpretation: These comparisons suggest that purchase-intention effects were driven by which comments were shown rather than by how many comments were shown.The stimuli varied the visibility of positive and harmful comments through separate engagement and moderation manipulations.

Discussion

Studies 3–5 show that hiding harmful comments improves ad-related outcomes, while platform transparency conditions this effect according to comment type. Brand transparency does not remove the positive moderation effect.

  • Study 3: Moderating harmful comments increased click intentions in a large replication, with means of 46.9 versus 34.1 when moderation was absent.The study found no main effect of brand engagement and no moderation-by-engagement interaction.
  • Discussion: The authors acknowledge that brand engagement might affect performance in actual social-media settings, although moderation produced the larger observed effect.They also identify possible benefits beyond ad performance, including social word of mouth on brands’ own pages.
  • Studies 4A and 4B: Studies 4A and 4B tested brand trust as a mechanism and platform transparency as a boundary condition across brand-directed and universally harmful comments.They also measured platform trust to assess whether transparency affects trust in the platform itself.

Results

Study 4A examined whether platform transparency changes the effects of hiding brand-directed harmful comments. Transparency reversed moderation’s purchase-intention benefit and operated primarily through brand trust.

  • Purchase intentions: Under opaque platforms, moderation increased purchase intentions from 40.6 to 52.6, but under transparent platforms it reduced them from 45.2 to 37.5.The moderation-by-transparency interaction was significant, F(1, 796) = 24.78, p < .001, η2 = 0.03.
  • Trust outcomes: Brand trust showed the same reversal: moderation raised trust under opacity but lowered it under transparency.Opaque means were 53.7 versus 44.9; transparent means were 41.1 versus 52.3.
  • Purchase intentions: Platform transparency significantly reversed the positive effect of moderating brand-directed harmful comments on purchase intentions.The authors describe this as a backfiring effect consistent with H4a.
  • Trust outcomes: Platform transparency also reduced trust in the platform itself, possibly because consumers penalized platforms for enabling impression management.The paper states that this inference did not translate into reduced purchase intentions.

Study 4B: Boundary On Platform Transparency of Universally Harmful Comments

Study 4B tested universally harmful comments, such as profanity or hostility, to determine whether platform transparency attenuates moderation’s benefits. Transparency flattened the benefit but did not reverse it.

  • Design: The study used the same general procedure as Study 4A but replaced brand attacks with harmful language unrelated to the brand or advertisement.The final sample included 1,173 female U.S. participants.
  • Purchase intentions: Across transparency conditions, moderation increased purchase intentions from 49.6 to 53.3 when comments were universally harmful.The main effect of moderation was significant, while transparency and the interaction were not.
  • Mediation: Both trust measures significantly mediated moderation’s effect on purchase intentions, but brand trust was the stronger pathway.The standardized coefficients were 0.66 for brand trust and 0.14 for platform trust.
  • Boundary condition: Across Studies 4A and 4B, transparency reversed moderation’s effect only for brand-directed comments and flattened it for universally harmful comments.The authors conclude that brands are still best off when platforms are not transparent.

Study 5: The Effect of Brand Transparency

Study 5 tested whether brands’ own transparency about hiding harmful comments changes moderation’s effect. The positive purchase-intention effect persisted when the brand disclosed its policy.

  • Design: The study used a 2 × 2 design crossing harmful-comment moderation with opaque versus transparent brand profiles.Transparency was conveyed through a community-policy section explaining whether the brand hid harmful language.
  • Implementation: The transparency manipulation reflected typical brand practice by pairing moderation disclosures with rationales about respectful or open conversations.The stimuli were modeled after Instagram’s interface and held positive-comment engagement constant.
  • Purchase intentions: Moderation increased purchase intentions from 45.8 to 57.9, with a significant main effect in Study 5.The positive effect persisted under brand transparency, where means were 56.4 with moderation and 48.9 without it.
  • Boundary condition: Brand transparency did not remove moderation’s positive effect on purchase intentions.The authors predicted this because brand disclosure may seem less manipulative than platform disclosure.
  • Brand trust: Moderation increased perceived brand trust, and brand transparency did not significantly interact with moderation.Trust means were 59.5 with moderation and 53.2 without it.

Mediation Analysis

Automated moderation improves advertising outcomes by hiding harmful comments, with brand trust mediating purchase intentions and transparency shaping when benefits persist or reverse.

  • Mediation Analysis: Brand trust mediates the positive effect of harmful-comment moderation on purchase intentions, with an indirect effect of 6.35 and 95% BootCI [4.07, 8.65].The mediation analysis used PROCESS model 4 with 10,000 bootstrapped samples.
  • Brand Transparency: Brand transparency about its own moderation practices does not undermine moderation’s benefits for purchase intentions or brand trust.A follow-up replication found the purchase-intention effect persisted with universally harmful comments, F(1, 1186) = 17.03, p < .001, η2 = 0.01.
  • Overall Findings: Automated comment management improves real advertising outcomes, including website registrations and return on ad spend, across field experiments and online studies.The research combines two large-scale field experiments with three preregistered online experiments.
  • Platform Transparency: Hiding harmful comments increases purchase intentions by improving perceived brand trust, but platform transparency can reverse this benefit when comments target the brand.This reversal does not occur for universally harmful comments.
  • Managerial Implications: The paper identifies harmful user comments as a managerially actionable social-media advertising context that can deteriorate brand trust and bottom-line performance.Comment moderation is presented as part of advertising performance management rather than solely reputational management.
  • Managerial Implications: A one-size-fits-all moderation strategy is not advisable because platform transparency and harmful-comment type jointly condition moderation’s advertising effects.Managers are advised to reconsider hiding brand attacks on platforms with transparency policies.
  • Limitation: The practical impact of moderation may be smaller in real settings if consumers discover hidden comments less often than in the studies.The authors note that platform transparency cues may be less salient in practice.

Limitations

The studies’ design choices constrain generalizability, and real-world transparency cues may be less salient than in the experimental settings.

  • Scope and generalizability: The lab experiments used a fictitious hair dryer advertisement targeting female participants, which may constrain generalizability despite no theoretical reason to expect product- or demographic-specific effects.The authors call for replication across product categories differing in involvement, risk, and symbolic meaning, and across more diverse consumers.
  • Transparency realism: Transparency manipulations explicitly disclosed comment moderation, whereas platform design and context may make moderation disclosures more or less salient in practice.Brands may disclose moderation tactics in profile community guidelines, while platforms control other disclosure features.
  • Transparency realism: On TikTok, users learn that comments are hidden only after scrolling through the entire comment section and must click to view them, illustrating how disclosure placement affects visibility.These platform-driven disclosures may be less apparent in practice than in the experiments.
  • Future research: Because real-world transparency cues may receive less attention, moderation may be less risky to brands in practice than the experimental settings suggest.The authors propose attention tracking and experiments varying disclosure prominence and placement to study how consumers process these cues.
Loading 2609.16005v1…