PaperlayerView exact revision on arXiv

Source-linked AI summary

Artificial intelligence in communication impacts language and social relationships

Jess Hohenstein, Dominic DiFranzo, Rene F. Kizilcec, Zhila Aghajari, Hannah Mieczkowski, Karen Levy, Mor Naaman, Jeff Hancock, Malte Jung

arXiv:2102.05756v1cs.HCcs.AI

TL;DR

AI increasingly shapes communication, but its effects on social relationships are not well understood. Using two randomized experiments, this paper finds that smart replies improve communication efficiency and partner perceptions while changing emotional language, and that suspected use is viewed negatively.

  • Problem

    Although AI increasingly shapes communication, research has largely focused on technical aspects and given limited attention to the social impacts of AI-generated messages.

    Source: Introduction · p. 1

  • Method

    Two randomized controlled experiments examined how displaying and using commercially deployed smart replies affects interaction and perceptions in real-time text-based communication.

    Source: Introduction · p. 1

  • Results

    Smart reply use increased communication efficiency, positive emotional language, and positive partner evaluations, while people suspected of using them were evaluated more negatively.

    Source: Discussion · pp. 3–4

  • Takeaways & Limitations

    Smart replies may improve efficiency and interpersonal perceptions, but these benefits are coupled with changes to emotional language and potential loss of personal expression.

    Source: Discussion · p. 5

    The conversations came from participants completing a contrived Mechanical Turk task, so future work should test whether the findings hold across other everyday messaging contexts.

    Source: Limitations · p. 8

Abstract

from arXiv · show

Artificial intelligence (AI) is now widely used to facilitate social interaction, but its impact on social relationships and communication is not well understood. We study the social consequences of one of the most pervasive AI applications: algorithmic response suggestions ("smart replies"). Two randomized experiments (n = 1036) provide evidence that a commercially-deployed AI changes how people interact with and perceive one another in pro-social and anti-social ways. We find that using algorithmic responses increases communication efficiency, use of positive emotional language, and positive evaluations by communication partners. However, consistent with common assumptions about the negative implications of AI, people are evaluated more negatively if they are suspected to be using algorithmic responses. Thus, even though AI can increase communication efficiency and improve interpersonal perceptions, it risks changing users' language production and continues to be viewed negatively.

Introduction

The introduction frames smart replies as a widely adopted form of AI in interpersonal communication whose social consequences remain unclear. The authors use randomized experiments to examine how displaying and using them affects communication and interpersonal perceptions.

  • Research has largely emphasized technical AI methods, leaving the social effects of integrating AI-generated messages into human communication underexamined.The introduction calls for research examining these systems within human populations.Source: Introduction · p. 1
  • Algorithmic responses made up 12% of Gmail messages, about 6.7 billion emails daily, while smart replies predict and offer suggested responses users can select.Systems draw on general and user-specific text corpora to generate suggested replies.Source: Introduction · p. 1
  • Prior studies found that smart replies can affect how people write and tend to express more positive emotions, but their social implications remain unclear.People also believe the presence of smart replies influences how they communicate.Source: Introduction · p. 1
  • Randomized experiments examine commercial smart replies in real-time text communication and find effects on communication efficiency, emotional tone, and interpersonal evaluations in both positive and negative ways.The experiments focus on how displaying and using AI-generated replies affects how people interact and perceive one another.Source: Introduction · pp. 1–2

Results

Instrumental-variable results show that actual smart-reply use improved communication efficiency, sentiment, and some interpersonal evaluations, while perceived use was associated with negative judgments. A second experiment found that positive and Google smart replies increased positive emotional content in conversations.

AI is Perceived Negatively but Improves Interpersonal Perceptions

  • Perceived partner use correlated with actual use (r=0.22) and predicted lower cooperation and affiliation ratings, even controlling for actual use.The negative association concerns perceived use; the IV estimates found that actual partner use improved cooperation ratings and affiliation.Source: AI is Perceived Negatively but… · pp. 2–3, Figure 2 · p. 3
  • Smart replies accounted for 14.3% of messages; greater self-use improved efficiency and sentiment, while greater partner-use improved sentiment, cooperation ratings, and affiliation.Efficiency increased only with self-use; partner-use did not improve communication efficiency.Source: AI is Perceived Negatively but… · pp. 2–3, Figure 1 · p. 2

AI Sentiment Affects Emotional Content in Human Conversations

  • Positive and Google smart replies caused more positive emotional content than negative or no smart replies; Google replies resembled positive, but not negative, replies.The Google-versus-positive comparison was not significant (p=0.61), while the Google-versus-negative comparison was significant (p=0.018).Source: AI Sentiment Affects Emotional Content… · p. 3

Discussion

Smart replies can make communication more efficient and positive, yet suspected AI use is judged negatively and actual AI use changes emotional language, raising risks for personal expression and social relationships.

Discussion

  • Actual smart-reply use increases communication efficiency and positive language and makes partners view users as more cooperative and affiliative, whereas suspected use lowers perceived cooperation and affiliation.The discussion distinguishes partners’ judgments of actual use from their negative perceptions of suspected use.Source: Discussion · pp. 3–4
  • The authors suggest negative judgments may reflect people’s greater trust in other people than computers, opaque AI mediation, and uncertainty about who authored a message.In current systems, senders may know their responses were modified or generated by AI while receivers do not.Source: Discussion · p. 4
  • AI-induced emotional language changes may affect relationships and risk homogenizing personal communication styles; AI already writes about 6.7 billion emails daily.The discussion notes that language shapes listeners’ characterizations of communicators and cautions that greater AI mediation could reduce personal expression.Source: Discussion · pp. 4–5
  • Because suggestions shift language in line with their linguistic qualities, developers could shape conversational outcomes by tuning reply wording, while considering the broader social consequences.The discussion frames this as both an opportunity to influence interactions and a risk as AI gains influence over social communication.Source: Discussion · pp. 4–5

Methods

The study procedures and materials received institutional review board approval, and the study was preregistered.

  • The study procedures and materials were approved by the Institutional Review Board (1610006732), and the study was preregistered on AsPredicted (40389).Source: Methods · p. 5

Web-Based AI-MC Platform

The researchers built a flexible web platform for studying how smart replies and their linguistic properties and presentation affect real-time interpersonal communication. It supported controlled reply conditions and recorded the replies participants saw.

  • The platform enabled online recruitment and real-time text conversations between participants with smart-reply support, while allowing researchers to control and record the replies shown.Figure 4 illustrates positive- and negative-sentiment reply sets; participants saw only one set during actual use.Source: Web-Based AI-MC Platform · pp. 5–6, Figure 4 · p. 5
  • The web application ran across major modern browsers and resized for desktops, tablets, and Android and iOS devices.It used Node.js and MongoDB on the backend and jQuery and Semantic UI on the client side.Source: Web-Based AI-MC Platform · p. 6
  • Four messenger implementations provided positive-sentiment, negative-sentiment, Google Reply API22, and no-smart-reply conditions.The platform resembled commercial messaging apps, with tappable suggestions alongside a standard text box.Source: Web-Based AI-MC Platform · p. 6
  • Positive and negative suggestions were randomly selected from prior work20 ratings, with three distinct utterances per instance and no immediate repetition; suggestions updated after each sent or received message.Examples included “I like it” and “I can’t agree more” in the positive condition, and “I don’t get it” and “No you are not” in the negative condition.Source: Web-Based AI-MC Platform · p. 6
  • Without smart replies, participants manually typed every message, providing the control condition.Source: Web-Based AI-MC Platform · p. 6
  • Because Google’s Reply model could not run on desktops, a stand-alone CPython library compiled on Linux generated suggestions when users sent or received messages.The model’s runtime preprocessing and postprocessing framework was built with C++ and compiled into an Android archive.Source: Web-Based AI-MC Platform · p. 6

Study 1

Study 1 paired Mechanical Turk participants in an anonymous discussion about unfairly rejected work and assessed their communication and perceptions of their partners. The analyses used condition assignment to instrument smart-reply use and examined message sentiment and post-conversation reports.

Participants & Procedure

  • 438 Mechanical Turk participants discussed with anonymous partners the top three changes to address unfairly rejected work.Participants completed the conversation through a web-based messaging platform and then reported perceptions of their partner and the interaction.Source: Participants & Procedure · p. 6

Analysis and Results

Statistics

  • Smart-reply use by participants and partners was instrumented by condition assignment in regression models without covariates.The study used instrumental-variable regression with cluster-robust CR2 standard errors.Source: Statistics · p. 7

Analysis and Results

  • Conversations with fewer than 10 total messages or fewer than 3 messages from either participant were excluded, as were incomplete surveys for self-report analyses.Individual messages were scored with VADER compound sentiment, ranging from -1 (most negative) to 1 (most positive).Source: Analysis and Results · p. 7

Study 2

In Study 2, LIWC analyses confirmed that positive and Google smart replies increased conversation affect relative to no smart replies, while negative replies reduced it. The robustness check supported the sentiment findings from VADER.

Study 2

  • Study 2 collected messaging conversations from 599 Mechanical Turk participants and compared smart-reply users told they were using AI with participants told they were using standard messaging.Participants were 37.2% female and ranged from 19 to 69 years old (M=35.6, SD=9.96); language measures were analyzed at the individual level using linear regression with cluster-robust standard errors and no covariates.Source: Study 2 · p. 7

Robustness Check for Sentiment Results

  • The robustness check used LIWC Affect to verify VADER sentiment findings and excluded conversations with fewer than 10 total messages or fewer than 3 messages from either participant.LIWC Affect and its Positive and Negative Emotion variables range from 0 to 100.Source: Robustness Check for Sentiment Results · p. 8
  • Positive and Google smart replies raised LIWC Affect versus control (t(124)=2.95, p<0.001, d=0.272), while negative replies lowered it (t(123)=-3.50, p<0.001, d=0.454).The effects of positive and Google smart replies on affect were statistically similar (t(150)=0.354, p=0.724). Figure 5 shows mean conversation affect by condition with cluster-robust standard-error bars.Source: Figure 5 · p. 8, Robustness Check for Sentiment Results · p. 8

Limitations

The findings are limited by the study’s contrived task and desktop setting, and by measuring perceptions in anonymous, one-time, real-time interactions rather than across varied relationships and communication contexts.

Limitations

  • The study used a contrived Mechanical Turk task, so findings may not generalize to other everyday messaging conversations.The authors call for testing the results in disparate contexts.Source: Limitations · p. 8
  • Because the web-based messenger was not robust enough for mobile use, the study examined desktop messaging and may not generalize to mobile contexts.The authors recommend examining interpersonal perceptions in mobile messaging with smart replies.Source: Limitations · p. 8
  • Perceptions were measured as momentary states in anonymous, one-time interactions, leaving their development over time and in closer relationships unknown.The authors also note that AI mediation may affect other communication contexts differently, including asynchronous communication such as email.Source: Limitations · pp. 8–9

Related papers

Loading 2102.05756v1…