Source-linked AI summary
Who Wrote this? How Smart Replies Impact Language and Agency in the Workplace
Kilian Wenker
TL;DR
Smart replies are designed to speed workplace communication, but their suggestions may shape the responses users write. The paper proposes a loss-of-agency theory and tests it with a crowdsourced experiment and interviews, finding response shifts associated with priming and time pressure.
Problem
Empirical studies of smart replies are scarce and have not directly examined potential effects through priming or time constraints.
Source: §1 · p. 2Method
The paper proposes a loss-of-agency theory and combines a crowdsourced experiment and interviews to investigate smart replies’ effects.
Source: §1 · p. 3Results
Smart-reply suggestions and time pressure were associated with shifts in written sentiment, which the study interprets as evidence of agency loss through response priming and time pressure.
Source: §4.2 · p. 15Takeaways & Limitations
Human and machine agency are not a zero-sum trade-off: their transfers can complement, replace, and reinforce one another.
Source: §4.2 · p. 15, §5 · p. 16External validity is most threatened by artificial writing conditions and possibly underspecified emails or low participant seriousness, although the scenarios came from common real-life situations.
Source: §4.2 · p. 15
Abstract
from arXiv · showhide
AI-mediated communication is designed to help us do our work more quickly and efficiently. But does it come at a cost? This study uses smart replies (SRs) to show how AI influences humans without any intent on the part of the developer - the very use of AI is sufficient. I propose a loss of agency theory as a viable approach for studying the impact of AI on human agency. This theory focusses on the transfer of agency that is forced by circumstances (such as time pressure), human weaknesses (such as complacency), and conceptual priming. Mixed methods involving a crowdsourced experiment test that theory. The quantitative results reveal that machine agency affects the content we author and the behavior we generate. But it is a non-zero-sum game. The transfers between human and machine agency are fluid; they complement, replace, and reinforce each other at the same time.
1. Introduction
Smart replies save time, but their suggestions may shape workplace responses when users cannot deliberate at length. The paper frames this as loss of agency and investigates priming and time constraints.
1. Introduction
- Smart replies offer quick responses to incoming messages, but their time-saving benefits raise questions about possible costs in workplace communication.They are short AI-suggested responses designed to save time and minimize interruptions.Source: §1 · p. 2
- Agency includes the capacity to decide and act, which circumstances such as time pressure, automation reliance, or complacency can constrain.When users cannot think at length about a response, their ability to retain agency is limited.Source: §1 · p. 2
- Priming occurs when prior smart-reply suggestions determine a response that the user would otherwise have given differently.Source: §1 · p. 2
- Empirical studies of smart replies are scarce and have not directly examined effects through priming or time constraints; the paper asks how replies are defined, affect language, and transfer agency.It also derives a definition of smart replies and discusses linguistic constraints from allowlists, blocklists, filters, canned responses, and biases as action stimuli.Source: §1 · p. 3
- The paper proposes a loss-of-agency theory and combines a crowdsourced experiment with interviews to study how smart replies can affect workplace agency.The proposed account focuses on response priming and time pressure.Source: §1 · p. 3
2. Background
The background defines smart replies as AI-generated reply suggestions in text-based communication and reviews how they may shape language and agency. It develops a loss-of-agency framework in which priming and situational constraints can shift decision-making power between people and AI.
2.1 Smart replies and the digital workplace
- Smart replies are a form of AI-mediated communication that offers short, complete, plausible reply suggestions in text-based communication, saving users time.They are a concept rather than an algorithm or underlying technology, and users must react to the suggestions.Source: §2.1 · pp. 4–5
2.2 Impact on language
- Prior work describes SR-related language biases, but whether biased suggestions in turn bias human behavior remains unclear.Potential influences include generic or simpler wording, sentiment, vocabulary, and emotional expression; linguistic alignment could lead users to adapt their own language to SR features.Source: §2.2 · p. 7
2.3 CMC theories and machine agency
- Machine-agency theory treats AI as able to decide and act in a technically pre-programmed way, while this study examines whether users confer agency on it without control.The review connects this question to CMC theories of communication constraints and affordances, including immediacy of feedback and procedural structuring.Source: §2.3 · pp. 7–8
2.4 Loss of agency theory and research hypotheses
- The proposed loss-of-agency theory concerns transfers of decision-making power that users do not control, under situational constraints, human weaknesses, or psychological priming.Priming is proposed as a mediator, while situational constraints, complacency, and automation bias are moderators; the hypotheses test sentiment, priming, and time pressure.Source: §2.4 · pp. 8–9
- The experiment could not test complacency or automation bias because the companies’ works councils did not approve questions related to personal work performance.Accordingly, the study forms no hypotheses about these constructs.Source: §2.4 · p. 9
3. Experiment
The experiment compared employee responses to workplace emails across human-written and AI-generated smart replies, including a timed AI condition. It used response sentiment as an indirect measure of agency-related decision skew.
3.1 Experiment design
- Employees responded to twelve workplace emails in randomly assigned human-SR, AI-SR, or timed AI-SR surveys; all groups could also compose their own replies.The emails came from an anonymized set supplied by a manufacturing-company sales director, while five software-company employees generated the human-written control suggestions.Source: §3.1 · p. 10
3.2 Measures
- Agency was inferred indirectly from response sentiment, analyzed with Google Cloud NLP; the time-constraint effect was gauged by comparing AI-SRs+TC with AI-SRs.The analysis used each full response rather than individual words.Source: §3.2 · pp. 10–11
3.3 Sample
- The sample comprised 346 German-writing employees from manufacturing and software companies in Germany; one participant was excluded from analysis for technical problems.Early and late responders showed no significant differences.Source: §3.3 · p. 11
4. Results and discussion
Suggested replies shifted the sentiment and uptake of workplace responses, with time pressure changing which suggestions people accepted. Participant accounts and the discussion show that these effects can coexist with agency-enhancing uses of SRs and are bounded by the study’s setting.
4.1.1. Quantitative results
- 0.58 for AI-SRs and 0.64 for AI-SRs+TC, versus 0.47 for H-SRs, were the average sentiment scores of given responses; the difference across all three groups was significant at p=0.01.The suggested AI-SRs scored 0.55, compared with 0.36 for H-SRs; Figure 4 depicts average sentiment scores for suggestions and responses.Source: §4.1.1 · p. 11, Figure 4 · p. 12
- When acceptance fell below an 80% threshold, it dropped faster in the time-pressure group; participants also rejected poor-fitting suggestions and typed their own responses.For email 3, the time-pressure group accepted the suggestion at a 100% rate; Figure 5 shows acceptance rates by email.Source: §4.1.1 · pp. 12–13, Figure 5 · p. 13
4.1.2 Participant experience
- Participants described time savings and quicker feedback, but also cited generic wording, limited usefulness, poor accuracy, and impersonal replies as reasons to reject SRs.Some participants said SRs helped with comprehension and composition, while others described complacency or relying on suggestions as a shortcut.Source: §4.1.2 · pp. 13–14
- Some participants deliberately copied suggestions into self-written replies, while one limited SR use to quick coordination with colleagues rather than customers or unfamiliar contacts.Others described using SRs to retain greater control over the wording.Source: §4.1.2 · p. 14
4.1.3 Qualitative interviews
- In interviews with eight selected participants, respondents rejected trading correctness for speed; experience, expertise, and situational awareness were described as safeguards against wrong decisions under pressure.Some workplace SRs were rejected because their wording seemed too informal.Source: §4.1.3 · p. 14
4.2. Discussion
- The discussion reports that more positive SR suggestions were associated with more positive written content, treating sentiment as a proxy for agency.The study also reports increased average positive sentiment under time pressure when suggestions were held constant.Source: §4.2 · p. 15
- The authors characterize human–machine agency as non-zero-sum: SRs may constrain users while also enhancing agency through procedural structure or immediate feedback.They describe the relationship as potential human–AI teaming rather than a simple transfer from human to machine.Source: §4.2 · p. 15
- The findings’ generalizability is uncertain because the sample consisted of employees in Germany and external validity to other populations could not be tested.The experiment also used artificial conditions, although the email situations came from real life and were described as common.Source: §4.2 · pp. 15–16
5. Conclusion
Smart replies can constrain language through algorithmic and training-data biases, while priming and time pressure shape human behavior. Machine and human agency interact fluidly, complementing, replacing, and reinforcing each other.
5. Conclusion
- Smart replies constrain and deform language through factors including word count, simplicity, politeness, gender, response selection, and biased training data.The study investigated whether these biases in smart replies lead to biased human behavior.Source: §5 · p. 16
- Response priming and time pressure are associated with biased human behavior, and time pressure makes participants pickier when smart-reply quality falls below a threshold.The study investigates whether biased smart replies lead to biased human behavior.Source: §5 · p. 16
- Machine agency affects authored content and the behavior people generate, while transfers between human and machine agency can complement, replace, and reinforce each other.The relationship is not a simple transfer from human to machine agency; it also includes cooperative elements.Source: §5 · p. 16