PaperlayerView exact revision on arXiv

Source-linked AI summary

AI Companions Reduce Loneliness

Julian De Freitas, Ahmet K Uguralp, Zeliha O Uguralp, Puntoni Stefano

arXiv:2407.19096v1cs.CYcs.AI

TL;DR

Behavioral research offers limited insight into whether AI companions designed for relationships alleviate loneliness. This paper uses observational analyses and experiments to assess their effects, finding reductions at cross-sectional and longitudinal scales, including over a day and a week.

Abstract

from arXiv · show

Chatbots are now able to engage in sophisticated conversations with consumers in the domain of relationships, providing a potential coping solution to widescale societal loneliness. Behavioral research provides little insight into whether these applications are effective at alleviating loneliness. We address this question by focusing on AI companions applications designed to provide consumers with synthetic interaction partners. Studies 1 and 2 find suggestive evidence that consumers use AI companions to alleviate loneliness, by employing a novel methodology for fine tuning large language models to detect loneliness in conversations and reviews. Study 3 finds that AI companions successfully alleviate loneliness on par only with interacting with another person, and more than other activities such watching YouTube videos. Moreover, consumers underestimate the degree to which AI companions improve their loneliness. Study 4 uses a longitudinal design and finds that an AI companion consistently reduces loneliness over the course of a week. Study 5 provides evidence that both the chatbots' performance and, especially, whether it makes users feel heard, explain reductions in loneliness. Study 6 provides an additional robustness check for the loneliness alleviating benefits of AI companions.

CONCEPTUAL FRAMEWORK

This framework examines whether AI companions alleviate loneliness, how feeling heard and chatbot performance may help explain any effect, and whether benefits persist over time. It also considers whether consumers underestimate those potential benefits.

  • The research extends chatbot literature from customer-service assistants to AI companions designed for social relationships, testing their effects on loneliness and underlying mechanisms.Source: CONCEPTUAL FRAMEWORK · pp. 6–7

Can AI companions Help Cope with Loneliness?

  • Conversational competence and making users feel heard could help AI companions alleviate loneliness, while their low-cost access and scalability suggest potential for broad use.The passages describe companions as tracking context, responding promptly, and making users feel heard; many offer basic conversations for free and do not require human involvement.Source: Can AI companions Help Cope with… · p. 8
  • Because prior correlational findings cannot rule out selection effects, the authors describe their work as the first causal assessment of whether representative AI companions reduce loneliness.Source: Can AI companions Help Cope with… · p. 9
  • Loneliness improvements can persist after interacting with an AI companion over multiple days, addressing whether benefits diminish with continued use.Source: Can AI companions Help Cope with… · p. 9

How could AI relationships alleviate loneliness?

CONCEPTUAL FRAMEWORK

  • The framework compares an empathetic AI companion with a nonempathetic assistant and a task-limited chatbot, hypothesizing that feeling heard matters more than performance.Performance includes timely responses, credibility, context tracking, response variability, and domain knowledge; the rationale is that loneliness involves perceived deficits in social and emotional support.Source: How could AI relationships alleviate… · p. 11
  • The study addresses the gap in human-only loneliness research by causally testing AI companions’ effectiveness and whether feeling heard and performance mediate their effects.It also explores short-term and potentially persistent effects of AI interactions.Source: How could AI relationships alleviate… · pp. 11–12
  • The framework asks whether consumers mispredict AI companions’ loneliness benefits, potentially avoiding apps that could alleviate loneliness.It frames this possibility as an affective forecasting error: consumers might fail to anticipate the relief an AI companion could provide.Source: Mispredicting AI’s… · p. 12

OVERVIEW OF STUDIES

The overview moves from observational examinations of app conversations and reviews to experiments testing loneliness changes, persistence, mechanisms, and robustness.

OVERVIEW OF STUDIES

  • Studies 1 and 2 examine loneliness-related talk in commercial AI-companion conversations and reviews of popular companion apps.The studies use observational, real-world material.Source: OVERVIEW OF STUDIES · p. 13
  • Study 3 tests single-session loneliness changes against doing nothing, interacting with a person, and watching online videos, and includes a human-framed chatbot condition.The human framing holds chatbot technology constant while testing the effect of believing the interlocutor is human.Source: OVERVIEW OF STUDIES · p. 13
  • Study 4 tracks loneliness over seven days and compares chatbot interaction with a control condition.Source: OVERVIEW OF STUDIES · p. 13
  • Studies 3 and 4 measure participants’ predictions about AI companions’ effects on loneliness to assess whether people are calibrated to those benefits.Source: OVERVIEW OF STUDIES · pp. 13–14
  • Study 5 compares a full-fledged companion with a non-empathic AI assistant and a basic chatbot to investigate feeling heard and chatbot performance as mechanisms.Source: OVERVIEW OF STUDIES · p. 14
  • Study 6 checks robustness by replicating the key effect using only a post-treatment loneliness measure.Source: OVERVIEW OF STUDIES · p. 14

STUDY 1

Study 1 examines loneliness in real Cleverbot conversations by using a fine-tuned LLM to identify explicit expressions of loneliness and comparing engagement in loneliness-related versus unrelated exchanges.

Approach

  • Fine-tuning an LLM addressed the risk that dictionary methods would miss variation in how consumers express loneliness.The model was trained to detect loneliness in conversations.Source: STUDY 1 · p. 15

Model training and evaluation

  • The classifier achieved F1 score 0.92 on the test set and precision of 0.87 in manual validation.The precision estimate came from author coding of model-identified messages outside the training set.Source: STUDY 1 · p. 17

Prevalence of loneliness-related conversations

  • The model flagged loneliness messages in 5.6% of conversations; both raters agreed (α = 0.84) that 221 of 250 predicted message pairs (88%) contained loneliness, across 156 of 3,201 conversations (4.9%).The authors say these percentages almost certainly underestimate the true percentage of lonely users on the app.Source: STUDY 1 · p. 18

Engagement of loneliness-related conversations

  • Loneliness-related conversations lasted longer, involved more turns, and featured more words than conversations without loneliness.The comparison was based on Wilcoxon signed rank tests.Source: STUDY 1 · p. 19

STUDY 1

  • Study 1 compares mean and median engagement for loneliness-related versus unrelated conversations, reporting Wilcoxon Z statistics and Cohen’s d effect sizes.The note defines Z as the Wilcoxon test statistic and marks p < .001 with ***; the supplied passages do not show the results.Source: Table 1 · p. 19, STUDY 1 · p. 19

STUDY 2

Study 2 examines whether app reviews suggest that consumers use AI companion apps to alleviate loneliness, including whether loneliness-related ratings may reflect feeling heard and users’ beliefs about the apps’ benefits.

STUDY 2

  • Loneliness appeared in 19.5% of Replika reviews and 0.4% of ChatGPT reviews, despite similar mean ratings (4.11 vs. 4.10), illustrating substantial variation across apps.Source: STUDY 2 · p. 21
  • Across six apps, loneliness-mentioning reviews had higher mean ratings than non-loneliness reviews: Replika 4.73 vs. 3.96, Chai 4.32 vs. 3.56, iGirl 4.62 vs. 4.01, Simsimi 4.65 vs. 3.78, Cleverbot 4.08 vs. 2.98, and ChatGPT 4.83 vs. 4.10.Source: STUDY 2 · pp. 21–22
  • Across all six apps, loneliness-mentioned reviews had a higher percentage of positive valence than other reviews.This pattern is consistent with users describing loneliness positively, including reports that apps helped them feel less lonely.Source: STUDY 2 · p. 22
  • Among reviews classified as expressing loneliness, a higher proportion mentioned feeling heard than among other reviews.This held even for apps with fewer loneliness mentions overall.Source: STUDY 2 · pp. 23–24
  • The authors propose that positive ratings may reflect loneliness relief partly through users feeling heard, while benefits may also depend on consumers believing the apps can help.The proposed explanations are examined in Studies 3–5, with consumer beliefs considered in Studies 3–4.Source: STUDY 2 · p. 24

STUDY 3

Study 3 examines whether AI-companion interactions change state loneliness and whether consumers anticipate that change, comparing the interaction with other activities and describing participants’ everyday coping choices. It reports that AI companions reduced loneliness on par only with interacting with another person, and that participants underestimated this reduction.

STUDY 3

  • Participants rated expected feelings before and state loneliness before and after 15-minute assignments to an AI chatbot, a chatbot framed as human, a human, YouTube, or doing nothing.The study compared predicted with actual post-interaction loneliness to assess whether participants underestimated the effect.Source: STUDY 3 · pp. 25–26
  • In a presurvey, no participant spontaneously named an AI companion as a way to cope with loneliness, despite social media and YouTube being popular technological choices.The authors note this contrast between everyday users and existing AI-companion users.Source: STUDY 3 · p. 25
  • After 305 of 601 recruits were excluded following comprehension checks, analyses without those exclusions still showed significant results in the same direction.The authors note that using the state-loneliness question both before and after the interaction may have caused confusion.Source: STUDY 3 · pp. 25–26
  • The chatbot-framed-as-human condition used deception to vary beliefs about the interlocutor while controlling for conversation quality.Participants were told they were interacting with a person, though they were actually chatting with a chatbot.Source: STUDY 3 · pp. 26–27
  • YouTube did not significantly change state loneliness, whereas doing nothing increased it.Participants did not accurately predict the YouTube outcome, despite the absence of a significant change in baseline loneliness.Source: STUDY 3 · p. 32
  • State loneliness fell after AI-chatbot interaction (MPre = 33.51 vs. MPost = 26.75) and interacting with a person (MPre = 38.40 vs. MPost = 31.29).The authors characterize AI companion improvement as on par only with interacting with another person; Figure 3 presents these results.Source: Figure 3 · p. 32, STUDY 3 · p. 33
  • Participants underestimated how much AI companions would reduce loneliness; expected versus actual loneliness after AI-chatbot interaction was 43.56 versus 34.46.Expectations also exceeded actual loneliness after YouTube and a chatbot acting like a person, with the largest effect sizes in the AI-chatbot and humanlike-chatbot conditions.Source: STUDY 3 · pp. 31, 33

STUDY 4

Study 4 used a seven-day longitudinal design to examine whether daily AI companion interactions reduce loneliness and whether consumers underestimate their effects. Loneliness decreased after chatbot interactions, was lower than control on four of seven days, and participants underestimated the reduction.

STUDY 4

  • Participants in the experience condition interacted with the same chatbot for 15 minutes daily over seven days and reported loneliness before and after each session.The control group reported loneliness daily without interacting with the AI; a separate prediction group estimated loneliness before and after imagined sessions on day one.Source: STUDY 4 · pp. 33, 35, 36
  • The crucial pre–post loneliness difference in the chatbot condition was not affected by attrition; attrition differed significantly, X2(1, N=420 + 406) = 10.29, p = .001, likely because control required less time and effort than experience.Source: STUDY 4 · p. 35
  • The study’s chatbot used GPT-4 with memory of prior conversations, alongside a caring and friendly response prompt and a two-minute inactivity check-in.It periodically summarized user messages and incorporated saved information into later chats.Source: STUDY 4 · pp. 36–37
  • Loneliness fell from 36.64 before interaction to 30.74 after interaction when data were aggregated across all days.Participants reported a significant decrease after every daily session (ps < .001).Source: STUDY 4 · p. 38
  • Loneliness gradually declined over the week in the experience condition (b = -0.92, p < .001) and control condition (b = -1.42, p < .001), likely reflecting the longitudinal nature of the study.Source: STUDY 4 · p. 38
  • More chatbot messages were associated with greater loneliness reduction (b = 0.08, p = .046), but participant word count was not (b = 0.03, p = .331).Source: STUDY 4 · pp. 40–41
  • Post-chat loneliness was significantly lower than control on four of seven days (ps < .020).The condition effect was significant (b = -5.46, p = .015), with no significant condition-by-day interaction (b = 0.07, p = .455).Source: STUDY 4 · p. 39, Figure 4 · p. 41
  • Across all seven days, participants underestimated the chatbot’s loneliness reduction: predicted improvement was 4.37 versus an experienced 5.91.The difference was significant (t(3177.7) = -2.96, p = .003, d = -0.10).Source: STUDY 4 · p. 40, Figure 4 · p. 41
  • The findings were consistent after propensity score matching and when analyses included participants who did not complete all seven days.Source: STUDY 4 · pp. 41–42

STUDY 5

Study 5 compared an AI companion with a generalist AI assistant and a limited chatbot, testing whether loneliness reduction differed and whether feeling heard or perceived performance helped explain it. Loneliness reduction was greater with the AI companion, and both feeling heard and performance mediated the differences.

STUDY 5

  • The experiment compared the companion with a generalist assistant that avoided emotional responses and a limited chatbot restricted to basic tasks.The same underlying chatbot was prompted differently across conditions; the companion was called Jessie, while the other conditions used the name AI assistant.Source: STUDY 5 · pp. 42, 44
  • 8.73-point loneliness reduction with the AI companion exceeded the AI assistant’s 2.10 and control’s -0.51.The comparisons were significant versus both conditions.Source: STUDY 5 · p. 44, Figure 5 · p. 46
  • Participants rated the AI companion significantly higher for feeling heard than the AI assistant and control, with means of 70.63, 24.85, and 11.51, respectively (both comparisons p < .001).Source: STUDY 5 · p. 45
  • Feeling heard and perceived performance mediated the AI companion’s loneliness-reduction advantage over both comparison conditions, with feeling heard showing the larger role.The feeling-heard coefficients were -7.86 versus control and -6.08 versus the AI assistant; performance coefficients were -1.16 and -2.70, respectively.Source: STUDY 5 · pp. 45–46

FIGURE 6

Figure 6 is identified as the mediation diagram for Study 5, and its note defines the significance symbols.

FIGURE 6

  • Figure 6 is the mediation diagram for Study 5.Source: FIGURE 6 · p. 47
  • The note defines *** as p < .001, ** as p < .01, and ns as not significant.Source: FIGURE 6 · p. 47

STUDY 6

Study 6 measured loneliness only after participants interacted with the AI companion or control chatbot, addressing concern that pre-interaction questions could affect responses. Loneliness was lower in the AI companion condition than in control.

STUDY 6

  • Study 6 removed the pre-interaction loneliness questions, feeling-heard and performance measures, and AI-assistant condition.It compared only the AI companion and control conditions, with loneliness measured after the interaction.Source: STUDY 6 · pp. 47–48
  • The earlier pre-post measurement approach may reduce external validity by making loneliness salient before interaction, unlike naturalistic situations.Study 6 addressed this concern by asking about loneliness only after the interaction.Source: STUDY 6 · p. 47
  • Post-interaction loneliness was lower with the AI companion (25.62) than control (36.91; d = 0.40).The difference was significant, t(698.8) = 5.37, p < .001.Source: STUDY 6 · p. 48

GENERAL DISCUSSION

The discussion presents evidence that AI companions reduce loneliness at day-to-week scales, with feeling heard more influential than chatbot performance. It also bounds these findings to friendly, caring app designs and a US context.

GENERAL DISCUSSION

Related papers

Loading 2407.19096v1…