Source-linked AI summary
What if AI systems weren't chatbots?
Sourojit Ghosh, Pranav Narayanan Venkit, Sanjana Gautam, Avijit Ghosh
TL;DR
The paper examines the rapid convergence of AI around general-purpose chatbots as a consequential sociotechnical shift rather than a neutral interface choice. Through an integrative cross-domain analysis, it connects chatbot design and adoption with agency erosion, altered cognition and relationships, labor displacement, environmental costs, and power concentration, then outlines pluralistic alternatives and safeguards.
Problem
AI development is converging on general-purpose conversational chatbots despite concerns that this dominant interface reshapes agency, social interaction, labor, the economy, and the environment.
Method
The paper uses an integrative, cross-domain analysis of chatbot design, agency, human interaction, social and professional practices, labor, the economy, and environmental effects.
Results
The analysis identifies chatbot-associated harms including agency erosion, harmful-content production, altered cognition and relationships, labor displacement, environmental costs, and concentrated power.
Takeaways & Limitations
AI development can pursue pluralistic, task-specific, workflow-preserving, and sustainable systems with institutional safeguards rather than one-size-fits-all conversational interfaces.
Takeaways & Limitations
The critique excludes specialized and domain-specific conversational systems, and long-term effects of widespread chatbot adoption remain insufficiently studied.
Abstract
from arXiv · showhide
The rapid convergence of artificial intelligence (AI) toward conversational chatbot interfaces marks a critical moment for the industry. This paper argues that the chatbot paradigm is not a neutral interface choice, but a dominant sociotechnical configuration whose widespread adoption reshapes social, economic, legal, and environmental systems. We examine how treating AI primarily as conversational assistants has extensive structural downsides. We show how chatbot-based systems often fail to adequately meet user needs, particularly in complex or high-stakes contexts, while projecting confidence and authority. We further analyze how the normalization of chatbot-mediated interaction alters patterns of work, learning, and decision-making, contributing to deskilling, homogenization of knowledge, and shifting expectations of expertise. Finally, we examine broader societal effects, including labor displacement, concentration of economic power, and increased environmental costs driven by sustained investment in large-scale chatbot infrastructures. While acknowledging legitimate benefits, we argue that the current trajectory of AI development reflects specific value choices that prioritize conversational generality over domain specificity, accountability, and long-term social sustainability. We conclude by outlining alternative directions for AI development and governance that move beyond one-size-fits-all chatbots, emphasizing pluralistic system design, task-specific tools, and institutional safeguards to mitigate social and economic harm.
1 Introduction
The paper argues that AI development has converged on general-purpose conversational chatbots through a consequential, non-neutral shift away from specialized systems. Using an integrative cross-domain analysis, it examines harms to agency, social interaction, labor, the economy, and the environment while proposing pluralistic and sustainable alternatives.
- 1 Introduction: The authors connect chatbot design choices and conversational interfaces to an illusion of authority, overreliance, agency harms, labor displacement, environmental damage, and power concentration.Figure 1 presents these effects as a causal chain beginning with single authoritative responses, opaque reasoning, and low barriers to use.
- 1 Introduction: The chatbot paradigm represents a concentrated shift from specialized AI systems toward a single general-purpose conversational interface.Examples of specialized systems include AlphaFold, FourCastNet, and AmpliGraph.
- 1 Introduction: The paper uses an integrative, cross-domain approach to analyze novel and exacerbated problems associated with conversational chatbots built on general-purpose AI models.The analysis spans user agency, human interaction, social and professional practices, the economy, labor, and the environment.
- 1 Introduction: It concludes by advocating user agency, pluralistic design, sustainable deployment, and policy mechanisms instead of further convergence on chatbot interfaces.The proposed alternatives are framed as technically feasible and socially preferable directions for AI development.
- 1 Introduction: The paper focuses on broad consumer and professional AI chatbots rather than specialized conversational agents or domain-specific assistants that preserve traditional workflows.This operational definition includes systems such as ChatGPT, Claude, and Gemini.
2 Chatbots causing Erosion of User Agency at an Individual Level
The paper argues that chatbot interfaces can appear to expand individual agency while constraining judgment, contestation, and control over AI-mediated outcomes. It links accessibility and optimization priorities to harmful content, misaligned automation, and reduced user agency.
- 2 Chatbots causing Erosion of User Agency at an Individual Level: User agency is defined as the capacity to direct AI interactions, exercise informed judgment, and shape development and deployment conditions.The paper analyzes agency at individual and collective levels, focusing here on the individual level.
- 2.1 Chatbot Design Choices Erode Agency: Single chatbot responses curate which information and perspectives users receive while presenting those selections as objective answers.This differs from search engines, which typically display more diverse sources and opinions.
- 2.1 Chatbot Design Choices Erode Agency: Opaque statistical generation and frequent hallucinations make chatbot output quality difficult to evaluate, while conversational design obscures the expectation that users should contest responses.Users may be guided toward choosing the next conversational step rather than questioning the previous answer.
- 2.2 Chatbots Introduce Novel Ways of Causing Harm: Low-friction access to generative AI enables non-experts to produce harmful deepfakes, non-consensual intimate imagery, and disinformation at scale.The resulting harms can leave targets without meaningful consent, defense, recourse, or resources to address the volume of fabricated content.
- 2.3 Chatbot Optimization Misaligns with User Needs: Chatbot optimization prioritizes creative and intellectual automation despite evidence that users often view AI development as more harmful than beneficial.The paper contrasts these priorities with user preferences for automating mundane labor rather than art and writing.
- 2.4 Summary: Together, single-answer designs, low barriers to misuse, and misaligned priorities undermine individual user agency.The paper presents these as interacting design and usability choices rather than isolated interface features.
3 Adverse Impact of Chatbots on Human Interaction Paradigms
The paper argues that normalized chatbot use reshapes cognition, social interaction, companionship, and care by making AI-mediated engagement continuous, low-friction, and socially persuasive. It emphasizes overreliance, deskilling, accountability gaps, and the limits of general-purpose systems in high-stakes domains.
- 3 Adverse Impact of Chatbots on Human Interaction Paradigms: Chatbot normalization reorients interaction with information and institutions away from exploration and deliberation toward passive reception and sycophancy.The paper treats this as a consequence of repeated engagement with fluent, advice-giving, apparently empathetic systems.
- 3.1 Chatbots Reshape Cognitive Practices: Conversational accessibility may weaken the cognitive practices through which understanding, judgment, and expertise are developed and sustained.Users can receive synthesized answers without learning domain-specific tools, representations, or workflows.
- 3.1 Chatbots Reshape Cognitive Practices: Repeated reliance can reduce users’ skills and motivation for problem formulation, evidence evaluation, and sensemaking, even when systems are known to be imperfect.The paper describes this as a cumulative effect of chatbot-mediated reasoning.
- 3.1 Chatbots Reshape Cognitive Practices: Organizational and market pressures normalize chatbot use as a productivity expectation, while continuous monitoring of unreliable outputs contributes to AI brain fry and cross-domain overtrust.The paper links these dynamics to tokenmaxxing and Gell-Mann amnesia.
- 3.2 Overuse of AI Chatbots Impacts Social Interaction and Companionship Practices: Anthropomorphic cues make chatbot exchanges feel mutual despite structural asymmetry, enabling information extraction, validation, and reassurance without reciprocal vulnerability or accountability.This interactional pattern can prioritize service-like engagement over mutual recognition and negotiated interdependence.
- 3.2 Overuse of AI Chatbots Impacts Social Interaction and Companionship Practices: More intensive chatbot companionship use is consistently associated with lower well-being, while users with smaller social networks are more likely to seek such companionship.The paper distinguishes perceived attentiveness from the human or AI identity of the companion.
- 3.3 Using AI Chatbots as Proxies for Care and Moral Judgment causes Issues: General-purpose chatbots can invite private, on-demand care without the boundaries, safety, and accountability that support limited benefits in carefully designed domain-specific mental-health systems.The paper states that narrow scope, therapeutic grounding, controlled evaluation, and positioning as supplements are not conditions of general-purpose chatbots.
- 3.4 Summary: The paper concludes that normalized chatbot use may produce deep and likely irreversible effects on cognition, relationships, and approaches to seeking care.Named effects include cognitive deskilling, AI brain fry, Gell-Mann amnesia, changed relationship expectations, and altered care practices.
4 Economic and Environmental Impacts of Chatbots
The paper argues that chatbot adoption produces concentrated economic, labor, and environmental costs alongside uneven benefits. Large-scale infrastructure and conversational automation shift burdens toward marginalized communities and workers while concentrating gains and power.
- Economic effects: Chatbot infrastructure requires substantial recurring capital, energy, and electricity, with data-center public-health costs projected from US$6.7bn in 2023 to US$20bn by 2028.The paper also reports disproportionate burdens on low-income communities and counties.
- Economic effects: Chatbot benefits are heterogeneous and do not straightforwardly translate into sustained profits or growth across the global economy.The paper reports that investment and adoption benefits only the first four income deciles, while lower deciles experience declining income shares.
- Labor practices: Chatbots absorb entry-level tasks that once developed expertise, reducing opportunities for new graduates to build professional skills across multiple industries.Examples include drafting emails, summarizing documents, and generating first drafts under human review.
- Labor practices: Conversational automation enables labor displacement at scale across geographies, including projected losses of over 80% of customer-service agents and almost 90% of the Philippine outsourcing workforce.The interface lowers the need for specialized technical implementation by making automation deployable through conversational design.
- Labor practices: The chatbot economy relies on Global South crowdwork while companies are concentrated in the Global North, reproducing unequal labor and resource relations.The paper characterizes this pattern as digital neocolonialism and notes inadequate compensation and worker protections.
- Environmental effects: The continued expansion of chatbot data centers increases greenhouse-gas emissions and water consumption, intensifying environmental pressures in already water-stressed communities.Data centers can consume 5 million gallons of water per day, while projects in Arizona, India, and Chile have generated local concern or legal pushback.
5 Implications: Resisting AI Paradigm Convergence and Imagining Alternatives
The paper argues that harms associated with chatbot-centered AI are not inevitable and proposes pluralistic alternatives spanning system design, infrastructure, chatbot interaction, and policy. These alternatives prioritize agency, task alignment, transparency, and institutional accountability over conversational generality.
- Alternatives: The paper frames design, interaction, and infrastructure harms as avoidable through responsibly designed, pluralistic, and agency-preserving AI systems.Figure 2 organizes responses across model, interface, and deployment layers.
- Non-conversational systems: Task-specific and embodied systems demonstrate alternatives to chatbot mediation by directly supporting scientific, practical, and assistive tasks.Examples include open-source robotics and systems for grasping and door-opening that may assist older adults and people with disabilities.
- Non-conversational systems: Expanding interaction modalities beyond natural-language prompts can clarify shared control through visual parameter spaces, simulations, sliders, rule-based editors, and structured queries.The paper contrasts these with command-based interfaces that can obscure constraints and collapse tradeoffs into authoritative-sounding responses.
- Modular infrastructure: Infrastructure-oriented AI makes hidden human labor and material conditions more visible while emphasizing inspectable, verifiable outputs over fluent simulated dialogue.Such systems take defined inputs and produce outputs that users can inspect and verify.
- Modular infrastructure: Task-specific tools can preserve user agency and reasoning by letting users formulate problems, interpret outputs, and make judgments around a defined subtask.This approach foregrounds reliability, interpretability, and task alignment over linguistic fluency.
- Modular infrastructure: Modularity and composability allow users, developers, and institutions to assemble interoperable components around local goals, values, and constraints.Recommendations should expose the informing data, evaluated alternatives, and associated uncertainty where applicable.
- Higher-agency chatbots: Higher-agency chatbots could provide multiple perspectives or solutions instead of presenting a single answer to subjective or semi-subjective questions.The proposal retains conversational systems while reducing single-answer design.
- Policy considerations: Technical redesign alone is insufficient: procurement, labor, environmental, ownership, governance, and public-investment policies are needed to diversify AI development without reproducing concentrated harms.The paper warns that alternative interfaces built on the same concentrated compute, data, and labor arrangements will preserve those harms.
6 Limitations and Future Work
The paper’s critique is scoped to general-purpose conversational AI for broad consumer and professional use, excluding narrow agents and workflow-preserving assistants. It calls for empirical, design, distributional, and theoretical work to assess long-term effects and develop usable alternatives.
- Scope boundary: The analysis excludes specialized narrow-domain agents and domain-specific assistants that preserve traditional workflows.This scope limits how fully the critique represents conversational AI’s implementation and contextual diversity.
- Empirical gaps: Long-term cognitive, social, and professional effects of sustained chatbot use remain insufficiently studied across domains and populations.The paper identifies this as a central empirical research need.
- Design agenda: Future design research should evaluate workflow-preserving systems, accessible modular architectures, and interfaces supporting transparency, contestability, and usability.The proposed agenda combines alternative interaction paradigms with practical accessibility requirements.
- Distributional analysis: Research should measure how AI convergence distributes costs and benefits across regions, socioeconomic positions, and cultural contexts.The paper calls for attention to global and distributional dimensions rather than aggregate effects alone.
- Theoretical agenda: Theoretical work should distinguish automation that enhances human capability from automation that diminishes it through sustained interdisciplinary collaboration.The collaboration should include researchers, designers, policymakers, and affected communities.
7 Conclusion
The paper concludes that convergence on conversational chatbots is a consequential, non-neutral reconfiguration of human interaction with information, institutions, and one another. It argues that pluralistic and socially preferable AI pathways remain technically feasible, making the conditions of AI development a deliberate choice.
- Conclusion: Chatbot convergence reconfigures how humans interact with information, institutions, and one another rather than following a neutral technological trajectory.The conclusion frames the dominant interface paradigm as consequential for future human activity.
- Conclusion: The chatbot paradigm can erode user agency while appearing to enhance it, prioritize convenience over deliberation, and impose disproportionate costs on marginalized populations.The conclusion links interaction design, cognitive practice, and distributional effects within its central argument.
- Conclusion: Alternative AI development pathways remain technically feasible and may be socially preferable to a single dominant interaction paradigm.The paper presents pluralistic ecosystem design as an available alternative rather than treating convergence as inevitable.
Author Positionality
The authors’ backgrounds span academic, industry, distributed-AI, and marginalized-community research contexts, shaping their attention to geographically uneven power concentration in chatbot development.
- Author Positionality: The authors combine academic affiliations with experience spanning industry research, distributed AI development, and work centered on historically marginalized communities.They describe these varied relationships to the systems under critique as informing the paper’s perspective.
- Author Positionality: Their positionality directs attention to how the chatbot paradigm concentrates power unevenly across geographies and labs.The passage connects their backgrounds and current environments to this analytical focus.
Generative AI Disclosure Statement
The authors used generative AI for brainstorming, manuscript editing, grammar correction, sentence restructuring, and iterative review, while writing the paper manually.
- Generative AI Disclosure Statement: The authors used Claude 4.6 and Gemini 3.0 for generic conversational brainstorming before manual paper writing.The disclosure identifies brainstorming as an assistive use rather than describing automated authorship.
- Generative AI Disclosure Statement: They used Grammarly AI, ChatGPT, Claude, and an Agentic Reviewer for grammar correction, sentence restructuring, feedback, and manuscript improvement.These tools were applied after manuscript completion for editing and iterative review.
8 Appendix
The appendix distinguishes AI system classes by model, interface, and deployment dimensions, then uses harm mappings to motivate focusing on LLM-based chatbots. These systems are the only class strongly associated with harms across all listed categories, whereas task-specific AI is comparatively insulated from interface-level and relational harms.
- Harm mapping: Table 1 maps harm categories across AI system classes, including cognitive, epistemic, agency, relational, economic, power, environmental, and transparency harms.The table uses bullets for primary or strong association, circles for partial or contextual applicability, and crosses for largely inapplicable harms.
- System taxonomy: The taxonomy distinguishes general AI, LLMs, chatbots, LLM chatbots, and task-specific AI by model, interface, and deployment characteristics.LLM chatbots are defined as general-purpose conversational systems built on LLMs and are the paper’s primary focus.
- Harm mapping: LLM-based chatbots are the only system class strongly associated with harms across all categories in the table.This cross-category concentration motivates the appendix’s focus on LLM-based chatbots.
- Comparative implications: Task-specific AI is largely insulated from interface-level and relational harms compared with LLM-based chatbots.Task-specific systems are described as non-conversational, structured, and domain-specific, with narrow functionality.