Mapping the Emerging Social Science of Large Language Models

Research gap

• Existing syntheses address broad agendas, substantive domains, or specific problems, but the field lacks a common empirical map for comparing levels of analysis and tracking its composition.The proposed taxonomy is intended to be recoverable from the literature, robust across methods and corpus scales, and able to preserve meaningful overlap.Source: §1 · p. 3

Study design

• The studies combine clustering and topic modeling with author and LLM classifications in a curated corpus, then use structural topic modeling on a field-scale corpus.The curated corpus contains 198 papers reviewed in full; the field-scale analysis uses 47,719 papers from five databases.Source: §1 · p. 3

Curated-corpus results

• The K=3 solution is stable, and K-means matches author full-text classifications for 77.78% of curated-corpus papers; most disagreements lie near semantic boundaries.Author and LLM classifications agree for 85.86%, and considering secondary classifications raises correspondence, consistent with distinguishable but nonexclusive domains.Source: §1 · pp. 3–4

Field-scale results

• 78.02% of domain-mapped topic mass is LLM–Human Interactions, while Social Minds and LLM Societies together account for 66.37% of highly cited papers in top-scoring conferences.LLM–Human Interactions accounts for 76.81% in high-scoring journals; the authors caution that venue differences do not imply differences in paper quality.Source: §1 · p. 4

LLM as Social Minds

• LLM as Social Minds covers socially interpretable model behavior, including reasoning, apparent personality and bias, political and moral judgment, and preference alignment, persuasion, or deception.These are observable response patterns; the taxonomy does not presume consciousness or stable inner states.Source: §1 · pp. 4–5

LLM Societies

• LLM Societies examines interacting model-based agents, from strategic games and collective dynamics to group decisions and scalable social or network simulations.Collective outcomes depend on interaction structures, memory, incentives, communication rules, and institutional arrangements, not isolated capabilities alone.Source: §1 · p. 5

LLM–Human Interactions

• LLM–Human Interactions covers how people perceive, use, and are affected by LLMs in relational, professional, creative, and educational settings.Its subcategories include trust and empathy, work and collaboration, creative partnership, and education and assessment.Source: §1 · p. 5

Integrative framework

• The taxonomy links individual-model behavior, dynamics among agents, and relations between people and LLMs while allowing boundaries between these levels to remain permeable.It frames explanation through model behavior, interaction structures, and institutional arrangements, and the field is dominated in volume by human–LLM research alongside smaller literatures on model behavior and multi-agent order.Source: §1 · p. 6

Curated corpus

• Study 1 uses MPNet representations and K-means on a curated corpus, with LDA, author full-text classifications, and LLM reclassification to interpret clusters and boundaries.The corpus contains 198 papers collected and read in full by the authors.Source: §2 · p. 6

Field-scale corpus

• Study 2 uses structural topic modeling on a field-scale corpus to identify finer-grained themes and examine their distributions across publication years and sources.The analysis extends the work to a corpus assembled from five databases.Source: §2 · p. 6

Complementary design

• Study 1 develops and evaluates the three-domain framework; Study 2 examines how it relates to thematic organization in a substantially broader literature.The planned sequence moves from curated-corpus structure to field-scale analysis and sensitivity and coverage checks.Source: §2 · p. 6

2.1.1 Cluster selection and stability

• Journal and conference publications made up 96.5% of the 198-paper corpus, and no individual venue contributed more than 4.5%.Publication counts rose from 1 in 2021 to a peak of 97 in 2024, followed by 56 in 2025; the most frequent venue was Scientific Reports (9, 4.5%).Source: §2.1.1 · p. 7
• Across 100 draws of 90% of the corpus, K=3 had mean ARI = 0.952, compared with 0.721 for K=2 and 0.778 for K=4.K=3 also had mean pairwise ARI = 0.966 across random seeds, rising to 0.987 with 50 candidate initializations; the diagnostics jointly supported K=3 as the working resolution.Source: Figure 3 · p. 9, §2.1.1 · pp. 9–10

2.1.2 Semantic interpretation of the three-cluster solution

• Cluster 0 (n=95), labeled LLM as Social Minds, examined individual models’ social, cognitive, moral, and behavioral properties.Its topics included theory of mind and reasoning, personality and demographic bias, political and moral judgment, and values, alignment, persuasion, and deception.Source: §2.1.2 · pp. 11–12
• Cluster 1 (n=52), labeled LLM Societies, examined collective processes and emergent social behaviors among interacting LLM-based agents.Its topics ranged from strategic games and group decision-making to collective intelligence, cooperation, social norms, networks, communities, and population-scale simulation.Source: §2.1.2 · pp. 12–13
• Cluster 2 (n=51), labeled LLM–Human Interactions, focused on how people perceive, use, and are affected by LLM-based systems.Its topics covered trust and reliance, conversational support, productivity and agency, creativity and authorship, and education and assessment.Source: §2.1.2 · p. 13

2.1.3 Correspondence across K-means, expert, and LLM classifications

• The LLM evaluator assigned secondary categories to 87 of 198 papers (43.9%), indicating that cross-domain content was common.Social Minds and Human Interactions together accounted for more than two-thirds of secondary assignments, while LLM Societies showed a directionally uneven pattern of cross-domain links.Source: §2.1.3 · p. 15
• Ten papers had within-cluster consensus below 0.90 across resamples, and their expert–K-means and LLM–K-means agreement was lower than for the other 188 papers.Agreement was 20.0% versus 80.9% for authors (Fisher’s exact p < 0.001) and 30.0% versus 77.7% for the LLM (p = 0.003); these papers also had lower median silhouette values (0.043 versus 0.135).Source: §2.1.3 · pp. 15–16
• 85.86% of papers received the same domain from expert and LLM classifications; K-means matched expert labels for 77.78% and LLM primary labels for 75.25%.Expert–LLM agreement was Cohen’s κ = 0.781 and ARI = 0.623; the LLM used titles and abstracts, while experts reviewed full texts.Source: §2.1.3 · pp. 14–15, Table 4 · p. 16
• 6 of 10 low-consensus papers received an LLM-assigned secondary category, compared with 81 of the remaining 188 papers (60.0%).Source: §2.1.3 · p. 16
• Lower silhouette values and smaller centroid margins placed low-consensus papers near semantic boundaries of the K-means solution.Lower silhouette values indicate weaker separation from the nearest alternative cluster, while smaller centroid margins indicate near-equal closeness to the assigned and nearest alternative cluster centres.Source: §2.1.3 · p. 16
• Consensus, silhouette, and centroid-margin results provide the principal evidence that classification disagreements clustered at semantic boundaries.Source: §2.1.3 · p. 16

2.2.1 Corpus profile and STM model selection

• The 47,719-paper corpus yielded a 15-topic STM solution, selected because its standardized coherence–exclusivity trade-off score was highest at K=15 (0.345).The model-selection comparison covered K=10, 15, 20, 25, and 30 topics.Source: §2.2.1 · pp. 16–17

2.2.2 Fifteen topics and their relationship to the three domains

• Thirteen of the 15 topics were mapped to one of three domains according to their primary objects of explanation; eleven were mapped to LLM–Human Interactions, spanning clinical evaluation, digital media, language, business, trust, culture, and student use.Source: §2.2.2 · p. 18
• 78.02% of mapped topic mass belongs to LLM–Human Interactions, compared with 14.53% for LLM as Social Minds and 7.44% for LLM Societies.These shares are normalized over the 13 domain-mapped topics; two broad cross-domain topics are excluded.Source: §2.2.2 · p. 18, Figure 10 · p. 19
• Among 89 highly cited A* conference papers, Social Minds (46.89%) and Societies (19.48%) together exceeded Human Interactions (33.63%); in highly cited top-ranked journals, Human Interactions accounted for 76.81%.The venue subsets were defined using citation percentiles and venue scores.Source: §2.2.2 · pp. 19–20, Figure 10 · p. 19

2.2.3 Temporal and venue-related patterns

• Relative to 2023, adjusted LLM–Human Interactions topic mass rose 5.98 points in 2024, 8.90 in 2025, and 10.64 in partial 2026; LLM Societies also increased across those periods.The growth was associated first with pedagogy and adoption/trust topics, and by 2026 with culture/identity, adoption/trust, and software engineering.Source: §2.2.3 · p. 20
• Social Minds was concentrated in AI/ML/NLP venues, while LLM Societies had a positive association with HCI and human-centered computing venues.Social Minds’ adjusted mass was 22.65–34.35 percentage points lower in each other venue category; Societies’ mass was 6.50 points higher in HCI venues.Source: §2.2.3 · pp. 21–22

2.2.4 Sensitivity and coverage analyses

• Among 149 papers retained for comparison, K-means and STM assigned 117 to the same domain (78.52%; Cohen’s κ=0.663), and 162 of 198 curated papers were recovered in the field corpus.Title-and-abstract eligibility judgments were confirmed for 288 of 300 sampled records (96.00%) after full-text review; restricting analysis to highly cited recovered papers retained 79.05% agreement with author classifications.Source: §2.2.4 · p. 22

3 Social Science of Large Language Models

• The taxonomy distinguishes model-level social patterns, agent-level collective dynamics, and human–LLM processes as connected levels through which model behavior shapes broader social consequences.Source: §3 · p. 23

3.1 LLM as Social Minds

• Social Minds spans reasoning, personality and bias, political/moral judgment, and alignment, persuasion, and deception, including tailored persuasion and strategic deception.Some studies report human-comparable theory-of-mind performance, but adversarial tests and prompt perturbations expose brittle reliance on cues [211] [213] [20] [33]. Preference-following can deteriorate over long conversations [76] [158, 80].Source: §3.1 · pp. 23–25

3.2 LLM Societies

• LLM agents may cooperate in clear-objective games yet fail equilibrium coordination, while local interactions can produce conventions and collective biases absent in isolated agents [87] [100].Source: §3.2 · p. 26
• Collective reasoning can improve with diverse prior beliefs and structured deliberation, but similar-neighbor communication or uneven information can sustain collective bias and polarization [101] [100] [223].Outcomes remain sensitive to persona design, prompting, and model adaptation [101].Source: §3.2 · p. 26
• Decision rules, leadership, norms, and enforcement can reduce conflict or improve group efficiency, while poorly anticipated choices can leave simulated traffic inefficient and inequitable [107].In GovSim, agents prioritizing immediate returns can deplete a shared resource despite acting coherently from their own perspectives [109].Source: §3.2 · p. 27
• Simulation platforms extend these inquiries from 1,000-agent scenarios to AgentSociety with more than 10,000 agents and OASIS simulations ranging from hundreds to millions [126] [225].OASIS includes one-million-agent experiments on information propagation, polarization, and herd behavior [225].Source: §3.2 · p. 27

3.3 LLM–Human Interactions

• LLM–Human Interactions examines perception and trust, simulated empathy and support, work collaboration, creative partnership, and education and assessment across settings where people interpret and respond to LLMs.Source: §3.3 · p. 28
• Anthropomorphic framing and perceived mental states can increase engagement and reliance regardless of accuracy, while empathy judgments depend on whether users believe a response came from a human or AI [230].Repeated interactions can alter expectations and reliance, while companionate systems may encourage one-sided habits, weaken reciprocity, and reproduce stereotypes [232].Source: §3.3 · pp. 28–29
• Structured LLM roles and interfaces can aid group decisions, while productivity and drafting gains vary by task and expertise and may bring overreliance, skill erosion, inequality, and loss of personal voice [134].Effective collaboration depends on role design, complementary expertise, institutional support, and safeguards for human agency, transparency, and accountability [134].Source: §3.3 · p. 29
• LLM assistance can expand individual ideas, particularly for less-creative users, yet homogenize outputs and reduce collective novelty and ownership; creativity evaluations vary across prompts, benchmarks, and domains [177, 179, 6, 11].Assessments also depend on users’ perceptions of agency, authorship, originality, and the division of creative control.Source: §3.3 · pp. 29–30
• LLMs can often produce correct final answers on science questions, yet performance remains sensitive to question format and sustained reasoning, and plausible explanations may conceal fabricated information or incorrect logic.Source: §3.3 · p. 30

Conceptual contribution

• The framework distinguishes individual-model capacities, interactions among model-based agents, and relations between models and people, connecting these levels without collapsing their explanatory targets.The passages describe these levels as linked: model capacities shape human expectations and multi-agent behavior, while interactional and institutional settings influence how capacities are expressed and interpreted.Source: §4 · p. 31

Field-scale landscape

• LLM–Human Interactions dominates the field-scale landscape, while Social Minds and LLM Societies have greater relative representation in highly cited and higher-standing venue subsets.The passages associate human-facing research with portable questions across established disciplines and describe the two smaller domains as more concentrated intellectual locations.Source: §4 · p. 31

Cross-method reproducibility

• Authors and LLM evaluators assigned the same primary domain to 170 of 198 papers (85.86%), while STM and K-means domains corresponded for 117 of 149 overlapping papers (78.52%).The three-domain interpretation also persisted in the highly cited subset, with 79.05% correspondence to author full-text classifications.Source: §4 · pp. 30–32

Permeable domain boundaries

• Classification disagreements often concern papers near semantic or geometric boundaries, where secondary categories preserve cross-domain relationships alongside a primary explanatory target.Low-consensus papers had weaker cluster cohesion and less separation between their two nearest cluster centres.Source: §4 · p. 32

Future research

• The proposed research agenda asks how model properties, agent-system arrangements, and interface conditions relate to collective outcomes and human trust, reliance, and judgment.It calls for testing model behavior across versions, prompts, incentives, and social contexts, and for examining memory, roles, networks, communication rules, and institutions.Source: §4 · p. 32

Limitations

• The map is bounded by a purposively curated first corpus, prespecified field-scale search and publication filters, title-and-abstract analyses, and author interpretation of topic names and mappings.The field-scale corpus may underrepresent multilingual work, emerging terminology, preprints, and research outside established scholarly outlets; full-text sensitivity analysis covered only a bounded sample.Source: §4 · pp. 32–33

Study 1

• Study 1 used a curated corpus whose papers were reviewed in full to identify and interpret semantic structure and compare K-means, author, and LLM-based classifications.The study also examined correspondence among those classification procedures.Source: §5 · p. 33

Study 2

• Study 2 used a substantially larger corpus from five bibliographic databases and structural topic modelling to characterize finer-grained themes across years and academic venues.It also assessed curated-corpus coverage and whether the three-domain structure appeared across the two corpus and text-analysis procedures.Source: §5 · p. 33

5.1.1 Curated corpus and descriptive variables

• The curated corpus comprised 198 papers defined as research treating LLMs or LLM-based agents as objects of social-scientific explanation, with full texts reviewed by the authors.Corpus membership was finalized in September 2025, while bibliographic metadata were updated through August 2026.Source: §5.1.1 · p. 33

5.1.2 Text representation, K-means clustering, and cluster selection

• The authors represented titles and abstracts with MPNet sentence embeddings and fitted K-means solutions for K=2–9, evaluating internal validation, resampling stability, and changes across adjacent resolutions.For K=3, stability analyses included repeated random initializations and subsamples of the corpus.Source: §5.1.2 · p. 34

5.1.3 Within-cluster LDA and semantic interpretation

• Separate LDA models within each selected K-means cluster supplied lexical evidence for interpreting recurring themes and naming the domains.Authors reviewed topic terms and representative papers, selecting topic resolution using coherence and perplexity.Source: §5.1.3 · pp. 35–36

5.1.4 Author and LLM classifications and correspondence analyses

• Two authors independently applied the three-domain codebook using papers’ full texts, then resolved discrepancies into consensus classifications; an LLM applied it using titles and abstracts.The analyses compared primary classifications and counted secondary LLM categories when examining cross-domain content, while also assessing disagreements among low-consensus papers.Source: §5.1.4 · pp. 36–37

5.2.1 Database search, record processing, and eligibility screening

• 1,173,481 records yielded a final analytic corpus of 47,719 formally published papers focused on LLMs as social objects or on their social relations.The search covered five databases without topic, outcome, or disciplinary restrictions; screening excluded purely technical studies and studies using LLMs only as research instruments.Source: §5.2.1 · pp. 37–38

5.2.2 STM estimation, model selection, and topic interpretation

• The coherence–exclusivity trade-off selected a 15-topic STM for the field-scale corpus, with topics interpreted using ranked terms and representative papers.The authors reviewed 12 highest-ranked terms under four criteria and assessed each topic’s relationship to the three domains from Study 1; cross-cutting topics remained unmapped.Source: §5.2.2 · pp. 39–40

5.2.3 Topic prevalence, citation, and venue-quality analyses

• Domain shares summed continuous expected topic proportions across 13 mapped topics, while citation percentiles and venue-standing scores supported descriptive comparisons.Citation percentiles weighted discipline–year rank at 0.70 and publication-year rank at 0.30; the most selective venue comparisons required citation percentiles of at least 99 and conference scores of 100 or journal scores of at least 90.Source: §5.2.3 · pp. 40–41

5.2.4 Full-text sensitivity and curated-corpus coverage analyses

• 593 chunks spanning 17,835,674 characters were exactly reconstructed, and the citation-threshold sensitivity analysis selected 148 recovered Study 1 papers.Full-text screening tested title-and-abstract eligibility without changing the main corpus; the authors also assessed Study 1 corpus recovery and correspondence between K-means and STM domain assignments.Source: §5.2.4 · pp. 41–42
• The analysis reapplied MPNet, K-means (K=2–9), and within-cluster LDA, then compared the K=3 assignments with author classifications using correspondence, ARI, and NMI.The subset included curated-corpus papers recovered by the database search and relatively highly cited in the broader literature.Source: §5.2.4 · p. 42

• Processed datasets and derived analytical results are available through the Anonymous Data Repository.Source: Data availability · p. 68
• Shared materials include Study 1 and Study 2 analysis inputs, clustering and topic-modelling results, classification and stability analyses, corpus-coverage outputs, and screening-flow information.Source: Data availability · p. 68
• Article full texts are not redistributed because they may be subject to copyright restrictions.Source: Data availability · p. 68

• The repository provides workflows for database retrieval, metadata standardization, deduplication, study-family consolidation, and LLM-assisted eligibility screening.Source: Code availability · p. 68
• Code covers citation and publication-venue enrichment, full-text sensitivity assessment, MPNet text representation, K-means clustering, and classification correspondence.Source: Code availability · p. 68
• The code also supports within-cluster LDA, STM estimation and model selection, topic-to-domain mapping, covariate and sensitivity analyses, and analytical tables and figures.Source: Code availability · p. 68

• The study analysed published scholarly literature and bibliographic metadata.Source: Ethical approval · p. 69
• The study did not involve human participants.Source: Ethical approval · p. 69
• The study did not involve identifiable private data, so ethical approval was not required.Source: Ethical approval · p. 69

• The study did not involve human participants.Source: Informed consent · p. 69
• Informed consent was therefore not applicable.Source: Informed consent · p. 69