Source-linked AI summary
Humans Introduce, Models Elaborate: Asymmetric Narrative Agency in Human-LLM Co-Writing
Halfdan Nordahl Fundal, Yuri Bizzoni, Charlotte Gjørup Bilde, Ida Bække Johannesen, Rebekah Baglini
TL;DR
Prior work on Human–LLM co-writing has focused largely on final outputs, leaving interactional dynamics less examined. This paper compares turn-based HH, HA, and AA storytelling using measures of alignment and narrative influence. It finds that HA is distinct, with humans tending to introduce persistent novel material and LLMs tending to elaborate and stabilize context.
Problem
Prior work has emphasized output outcomes in Human–LLM co-creation, leaving open whether its turn-by-turn coordination resembles human-human, LLM-LLM, or a distinct regime.
Source: §1 · pp. 1–2Method
The study compares matched HH, HA, and AA turn-based storytelling and measures affective adaptation and directional narrative influence.
Source: §1 · p. 2, §4.4 · p. 5Results
Takeaways & Limitations
Human–LLM co-writing is a distinct interactional regime with asymmetric narrative agency and adaptive alignment, differing from both HH and AA co-writing.
Source: §6 · p. 9Generalizability is limited by mostly student participants from one university and a storytelling paradigm capped at ten exchanges.
Source: Limitations · p. 10
Abstract
from arXiv · showhide
Human-LLM co-writing is increasingly used for open-ended text generation, but much prior work focuses on final outputs rather than the interactional dynamics through which stories are produced. We study turn-based collaborative storytelling across three matched conditions: Human-Human (HH), Human-LLM (HA), and LLM-LLM (AA). Using a shared storytelling paradigm, we measure how agents align, introduce novel material, and influence narrative development through turn-level measures of valence adaptation, semantic novelty, transience, and resonance. Our results show that HA co-writing is not intermediate between HH and AA collaboration. Instead, it displays a distinctive asymmetry where humans tend to introduce more novel and persistent narrative material, while LLMs tend to elaborate and stabilize the existing context. These findings suggest that, in this setting, LLMs function less as human co-authors and more as adaptive narrative amplifiers that reshape how agency is distributed in collaborative writing.
1 Introduction
Prior work on Human–LLM co-creation has emphasized output outcomes, leaving open how narrative coordination unfolds turn by turn. The study compares matched HH, HA, and AA storytelling to examine whether HA is distinct and how narrative influence is distributed.
1 Introduction
- Prior work emphasizes output quality, creativity, productivity, and user experience, leaving open whether HA resembles HH, AA, or a distinct coordination regime.The paper frames turn-based storytelling as a controlled setting for studying how contributions respond to context and shape what follows.Source: §1 · p. 1
- The study compares HH, HA, and AA on a shared task structure using turn-level measures of affective alignment and semantic influence.Matched HH and AA conditions are added to an existing Human–LLM corpus.Source: §1 · p. 1, Figure 1 · p. 1
- The research questions ask how affective alignment and semantic distance vary by pairing, how narrative influence is distributed, and whether HA is intermediate or distinct.Source: §1 · p. 2
- Humans tend to contribute more novel and persistent material in HA co-writing, while LLMs tend to elaborate the evolving context and redistribute narrative agency.The introduction presents semantic influence as the clearest asymmetry in Human–LLM co-writing.Source: §1 · p. 2
2 Related Work
Related work examines creative outcomes, interactional alignment, sentiment modeling, and information propagation. The paper positions turn-level narrative influence in co-creative human–LLM stories as comparatively underexplored.
Human–LLM co-creation
- Human–LLM creativity research has assessed output quality, originality, productivity, ideation, and user experience, with mixed findings across benefits and drawbacks.Reported outcomes include more novel artifacts and enjoyment, but also narrower diversity, reduced self-perceived creativity, and possible underperformance versus human collaboration.Source: §2 · p. 2
Alignment and affective adaptation
- Dialogue research documents linguistic and emotional coordination, while lexical matching captures only one dimension and may miss emotional contagion.Prior approaches include polarity time-series matching and predefined emotion labels.Source: §2 · p. 2
Sentiment analysis in digital humanities
- Sentiment analysis supports modeling narrative affect, but dictionary and transformer methods remain challenging as general-purpose tools for literary text.Concept vector projection has been used to derive continuous sentiment scores and model alignment between human–LLM interlocutors.Source: §2 · p. 2
Information propagation in static discourse and dialogue
- Information-theoretic work distinguishes discourse novelty, transience, and resonance, but their operation in co-creative human–LLM narratives remains less explored.Prior work applied surprisal-based measures to speeches and dialogue, including information flow, backchannels, and local stability.Source: §2 · pp. 2–3
3 Datasets
The study uses turn-based storytelling in three dyadic conditions with a constant task structure. It filters incomplete or malformed data before analysis, with HA stories contributing nine full exchanges.
3 Datasets
- Turn-based storytelling across three dyadic conditions holds the task structure constant for comparing collaborative dynamics.The paradigm treats stories as sequences of local responses that can reveal adaptation to preceding contributions.Source: §3 · p. 3, §3.1 · p. 3
3.1 Shared task design
- All conditions use identical symmetric instructions for alternating turns and continuing a shared story for ten exchanges.The prompt is intended to elicit open-ended narrative continuation; the fixed format supports analysis of response, alignment, and influence.Source: §3.1 · p. 3
- HA includes 97 human participants, HH includes 72 participants, and AA pairs two agents instantiated as the same LLM under matched turn-taking constraints.The HA and AA conditions use four LLM families; HH participants were university or graduate students at Aarhus University.Source: §3.1 · p. 3
3.2 Preprocessing and filtering
- After incomplete sessions, empty turns, and malformed outputs were removed, HA stories had nine full exchanges and fewer exchange-level observations per story.The reported statistics describe the final analytic dataset.Source: §3.2 · p. 3
4 Methods
The methods operationalize narrative agency through turn-level affective alignment, semantic distance, and narrative influence, then model how these measures differ across co-writing conditions.
4 Methods
- Narrative agency is measured through affective adaptation, semantic distance between adjacent contributions, and whether contributions introduce material that shapes what follows.The framework also treats agency as tracking partners’ valence and semantic trajectories.Source: §4 · p. 4
4.1 Baseline valence difference
- Each story’s baseline valence asymmetry is the absolute difference between agents’ mean valence scores, with cross-condition analyses using unsigned asymmetry magnitudes.Turn valence is the projection of an embedding onto a pretrained sentiment concept vector; lower and higher values indicate negative and positive tones, respectively.Source: §4.1 · p. 4
4.2 Directional Alignment
- Directional alignment is estimated from within-story correlations between one agent’s turns and the other’s following responses; HH and AA use absolute asymmetry, while HA also receives signed agent contrasts.The HA interaction’s positive values indicate stronger LLM alignment to prior human valence than the reverse.Source: §4.2 · p. 4
4.3 Semantic distance
- Semantic distance is the cosine distance between adjacent agents’ turn embeddings and is modeled by condition with a story-level random intercept.The turn embeddings use Kingsoft-LLM/QZhou-Embedding rather than the sentence encoder used for valence projection.Source: §4.3 · pp. 4–5
4.4 Novelty, transience and resonance
- Novelty measures a turn’s excess surprisal against a beginning-of-sequence baseline, while transience measures its marginal effect on the immediately following counterpart turn’s predictability.Resonance is novelty minus transience, expressing the contribution’s net novelty gain in future discourse; transience analyses account for the responding partner’s identity.Source: §4.4 · p. 5
- Turn surprisal is the mean token surprisal conditioned on prior story context and within-turn tokens, scored with google/gemma-4-31B; proportional surprisal reductions scale metrics by prior baselines.The scorer is from a different model family than the four experimental LLMs.Source: §4.4 · pp. 5–6
- The analysis models absolute asymmetry by condition and tests whether novelty predicts resonance differently across conditions, with additional within-HA agent contrasts and agent-specific innovation-bias slopes.A novelty-to-resonance slope above 1 indicates innovation bias; the HA agent-by-novelty interaction estimates whether one agent shows stronger bias.Source: §4.4 · p. 6
5 Results
HA shows its clearest difference from HH and AA in narrative-role asymmetry: humans contribute more novel, persistent material while LLMs elaborate context. Affective alignment differences are less conclusive, while semantic distance and novelty–resonance coupling are higher in HA.
5.1 Affective alignment
- HA baseline valence asymmetry was 0.060, versus 0.041 for HH and 0.028 for AA (planned contrast Δ=0.025, p < .001).The result indicates a larger average affective gap between partners in HA stories.Source: Figure 2 · p. 6, §5.1 · p. 6
- Directional valence alignment was positive in all conditions, but cross-condition asymmetry tests were unreliable; within HA, LLM alignment was stronger (Δ=0.143, p=.008).The signed per-story diagnostic showed the same direction, while corresponding signed slot tests were not reliable for HH or AA.Source: §5.1 · p. 6
HA exchanges show high semantic distance
- Adjacent semantic distance was highest in HA (0.316), followed by AA (0.296) and HH (0.287); HA exceeded both in mixed-model comparisons (p < .0001).The planned HA-versus-average-of-same-type-baselines contrast was also significant (Δ=0.0248, p < .0001).Source: HA exchanges show high semantic distance · p. 6, Figure 3 · p. 7
Novelty couples with resonance in HA stories
- Within HA, human turns had more context-explained information than LLM turns (30.2% vs. 52.8%; mean Novelty −1.82 vs. −2.65) and higher mean resonance (−0.796 vs. −2.07).The reported sign convention means humans introduced more contextually surprising material that was more strongly taken up by the LLM than vice versa.Source: Novelty couples with resonance in HA… · p. 7
- More novel turns tended to be more resonant across conditions, with the strongest novelty–resonance slope in HA (1.217), versus HH (1.066) and AA (0.956).Both HH and AA slopes were flatter than the HA slope.Source: Figure 4 · p. 7, Novelty couples with resonance in HA… · p. 7
- With speaker type held constant, human turns had 12.8 percentage points more uptake with an LLM partner than a human partner, while LLM turns had 26.4 points less with a human than an LLM partner.When target-text baseline entropy was held constant, turns followed by human writers showed 12.0%–17.2% uptake, versus 30.0%–38.4% for turns followed by LLMs.Source: Novelty couples with resonance in HA… · p. 7, Table 3 · p. 8
5.3 Narrative influence
- HA slot asymmetry exceeded HH and AA for novelty (0.828), transience (0.484), and resonance (1.290), with all planned contrasts significant (pBH < .001).The result supports HA as a distinct interactional regime rather than an intermediate condition.Source: §5.3 · p. 7, Table 2 · p. 8
Novelty couples with resonance in HA stories
- Shuffled-context gains collapsed while preserving the HA gap, lexical and surface checks did not explain the dynamics, and AA proportional context gains remained 50%–64% across model families.The shuffled-context control used a 12B rather than 31B scorer; comparisons were within the same scorer.Source: Novelty couples with resonance in HA… · p. 8
6 Discussion
Affective adaptation appears across co-writing conditions, while Human–LLM collaboration is most distinctive in semantic distance and narrative influence. Humans tend to introduce surprising material that persists, while LLMs elaborate and stabilize the existing narrative.
6 Discussion
- Role differentiation is most pronounced in human–LLM collaboration, supporting its characterization as distinct from HH and AA.Source: §6 · p. 9
- Affective adaptation appears across all conditions, but HA suggests models accommodate more while humans retain greater emotional autonomy; cross-condition differences are not reliable.Source: §6 · p. 9
- In HA, humans contribute more surprising material that persists, while LLMs consolidate the narrative direction; this role asymmetry is weaker in HH and weaker still in AA.Source: §6 · p. 9
- HA shows a steeper novelty–resonance slope, indicating stronger downstream integration of surprising turns, whereas AA shows reduced future integration.Source: §6 · p. 9
7 Conclusion and Future Work
The paper examines turn-based storytelling across human–human, human–LLM, and LLM–LLM pairings. It concludes that human–LLM co-writing is distinct, with humans tending to introduce persistent novelty and LLMs actively elaborating the emerging context.
7 Conclusion and Future Work
- The study compares turn-based collaborative storytelling in HH, HA, and AA using a shared experimental paradigm.Source: §7 · p. 9
- HA is qualitatively distinct from HH and AA, with humans tending to introduce surprising material that persists and LLMs elaborating and stabilizing the narrative context.Source: §7 · p. 9
- LLM contributions are not entirely derivative, showing non-zero novelty, transience, and semantic dissimilarity.Source: §7 · p. 9
- Future work should test whether these dynamics hold across longer interactions, more diverse participants, other genres, and dyadic text settings such as argument or negotiation.Source: §7 · p. 9
Limitations
The findings are bounded by sample and paradigm constraints, model-dependent measures, and an operationalization of agency through discourse influence rather than direct observation.
Limitations
- The conditions differ in sample size, participants are mostly students from one university, and the ten-exchange paradigm limits the study’s scope.The authors say expanding the HH sample remains necessary, despite asymmetry effects being consistent across four estimators.Source: Limitations · p. 10
- Pooling stories across four LLM families increases ecological validity but may obscure model-specific dynamics.Source: Limitations · p. 10
- Valence, novelty, transience, and resonance depend on model choices, while discourse-based agency measures cannot capture authorial intention or subjective agency.Spell-correction of human contributions may also have attenuated some human–LLM lexical differences.Source: Limitations · p. 10
Ethics Statement
Participants consented to research use of their text, and the study collected only anonymized age and gender data. The authors also report computational and API-related environmental costs.
Ethics Statement
- Participants could withdraw, were informed their text would be used for research, and provided consent; only anonymized age and gender data were collected.The dataset comprises English-language creative fiction by adults recruited in a university context in Denmark.Source: Ethics Statement · p. 10
- Embedding and causal-surprisal analyses used five GPU hours on an NVIDIA A100, with an estimated footprint of 0.23 kg CO2 eq.The estimate assumes a carbon intensity of 0.115 kg CO2eq/kWh.Source: Ethics Statement · p. 10
- Story generation used commercial APIs and hosted endpoints, but their precise energy use could not be verified.Source: Ethics Statement · p. 10
A.1 Recruitment and Preprocessing
HA shows greater slot-level asymmetry in novelty, transience, and resonance than HH and AA. Illustrative exchanges depict humans steering affect and introducing persistent redirections, while LLMs follow affective shifts and stabilize context.
A.1 Recruitment and Preprocessing
- HH participants were unpaid volunteers from English-taught Aarhus programs paired anonymously; HA participants were excluded, and AA paired HA-condition models under the same prompt and turn structure.The HA condition is drawn from Fundal and Bizzoni (2026).Source: §A.1 · p. 12
- Before analysis, incomplete sessions, empty turns, and malformed model outputs were removed; stories with missing turns contributed only valid adjacent exchange pairs.Human turns were spell-corrected before valence, embedding, semantic-distance, and surprisal measures were computed.Source: §A.1 · p. 12
A.2 Generation and scoring configuration
- HA and AA turns used temperature 1.0 and a 40-token cap; AA included 20 stories per model across four named language models.Valence, semantic distance, and surprisal were scored using specified embedding and language models; analyses used R, with permutation and bootstrap procedures using 10,000 resamples.Source: §A.2 · p. 12
A.3 Qualitative Examples of Measures
- Illustrative exchanges show LLMs following human affective shifts while humans steer tone; humans also introduce persistent novel redirections as LLMs stabilize context.The table defines resonance as novelty minus transience.Source: Table 5 · p. 13, Table 6 · p. 14
- HA shows substantially larger per-story slot asymmetry in novelty, transience, and resonance than both HH and AA.Cliff’s δ values compare HA against each baseline condition.Source: Table 8 · p. 15