Source-linked AI summary
Synthetic Child-Adult Interactions Are Not Developmentally Plausible
Bastian Bunzeck, Francesca Padovani
TL;DR
The paper examines whether synthetic child-adult dialogue resembles natural interaction, comparing TinyDialogues with natural dialogue across linguistic measures. It finds lexical alignment but substantial divergence in spoken interaction patterns, and pilots instruction-tuning on real dialogues as an alternative.
Problem
It is unclear whether language-model-generated child-adult interactions resemble natural dialogue, despite synthetic data’s growing use in developmentally plausible language modeling.
Source: §1 · p. 1Method
The authors compare TinyDialogues with natural dialogue across linguistic measures and pilot instruction-tuning using data created from real dialogues.
Source: §1 · p. 1Results
TinyDialogues matches natural data lexically but diverges on other key measures, including overly complex and grammatical text and a lack of spoken fragments; a pilot finds real-transcript fine-tuning brings a small 4B model closer to natural construction distributions.
Source: §1 · p. 1, §6 · pp. 7–8Takeaways & Limitations
Moderate parameter-efficient fine-tuning on real transcripts can bring a small model closer to natural construction distributions than naive prompting of a much larger model.
Source: §6 · p. 8
Abstract
from arXiv · showhide
Synthetic data has become a common resource for training language models, and this trend has spilled over into research on developmentally plausible language modeling. However, it remains unclear whether LM-generated child-adult interactions actually resemble natural ones. We compare the synthetic TinyDialogues dataset to natural dialogue across a broad range of linguistic measures. While synthetic data matches natural data on the lexical level, it drastically underestimates the messy, fragmented nature of spoken interaction. To mitigate this, we propose instruction-tuning with data created from real dialogues, and show in a pilot study that a small model approximates the statistical tendencies of natural data after moderate parameter-efficient fine-tuning.
1 Introduction
The introduction asks whether language-model-generated child-adult dialogue resembles natural interaction, then compares TinyDialogues with natural dialogue and proposes fine-tuning on real-dialogue data.
1 Introduction
- The authors describe the project as work-in-progress and submit it as non-archival to seek community feedback, particularly on the fine-tuning approach and synthetic-data desiderata for BabyLM.Source: §1 · p. 1
- TinyDialogues matches natural data lexically but underestimates spoken fragmentation on syntactic and construction measures, with fewer grammatical errors and greater semantic alignment.The paper also proposes real-dialogue instruction-tuning; a pilot finds a small model can approximate natural-data statistical tendencies after moderate parameter-efficient fine-tuning.Source: §1 · p. 1
- Synthetic data offers a quick way to create large quantities of text, but its resemblance to natural language remains uncertain and matters for experiments based on it.The introduction notes that real child-adult transcripts are costly to produce, transcribe, and annotate, and may be unavailable in sufficient quantity or quality.Source: §1 · p. 1
- Real-dialogue instruction-tuning lets a small 4B-parameter model approximate natural-data statistical tendencies after moderate parameter-efficient fine-tuning in a pilot study.The approach creates instruction-tuning data from real dialogues.Source: §1 · p. 1
2 Related work
Prior work has generated child-directed or child-caregiver dialogue and evaluated synthetic language, but its findings depend on task, interaction length, and register.
Stochastically generated child-caregiver interactions
- TinyDialogues was created by prompting GPT-4 for age-appropriate child-caregiver dialogues containing specified words; prior benchmark comparisons did not analyze linguistic differences between its data and CHILDES.The datasets also had sharply different speaker distributions: CHILDES was 45.7% mother and 38.2% child, while TinyDialogues was 46.6% child and 8.4% mother.Source: §2 · pp. 1–2
- GILES generates child-directed speech, not child responses or interaction, and reports age-dependent alignment on measures such as utterance length and parts of speech (Räsänen and Kocharov, 2024, 2025).Natural data was generally more complex for type-token ratio and perplexity under a reference language model.Source: §2 · p. 2
Evaluating the linguistic properties of synthetic data
- Work on generating simple or child-appropriate stories is outside this paper’s scope (Eldan and Li, 2023).Source: §2 · p. 2
- Liu and Fourtassi (2024) found single-turn generations reasonably close to human data on word, utterance, and dialogue measures, while multi-turn outputs diverged more, particularly on utterance and discourse measures.Source: §2 · p. 2
- Fine-tuning small Llama models on natural dialogue continuations better approximated discourse coherence than fine-tuning on synthetic data.Source: §2 · p. 2
- Other evaluations report reduced distributional diversity in regenerated text (Ju et al., 2025) and register-dependent closeness to human language, with spoken data especially difficult to approximate (Nieth et al., 2026).Nieth et al. report that closeness depends on register rather than model size.Source: §2 · p. 2
3 Data pre-processing
The authors construct comparable synthetic and natural dialogue datasets by matching dialogue length and total word count where data availability permits.
3 Data pre-processing
- TinyDialogues’ evaluation split supplies 19,000 dialogues and 280,000 utterances; natural comparison data comes from CHILDES, TalkBank, and Class-Bank.The natural corpora represent child-caregiver, adult-adult, and classroom interaction, respectively.Source: §3 · p. 2
- Comparison datasets target approximately 201 words per dialogue and 940,000 words overall, with longer natural dialogues segmented and sampled to reach the word budget.Classroom data was insufficient for the target, so all available data was used; CHILDES was divided into toddler and older-child groups.Source: §3 · pp. 2–3
- The expected age-group correspondences are TD-2 with toddlers, TD-5 with children, TD-10 with classroom data, and TD-15 with adult interactions.Source: §3 · p. 3
4 Analyses
The analyses compare lexical diversity, word-level measures, and dialogue structure across synthetic and natural datasets.
4.1 Type-token ratio
- Natural and synthetic type-token ratios are relatively aligned; natural corpora are more lexically diverse for younger groups, but that difference vanishes as simulated age increases.Synthetic TTR rises with simulated age, with TD-10 and TD-15 converging toward natural adult data; early natural lexical diversity may be inflated by phonetic transcriptions of nonstandard child forms.Source: §4.1 · p. 3
4.2 Word density and concreteness
- Word density is the proportion of content words relative to other words in an utterance, while concreteness averages human ratings for content-word lemmas.Density and concreteness are expected to decrease with age; concreteness ratings use a 1–5 scale from abstract to concrete.Source: §4.2 · pp. 3–4
- TD-2 word density is 0.52 versus Toddler at 0.44, while older synthetic ages converge toward natural values and concreteness follows a similar age-related decline.Synthetic concreteness is consistently, though slightly, higher than natural data at every age.Source: §4.2 · p. 4
4.3 Average utterance length and average turns
Utterance length rises with age in both natural and synthetic data, but TinyDialogues overshoots natural lengths at every stage. The source of that excess shifts from caregivers at TD-2 to both speaker roles from TD-5 onward.
4.3 Average utterance length and average turns
- Within each data type, average utterance length increases with age, while matching total word counts means turn count and utterance length trade off.Dialogue-level statistics also include turn counts and their minimum and maximum; natural segments were sampled toward a 940,000-word budget, except Classroom, which used all available data.Source: §4.3 · p. 4
- Synthetic utterances are longer than natural ones at every stage; TD-2 averages 8.2 tokens, exceeding natural Adult and Classroom averages.From TD-5 onward, both synthetic speaker roles exceed the longest natural category.Source: §4.3 · p. 4
- At TD-2, excessively long caregiver turns drive the high overall average, while synthetic child utterances are shorter than real toddlers’ utterances.In natural data, caregiver utterances are only modestly longer than children’s, with the asymmetry remaining fairly stable and reversing slightly in Classroom.Source: §4.3 · p. 4
4.4 Average parse tree depth
Synthetic dialogues have deeper dependency trees than natural data at every stage, with the TD-2 excess concentrated in caregiver speech. Construction types likewise reveal a contrast: synthetic data has far fewer fragments and more complex utterances.
4.4 Average parse tree depth
- Average tree depth is the mean, by dataset and speaker role, of the longest root-to-leaf path in each utterance’s dependency tree.The study adds Universal Dependencies annotations with CAIT, which combines a dependency parser and POS tagger trained on CHILDES data.Source: §4.4 · p. 4
- Synthetic dependency-tree depth is consistently higher than natural depth at every stage; at TD-2, caregiver depth is 3.5 versus 1.4 for children.From TD-5 onward, the excess depth affects both speaker roles; overall depth rises with age in both data types.Source: §4.4 · p. 4, Figure 3 · p. 5
4.5 Utterance-level constructions
- CAIT assigns utterances to construction types including fragments, questions, copulas, imperatives, subject-predicate constructions, and complex utterances.The tagger also includes formulaic constructions; the reported distributions are per dataset.Source: §4.5 · p. 5
- Natural corpora contain 31.31–46.07% fragments, versus 0.01–4.97% in synthetic data except TD-2 at 29.99%; complex utterances are 51.34–83.12% synthetic versus 9.02–26.84% natural.Synthetic data also has more frequent questions with age, whereas questions become less frequent in natural data.Source: §4.5 · p. 5
4.6 Semantic alignment
Semantic and linguistic alignment measures show some shared age trends across natural and synthetic dialogue, but also systematic differences. Synthetic dialogue has higher turn-by-turn semantic alignment and syntactic alignment, while its lexical alignment remains stable across age groups.
4.6 Semantic alignment
- Inter-speaker similarity averages cosine similarity between speaker-level mean utterance embeddings; turn-by-turn alignment averages cosine similarity for adjacent turns from different speakers.Both measures use 384-dimensional sentence-transformer embeddings; only turn-by-turn alignment is sensitive to conversational order.Source: §4.6 · p. 5
- Turn-by-turn semantic alignment declines with age in both data types, yet synthetic dialogues score consistently higher than natural ones at every age.Inter-speaker similarity also declines with age in both; it measures shared topic across a dialogue, unlike order-sensitive turn-by-turn alignment.Source: §4.6 · pp. 5–6, Table 2 · p. 6
4.7 Linguistic alignment
- Lexical and syntactic alignment are computed between adjacent speaker turns, averaged within dialogues and then across each dataset.Lexical alignment measures repeated word forms and syntactic alignment repeated part-of-speech sequences, using lemma-level unigram and bigram scores.Source: §4.7 · p. 6
- Synthetic lexical alignment stays stable across age groups, while syntactic alignment increases with age in both data types and remains higher in synthetic dialogue.The stable synthetic lexical trend contrasts with the decline in natural lexical alignment.Source: §4.7 · p. 6, Table 3 · p. 6
4.8 Children’s grammatical errors
A CHILDES-trained context-dependent classifier finds that predicted child grammaticality rises with age in both natural and synthetic data, but the rise is sharper in synthetic dialogue. The authors caution that the classifier may not capture older speakers’ utterance complexity.
4.8 Children’s grammatical errors
- The study retrains the context-dependent grammaticality classifier from scratch on released CHILDES annotations and predicts grammatical, ungrammatical, or ambiguous proportions by dataset and speaker role.The authors could not access the original model checkpoints.Source: §4.8 · p. 6, Figure 5 · p. 7
- Child grammaticality rises with age in both data types, modestly in natural data but sharply in synthetic data, peaking at age 10 before dropping slightly at 15.Natural data retains substantial ungrammatical or ambiguous speech among older age groups, unlike synthetic data, where ungrammatical utterances become rare.Source: §4.8 · p. 6, Figure 5 · p. 7
- The classifier may label older speakers’ complex utterances ambiguous because its CHILDES training data primarily covers children aged 2–5.For older synthetic age groups, ambiguous utterances become comparatively more frequent as ungrammatical ones become rare.Source: §4.8 · pp. 6–7
- Caregiver grammaticality is usually higher than child grammaticality when children are younger, but Classroom reverses this pattern: CAR is 61% and CHI is 78%.Caregiver speech follows a broadly similar developmental pattern to child speech once children are above age 2.Source: §4.8 · p. 7, Figure 5 · p. 7
5 Fine-tuning approach
A pilot fine-tunes a small model on instruction data built from real dialogues; its generated dialogue approximates natural construction statistics, though only one evaluation is reported.
5 Fine-tuning approach
- Prompts combine CHILDES metadata and an LM-generated dialogue summary, with the original transcript serving as the gold response.Metadata includes child age, number of turns, and participants.Source: §5 · p. 7
- The model generated 250 dialogues of up to 300 novel tokens from 25 hand-written summary prompts, using nucleus sampling at temperature 1.2.Lower temperature settings produced overly repetitive dialogue, including single-word repetitions at 0.8.Source: §5 · p. 7
- After moderate parameter-efficient fine-tuning, the 4B model reproduces natural rates of fragments, imperatives, and subject-predicate constructions without TinyDialogues’ overshoot in complex utterances.The authors report only one evaluation of the generated dialogue.Source: §5 · p. 7
6 Discussion and conclusion
Synthetic dialogue diverges from natural interaction on several linguistic dimensions, while a pilot suggests real-dialogue fine-tuning can bring a small model closer to natural construction distributions. The authors stress that these findings remain preliminary and outline broader analyses for future work.
6 Discussion and conclusion
- Synthetic dialogue is verbose, highly grammatical, syntactically complex, and less diverse in constructions, while lacking natural utterance-level messiness and aligning excessively on the semantic level.The authors suggest TinyDialogues may drift toward written, complete-sentence text as simulated children age; Table 4 reports utterance-level construction distributions.Source: §6 · pp. 7–8, Table 4 · p. 8
- The pilot suggests moderate parameter-efficient fine-tuning on real transcripts brings a small 4B model closer to natural construction distributions than naive prompting of a much larger model.Whether outputs are semantically coherent, faithfully age-conditioned, or useful as training data remains open.Source: §6 · p. 8
- Future analyses could broaden the linguistic measures, languages, and genres examined, including code-switched data and synthetic stories alongside natural counterparts.The authors identify these extensions as ways to shed more light on the linguistic dimensions and human-likeness of LM-generated language.Source: §6 · p. 8
A Prompts for generation
The appendix provides hand-written dialogue-summary seeds spanning everyday child-caregiver situations, playful pretend, learning, and minor conflicts.
A Prompts for generation
- The seeds also depict frustration, fear, and mishaps, with caregivers comforting children or working through a problem together.Examples include a lost toy, a stuck zipper, a fallen block tower, and a bumped supermarket cart.Source: A Prompts for generation · pp. 11–12
- Several seeds use imaginative play to turn ordinary activities into stories or games, such as baking, shopping, puddles, and household tasks.Examples include broccoli as dinosaur trees, a cart as a race car, and socks as puppets.Source: A Prompts for generation · p. 11
- Many prompts center on everyday routines and negotiation, including bedtime, meals, dressing, chores, and screen-time limits.Examples include a bedtime-story compromise, choosing shoes, tidying toys before Minecraft, and helping with laundry.Source: A Prompts for generation · p. 11
- Other prompts pair a child's questions or discoveries with a caregiver's explanation, covering phones, shadows, plants, the moon, and a garden snail.The moon prompt includes a silly answer followed by an attempt at a proper explanation.Source: A Prompts for generation · p. 11