A Model-Agnostic Data Manipulation Method for Persona-based Dialogue Generation

1 Introduction

• Persona-based dialogue datasets are small and costly to expand, while their samples are harder to learn from because they combine dialogue history, persona sentences, and responses.PersonaChat has about 162,000 utterances and fewer than 5,000 unique persona profiles, compared with millions of utterances in conventional datasets.Source: §1 · p. 1
• Existing augmentation can fail to preserve both dialogue-history coherence and persona-response consistency, and prior pretrained-model approaches are model-specific.The paper therefore seeks a model-agnostic approach for both data scale and data difficulty.Source: §1 · p. 2
• The proposed method distills samples, diversifies the easier data, and trains first on augmented distilled data and then on original data.The method is presented as model-agnostic and evaluated with Transformer encoder-decoder and GPT2 models.Source: §1 · p. 2

Persona-based dialogue generation

• Persona-conditioned dialogue models have added persona embeddings, profile memory, copying, CVAEs, or meta-learning for low-resource personas.These approaches primarily modify how dialogue models use persona information.Source: §2 · p. 2
• Pretrained-model approaches often depend on model-specific network modifications and fine-tuning losses, limiting transfer across pretrained models.Many also concatenate persona text and dialogue history into one input sequence.Source: §2 · p. 2

Text data manipulation

• Prior dialogue augmentation generates utterances or retrieves responses, but studies pairwise query-response relationships rather than auxiliary persona information.Other data manipulation work filters noisy samples, reweights data, or uses dialogue-cluster memory.Source: §2 · pp. 2–3

Curriculum learning

• Unlike curricula based on response coherence and diversity, this work treats newly distilled data as an easy curriculum.The cited curriculum-learning studies train successively from easy to hard.Source: §2 · p. 3

3 Our Data Manipulation Method

• D3 is model-agnostic: it changes training data rather than the dialogue model, and combines distillation, diversification, and curriculum training.The curriculum presents easier augmented distilled data before the harder original data.Source: §3 · p. 3, Figure 2 · p. 4

3.1 Data Distillation

• Distillation keeps a persona sentence that entails the response and the latest history utterance; one original sample can yield none, one, or multiple distilled samples.Persona-response consistency is determined as an NLI task; experiments find that the last utterance is enough for the method to work well.Source: §3.1 · pp. 3–4

3.2 Data Diversification

• Diversification edits persona sentences at token or phrase level, aligns responses by token replacement or GPT2 prediction, and creates history variants through back translation.Edited personas are scored using normalized perplexity and BERTScore F1, with lower values preferred; response samples are evaluated for fluency, persona consistency, and history coherence, then filtered by a threshold.Source: §3.2 · pp. 4–5

3.3 Data Curriculum

• Curriculum training uses augmented distilled data as the easy stage and original data as the hard stage, successively training on both until convergence.Training only on augmented distilled data is not proper because the persona/history-response relationship differs from the original data.Source: §3.2 · p. 4, §3.3 · p. 5

4.1 Experimental Setup

• Evaluation uses PersonaChat with TRANSFORMER trained from scratch and GPT2 finetuned, assessing automatic response metrics and human fluency, coherence, and persona consistency.Human evaluation uses 200 test samples rated by five annotators.Source: §4 · p. 5, §4.1 · p. 6

4.2 Results

• D3 performs best on TRANSFORMER except for Ent-1; on GPT2, only D3 and ENTROPY FILTER exceed the original-data baseline on Pcon.Table 2 reports comparisons across both base models; other GPT2 methods do not consistently improve the various metrics.Source: Table 2 · p. 6, §4.2 · p. 7

4.3 More Analysis

• Removing diversification lowers performance across aspects, while removing distillation worsens it further, especially C-score; distilled-only training improves C-score but degrades other aspects.The authors conclude distilled data should warm-start training on original data rather than replace it.Source: §4.3 · p. 7
• With an NLI model trained on 200 labeled samples, D3* is slightly worse than full-data D3 but remains superior to most compared methods, with response-diversity metrics nearly unchanged.The authors report this setting can be useful when only limited in-domain NLI labels are available for distillation.Source: §4.3 · p. 7
• The designed curriculum outperforms the compared variants overall; Only augment and Reverse score highly on C-score but worse on n-gram accuracy, while Shuffle beats Original yet remains less efficient.The variants compare original-only, augmented-only, mixed, and reversed training curricula.Source: Table 4 · p. 8, §4.3 · p. 8
• More than 80% of diversified samples use model-predicted responses, while diversification adds new persona content and prioritizes persona novelty over dialogue-history novelty.Phrase-level editing contributes personas with fluency and semantic novelty.Source: §4.3 · pp. 7–8, Figure 4 · p. 7
• D3 yields the highest token- and sentence-level consistent attention weights across the selected decoder layers, with a stronger advantage in higher layers.These measures track attention to persona tokens entailed by the response; the case study also reports greater attention to proper persona text during generation.Source: §4.3 · pp. 8–9
• Case-study visualizations show D3 directing more attention to relevant persona text while generating responses.The figure compares TRANSFORMER and TRANSFORMER-D3 responses and persona attention weights.Source: Figure 6 · p. 9, §4.3 · p. 9

5 Conclusion

• The method distills and augments persona-based dialogue data, then uses curriculum training; it improves Transformer encoder-decoder and GPT2 performance.The curriculum trains first on augmented distilled samples and then on original data.Source: §5 · p. 9

A.1 Details of Distillation

• RoBERTa-Large-MNLI is finetuned on DialogueNLI and reaches 90.8% accuracy on its dev set for persona-consistency judgments.The model also calculates entailment probability for response filtering and C-score in the experiments.Source: §A.1 · p. 12
• In the few-shot setting, an NLI model trained on 200 samples reaches 79.3% accuracy on the dev set.It is trained with a learning rate of 2e-5.Source: §A.1 · p. 12

A.2 Details of Diversification

• BERT-uncased and GPT2-base are finetuned on persona sentences, while GPT2 is separately finetuned on responses for the diversification stage.The persona finetuning runs for 100 steps and response finetuning for 200 steps, each with batch size 32 and learning rate 1e-4.Source: §A.2 · p. 12
• Persona editing uses BERTper for token-level edits and GPT2per for phrase-level edits, generating 10 unique persona sentences per original sentence.Token masking targets informative POS tags; token-level and phrase-level mask ratios are 0.8 and 0.3–0.6, respectively.Source: §A.2 · p. 12, Table 6 · p. 13
• Response alignment restricts overlapping token POS tags and trains Multi-GPT2 on distilled data; its dev-set performance is reported as having high n-gram accuracy and persona consistency.The supplied passage does not give the underlying metric values.Source: §A.2 · p. 13
• Dialogue-history augmentation translates utterances English-to-French and back, producing 25 candidates per original and selecting the lowest-BLEU candidate for divergence.The translation model uses beam search of size 5 and selects Np = 1 candidate.Source: §A.2 · p. 13
• Quality filtering combines normalized GPT2 response perplexity and an entailment probability from a RoBERTa model finetuned on InferConvAI.The passage sets β = 0.2 and γ = 0.6 in Eq. 2.Source: §A.2 · p. 13
• Edited persona sentences have GPT2-PPL 6.427 versus 10.426 for original persona sentences.The passage also describes comparisons of response fluency and coherence in Table 8.Source: §A.2 · p. 13

B Details of Experiment

• The Transformer uses six 300-hidden-size encoder and decoder layers with four heads, while GPT2-base has 12 layers and a 768-dimensional hidden state.Figure 7 shows the input and output formats; both models use beam size 3 at inference.Source: B Details of Experiment · p. 13, Figure 7 · p. 14
• Both models use batch size 256, with learning rates 2e-4 for Transformer and 6.25e-5 for GPT2; curriculum training initializes the hard stage from the best easy-stage model.Early stopping uses 15 epochs for Transformer and 5 for GPT2 without dev-loss improvement.Source: B Details of Experiment · p. 13
• Table 7 reports the performance of Multi-GPT2 on a dev set distilled from PersonaChat.The caption identifies the evaluation scope but does not provide the results in the supplied passage.Source: Table 7 · p. 14
• Hyperparameters are selected by coarse grid search on Transformer, and τ has more impact than the other parameters because it determines distilled-sample quality.GPT2 follows the same settings; the candidate values are listed in Table 9.Source: B Details of Experiment · pp. 13–14, Table 9 · p. 14
• Baseline comparisons include back translation, CVAE, and entropy filtering, with training sample counts listed in Table 10.All models are trained until loss fails to decrease for the same number of epochs.Source: B Details of Experiment · p. 14, Table 10 · p. 14
• Evaluation uses BERTScore and C-score alongside human ratings, while Table 8 reports GPT2 perplexity and NLI-based coherence scores for original and generated responses.The appendix provides instructions for human annotators in Tables 14 and 15.Source: B Details of Experiment · p. 14, Table 8 · p. 14

C.1 Attention on Dialogue History

• Transformer attention on the last dialogue-history utterance is significantly higher than on other utterances, motivating history distillation that removes former utterances.Figure 8 compares decoder-layer attention with a red line marking uniform attention across tokens.Source: §C.1 · p. 14

C.2 Statistical Results of Table 2

• The significance tests compare D3 against each baseline using human ratings and applicable automatic metrics, including C-score and BERTScore.The passage describes Student’s t-tests.Source: §C.2 · p. 14
• With Transformer, nearly all baseline results are reported as significantly different from D3; with GPT2, such differences appear less often except for CVAE.The authors state that the pattern suggests data-manipulation methods may have less impact with a pretrained model.Source: §C.2 · pp. 14–15
• Table 11 covers distillation and diversification variants on GPT2, while Table 12 compares curriculum variants on GPT2.The supplied captions establish the tables’ scope but do not state comparative outcomes.Source: Table 11 · p. 15, Table 12 · p. 15
• Figure 8 plots average token-level attention weights by decoder layer for the last utterance and other dialogue history, with uniform attention marked in red.The caption identifies the comparison but supplies no outcome for this request.Source: Figure 8 · p. 15

• GPT2's diversification and distillation results show fewer impacts than the corresponding effects on TRANSFORMER.The passage attributes this reduced sensitivity to GPT2's strong pretraining.Source: §C.3 · p. 15
• GPT2 is less vulnerable to differences in data-sample counts because it is a strong pretrained model.Source: §C.3 · p. 15
• GPT2's results for different curriculum variants demonstrate a conclusion similar to TRANSFORMER's.Source: §C.3 · p. 15

• D3-trained models have the most accurate persona attention at both token and sentence levels.Source: §C.4 · p. 15
• GPT2 shows greater divergence between layers than Transformer in the visualized attention weights.Source: §C.4 · p. 15

• Extending Ddiv to about 200% of Ddis brings only a very slight promotion but requires longer training.The comparison is against the original method, where Ddiv is nearly the same size as Ddis.Source: §C.5 · p. 15, Table 13 · p. 16
• Squeezing Ddiv to about 50% of Ddis has a more obvious effect on performance than extending it.Source: §C.5 · p. 16
• Using Ddiv at a size similar to Ddis is presented as a trade-off between resource cost and performance.The authors also say this supports a fair comparison with previous methods.Source: §C.5 · p. 16

• In the additional cases, D3 helps models attend more to suitable persona parts, and the generated responses have better persona consistency.Source: §C.6 · p. 16, Figure 10 · p. 17
• Figure 10 colors persona text by attention weight, with darker colors indicating higher weight; response text colors mark persona consistency.Source: Figure 10 · p. 17