Towards Coherent and Cohesive Long-form Text Generation

1 Introduction

• Cohesion concerns local connections between consecutive sentences, whereas coherence concerns how multiple sentences hold together globally.Source: §1 · p. 1
• Long-form generation lacks formal specifications and widely accepted measures of coherence and cohesion, while neural models can miss cross-sentence dependencies despite producing plausible individual sentences.Such outputs may be detached from their surrounding dialogue or text.Source: §1 · p. 1
• The proposed coherence discriminator scores compatibility across all sentences in a paragraph, while the cohesion discriminator scores consecutive sentence pairs.Their scores serve as reward signals for an RNN-based language model.Source: §1 · p. 1
• Negative-critical sequence training uses negative samples to estimate a reward baseline, eliminating the need for a separate critic function.Source: §1 · pp. 1–2
• The authors develop a model for more coherent and cohesive long-form text and validate it on TripAdvisor and Yelp English reviews.Source: §1 · p. 2

2 Related work

• In the reinforcement-learning framing, a generator is a policy choosing each word as an action conditioned on the preceding text, with expected reward accumulated over the sequence.The paper formulates generation as an episodic problem with reward available only at the episode's end and γ = 1.Source: §2 · p. 2
• Prior work directly optimizes metrics such as BLEU or ROUGE, while later consistency and repetition objectives rerank candidates rather than reward or penalize them.Source: §2 · p. 2
• Dialogue reward models have combined objectives such as information flow and semantic coherence, but their individual contributions were not ablated and their generalization to long-form tasks was unclear.Source: §2 · p. 2
• Order-based rewards may not capture coherence because a paragraph admits many negative orderings; the authors also model cohesion between consecutive sentence pairs with word-level features.Source: §2 · p. 2
• Text GAN discriminators typically distinguish real from generated text rather than assess linguistic properties, and a language-model discriminator still does not model cohesion or coherence.Source: §2 · p. 2
• The authors argue that inadequate long-form generators lack effective mechanisms to measure and control local and global consistency; prior neural work jointly exploring coherence and cohesion in long-form generation had not come to their attention.Source: §2 · p. 2

3 Coherence and Cohesion Models

• Coherence concerns how sentences are organized across a paragraph, while cohesion concerns local connections between neighboring sentences.The model treats the two properties with separate discriminators operating at sentence and word levels, respectively.Source: §3 · p. 3

3.1 Coherence discriminator: Dcoherence

• Dcoherence encodes sentence bag-of-words vectors with separate source and target encoders, then scores their text-chunk vectors by cosine similarity.The encoders share an architecture but not parameters, so the score is not symmetric; bag-of-words inputs preserve high-level sentence semantics while ignoring word order.Source: §3.1 · p. 3, Figure 1 · p. 3
• Dcoherence learns to score coherent text-chunk pairs above incoherent pairs using a margin-based pairwise ranking loss.Negative pairs include rotated targets, shuffled sentence order, and combinations of rotation and shuffling; a weighted average emphasizes harder negatives and empirically stabilizes training and speeds convergence.Source: §3.1 · pp. 3–4

3.2 Cohesion discriminator: Dcohesion

• Dcohesion encodes consecutive sentences from raw word embeddings with separate source and target networks and scores them by cosine similarity.Unlike the coherence discriminator's bag-of-words sentence representations, its CNN or RNN encoders retain word order, which matters for judging neighboring-sentence cohesion.Source: Figure 1 · p. 3, §3.2 · p. 4
• Dcohesion is trained to rank real consecutive-sentence pairs above incohesive alternatives with a margin-based pairwise ranking loss.Its negative pairs are produced by replacing the next sentence, shuffling its words, and combining those approaches; positive pairs are sampled from consecutive sentences in paragraphs.Source: §3.2 · pp. 4–5

4.1 Long-form text generator: G

• The attention-based bidirectional sequence-to-sequence generator is pre-trained by maximum likelihood, yet its long-form outputs often miss the desired coherence and cohesion standards.The pre-trained model is denoted GMLE.Source: §4.1 · p. 5
• The pre-trained coherence and cohesion discriminators provide reward or penalty signals that adjust GMLE through negative-critical sequence training, a policy-gradient variant.Source: §4.1 · p. 5

4.2 Negative-critical sequence training

• For generated text conditioned on a source, the method calls both discriminators and computes overall cohesion as the mean score over consecutive sentence pairs, including the source's last sentence.Source: §4.2 · p. 5
• Raw discriminator scores are not interpretable; the differences between positive-pair scores and maximal or average negative-pair scores indicate how well each discriminator separates positive from negative pairs.Source: §4.2 · p. 5
• The score margin functions as reward minus baseline, so negative samples provide a baseline without requiring a separately trained critic.Source: §4.2 · p. 5
• The negative-pair expectations used in the reward are computed by averaging over an ensemble of negative pairs.Source: §4.2 · p. 5
• The reward baseline averages negative-pair scores rather than using their maximum or weighted mean, which the authors report can be noisy.They report the mean baseline as an empirically better alternative for reducing reward noise.Source: §4.2 · p. 5

5.3 Results

• Discriminator retrieval evaluation uses R@K over 100 negative candidates, with reported ratios averaged across 20 evaluations.The experiments also show examples of positive and negative pairs and their rewards in Table 1.Source: Table 2 · p. 6, Table 1 · p. 6, §5.3 · p. 8
• Sample generations contrast GMLE’s repetition and inconsistent opinions with GMLE+RL’s more interesting, sentiment-consistent continuations.The comparison is presented in Table 3.Source: Table 3 · p. 7

5.1 Dataset

• The datasets are TripAdvisor and Yelp reviews; retained reviews have at least 10 sentences, each 5–30 words, with five sentences as input and the next five as target.The selected subsets contain roughly 60,000 TripAdvisor reviews and 220,000 Yelp reviews, split 0.8/0.1/0.1 for train, development, and test.Source: §5.1 · pp. 6–7

5.2 Implementation details

• Discriminators use fixed GloVe embeddings with CNN or bidirectional-GRU encoders, selecting checkpoints by validation R@1.The CNN encoder uses 512 filters at each specified filter size; discriminator training uses Adam with learning rate 1e-5.Source: §5.2 · p. 7

5.3 Results

• Human judges preferred GMLE+RL to GMLE in evaluation of 200 TripAdvisor samples after poor judges were removed.Judges ranked outputs for coherence and cohesion; Table 5 reports preferences after adjustment for spamming.Source: Table 5 · p. 8, §5.3 · p. 8

5.2 Implementation details

• GMLE+RL is fine-tuned for up to five epochs with equally weighted coherence and cohesion rewards, while supervised learning continues and decoding remains greedy.The fine-tuning learning rate is 1e-5.Source: §5.2 · pp. 7–8

5.3 Results

• Discriminator-guided GMLE+RL significantly improves BLEU, NLL, and PPL, with marginal differences in diversity.Table 4 reports the automated evaluation metrics as an ablation study.Source: Table 4 · p. 7, §5.3 · p. 8

• Pre-trained discriminators score generated texts, while artificially constructed negative-pair scores provide policy-gradient baselines in negative-critical sequence training.Source: §6 · p. 8
• Human and automatic evaluations agree that the method generates more locally and globally consistent texts on two long-form generation tasks.Source: §6 · p. 8
• The authors say the method still needs significant improvement to meet the goal of generating meaningful and logical long-form texts.Source: §6 · p. 8

• Crowd-sourced judges choosing GMLE+RL over Human more than 40% of the time were removed as probable spammers.Source: A Human evaluation un-adjusted scores · p. 12
• The authors considered the 40% threshold appropriate for identifying poor judges.Source: A Human evaluation un-adjusted scores · p. 12
• Table 6 presents human-evaluation results before accounting for the poor judges.Source: A Human evaluation un-adjusted scores · p. 12

• Sequence-level rewards are available only after a generation is complete, making them sparse end-of-sequence signals for the generator.Source: B Sparse end-of-sequence rewards · p. 12
• For this task, the authors observed that exploration caused most training noise, while learning generalized to test data according to human and automatic evaluations.Source: B Sparse end-of-sequence rewards · p. 12
• Reward shaping was unnecessary for this task, unlike prior work that provided signals for partially generated sequences.Source: B Sparse end-of-sequence rewards · p. 12