Source-linked AI summary
Deep Communicating Agents for Abstractive Summarization
Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, Yejin Choi
TL;DR
Long-document abstractive summarization requires an effective way to represent information across the input. The paper distributes encoding across communicating agents connected to one decoder and trained with reinforcement learning, and reports higher-quality, more focused summaries than single-agent and non-communicating baselines. The conclusion attributes the improvement to covering salient concepts while maintaining semantic coherence.
Problem
Neural models that encode short text well do not generalize effectively to long documents, and representing a long document remains challenging.
Method
The model assigns document subsections to collaborating encoders that exchange information across layers and use contextual agent attention to guide a shared decoder.
Results
Multiple communicating encoders produce higher-quality summaries than single-encoder and non-communicating baselines, with human evaluations indicating more focused summaries.
Takeaways & Limitations
The results support using communicating agents to cover salient information across long documents while maintaining semantic coherence and avoiding unnecessary details.
Takeaways & Limitations
The semantic cohesion loss depends on a tunable hyperparameter λ.
Abstract
from arXiv · showhide
We present deep communicating agents in an encoder-decoder architecture to address the challenges of representing a long document for abstractive summarization. With deep communicating agents, the task of encoding a long text is divided across multiple collaborating agents, each in charge of a subsection of the input text. These encoders are connected to a single decoder, trained end-to-end using reinforcement learning to generate a focused and coherent summary. Empirical results demonstrate that multiple communicating encoders lead to a higher quality summary compared to several strong baselines, including those based on a single encoder or multiple non-communicating encoders.
1 Introduction
The paper addresses the difficulty of encoding long documents for abstractive summarization by distributing the text across communicating encoder agents. These agents share context across layers and provide a single decoder with coordinated information to generate focused summaries.
- 1 Introduction: Long-text encoding remains difficult because neural models that perform strongly on short text do not generalize well to long documents.
- 1 Introduction: Effective representation of a long document remains challenging despite improved attention, coverage mechanisms, and coherence-focused training objectives.
- 1 Introduction: The model divides a document across collaborating encoder agents, each responsible for a different subsection and sharing global context with the others.
- 1 Introduction: Agents repeatedly adapt their subsection encodings using global context, then deliver information to a decoder through contextual agent attention.The decoder is trained end-to-end with self-critical reinforcement learning to generate focused and coherent summaries.
- 1 Introduction: Multiple communicating encoders produce higher-quality summaries than strong single-encoder and non-communicating multi-encoder baselines on CNN/DailyMail and New York Times.Human evaluations also find the summaries more focused, while analysis links the gains to gathering salient information across document regions.
2 Model
The model uses multiple encoders that process separate paragraphs, communicate through stacked contextual layers, and feed a decoder with hierarchical attention. It combines agent attention with pointer-style generation so agents can jointly select vocabulary and document words.
- 2 Model: Each document is split across multiple agents, with each agent encoding one paragraph or subsection of the input.
- 2.1 Multi-Agent Encoder: Each agent first applies a local bidirectional LSTM encoder, whose output feeds stacked contextual encoder layers.
- 2.1 Multi-Agent Encoder: At every contextual layer, each agent receives an averaged message from the other agents and updates both its local representation and outgoing information.The received message is based on the other agents’ last hidden-state outputs.
- 2.2 Decoder with Agent Attention: The decoder uses hierarchical attention: token-level attention produces a context for each agent, followed by agent-level attention that combines those contexts.The resulting fixed-length agent context is concatenated with the decoder state to produce the vocabulary distribution.
- 2.2 Decoder with Agent Attention: Contextual agent attention uses the previous agent attention as additional decoding information to reduce frequent switching between agents.
- 2.3 Multi-Agent Pointer Network: Agent-specific generation probabilities combine vocabulary generation with copying from the corresponding paragraph, producing an extended vocabulary that includes out-of-vocabulary document words.The final distribution weights each agent’s distribution by its agent attention, allowing agents to vote for different OOV words.
3 Mixed Objective Learning
The model combines maximum-likelihood, reinforcement-learning, semantic-cohesion, and sentence-level intermediate rewards to improve summary quality while preserving fluency and reducing repetition.
- Mixed Loss: The training objective jointly optimizes multiple losses for deep communicating agents.The framework combines maximum likelihood, reinforcement learning, and optionally semantic cohesion losses.
- Semantic Cohesion: Semantic cohesion loss minimizes cosine similarity between consecutive sentence-ending decoder states to encourage informative, non-repetitive sentences.The loss uses hidden states at sentence boundaries identified by end-of-sentence tokens.
- Reinforcement Learning: Self-critical reinforcement learning compares sampled sequences with greedily decoded baselines to increase expected reward.The sampled sequence is rewarded when it outperforms the greedy baseline.
- Mixed Loss: MLE improves language modeling, whereas RL directly optimizes task-level rewards but can reduce readability and fluency when used alone.The mixed objective balances these objectives with a tunable γ parameter.
- Intermediate Rewards: Sentence-based differential ROUGE rewards each sentence for the additional ROUGE it contributes, promoting diverse sentences and novel information.The reward is the ROUGE difference between the full prefix and the preceding sentence prefix.
4 Experimental Setup
Experiments evaluate deep communicating agent models on CNN/DailyMail and New York Times summarization using ROUGE-based test evaluation and comparisons with published baselines and ablations.
- Datasets: Experiments use the CNN/DailyMail and New York Times summarization datasets with replicated preprocessing and document partitioning across agents.Documents are partitioned among a fixed number of agents, such as three agents for three paragraphs.
- Training Details: Articles are truncated to 800 tokens, summaries are limited to 100 training tokens and 110 test tokens, and vocabularies contain the 50,000 most frequent training tokens.Multi-agent models distribute truncated articles while preserving paragraph and sentence boundaries where possible.
- Evaluation: Evaluation reports ROUGE-1, ROUGE-2, and ROUGE-L, selecting MLE models by validation negative log-likelihood and MLE+RL models by validation ROUGE-L.Final predictions use beam search with width 5.
- Baselines: The study compares deep communicating agents with published extractive, pointer-network, reinforcement-learning, and controllable abstractive summarization systems.The baselines include models with and without coverage and models using intra-decoder attention.
- Ablations: Seven ablations progressively test single-agent training, semantic cohesion, mixed loss, multi-agent pointer networks, communication, contextual agent attention, and combined objectives.The final m7 model uses three agents with contextual agent attention and mixed MLE+RL+SEM loss.
5 Results
The communicating-agent models achieve stronger automatic and human-evaluated summaries than single-agent and non-communicating baselines. Communication, distributed attention, contextual agent attention, and repetition-oriented objectives are associated with better coverage, focus, coherence, and ROUGE performance.
- The strongest multi-agent models with pointer generation and communication perform best on ROUGE-1 and ROUGE-2 across CNN/DailyMail and NYT.
- All multi-agent models improve over single-agent baselines, while three-agent models generally outperform two- and five-agent variants on CNN/DailyMail.The documents are truncated before training, which may make larger agent counts more suitable for multi-document summarization.
- Communication produces the largest ROUGE improvement over independent agents on CNN/DailyMail by helping encoders identify key facts and avoid unnecessary details.
- Contextual agent attention improves ROUGE by stabilizing the decoder’s agent selection and reducing frequent switches that dilute topical signals.
- Semantic cohesion penalties and sentence-based intermediate reinforcement-learning rewards improve ROUGE scores while targeting redundancy and semantic diversity.
- Human evaluators prefer communicating-encoder summaries across all rating metrics and head-to-head comparisons, with the largest improvement on focus.The evaluation used 100 CNN/DailyMail test samples and Likert ratings from 1 to 5.
- Distributed attention across agents yields higher ROUGE-L, and communicating multi-agent summaries outperform corresponding single-agent summaries even when one agent dominates attention.
- Qualitative comparisons show the best multi-agent model captures all human-summary facts with few extra details, whereas the single-agent model misses facts and includes unnecessary content.
6 Related Work
Related work develops attention, pointer, coverage, coherence, and repetition-control mechanisms for sequence generation and summarization. This paper instead applies deep communicating agents to encode long text collaboratively.
- Attention mechanisms guide encoder-decoder models toward relevant input context and have been applied across translation, classification, dialog, and summarization.
- Pointer and coverage networks address redundancy and saliency in generated summaries, while other approaches use intra-decoder attention to penalize repeated words.
- The paper uniquely applies the CommNet deep communicating-agent framework to long-text encoding for summarization.
- The authors introduce semantic coherence loss and intermediate sentence-based rewards to discourage semantically similar generations.
7 Conclusions
The paper concludes that deep communicating agents improve long-text abstractive summarization under both automatic and manual evaluation. The analysis attributes the improvement to better coverage of salient concepts and semantic coherence.
- Deep communicating agents improve long-text abstractive summarization in both automatic and manual evaluations.
- The improvement is associated with covering all and only salient concepts while maintaining semantic coherence in summaries.
A Supplementary Material
Table 6 summarizes the CNN/DailyMail and New York Times datasets used in the experiments.
- The supplementary material includes summary statistics for the CNN/DailyMail and New York Times datasets.
A.1 Datasets
The experiments use CNN/DailyMail and New York Times datasets, with preprocessing choices that preserve non-anonymized named entities.
- CNN/DailyMail contains online news articles paired with multi-sentence summaries.
- The CNN/DailyMail data uses the same splits as Nallapati et al. (2017) and retains non-anonymized entities.
- The New York Times dataset contains articles published between 1996 and 2007 and is split chronologically into 90% training, 5% validation, and 5% testing.
- Unlike related preprocessing, the study does not anonymize NYT named entities to reduce preprocessing.
A.2 Training Details
Training uses fixed model and vocabulary settings, optimization choices for maximum-likelihood and reinforcement-learning models, and decoding constraints against repetition and unknown words.
- Encoder and decoder hidden states have size 128, and models are trained on an NVIDIA P100 GPU machine.
- The vocabulary is limited to the 50,000 most frequent training tokens, with 200-d GloVe embeddings initialized and then fine-tuned.
- Adam uses learning rates of 0.001 for MLE models and 10^-5 for MLE+RL models.
- The mixed-loss gamma is tuned over {0.95, 0.97, 0.99}, with 0.97 yielding the best gains in almost all DCA models.
- During testing, the decoder cannot generate the same trigram twice, and UNK tokens are replaced using the source word with the largest cascaded attention.
B Generated Summary Examples
The appendix compares human summaries with single- and multi-agent outputs, highlighting factual errors, unnecessary details, and captured key facts. Examples show multi-agent models selecting salient information from more distant document sections and sometimes simplifying content.
- The examples compare golden summaries with a single-agent MLE+RL baseline and a best multi-agent model trained with mixed MLE+SEM+RL loss.
- Red highlights mark details that should not appear in summaries but were generated by the models.
- Red markings identify factual errors in generated summaries.
- Green highlights identify key facts from the human summary captured by only one model.
- In Table 7, the multi-agent model selects salient sentences from farther down the document, specifically sentences 8 and 10, while the single model selects sentences 2 and 3.
- In Table 8, the multi-agent model captures the main character and key facts, while also simplifying a long source sentence.
- In Table 9, the multi-agent model produces a concise summary with several key facts, although it also misses the player who scored the only goal.