RAG-RST: Retrieved Exemplars as In-Context Discourse Schema for RST Parsing

1 Introduction

• A strong LLM given only a target EDU pair reaches 0.794 F1 on nuclearity and 0.738 F1 on relation, despite exposure to discourse-rich pretraining text.The passage distinguishes knowledge of discourse phenomena from knowledge of the annotation convention that categorizes them.Source: §1 · p. 2
• The open design question is how many labeled neighbors to show, how consistent they should be, and where the target pair sits in discourse structure.The cited prior work frames labeled neighbors as demonstrations of an annotation scheme [Mei et al., 2025].Source: §1 · p. 2
• RAG-RST retrieves annotated EDU pairs as demonstrations of the RST schema at inference time, rather than supplying that convention through parameter updates.Its retrieval index includes gold-labeled RST-GUM examples and unlabeled CNN/DailyMail sentence pairs used only to improve neighbor coverage.Source: §1 · p. 2
• Precomputed labeling generates labels for all EDU pairs before parsing, whereas dynamic labeling retrieves at each reduce step using the reductions already committed to.The latter makes the exemplars shown to the generator depend on parsing progress.Source: §1 · p. 2
• Both RAG-RST strategies outperform existing RST parsers across all four standard evaluation metrics, with dynamic retrieval consistently strongest.The contribution is scoped to label prediction: the shift/reduce action policy is adopted unmodified from prior work.Source: §1 · p. 2

2 Related Work

• RST parsing has used transition-based and chart methods, with statistical models predicting nuclearity and relation types between discourse units.The cited approaches include transition-based parsers [Soricut and Marcu, 2003, Joty et al., 2015] and statistical label-prediction models [Hernault et al., 2010, Liu et al., 2016, Wang et al., 2018].Source: §2 · p. 2
• Neural span representations and deep encoders improve parsing, but accurate label prediction remains challenging in complex discourse structures.The cited neural enhancements are associated with improved performance and robustness [Nguyen et al., 2021], while label prediction challenges are reported in [Maekawa et al., 2024, Zhang et al., 2025].Source: §2 · p. 2
• Unlike Liu et al. [2026], which retrieves during pretraining, RAG-RST retrieves at parse time and conditions exemplar selection on the structure built so far.The same EDU pair can retrieve different demonstrations depending on how earlier reductions grouped it.Source: §2 · p. 3

3 RAG-Enhanced RST Discourse Parsing Framework

• RAG-RST integrates a FAISS-based retriever and LLM label generation with a shift-reduce parser in precomputed and dynamic configurations.Both configurations use the retrieved examples to predict nuclearity and relation labels.Source: §3 · p. 3

3.1 Method 1: Precomputed Labeling (Outside the Parser)

• Precomputed labeling generates labels for EDU pairs before parsing, stores them in a lookup table, and supplies them to the parser during reduce actions.The parser constructs the discourse tree without invoking retrieval or generation during parsing.Source: §3.1 · pp. 3–4

3.2 Method 2: RAG During Reduce Action (Inside the Parser)

• Dynamic RAG retrieves and generates labels at each reduce action using the current stack-top spans, then applies the labels immediately as parsing proceeds.For a composite span, its pooled representation reflects earlier parser groupings; the query excludes the buffer and deeper stack.Source: §3.2 · pp. 4–5
• The configurations differ only when a reduce action involves at least one composite span; when both operands are single EDUs, their retrieval queries are identical.At composite-span reductions, retrieval reflects the grouping already committed to by the parser.Source: §3.2 · pp. 4–5

• The section describes the datasets, baselines, experimental setup, and results.Source: §4 · p. 5

4.1.1 Dataset

• RST-DT provides 347 training and 38 test documents, while RST-GUM supplies annotated retrieval examples and CNN/DailyMail adds unlabeled retrieval coverage.CNN/DailyMail is not used for training, evaluation, or RST-label supervision.Source: §4.1.1 · pp. 5–6

4.1.2 Baselines for RST Parsing

• Baselines span CRF, SVM, neural, transition-based, sequence-labeling, and LLM-based RST parsers, including a supervised LLM method that learns the schema through parameter updates.RAG-RST instead supplies the schema through retrieved exemplars at inference time; Table 4 compares full parsing performance, not label prediction.Source: §4.1.2 · p. 6

4.1.3 Retrieval and Generator Setup

• The retrieval pipeline uses MiniLM EDU-pair embeddings, a FAISS index, and the top K neighbors selected by cosine similarity with a 0.6 threshold.K and the threshold were selected on a development split from the RST-DT training documents; the 38 test documents are reserved for final parsing evaluation. CNN/DailyMail neighbors are excluded from labeled generator context, and test documents are screened for near duplicates.Source: §4.1.3 · p. 6
• The generator receives the target EDU pair and retrieved labeled neighbors, then predicts nuclearity followed by an RST relation; reported results average three runs with variance of at most ±0.005 F1.Source: §4.1.3 · p. 6

4.1.4 Parser Configurations

• Precomputed labeling predicts every adjacent EDU pair before parsing, while Dynamic RAG retrieves and generates labels at each reduce action using current stack-top spans.Both configurations use LLaMA-3 and K = 50 retrieved neighbors.Source: §4.1.4 · p. 7

4.2.1 Schema Recoverability and the Effect of Retrieval

• Retrieval raises nuclearity from 0.794 to 0.846 F1 and relation from 0.738 to 0.788 F1; adding CNN/DailyMail yields a further +0.017 nuclearity and +0.020 relation over RST-GUM alone.CNN/DailyMail pairs are unlabeled and excluded from the labeled context; their presence changes which labeled neighbors reach the generator.Source: §4.2.1 · p. 7, Table 1 · p. 7

4.2.2 Retrieval Size and Generator Results

• K = 50 achieves the best performance across all tested models; performance degrades beyond fifty, and LLaMA-3 reaches 0.846 nuclearity F1 and 0.788 relation F1.Top-50 retrieval was selected on development data and fixed for subsequent experiments.Source: §4.2.2 · pp. 7–8, Table 2 · p. 8
• LLaMA-3 achieves the highest F1 scores among six generators: 0.846 nuclearity, 0.788 relation, and 0.817 combined.LLaMA-2 and Mistral-7B perform comparably and both substantially outperform encoder-only and sequence-to-sequence models.Source: §4.2.2 · p. 8, Table 3 · p. 8

4.2.3 RST Parsing Results

• 0.892 span, 0.791 nuclearity, 0.711 relation, and 0.674 full tree: Dynamic RAG outperforms every baseline and Precomputed RAG across all four metrics.Precomputed RAG scores 0.885 span, 0.782 nuclearity, 0.694 relation, and 0.663 full tree.Source: §4.2.3 · p. 8, Table 4 · p. 9
• Dynamic retrieval conditions exemplar selection on reductions already made, because its query represents the pair within the partially constructed tree rather than in isolation.The authors report that this structural conditioning yields more coherent label sequences, particularly for complex or long-distance relations.Source: §4.2.3 · pp. 8–9

5 Limitations

• Results are established on English discourse; extending the framework to other languages remains an important direction.The framework depends on retrieved exemplars and is described as transferable to languages with annotated RST corpora.Source: §5 · p. 9
• The retrieval sweep covers retrieval size and generator architecture, while MiniLM was chosen for encoding cost and decoding was greedy.The authors expect stronger retrievers to improve neighbor quality and describe the reported figures as conservative in that respect.Source: §5 · p. 9
• Retrieval is evaluated through label-prediction effects rather than intrinsic neighbor quality; neighbor-label agreement is proposed as a further characterization.Source: §5 · p. 9

6 Conclusion and Future Work

• RAG-RST supplies the RST annotation schema as in-context demonstrations without parameter updates, using Precomputed labeling or Dynamic retrieval at each reduce step.Source: §6 · p. 9
• Both configurations outperform existing parsers across span, nuclearity, relation, and full-tree metrics, with retrieval at the point of decision consistently stronger.Source: §6 · p. 9
• Retrieved-schema utility saturates and declines beyond fifty exemplars, and exemplar relevance depends on structural position.Future work includes stronger retrievers and larger or instruction-tuned generators, integration with other RST parsers, cross-language evaluation, and downstream tasks such as summarization.Source: §6 · p. 9

A Full Retrieval Results

• The results cover all three generator models across the full range of evaluated Top-K values.Table 5 extends the F1 figures summarized in Table 2 with precision and recall.Source: A Full Retrieval Results · p. 12
• Table 5 reports nuclearity and relation prediction performance across evaluated Top-K values using FAISS retrieval on the RST-DT development split.It includes full precision, recall, and F1 results for all three generator models.Source: Table 5 · p. 12, A Full Retrieval Results · p. 12
• Table 5 reports performance for both nuclearity and relation prediction.Source: Table 5 · p. 12