Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue

• Neural dialogue generators struggle with sentence planning, discourse structuring, complex inputs, and semantic errors such as hallucinated content.The paper frames these as limitations of current neural NLG models.Source: §1 · p. 1
• Tree-structured inputs and outputs support constrained decoding that helps ensure generated text contains all and only the specified content.The approach incorporates discourse relations into neural generation, extending beyond control over their mere presence or absence.Source: §1 · p. 1
• The paper releases a weather-domain dataset with discourse-level structures, alongside its results on E2E.The introduction identifies JUSTIFY, JOIN, and CONTRAST as relations included in the weather dataset.Source: §1 · pp. 1–2
• Constrained decoding with enriched inputs improves semantic correctness, generalizability, and data efficiency on both E2E and weather datasets.The paper also reports better control over CONTRAST relations on E2E and improved diversity and expressiveness over standard baselines.Source: §1 · pp. 1–2

2.1 Limitations of Flat MRs

• Flat E2E meaning representations omit discourse relations such as CONTRAST, leaving crowdworkers to decide when to use them and neural models rarely expressing them.Only 43 of 10,080 released system outputs contained contrastive tokens, despite about 300 contrastive test samples.Source: §2.1 · p. 2
• Flat representations provide no means to control discourse behavior for users whose preferences may differ from general trends in the data.The authors argue that live systems may use user models to inform contrast, grouping, or justifications.Source: §2.1 · p. 2

2.2 Tree-Structured MRs

• The proposed tree-structured representations combine arguments, dialog acts, and discourse relations that can nest to arbitrary depths.Examples of discourse relations include JOIN, JUSTIFY, and CONTRAST.Source: §2.2 · p. 3
• Tree-structured MRs make argument-to-dialog-act correspondence, grouping, and contrast easier to represent and encourage argument reuse across dialog acts.This supports more atomic definitions of dialog acts and arguments than flat MRs.Source: §2.2 · p. 3

• The weather domain was chosen for the complexity of long forecast summaries that reason over disjoint information, with collection focused on date and time ranges.The dataset was collected in a conversational setup.Source: §3 · p. 3
• Crowdworkers supplied sample weather queries, which were parsed with rules to identify locations, dates, and requested attributes.The extracted information was used in the dataset creation pipeline.Source: §3 · p. 3
• The authors generated artificial forecasts from query arguments and applied automatic rules to create tree-structured MRs, including CONTRAST for opposing conditions.The forecasts were intended to mimic a system with access to a weather API.Source: §3 · p. 3
• Trained annotators wrote responses expressing the tree-structured MRs while using the user's query to make responses natural.Annotators were allowed to elide information.Source: §3 · p. 3

Our MR

• Table 2 contrasts a sample flat MR and reference with the proposed tree-structured MR.Table 3 gives the collected weather-domain ontology; Table 4 lists defined subfields for nested arguments.Source: Table 2 · p. 4, Table 3 · p. 4, Table 4 · p. 4
• The annotation process includes labeling response spans for each argument, dialog act, and discourse relation.A separate group evaluated fluency, correctness, naturalness, and annotation correctness.Source: Our MR · p. 4

3.1 Dataset statistics

• 33,493 examples make up the weather dataset, with tree-structured MRs and matching tree-structured response annotations.Responses range from length 8 to 151, averaging 40.6 tokens; vocabulary size is 1,485.Source: §3.1 · p. 4
• The weather dataset includes user queries alongside the responses, supporting studies of naturalness or relevance to the query.The passage notes potential relevance to recommendations and justifications, as well as YES and NO dialog acts.Source: §3.1 · p. 4
• The weather training set contains 25,390 examples and 11,879 unique delexicalized MRs; 1.1K test examples (35%) have unseen MRs.The test set contains 3,121 examples.Source: §3.1 · p. 4

3.2 Enriched E2E Dataset

• ~51K converted E2E examples include JOIN in ~24K (47%) and CONTRAST in 2,237 (4.3%).Tree structures were induced using slot-span alignment, Viterbi segmentations, parsing for coordinated arguments with “but,” and sentence boundaries for JOIN.Source: §3.2 · pp. 4–5

4.1 Seq2Seq with Linearized Trees

• A Seq2Seq model predicts a linearized response tree from a linearized tree MR, with non-terminals marking discourse relations and arguments and their span boundaries.The predicted tree provides supervision on MR-response alignment and is used to verify response structure.Source: §4.1 · p. 5

4.2 Constrained Decoding

• During beam search, opening nodes must be valid children not already used in the subtree, and closing brackets or EOS require all parent children to be covered, directly or by ellipsis.The decoder masks constraint-violating candidates so they cannot continue through the beam.Source: §4.2 · pp. 5–6
• Because non-JOIN children may occur in any order, decoding tracks multiple possible alignments and removes states that violate constraints.JOIN children must appear in order; the authors report that alignment nondeterminism quickly resolves in practice.Source: §4.2 · p. 6
• The constraints do not prevent every hallucination: the model can skip a disallowed slot annotation yet generate words expressing that slot, especially for unusual inputs.Figure 1 illustrates outputs that pass or fail tree constraints, including a failure with a missing CONTRAST node and illegal INFORM children.Source: Figure 1 · p. 5, §4.2 · p. 6

5.1 Experimental Setup

• The setup uses aligned references to filter unexpressed input information while retaining repeated arguments elided for redundancy, and evaluates tree structure with a metric that accounts for grouping and ellipsis.Human ratings separately assess grammaticality and semantic correctness.Source: §5.1 · pp. 6–7

5.2 Constrained Decoding Analysis

• Human evaluation measures correctness on full test sets and on challenging subsets: E2E examples with contrasts and weather examples with CONTRAST or JUSTIFY, including unseen argument combinations.The experiments also filtered examples where S2S-CONSTR failed to generate a valid response from model computations.Source: §5.2 · p. 7

5.3 Results

• Across E2E and weather, S2S-CONSTR significantly improves tree accuracy over S2S-TREE, while tree-aware models beat S2S-TOKEN and S2S-FLAT on human correctness; Table 7 reports these results.S2S-CONSTR also fails to generate a valid weather response for about 1.5% of test examples.Source: §5.3 · p. 7, Table 7 · p. 8

5.5 Data Efficiency and Generalizability

• S2S-CONSTR exceeds 90% tree accuracy with 2K samples and 95% with 5K on both datasets.Figure 2 plots test-set performance as the training-set size varies.Source: Figure 2 · p. 8, §5.5 · p. 8

5.3 Results

• Tree-based models show a slight grammaticality drop on weather, and 22 of 35 tree-accuracy/human-evaluation disagreements involved missing information in references, often attributed to annotation noise.The analysis also found annotator confusion in 6 of 35 cases (17%).Source: §5.3 · pp. 7–8

5.4 Diversity Metrics

• Tree-structured models show higher diversity than neural baselines without diversity considerations, alongside improved BLEU scores; Table 8 reports the E2E diversity metrics.The authors note that this pattern suggests discourse-relation inputs can increase diversity without losses on automatic metrics.Source: §5.4 · p. 8, Table 8 · p. 9

5.5 Data Efficiency and Generalizability

• With 5% compositional E2E training data, E2E-WEATHER reaches 76% tree accuracy versus 53.72% for E2E-only; final E2E-WEATHER accuracy is also higher.Without compositional E2E examples, neither model produces valid sequences.Source: Figure 3 · p. 9, §5.5 · p. 9

6 Related Work

• Moryossef et al. (2019) use sentence plans for input-structure control but do not address discourse relations or constrained decoding; Puduppully et al. (2018) combine content planning and realization for structured tables without discourse structure.Reed et al. (2018) is identified as the most closely related work.Source: §6 · p. 9
• Unlike neural checklist coverage of ingredient lists, this approach constrains ordering and grouping of tree structures; grammar-based realizers can prevent hallucination entirely but require an explicit grammar.The paper also contrasts constrained decoding in machine translation, which enforces specific output words rather than tree-structure constraints.Source: §6 · p. 9

7 Conclusions

• Tree-structured meaning representations and constrained decoding improve expressiveness and semantic correctness while enhancing data efficiency and generalizability.The paper also releases a weather dataset and an enriched E2E dataset with tree-structured representations.Source: §7 · p. 9

A Weather Forecast Generation

• Forecasts were sampled at hourly or daily granularity according to the requested date range, with plausible weather values and randomly selected additional conditions.The authors prioritized creating complex meaning representations over precise forecast values.Source: A Weather Forecast Generation · p. 12

B Tree-Structured Weather MR Creation

• Weather meaning representations encode errors, aggregation, contrasts, yes/no answers, and recommendations with justifications using dialog acts and discourse relations.Similar weather attributes are grouped with JOIN; recommendations are the nucleus and their justifications the satellite of JUSTIFY.Source: B Tree-Structured Weather MR Creation · p. 12

C Dataset Creation Quality

• Filtering removed 3,404 of 37,162 examples rated below 3 on any of four quality dimensions.Responses were double annotated on a 1–5 scale for fluency, correctness, naturalness, and annotation correctness.Source: C Dataset Creation Quality · p. 12

D Data Preprocessing

• Delexicalization replaces selected argument values with placeholders in both inputs and outputs, reducing vocabulary size and sparsity because these values are often copied verbatim into responses.The authors did not explore a copy mechanism for these arguments.Source: D Data Preprocessing · pp. 12–13
• Numerical arguments delexicalized include temperature-related values, precipitation chance, and date components such as day, month, and year.Source: D Data Preprocessing · p. 13
• Named entities delexicalized include restaurant names and landmarks in E2E, plus cities, regions, countries, and date weekdays.Source: D Data Preprocessing · p. 13

E Additional Experiments

• Reranked S2S-TREE achieved tree accuracy of 97.6% on E2E and 95.4% on weather.Source: E Additional Experiments · p. 13
• For E2E, RNNG filtering raised correctness on filtered test sets from 85.89% to 87.44%.Source: E Additional Experiments · p. 13
• For weather, RNNG filtering raised correctness on filtered test sets from 91.82% to 93.84%.Source: E Additional Experiments · p. 13

F Human Evaluation of Models

• Correctness raters compared each response against its reference rather than directly against the tree-structured meaning representation.The authors report that this increased annotation speed and agreement, while risking confusion from noisy references.Source: F Human Evaluation of Models · p. 13
• Weather responses were automatically marked incorrect when a CONTRAST or JUSTIFY meaning-representation relation lacked corresponding contrastive or justification markers.Source: F Human Evaluation of Models · p. 13
• Automatic rejection eliminated noise from 2.8% of all responses.Source: F Human Evaluation of Models · p. 13

G Model Training Details

• The constrained-decoding experiments use the S2S-FLAT seq2seq model with 300-dimensional GloVe embeddings, 0.2 dropout, 128-dimensional encoder and decoder hidden states, and beam size 10.Training uses Adam at learning rate 0.002, reduced by a factor of 5 when validation loss stops decreasing.Source: G Model Training Details · p. 13

H Constrained Decoding Algorithm

• The algorithm builds parent-child maps, tracks valid nonterminal nodes and ellipsis options, and initializes decoding state with node coverage and elided-node records.A child is considered covered when it is covered directly or through ellipsis; nodes already elided cannot cover other nodes through ellipsis.Source: H Constrained Decoding Algorithm · p. 14
• Structural tokens are accepted only when they match an available child or close a node whose children are covered directly or through ellipsis.Accepted tokens update the decoding states; the score-masking function returns a score only when the token is accepted.Source: H Constrained Decoding Algorithm · p. 14