GLUCOSE: GeneraLized and COntextualized Story Explanations

1 Introduction

• Humans connect story events to background knowledge to infer what happened and why, while AI systems remain far from comparable commonsense reasoning.The paper frames knowledge acquisition and incorporation into AI systems as two bottlenecks.Source: §1 · p. 1
• GLUCOSE captures ten dimensions of implicit causal explanation about story events and states, with each specific statement paired with a general inference rule.The dimensions include causes and effects that existing resources largely miss.Source: §1 · pp. 1–2
• A multi-stage crowdsourcing platform produced more than 670K annotations in children’s stories, and the explanations extend substantially beyond existing knowledge resources.Source: §1 · p. 2
• Pretrained neural models perform poorly on the evaluation task, but models fine-tuned on GLUCOSE generate commonsense explanations that rival humans’ on novel stories.The curated test set uses stories excluded from training.Source: §1 · p. 2

2 Related Work

• ConceptNet [Speer et al., 2017] connects phrases through semantic relations, but its knowledge is mostly taxonomic, can be noisy, and lacks context for inference.The passage gives “kill requires eat breakfast” as an example of a context-free relation that is difficult to interpret.Source: §2 · p. 3
• ATOMIC [Sap et al., 2019] consists of 877K textual if-then descriptions, each describing a likely cause/effect of one of 24K+ events, but it does not encode xIntent–xEffect dependencies.Source: §2 · p. 3
• ATOMIC’s person-centric relations omit agentless events and many entities and causal changes covered by GLUCOSE, including changes in location.Source: §2 · p. 3

3 The Knowledge Model of GLUCOSE

• Each story is explained across ten causal dimensions, with both a specific statement and a general rule for each dimension.Source: §3 · p. 3, §3.3 · p. 4

3.1 Causal Dimensions of Explanation

• The ten dimensions cover causes and effects before and after an event or state, including event chains, motivations, emotions, naive psychology, and attribute changes.Pilot studies aimed to make dimensions intuitive and distinguishable while minimizing overlap and maximizing worker agreement.Source: §3.1 · p. 4

3.2 Semi-structured Inference Rules

• Semi-structured rules use syntactic slots for an antecedent, connective, and consequent, balancing free-text expressivity with constraints that support processing and evaluation.Indexed variables refer to the same entities on both sides, and relative-clause attributes can further specify them.Source: §3.2 · p. 4

3.3 Generalized and Contextualized

• Each explanation links a context-relevant statement to a general rule, expressing a mini-theory intended to support generalization and causal reasoning.Source: §3.3 · p. 4

4.1 Data Acquisition Platform

• The three-stage pipeline qualifies workers, collects all ten dimensions per story-sentence pair, and has an expert review submissions and provide feedback.The same workers answer all dimensions for a pair; each must score at least 90% on the qualification test.Source: §4.1 · p. 5

4.2 Dataset Composition and Statistics

• Stories were sampled from the estimated 5–8 target-age group, but vocabulary-based age estimates do not ensure all content is appropriate for that group.Source: §4.2 · p. 5
• Dimensions 1 and 6 account for 18.1% and 16.4% of rules, while dimensions 9 and 10 are most often marked not applicable.Source: §4.2 · p. 5, Figure 1 · p. 5
• In sampled specific rules, at least one annotator contributed non-story-content inferences in 66% of dimension 1 samples and 63% of dimension 6 samples.These included likely follow-on events and world knowledge about story entities.Source: §4.2 · p. 5

4.3 Comparison to Other Resources

• The resource comparison maps GLUCOSE dimensions to related resource relations, then filters verb-matched entries for more than 50% lexical overlap.Because exact overlap cannot be quantified automatically, the reported overlap is treated as a ceiling.Source: §4.3 · pp. 5–6
• Table 3’s ceiling-overlap analysis indicates GLUCOSE captures extensive commonsense knowledge unavailable in existing resources, though some ATOMIC event knowledge is absent from its stories.GLUCOSE’s dimensions encompass ATOMIC’s nine inference categories, but some ATOMIC events may not appear in the GLUCOSE stories.Source: Table 3 · p. 6, §4.3 · p. 6

• For a story, sentence, and dimension, the task asks models to provide an explanation in both specific and general forms.Source: §5 · p. 6
• The curated test set contains 500 story/sentence pairs, retaining dimensions with three-worker agreement and round-robin BLEU above 0.75 before manual error removal.Each pair has 1–5 dimensions answered.Source: §5 · p. 6
• Human judges rate shuffled system answers on a four-point correctness scale, which is mapped to scores from 0 to 3 and averaged.Source: §5 · p. 6
• Spearman = 0.891, Pearson = 0.855, and Kendall’s τ = 0.705 (all p-value < 0.001) show strong human–BLEU correlation on the test set.The authors attribute the correlation to semi-structured rules that constrain variation and a calibrated human-evaluation interface.Source: §5 · p. 6

6 Models

• Models train on the initial 440K annotations from 3,360 stories, excluding entries whose context story overlaps with a test instance.Source: §6 · p. 7

6.1 Pretrained Language Model (PT-LM)

• PT-LM prompts GPT-2 with the story, target sentence, and dimension trigger, using constrained decoding when a dimension template limits allowable words.Source: §6.1 · p. 7

6.2.1 Language Models

• One-sided models generate only the target side of a rule, while Full-LM generates the complete rule; separate models for each dimension performed much worse.Source: §6.2.1 · p. 7

6.2.2 Encoder-Decoder Model (Enc-Dec)

• The encoder-decoder jointly predicts specific and general rules using a 770M-parameter T5 model, with the dimension and marked story sentence as input.Source: §6.2.2 · p. 7

7 Results and Discussion

• Enc-Dec uniformly outperforms the other evaluated models, with the comparison reported across GLUCOSE evaluation results.Table 4 reports human evaluation scores and BLEU scores for general and specific rules.Source: §7 · p. 7, Table 4 · p. 8
• Enc-Dec performs competitively with humans in many dimensions, while its best model still trails human performance on specific statements and general rules.The authors identify a consistent performance gap as headroom for improving generalizable commonsense reasoning.Source: §7 · pp. 7, 9
• Performance is worst on general rules for dimensions 5 and 10, which have the fewest training points and the most diverse content.The Table 4 caption notes that dimensions with nonsignificant performance differences strongly correlate with those having the least data.Source: §7 · p. 8, Table 4 · p. 8
• Generating specific statements is easier for the best models than generating general rules, which additionally requires appropriately generalizing the paraphrase and rule components.For general rules, the resulting statement must remain generally valid.Source: §7 · p. 8
• Generalization can require context-sensitive transformations, such as replacing “turning his bike” with “moves away from Something (that is dangerous).”For the lasagna example, the generalization differs by dimension: one replaces the person and action, while another adds a food-type constraint.Source: §7 · p. 9
• The evaluation scores each sentence and dimension independently, so it does not test consistency across dimensions or across sentences.The authors describe joint prediction across a story as a more challenging task that could potentially yield more accurate downstream predictions.Source: §7 · p. 9

8 Conclusions

• GLUCOSE contains implicit commonsense knowledge encoded as explanatory mini-theories grounded in narrative contexts and organized into ten causal dimensions.The dimensions are inspired by cognitive psychology.Source: §8 · p. 9
• Training on GLUCOSE data significantly improves model performance on unseen stories, and the paper reports that these results support using carefully crafted data with pretrained neural architectures.The authors also report that GLUCOSE rules capture knowledge absent from existing resources and pretrained models.Source: §8 · p. 9
• The paper defines an evaluation task for predicting specific statements and general inference rules from a story, sentence, and dimension.The authors describe a doubly vetted test set, a human-judgment platform, and BLEU as a validated automated metric.Source: §8 · p. 9
• The authors hope the models17 will enable the AI research community to explore effective approaches for incorporating commonsense reasoning capabilities into various downstream tasks.Source: §8 · p. 9

Semi-structured Inference Rules

• Each story-specific statement and general rule follows an antecedent-connective-consequent template, with antecedent and consequent built from subject, verb, object, and preposition slots.Some slots have predefined options to shape the semantics of the collected knowledge.Source: Semi-structured Inference Rules · p. 13
• Some predefined slot options allow custom entries, creating soft constraints, while others impose hard constraints.General-rule subject and object slots also have predefined choices such as SomeoneA and Some PeopleC.Source: Semi-structured Inference Rules · p. 13

Comparison to Other Resources

• The comparison with ConceptNet and ATOMIC uses a best-effort mapping between GLUCOSE dimensions and relations in those resources.Source: Comparison to Other Resources · p. 13

Appendix B: Data Collection Pipeline

• Workers first take a qualification test and must score at least 90% on ten multiple-choice questions before entering the main task.The qualification questions assess slot use, generalization level, and causal and temporal understanding; generalization is identified as the most difficult and important aspect.Source: Appendix B · p. 13
• In the main task, qualified workers address all ten dimensions for a story and sentence, with the option to mark a dimension not applicable and skip it.The UI presents structured rule slots when workers choose to answer.Source: Appendix B · p. 13
• Expert review monitors a percentage of submissions and average dimensions completed, adjusts worker ratings, and can disqualify workers when quality risks degrading the dataset.Workers can advance to higher ratings and top-rated batches as proficiency improves.Source: Appendix B · p. 13

ATOMIC-trained Model

• The ATOMIC-trained GPT-1 model predicts only general rules and performs significantly worse than even the worst-performing PT-LM model.The authors attribute this expected gap to little overlap between ATOMIC and GLUCOSE knowledge.Source: ATOMIC-trained Model · p. 16

K-Nearest Neighbor (KNN)

• The KNN baseline predicts general rules by retrieving similar training instances, but performs significantly worse than the worst-performing PT-LM model.The authors say this performance highlights the importance of generalizing beyond training data.Source: K-Nearest Neighbor (KNN) · p. 16

Pretrained Language Model (PT-LM)

• PT-LM prompts GPT-2 directly to predict GLUCOSE dimensions using natural-language templates.The appendix specifies a 774M-parameter GPT-2 model and top-K sampling with K = 15.Source: Pretrained Language Model (PT-LM) · p. 16, Table 8 · p. 18

1S-LM and Full-LM

• The model uses the exact same underlying model as PT-LM and was fine-tuned on 8 NVIDIA Tesla V100 GPUs for 10K steps.Source: 1S-LM and Full-LM · p. 16

Enc-Dec Model

• Non-contextual variants remove the story from the input and underperform their contextual counterparts.The authors say this further validates the importance of context for commonsense inference.Source: Enc-Dec Model · p. 17