Embedding Models for Episodic Knowledge Graphs

1. Introduction

• Semantic knowledge graphs describe current facts, whereas episodic graphs preserve event context over time.The paper frames these graph types as counterparts to semantic and episodic memory, respectively.Source: §1 · pp. 2–3
• The paper extends leading static knowledge-graph embedding models to temporal graphs to represent episodic data and generalize to new facts.The models include constrained Tucker, DistMult, RESCAL, HolE, and ComplEx.Source: §1 · p. 2
• The experiments assess generalization to missing temporal facts and whether factorized knowledge graphs can serve as explicit memory.They use ICEWS, describing interactions between nations over several years, and GDELT, which monitors worldwide news media for more than 30 years.Source: §1 · p. 2
• The authors propose that semantic memory can be derived from episodic memory by marginalization.They study this hypothesis using ICEWS, which contains many events with start and end dates.Source: §1 · pp. 3–4
• The tensor representation distinguishes semantic triples from episodic quadruples by adding a timestamp to episodic facts.The paper describes semantic knowledge graphs as 3-way tensors and episodic knowledge graphs as 4-way tensors.Source: §1 · p. 4

Tucker

• Episodic Tucker uses a four-dimensional core tensor, extending the semantic Tucker decomposition while retaining unique entity representations independent of subject or object role.The paper assumes equal latent ranks for entities, predicates, and time indices in its experiments for fair model comparison.Source: §2 · pp. 7–9

Tree

• Tree is proposed to address episodic Tucker’s training cost by replacing its four-way core with two three-dimensional tensors joined through predicate-related cores.Its two subtrees partition subject and time from object and an auxiliary time representation.Source: §2 · pp. 8–9

ConT

• ConT generalizes RESCAL to episodic tensors by contracting time into a three-way core G_t, reducing complexity to approximately r^3; the authors attribute its superior modeling to G_t’s representational flexibility.G_t is a three-dimensional core tensor for each timestamp, analogous to RESCAL’s relation-specific matrix.Source: §2 · p. 9

Compositional models

• To address possible overfitting from the three-dimensional cores in Tree and ConT, the paper turns to episodic generalizations of compositional models, including DistMult, HolE, and ComplEx.DistMult constrains entities to unique representations and is a Tucker special case with a diagonal core.Source: §2 · pp. 9–10

HolE

• HolE uses holographic reduced representations to store and retrieve predicates, while episodic HolE is intended to retrieve timestamps from subject, predicate, and object representations.Source: §2 · p. 10
• A compositional operator motivates projecting episodic memory to semantic memory, while higher-order circular correlation or convolution increases retrieval inaccuracy.Source: §2 · p. 10

ComplEx

• ComplEx models symmetric and antisymmetric relations; HolE is its conjugate-symmetric special case [19], so ComplEx has more degrees of freedom.Source: §2 · p. 10
• The paper defines an episodic complex embedding to remain consistent with episodic HolE.Source: §2 · p. 11

GDELT inference results

• On GDELT, tensor models outperform compositional models on entity and predicate prediction; ConT performs best on entity tasks, while Tucker performs better on predicate tasks.ConT’s superior Hits@1 for entity prediction indicates that the dataset is unbalanced, with events involving some entities easier to fit.Source: §3 · p. 13, Table 4 · p. 14

ICEWS inference results

• On ICEWS, tensor models outperform compositional models on both missing-entity and predicate inference; ConT’s superior entity Hits@1 again indicates that events involving certain entities dominate.The passage attributes this pattern to dataset imbalance.Source: §3 · pp. 14–15, Table 5 · p. 14

ICEWS rare-event recollection

• On ICEWS (rare), which contains temporal events occurring fewer than three times, ConT outperforms other compositional models in recollection, underscoring the value of its large-dimensional time representation.For ConT, the real time-latent dimension is r̃^3 after flattening G_t; this flexible representation could compress almost all semantic triples occurring at an instance.Source: §3 · p. 15

• Figure 4 shows filtered MRR scores versus rank for entity and timestamp recollection on the ICEWS rare training dataset.Source: Figure 4 · p. 16, §3 · p. 16

4. Semantic Memory from Episodic Memory with Marginalization

• The proposed projection marginalizes time by summing time representations to construct current semantic facts from an episodic knowledge graph.Probabilistic tensor models permit this integration over time representations.Source: §4 · p. 17
• Because events with start and end timestamps cannot be integrated directly into current semantic facts, the projection subtracts end-time representations to exclude terminated events.The method assigns separate latent time representations to event starts and ends.Source: §4 · p. 17
• The projection experiment trains HolE, DistMult, ComplEx, ConT, and Tucker on ICEWS start-time and end-time episodic tensors and a semantic tensor, then evaluates recall and AUPRC across ranks.For end-time training, subject, predicate, and object representations are fixed from start-time training, assuming they do not change during a consecutive event.Source: §4 · pp. 17–18
• Only Start-End marginalization yields reasonable episodic-to-semantic projection; ConT performs best, with recall saturating near rank 15, while HolE has sizable errors, especially at small ranks.The authors attribute HolE’s errors to higher-order encoding noise.Source: Figure 5 · p. 18, §4 · p. 20
• The experiment supports viewing semantic memory as long-term storage for episodic memory in which exact timing information is lost.Source: §4 · p. 20
• Because ConT lacks a direct semantic counterpart, the projection experiments use RESCAL semantic results, consistent with viewing ConT as a high-dimensional episodic generalization of RESCAL.Source: §4 · pp. 20–21
• ConT has the best projection performance: its projected recall scores on the genuine dataset are much higher than on the false semantic dataset, and only its Start-End projection achieves similar results to the corresponding semantic model.Table 7 reports filtered and raw Hits@10 scores for the episodic-to-semantic projection.Source: §4 · p. 21, Table 7 · p. 21
• Experiments compare Start (Eq. 9) and Start-End (Eq. 10) projections on genuine and false semantic ICEWS datasets, with direct modeling of genuine semantic data as a reference.Projection hyperparameters are selected by monitoring the difference between Hits@10 recall on the genuine and false datasets.Source: §4 · p. 21
• Projected compositional models can identify whether a semantic triple was ruled out before the last timestamp, but cannot provide good inference results on genuine data.Source: §4 · pp. 21–22

5. Conclusion

• The paper presents its models as the first mathematical models for declarative semantic and episodic memory functions.Source: §5 · p. 22
• The authors generalize Tucker, RESCAL, HolE, ComplEx, and DistMult to four-dimensional temporal or episodic knowledge graphs and introduce Tree and ConT as RESCAL generalizations.Source: §5 · p. 22
• ConT has superior overall performance, with high-dimensional time representations important for sparse episodic tensor reconstruction and generalization.The projection to current semantic facts requires considering both the starting and terminal points of consecutive events.Source: §5 · p. 22
• The authors propose that temporal knowledge graph embeddings may model cognitive episodic memory and that semantic memory can be generated from it by marginalization.Source: §5 · p. 22
• The ICEWS experiments indicate that current semantic facts can be derived from the episodic tensor only by a projection considering both starting and terminal points of consecutive events.Source: §5 · p. 22