Source-linked AI summary
BehaviorFormer: Discovering Behavioral Patterns via Clusterable Next-Token Prediction
Tobias Schneider, Danial Fahrner, Soren Antebi, Daniel Uedelhoven, Lennard Bodden, Sandra Halscheidt, Rafet Sifa
TL;DR
Open-ended games lack clear goals and ground-truth behavioral annotations, making behavioral pattern discovery challenging. The paper combines a next-token prediction Transformer with DCSS to learn representations and clusters jointly. In a controlled Minecraft session, it identifies 23 patterns grouped into seven categories that capture shared behavior across players.
Problem
Open-ended games lack clear goals and ground-truth behavioral labels, making behavioral pattern discovery challenging.
Source: §I · p. 1Method
Results
The controlled Minecraft session yielded 23 behavioral patterns grouped into seven interpretable categories, capturing shared behavior across players.
Source: §I · p. 1, §V · p. 4, B. Qualitative Analysis · p. 4Takeaways & Limitations
The discovered patterns refine structures present in pure next-token prediction representations into clearly separable clusters.
Source: §V · p. 4The findings come from a controlled Minecraft session; further work is needed to validate generalization across games and longer sessions.
Source: §V · p. 4
Abstract
from arXiv · showhide
Discovering behavioral patterns in open-ended games is challenging due to the lack of clear objectives or ground-truth annotations. In this work, we extend a next-token prediction Transformer with the deep clustering algorithm DCSS to learn meaningful representations with well-separated clusters. In a controlled Minecraft session with 26 players, the method discovers 23 distinct behavioral patterns that generalize across the studied players, further refining structures present in self-supervised representations into clearly separable clusters. We group the clusters into seven interpretable categories covering stationary, roaming, travel, mining, base work, logging, and farming behavior.
I. INTRODUCTION
Open-ended games lack clear goals and ground-truth behavioral labels, making manual analysis dependent on predefined features and annotations. The paper combines a Transformer with DCSS to discover shared behavioral patterns from gameplay logs, finding 23 patterns grouped into seven interpretable categories.
I. INTRODUCTION
- Open-ended games lack clear goals and ground-truth behavioral labels, complicating manual analysis based on hand-crafted features and self-reported annotations.Such approaches require predefined patterns, impose prior assumptions, and may miss unexpected behavior or complex temporal structure.Source: §I · p. 1
- Transformers capture temporal dynamics but do not explicitly model behavioral patterns, so the paper combines one with DCSS.The combined algorithm jointly learns representations and compact, well-separated clusters.Source: §I · p. 1
- The method finds 23 global behavioral patterns in a controlled Minecraft session with 26 players, grouped into seven interpretable categories.The patterns capture shared behavior across players rather than individual playstyles.Source: §I · p. 1
II. RELATED WORK
Prior player-modeling work uses hand-crafted features or labels, while self-supervised sequence models learn representations without explicit behavior labels. Deep clustering offers a way to optimize cluster structure directly alongside representations.
II. RELATED WORK
- Müller et al. classify four Minecraft behaviors with 65% accuracy, but require categories and features to be defined in advance.The passage notes that hand-crafted representations only partially capture open-ended behavior's ambiguity and temporal structure.Source: §II · p. 1
- Player profiling methods – rely on hand-crafted features, while Player2Vec [7] and Pu et al. learn from event sequences without behavior labels.Because their objectives focus on sequence modeling, cluster structure emerges only implicitly.Source: §II · p. 1
- DEC and IDEC encourage compact clusters, whereas DCSS additionally optimizes inter-cluster separation with a soft silhouette objective.The passage presents DCSS as promising for behavioral pattern discovery.Source: §II · p. 1
A. Clustering Objective
The clustering objective combines next-token reconstruction with soft silhouette clustering and entropy regularization to avoid trivial solutions. A Transformer supplies sequence representations to the clustering network, with next-token cross-entropy replacing the autoencoder’s MSE loss.
A. Clustering Objective
- The DCSS objective combines reconstruction, soft silhouette clustering, and entropy regularization to prevent trivial solutions.The mean entropy and silhouette terms operate on cluster assignment probabilities, while λ1 and λ2 balance the terms.Source: A. Clustering Objective · pp. 1–2
- The objective uses the chosen reconstruction loss, soft silhouette score, and mean assignment entropy, balanced by λ1 and λ2.The silhouette score and entropy are computed over cluster assignment probabilities.Source: A. Clustering Objective · p. 2
- A Transformer replaces DCSS’s autoencoder, feeding its final-token hidden state to the clustering network and using next-token cross-entropy instead of MSE.Source: A. Clustering Objective · p. 2
B. Dataset Construction
The dataset records open-ended behavior during a controlled six-hour Minecraft session with 26 players. Logged events and discretized movement are tokenized into gameplay sequences for Transformer training, with positional embeddings representing token order and shared time windows.
B. Dataset Construction
- A controlled six-hour Minecraft session recorded 26 players with freedom to choose activities, apart from forming groups of 2–6.The logs include block and item events, interactions with chests or furnaces, entity deaths, and player positions sampled every 0.5 seconds.Source: B. Dataset Construction · p. 2
- The tokenized vocabulary contains 532 event and movement tokens, and 11,234 sequences of 64 tokens each represent approximately 30 seconds of gameplay.Movement direction uses eight bins and magnitude categories suited to Minecraft; validation holds out 60 sequences per player, leaving 9,674 training sequences.Source: B. Dataset Construction · p. 2
- Two learned additive positional embeddings encode token index and the shared 0.5-second tick for tokens occurring simultaneously.Source: B. Dataset Construction · p. 2
C. Hyperparameter Selection
The authors sweep Transformer and DCSS hyperparameters and assess configurations across clustering quality, stability, coverage, and next-token prediction fidelity.
C. Hyperparameter Selection
- The sweep varies Transformer embedding dimension, depth, and DCSS-stage learning rate, plus cluster count, clustering-loss weight, pretraining epochs, and clustering temperature.Training uses batch size 256, a pretraining learning rate of 10^-4, two attention heads, no dropout, and early stopping with patience 10.Source: C. Hyperparameter Selection · p. 2
- Four metrics evaluate configurations: silhouette score for cluster quality, pairwise ARI for stability, cluster utilization for coverage, and validation cross-entropy loss for representation fidelity.Mean pairwise ARI is computed across 10 independent runs.Source: C. Hyperparameter Selection · p. 2
A. Hyperparameter Analysis
Hyperparameter selection trades off clustering quality and representation fidelity, while DCSS refines structure in next-token representations into more separable clusters.
A. Hyperparameter Analysis
- No configuration is optimal across all four objectives, but multi-objective optimization selects a robust configuration with near-optimal values; Table I compares one-at-a-time hyperparameter changes against it.The selected configuration uses 24 of 25 clusters.Source: A. Hyperparameter Analysis · p. 2, Table I · p. 3
- Stability and coverage depend strongly on cluster count; dimension 64 supports performance across metrics, deeper models improve fidelity, and longer pretraining improves stability and coverage.Both lower and higher learning rates show instability and inconsistent coverage.Source: A. Hyperparameter Analysis · p. 2
- The authors needed a much higher soft-silhouette loss weight than the DCSS paper’s 0.01 for good separability and stability, and slightly lower clustering temperature may prevent low cluster utilization.They suggest dataset and architecture differences may explain the higher loss weight.Source: A. Hyperparameter Analysis · p. 2
- 1%: The selected configuration’s loss increases by 1% over raw NTP while achieving highly separable clusters, high stability, and good coverage.The authors say this suggests meaningful clustering but requires further qualitative analysis.Source: A. Hyperparameter Analysis · pp. 2–3
- The NTP embedding space has cluster-like structures with unclear boundaries, whereas DCSS produces 24 well-separated clusters and refines much of the existing structure with negligible impact on representation fidelity.Some DCSS clusters contain HDBSCAN-detected sub-clusters.Source: A. Hyperparameter Analysis · p. 3, Figure 1 · p. 3
B. Qualitative Analysis
The qualitative analysis interprets 23 clusters, grouped into seven categories, as behavioral patterns shared across players. Stationary and roaming dominate the session, while focused work is relatively rare but highly active.
B. Qualitative Analysis
- The authors interpret clusters using action and motion distributions, player position and proximity, timing, and transition statistics, then assign post-hoc labels and group them into seven categories.One of 24 utilized clusters was excluded for having only six assigned samples, leaving 23 for analysis.Source: B. Qualitative Analysis · p. 3, Table II · p. 4
- Stationary and Roaming together account for over 85% of samples; Stationary alone exceeds half and has a 0.71 self-transition rate.Roaming accounts for the second-largest portion and has self-transition rates below 0.1.Source: B. Qualitative Analysis · p. 3
- Focused-work clusters—Mining, Base work, Farming, and Logging—make up less than 10% of samples but average 11 to 17 action tokens per sequence.Farming is the most action-intensive cluster overall; mining includes systematic tunneling and natural cave exploration, while base work includes near-base construction and maintenance.Source: B. Qualitative Analysis · p. 4
- Travel clusters range from short overland sprints to steady boat travel, consist almost entirely of motion tokens, and appear especially in the session’s second half.Boat travel has a 0.38 self-transition rate, the second-highest persistence.Source: B. Qualitative Analysis · p. 4
- Nearly half the clusters cover at least 24 of 26 players, and every cluster covers at least 10, indicating shared patterns rather than individual playstyles.Combat clusters are absent because relevant game events were not logged.Source: B. Qualitative Analysis · p. 4
V. CONCLUSION
The Transformer–DCSS method discovers global behavioral patterns from raw gameplay logs, producing 23 well-separated clusters across seven interpretable categories. Careful hyperparameter selection is crucial, and the discovered clusters refine structure already present in pure next-token prediction representations.
V. CONCLUSION
- Combining a Transformer with the DCSS clustering objective enables automatic behavioral pattern discovery from raw gameplay logs in open-ended games.The authors report that careful hyperparameter selection is crucial for high-quality, stable results.Source: §V · p. 4
- 23 well-separated clusters across seven interpretable categories capture shared behavior across players rather than individual playstyles.The clusters refine structures already present in pure next-token prediction representations.Source: §V · p. 4
- The authors identify validation across games and longer sessions, sequence construction, tokenization, architecture choices, and dynamic cluster-count selection as directions for further work.Source: §V · p. 4