STEVE-1: A Generative Model for Text-to-Behavior in Minecraft

• STEVE-1 combines a VPT policy trained toward MineCLIP visual goals with a prior that translates text instructions into MineCLIP visual embeddings.Source: §1 · p. 2, Figure 1 · p. 2
• The paper proposes instruction-tuning generative behavior models without relying on a large dataset of expensive instruction labels.Source: §1 · pp. 1–2
• 12 of 13 early-game goal-conditioned control tasks are completed robustly using only $60 of compute.Source: §1 · p. 2
• STEVE-1 robustly completes 12 of 13 goal-conditioned control tasks in Minecraft’s early-game evaluation suite, while basic prompt chaining dramatically improves long-horizon crafting and building performance.Source: §1 · p. 2
• Classifier-free guidance and unCLIP are identified as essential for strong sequential decision-making performance, alongside pretraining and data scaling.Source: §1 · p. 2

Minecraft as a Test-bed for AI

• Minecraft provides a complex visual test bed for open-ended instruction-following with only low-level mouse and keyboard actions.Source: §2 · p. 3
• Unlike prior Minecraft agents trained on fixed tasks or requiring further MineCLIP-reward reinforcement learning, STEVE-1 learns goal-reaching behavior self-supervised from large gameplay data.Source: §2 · p. 3
• VPT supplies a behavior foundation model pretrained on 70k hours of Minecraft gameplay, while MineCLIP aligns Minecraft videos with text captions.Source: §2 · p. 3

Foundation Models for Sequential Decision-Making

• Prior sequential decision-making work uses supervised reinforcement learning and hindsight relabeling, whereas STEVE-1 relabels goals with future trajectory-segment MineCLIP embeddings.Source: §2 · p. 3

Text-Conditioned Generative Models

• STEVE-1 adapts unCLIP’s text-to-embedding prior and classifier-free guidance from text-conditioned image generation to behavior generation.Source: §2 · p. 3

3 Method

• The method decomposes text-conditioned behavior into a policy p(τ|zτgoal) and a prior p(zτgoal|y), producing trajectories through an intermediate MineCLIP goal latent.Source: §3 · p. 4

3.1 Policy

• The policy fine-tunes VPT by conditioning its visual processing on MineCLIP goal embeddings and autoregressively predicting actions.Source: §3.1 · pp. 4–5
• Packed hindsight relabeling samples future goal timesteps and assigns each goal to the preceding interval, creating self-supervised goal-conditioned action data.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 Prior

• The prior is a CVAE trained to map frozen MineCLIP text representations to visual-goal embeddings using a small text-labeled dataset.Source: §3.2 · p. 5

3.3 Datasets

• Policy training uses 54M gameplay frames from contractor and VPT-generated data, while prior training uses 2,000 human-labeled instructions plus MineCLIP and API-based augmentation.Source: §3.3 · p. 5

3.4 Inference

• At inference, classifier-free guidance combines conditional and unconditional policy logits, with λ controlling the strength of goal-following behavior.Source: §3.4 · pp. 5–6

3.5 Evaluation

• Evaluation combines programmatic metrics such as travel distance and item counts with minimum MineCLIP cosine distance for broader textual and visual goals.Source: §3.5 · p. 6

4.1 Training Setup

• Training uses the official VPT codebase with distributed data parallelism on four A40 GPUs for 160M frames, or just under three dataset epochs.The learning rate is 4e-5, while other hyperparameters match Baker et al. [5].Source: §4.1 · p. 6

4.2 Performance on Textual and Visual Goals

• STEVE-1’s visual-goal agent achieves the shortest MineCLIP goal distance on 11 evaluation tasks except for confusion between digging and dirt and between swimming and going underwater.The evaluation uses programmatic metrics and automatic MineCLIP embedding distances across short-horizon tasks.Source: Figure 3 · p. 7, §4.2 · p. 7
• STEVE-1 collects 75× more dirt, 4.9× more wood, and 22× more seeds than the unconditional VPT agent, while traveling 4.3× farther on relevant text instructions.Against the text-conditioned VPT agent, it collects 3.3× more dirt, 4.4× more wood, and 8.1× more seeds, while traveling 2.2× farther.Source: §4.2 · p. 7

4.3 Prompt Chaining

• Prompt chaining raises success to 50% for building a tower and 70% for crafting wooden planks, whereas directly prompting with the final tasks produces near-zero success.The agent first gathers the required materials, then switches to the construction or crafting prompt.Source: §4.3 · p. 8, Figure 4 · p. 8

4.4 Scaling

• Scaling produces task-dependent behavior: seed collection and travel distance stop improving significantly past 20M frames, while log and dirt collection improve rapidly after 60M frames.Chained-prompt performance steadily increases with more data, whereas unchained multi-step tasks remain challenging throughout training.Source: §4.4 · pp. 8–9

4.5 What Matters for Downstream Performance?

• Pretrained weights outperform training from scratch, with rl-from-foundation-2x generally performing best after fine-tuning and pretrained agents succeeding on tree chopping that scratch training cannot handle.The comparison uses models fine-tuned on 100M frames from four initializations.Source: §4.5 · p. 9, Figure 5 · p. 10
• Classifier-free guidance improves collection performance, with 7.5× more dirt and 15× more wood than λ = 0 and a maximum around λ = 5.0 to λ = 7.0.Performance declines after the guidance parameter exceeds this range.Source: §4.5 · p. 9, Figure 5 · p. 10
• Longer, more specific prompts improve STEVE-1’s performance, producing 1.6× more wood, 2× more dirt, and 3.3× more seeds.The experiment changes a simple “get dirt” instruction into a more detailed behavioral specification.Source: §4.5 · p. 9, Figure 6 · p. 10

5 Limitations and Conclusion

• STEVE-1 uses pretrained VPT and MineCLIP models to follow open-ended Minecraft instructions with low-level controls and raw pixels, completing 12 of 13 early-game tasks for $60 of compute.The authors report that it far outperforms previous baselines.Source: §5 · p. 10
• STEVE-1 is mostly proficient at short-horizon tasks, while longer-horizon tasks remain difficult despite prompt chaining's promise.The authors identify improving long-horizon performance as future work.Source: §5 · p. 10

A Broader Impact

• Pretraining biases from large internet datasets may persist after instruction-tuning and create societal risks if not addressed carefully.The paper also discusses potential benefits for helpful agents and AI democratization.Source: A Broader Impact · p. 17

B.1 Goal Misgeneralization

• STEVE-1 can overgeneralize visual instructions, although more training data appears to reduce this tendency.The authors attribute powerful generalization partly to MineCLIP embeddings, especially when exact nearby items are unavailable.Source: §B.1 · p. 17

B.2 Random Selection of Hindsight Goals

• The method randomly selects future episode timesteps as hindsight goals to maintain simplicity and diverse, unbiased short-horizon goal coverage.The authors leave semantically informed timestep selection for future work.Source: §B.2 · p. 17

B.3 Difference in Performance Between Text and Visual Goals

• Visual goals often outperform text goals converted through the prior because text may be less accurately represented and is inherently more ambiguous.More prior data, stronger architectures, observation context, and detailed prompts could narrow this gap.Source: §B.3 · p. 17

B.4 Challenges in Evaluating STEVE-1

• Capability evaluation depends strongly on preconditions, prompt engineering, and the diversity and difficulty of chosen tasks.The authors describe open-ended generalist-agent evaluation as an open research problem.Source: §B.4 · pp. 17–18
• STEVE-1 cannot currently accomplish several long-horizon or complex crafting tasks, including obtaining a diamond, building a house, and crafting an enchanting table.The paper calls for more sophisticated evaluation methods spanning short- and long-horizon tasks.Source: §B.4 · p. 18
• STEVE-1 and VPT cannot currently be tested in MineDojo because their action spaces are incompatible.Bridging this incompatibility is identified as beneficial for generalist-agent research.Source: §B.4 · p. 18

B.5 Towards Improved Long-Horizon Performance

• Hindsight goals extend at most 200 timesteps, making longer tasks out-of-distribution and creating a trade-off between long-horizon gains and short-horizon performance.The authors suggest scaling, reinforcement learning, or automatically generated prompt chains as possible directions.Source: §B.5 · p. 18

B.6 Applying the STEVE-1 Approach to Other Domains

• The STEVE-1 method is not specific to Minecraft and could be investigated in other domains once suitable behavioral and visual-language foundation models exist.The paper names robotics, video games, and the web as potential application areas elsewhere.Source: §B.6 · p. 18

C Additional Ablations

• Additional ablations compare classifier-free guidance, text augmentation, VAE variants, chunk sizes, and other design choices with programmatic evaluation metrics.Source: C Additional Ablations · p. 18

C.1 Classifier-Free Guidance During Training

• 2–3× better inventory-collection performance is achieved by the guidance model at λ = 3, although travel-distance performance is similar across variants.The no-guidance model slightly outperforms guidance at λ = 0, but cannot use λ > 0 at inference.Source: §C.1 · pp. 18–19, Figure 7 · p. 19

C.2 Text Augmentation

• Visual embeddings with the CVAE outperform directly using MineCLIP text embeddings in the other text-augmentation baselines.Even with augmentation, only around 2% of the dataset contained text embeddings.Source: §C.2 · p. 19, Figure 8 · p. 20

C.3 VAE Variants

• Augmenting the CVAE prior dataset helps dirt and seed collection but slightly hurts wooden-log collection and travel-distance metrics.The comparison uses the same policy with conditional scale λ tuned separately for each CVAE variant.Source: §C.3 · p. 19, Figure 9 · p. 20

C.4 Chunk Size

• A max_btwn_goals chunk size around 200 performs best, while larger values introduce noise and reduce performance.The ablation varies max_btwn_goals over [100,200,300,400] with min_btwn_goals fixed at 15.Source: §C.4 · p. 20, Figure 10 · p. 20

C.5 Generalization to Novel Text Instructions

• Removing every instruction containing “dirt” or “dig” from training still leaves STEVE-1 able to dig holes and obtain dirt.The removed instructions represented around 10% of the instruction set, and performance slightly decreased across tasks because the prior-training set was smaller.Source: §C.5 · p. 21, Figure 11 · p. 21
• The generalization result is attributed to MineCLIP providing most text understanding, while the prior learns a mapping from text embeddings to visual embeddings.The policy follows instructions in visual MineCLIP space, the MineCLIP text encoder aligns text and visual embeddings, and the prior translates between them.Source: §C.5 · p. 21

D.1 Gameplay Dataset

• The gameplay dataset combines 7,854 contractor episodes with 2,267 episodes generated by pretrained VPT agents.Agent switching and random spinning were added to increase data diversity and downstream steerability.Source: §D.1 · pp. 21–22

D.2 Text-Video Pair Dataset

• 2,000 human-labeled trajectory segments contain only 32,000 labeled frames, or 0.06% of the full dataset.Each segment contains 16 video frames, and the labeled data represents 27 minutes of gameplay.Source: §D.2 · p. 22

D.3 Prompt Design

• Longer prompts are constructed by combining many task-related terms, and this text-to-image-inspired prompt design works well in STEVE-1.Short prompts come from prior literature or initial trial prompts.Source: §D.3 · p. 22

E.1 Policy Training

• Packed hindsight relabeling samples an episode segment, changes goals at randomly selected timesteps, and fills the episode with embeddings from corresponding future goal segments.This creates multiple relabeled sequences within one sampled sequence.Source: §E.1 · p. 22

E.2 Prior Training

• The prior is a conditional VAE that models visual embeddings conditioned on frozen MineCLIP text embeddings.Its role is to translate text representations into visual goal embeddings for the policy.Source: §E.2 · p. 22
• 50,000 automatically mined text-video pairs are added to 2,000 hand-labeled examples for prior training.The mined pairs come from 8,000 GPT-generated labels and nearest visual-embedding searches across sampled gameplay episodes.Source: §E.2 · p. 23
• Embedding selection uses a timestep offset so automatically discovered clips are more likely to show follow-through when breaking blocks.This addresses clips where the agent hits a block but does not complete the break.Source: §E.2 · p. 23

E.3 Training Costs

• $60 covers renting one 8xA10g AWS spot-instance node for 12 hours.The estimate uses instance prices from May 2023.Source: §E.3 · p. 24

F.1 MineCLIP Evaluation

• MineCLIP evaluation is conducted on both text and visual prompts, with results reported in Figure 12.Source: §F.1 · p. 24

F.2 Steerability with Programmatic Metrics

• The programmatic comparisons use 10 trials per conditioning variant, 3000-timestep episodes, and conditional scaling λ = 7 for visual goals and λ = 6 for text goals.Inventory counts record the maximum observed inventory, which is a lower bound on items collected.Source: §F.2 · p. 24
• STEVE-1 is more steerable than unconditional and text-conditioned VPT, collecting more relevant items under matching instructions and outperforming VPT despite half-length rollouts.For unrelated instructions, it pursues the measured item less than the unconditional agent.Source: §F.2 · p. 24

F.3 Prompt Chaining Visualization

• Across 11 text MineCLIP tasks, STEVE-1 usually achieves the shortest episode-to-goal distance when prompted appropriately.The MineCLIP metric records the minimum cosine distance between the goal embedding and video embeddings during each episode.Source: Figure 12 · p. 25
• Across 11 visual MineCLIP tasks, appropriate prompts usually produce the shortest distance, with confusion between digging and dirt and between swimming and going underwater.These are the two reported exceptions in the visual evaluation.Source: Figure 12 · p. 25
• Classifier-free guidance improves programmatic performance over λ = 0, yielding 7.5× more dirt, 15× more wood, 23× more seeds, and 1.7× farther travel.The conditional scale λ is tuned separately to improve each task.Source: Figure 17 · p. 28
• Prompt chaining switches visual goals within an episode, enabling sequential behaviors such as collecting dirt before building a tower and gathering logs before crafting planks.The visualizations use 1500 timesteps for each stage.Source: Figure 18 · p. 30, Figure 19 · p. 30