Source-linked AI summary
DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training Data
Peter Schneider-Kamp, Jacob Nielsen, Gianluca Barmina, Kenneth Enevoldsen, Lukas Galke Poech
TL;DR
Open LLM development often depends on massive, non-permissible datasets that raise barriers for ethically sourced, open research. Mimir v1 addresses this by training a 1-billion-parameter HRM model from scratch on permissible post-training data, achieving competitive English and Math & Code performance and state-of-the-art Danish results.
Problem
Massive, non-permissible training pipelines create a prohibitive barrier for researchers developing capable models from permissible, openly sourced data.
Method
Mimir v1 is a 1-billion-parameter HRM model trained from scratch on a curated mixture of 161 datasets, including synthetic transplant datasets replacing non-permissible data.
Results
Across English, Math & Code, and Danish benchmarks, Mimir is highly competitive, with superior average Danish performance and strong results against larger models.
Takeaways & Limitations
Synthetic permissible alternatives can support small, capable models with competitive instruction-following and reasoning performance without compromising data rights.
Takeaways & Limitations
Mimir still lags behind Gemma 4 on Math & Code and has limited assistant capabilities compared with the state of the art.
Abstract
from arXiv · showhide
Current large language model development relies on massive, often non-permissible datasets, creating a high barrier for researchers committed to open-source and ethically sourced data. We introduce Mimir v1, a 1-billion-parameter language model based on the Hierarchical Reasoning Model (HRM) architecture, that is trained from scratch and delivers highly competitive performance for English and sets a new state of the art for Danish using only permissible post-training data. Trained on a mixture of 161 datasets, Mimir v1 outperforms the original HRM-Text 1B and competes with larger frontier models like Qwen 3.5 4B and Gemma 4 E2B, tested across 20 benchmarks for English, Math & Code and Danish. The model is available on the Hugging Face Hub: https://huggingface.co/danish-foundation-models/DFM-Mimir
1 Introduction
Mimir v1 addresses the computational and data-access barriers of contemporary LLM development by applying HRM-Text to train a 1-billion-parameter model from scratch on post-training data. Optimized for Danish and English, its synthetic-data approach achieves comparable or superior performance while preserving data rights and enabling small, capable, permissible models.
- Motivation: Current LLM development relies on massive multi-stage pipelines and exorbitant data volumes, demanding extensive computation and creating prohibitive barriers.The passage frames this approach as a “monolithic recipe” for modern model development.
- Approach: Mimir v1 is a 1-billion-parameter HRM trained from scratch with HRM-Text, focusing on post-training data to create a viable base model.The model uses the architecture proposed by Wang et al. [2026] and is optimized for Danish and English tasks.
- Results: Synthetic alternatives achieve comparable or superior performance without compromising data rights, supporting low-resource communities with small, capable, fully permissible models.The passage also emphasizes low training and inference requirements.
2 Datasets
Mimir v1’s training corpus combines 161 mostly freely available datasets totaling 70,479,308,606 tokens per epoch, spanning instruction, knowledge, mathematics, reasoning, and agentic-style post-training data. The corpus emphasizes Danish and English data while shifting training toward free-form answer generation rather than multiple-choice selection.
- Corpus composition: 161 datasets contribute 70,479,308,606 tokens per epoch, covering English and Danish instruction and knowledge, mathematics, and agentic-style post-training objectives.Almost all datasets are freely available on the Hugging Face Hub.
- Functional categories: Danish instruction & knowledge (22.07%), English instruction (19.26%), and Sapient mixed data (17.02%) together exceed 58% of the corpus.Math & reasoning contributes 14.8%, bringing the top four categories to 73%.
- Language distribution: English accounts for 68.5% of tokens, Danish for 24.7%, bilingual Danish–English data for 6.4%, and other bilingual translation data for 0.2%.Six of the eight functional categories are entirely English.
- Data format and task design: The corpus shifts training away from multiple-choice classification toward free-form generation, with regenerated Sapient examples asking models to produce answers instead of selecting candidates.This aligns training with evaluations emphasizing exact-match scoring, including GSM8k, MATH, and DROP.
- Dataset concentration: 66.5% of all tokens come from the ten largest datasets, while the top three contribute 38.1% and the remaining 151 datasets contribute 33.5%.The Sapient mega-repository contributes 16.9% and lærebogen 11.8%.
3 Architecture
Mimir uses the HRM-Text architecture with a 1,536-dimensional hidden size and hierarchical reasoning configured through H- and L-cycles. Its configuration also specifies attention, feed-forward, optimization, positional encoding, and normalization settings.
- Architecture: 1,536 hidden size defines Mimir’s HRM-Text architecture.The model uses 12 attention heads per layer and a feed-forward expansion factor of 4.
- Architecture: 2 H-cycles and 3 L-cycles configure hierarchical reasoning.Truncated backpropagation is limited to 5 steps, with a warmup ratio of 0.2.
- Architecture: 10,000 is the RoPE positional encoding θ value.Mimir applies Rotary Position Embedding (RoPE).
- Architecture: 10−6 is the pre-norm layer normalization ϵ value.The model applies pre-norm layer normalization.
4 Training
Mimir is trained from scratch with the Gemma-4 tokenizer and conversational chat-template conventions, using FSDP with mixed-precision computation. Training ran for 1.65M steps on eight NVIDIA B200 GPUs in just under three weeks.
- Model training setup: Mimir is trained from scratch using the Gemma-4 tokenizer, unlike HRM-Text’s custom tokenizer.The model also uses a chat template to learn structural conventions and behavioural patterns of modern conversational AI.
- Model training setup: FSDP training uses bfloat16 for computation and fp32 for gathering precision.This is described as a conventional setup.
- Training scale and duration: 1.65M steps were completed on 8 NVIDIA B200 GPUs with 180 GB HBMe3 in just under 3 weeks.The average step time was just under 1.1 seconds.
5 Results
Mimir is evaluated across English, Math & Code, and Danish benchmarks against models spanning 1B to 5B total parameters. It outperforms competitors on several English and Danish tasks, leads its weight class on selected Math & Code tasks, and uses standardized greedy-decoding evaluation.
- Evaluation scope: Mimir is compared across English, Math & Code, and Danish benchmark suites against 1B-parameter, 2–3B, and 4–5B models.The comparisons include HRM-Text, Qwen 3.5, Gemma 3, OLMo2, and Gemma 4 E2B.
- English results: Mimir outperforms all considered competitors on BoolQ, Winogrande, and DROP in the English benchmarks.
- Math & Code results: Mimir leads its weight class on GSM8K and HumanEval, ranks second overall on GSM8K, and surpasses Qwen3.5 2B on HumanEval.
- Danish results: Mimir outperforms all competitors on Danish grammatical tasks, question-answering tasks, and achieves a near-best result on Nordjylland News summarization.The cited grammatical tasks are DaLA and GEC, while the question-answering task is WikiQA.
- Evaluation protocol: All benchmarks use temperature 0 with greedy decoding, shuffle seed 4242, full datasets, and vLLM-served endpoints, while Mimir additionally requires FlashAttention.vLLM with FlashAttention4 and Hugging Face Transformers produced comparable results up to numerical stability.
6 Conclusion, Limitations and Future Directions
Mimir v1 is a 1-billion-parameter HRM-Text model delivering frontier-level performance using only permissible data. Despite competitive results, it trails Gemma 4 on Math & Code, motivating future work on scaling, assistant capabilities, reinforcement learning, and dataset openness.
- Conclusion: Mimir v1 is a 1-billion-parameter language model using HRM-Text architecture and only permissible data.The data excludes personal information and copyright infringement, relying on open licenses, agreements, or the EU research text-and-data-mining exception.
- Limitations: Mimir v1 still lags behind Gemma 4 (5B, effective 2.3B) on Math & Code.This gap leaves room for improvement in future iterations.
- Future Directions: Future work will investigate HRM scaling, improve assistant capabilities, explore reinforcement learning, and further open the dataset’s licensing.Reinforcement learning remains unexplored for this architecture, while dataset development also targets improved model performance.
7 Contributors
The paper attributes contributions across conceptualization, data curation, analysis, methodology, software, project administration, resources, supervision, validation, visualization, funding, and writing. Peter Schneider-Kamp has the broadest listed contribution profile, while other contributors are assigned more specialized roles.
- Contributors: Peter Schneider-Kamp contributed across conceptualization, data curation, formal analysis, funding acquisition, investigation, methodology, project administration, resources, software, supervision, validation, visualization, and writing.His listed writing roles include both original drafting and review and editing.
- Contributors: Jacob Nielsen contributed to formal analysis, investigation, methodology, software, validation, visualization, and writing.His listed writing roles include original drafting and review and editing.
- Contributors: Lukas Galke Poech contributed to formal analysis, investigation, methodology, software, supervision, validation, visualization, and writing review and editing.His listed roles include supervision but not original drafting.
- Contributors: Other contributors covered data curation, resources, investigation, project administration, formal analysis, software, visualization, funding acquisition, and writing review and editing.The listed contributors include Gianluca Barmina, Kenneth Enevoldsen, Mogens Henrik From, Andrea Blasi N´u˜nez, Annemette Brok Pirchert, Stine Lyngsø Beltoft, Torben Blach, Sofie Helene Bruun, Oliver Kinch, Rasmus Larsen, Dan Saattrup Smart, and Kristoffer Laigaard Nielbo.
A List of Training Datasets
Mimir’s corpus comprises 161 datasets, listed with Hugging Face identifiers, formatting type, token counts, and percentage shares. Table 10 orders them by sampled tokens per epoch and includes curated, reformatted, translated, audited, agreement-supplied, and tool-call-formatted sources.
- Corpus overview: 161 datasets are catalogued in Table 10, sorted by sampled tokens per epoch.The table records each dataset’s Hugging Face identifier, form type, token count, and percentage share.
- Formatting types: 3.49B tokens and 4.95% come from allenai/Dolci-Instruct-SFT-No-Tools, listed as reformatted.Other listed sources include tool-call-formatted datasets such as allenai/Dolci-Instruct-SFT-Tool-Use and glaiveai/glaive-function-calling-v2.
- Source and task diversity: The corpus also includes translated-and-audited sources, agreement-supplied material, and datasets for Danish-English transformations.Examples are dfm8-openhermes-da, DBC, Lex.dk articles, and transformationsdanish-english.
B List of Evaluation Datasets
The evaluation-dataset appendix documents the benchmarks used, their precise Hugging Face sources, and the in-context shots provided to models. It also specifies a standardized evaluation configuration with temperature 0 and a fixed seed.
- Evaluation dataset coverage: Table 11 lists all evaluation datasets considered in the study.It presents an overview of the benchmarks and their configuration.
- Dataset provenance: Each dataset is identified by its precise Hugging Face source.The sources are given as Hugging Face identifiers.
- Evaluation configuration: The table records the number of in-context shots provided to models during evaluation.This documents the prompting configuration for each benchmark.
- Evaluation configuration: All benchmarks are evaluated with temperature 0 and a fixed seed.These settings standardize the evaluation procedure.