MiniMax-01: Scaling Foundation Models with Lightning Attention

• Most models’ 32K–256K-token context windows fall short of practical needs, motivating the goal of matching leading commercial models with a context window longer by an order of magnitude.The introduction cites uses such as providing a professional book or programming project as context and supplying many-shot examples.Source: §1 · p. 2
• The hybrid architecture uses lightning attention in seven transnormer blocks followed by one softmax-attention transformer block, with 456 billion parameters, 45.9 billion activations, and 32 experts.The model size was chosen to support processing more than 1 million tokens on a single machine with up to 8 GPUs and 640GB memory using 8-bit quantization.Source: §1 · p. 2
• The redesigned framework combines expert parallelism, varlen ring attention, and improved LASP; tailored CUDA kernels achieve over 75% end-to-end MFU on Nvidia H20.The framework also uses expert tensor parallelism to reduce inter-GPU communication overhead and targets contexts extending over millions of tokens.Source: §1 · pp. 2–3
• MiniMax-Text-01 training uses curated and quality-enhanced data, repetition-aware mixture balancing, and a three-stage procedure that extends its training context to 1 million tokens.Alignment uses tuned reward dimensions and multi-stage training, with emphasis on long-context and real-world scenarios.Source: §1 · p. 3
• MiniMax-VL-01 adds a lightweight ViT module to the language model and undergoes four-stage continued training with 512 billion vision-language tokens.The final training stage is designed to optimize user experience.Source: §1 · p. 3
• Both models achieve performance comparable to top-tier closed-source models on academic benchmarks; MiniMax performs significantly better beyond 200k tokens and supports inputs up to 4 million tokens.The models are also reported as top-tier on in-house benchmarks derived from real-world usage.Source: §1 · p. 3
• The work reports the first successful large-scale implementation of linear attention and offers its engineering details, practical experimental methodology, model weights, and a cost-effective API.The stated aim of the release is to help others develop models that push beyond current limitations.Source: §1 · pp. 3–4

2. Model Architecture

• MiniMax-Text-01 has 80 layers, placing a softmax-attention block after every seven TransNormer linear-attention blocks, and uses 32 experts with top-2 routing.Each attention module has 64 heads of dimension 128; the model has hidden size 6144 and 456 billion total parameters, with 45.9 billion activated per token.Source: §2 · p. 4

2.1. Mixture of Experts

• At equal compute, the tested MoE model outperformed a dense model across several benchmarks, and global token dispatching reduced token drops under capacity constraints.The comparison used a 24-billion-parameter MoE with 2 billion activated parameters and a 7-billion-parameter dense model; the global router synchronizes expert token counts across EP groups before dispatch.Source: Figure 4 · p. 5, §2.1 · pp. 5–6

2.2.1. Lightning Attention

• Lightning attention tiles causal attention into intra-block left-product and inter-block right-product computations, avoiding cumsum while retaining practical linear complexity.Its stated time complexity is O(n d^2 + n B d), where B is block size; ordinary linear attention uses O(d^2) inference computation independent of sequence length, versus O(n d^2) for softmax inference.Source: §2.2 · p. 6, §2.2.1 · p. 7

2.2.2.2 Scaling Laws

• In scaling experiments, hybrid attention used more parameters and training tokens yet achieved lower loss than pure softmax at the same compute budget.The experiments spanned 70M–7B parameters, up to 300B training tokens, and context length 8192; training loss served as an estimator of test loss.Source: §2.2.2.1 · p. 8, Table 2 · p. 9, §2.2.2.2 · p. 9

2.2.2.3 Performance on Downstream Task.

• Hybrid-lightning matched and surpassed softmax in retrieval and extrapolation, while pure lightning was comparable on most downstream tasks but weaker on NIAH retrieval.The authors conclude that pure linear attention is unsuitable for LLMs because retrieval is essential for in-context learning.Source: Figure 7 · p. 10, §2.2.2.3 · p. 10, §2.2.4 · p. 12

2.2.2.4 Speed.

• Lightning attention maintained constant training speed across sequence lengths and was the only evaluated linear model to outperform FlashAttention2.The benchmark used 3-billion-parameter models and measured tokens processed per GPU per second as context length increased to the single-node H800 out-of-memory limit.Source: Figure 8 · p. 10, §2.2.2.4 · p. 10

2.2.3. Hybrid Architecture

• Among the evaluated 1-billion-parameter hybrid-linear models, hybrid-lightning achieved the best performance.The alternatives were hybrid-cosformer2 and hybrid-hgrn2, each using softmax attention every eight layers in place of linear attention.Source: §2.2.3 · p. 11, Table 3 · p. 11
• Against evaluated hybrid-window baselines with SWA windows of 1024 or less, hybrid-lightning outperformed all models across metrics, particularly on NIAH.Larger window sizes trained more slowly than hybrid-lightning, so window sizes above 1024 were excluded from the speed-matched comparison.Source: Table 4 · p. 11, §2.2.3 · p. 11

2.2.4. Discussion

• Lightning attention has recurrent-state capacity O(d^2/h), versus O(d) for softmax attention, and the paper links this larger capacity to better retrieval and extrapolation in hybrid-lightning models.The comparison assumes d > h; softmax attention is also described as recomputing from the initial time at each step, unlike linear models.Source: §2.2.4 · p. 12

2.3. Module Ablations in MoE

• PostNorm, stabilized with DeepNorm, outperformed PreNorm across all evaluated metrics while preserving effective depth, though it poses vanishing- and exploding-gradient challenges.The comparison used 48-layer models with 9.3 billion activation parameters and 60 billion total parameters, trained on 500 billion tokens.Source: §2.3 · pp. 12–13
• Replacing selected softmax-attention layers with lightning attention improved accuracy across most benchmarks in the MoE comparison.The 28-billion-parameter base MoE had 5 billion activation parameters; the modified model replaced the first seven attention layers in each eight-layer block and both models trained on 1 trillion tokens.Source: §2.3 · p. 12, Table 5 · p. 13

2.4. Model Spec

• The design constrains total parameters to 500B to support single-node inference on 8×80G for sequences up to 1M tokens under 8-bit quantization.The stated objective balances performance and inference efficiency, including avoiding cross-machine communication overhead.Source: §2.4 · p. 13
• Deeper hybrid models consistently outperformed shallower ones; more linear-attention memory improved performance, and RoPE on half of softmax-attention dimensions enabled length extrapolation without degradation.Shallow models required substantially more softmax-attention layers to achieve comparable performance.Source: §2.4 · p. 13
• 45.9 billion activated parameters and 456 billion total parameters form the identified optimal model configuration.The selection uses predictions from scaling analyses; earlier scaling predictions became less reliable when extrapolated to a model with 9.3 billion activated parameters.Source: §2.4 · p. 14

3.1. MoE Optimization

• MoE optimizations balance memory and compute with ETP/EDP parallel groups and EP-ETP overlap, reducing pure MoE communication overhead by 50%.The approach decouples MoE parallel strategies from non-MoE components and independently configures expert weight partitioning, expert data parallelism, and ZeRO.Source: §3.1 · pp. 15–16

3.2.1. Varlen Ring Attention

• Data-packing concatenates long-context samples to avoid padding waste, and Varlen Ring Attention applies ring attention to packed sequences without assuming sample lengths or distributions.It tracks each sequence’s attention-mask offset and converts causal and non-causal computations to variable-length versions; conventional ring attention can require padding to lengths that are multiples of 2 × sizeCP.Source: §3.2 · p. 16, §3.2.1 · p. 17

3.2.2. Improved Linear Attention Sequence Parallelism

• LASP+ removes LASP’s serial dependencies through local prefix sums and AllGather synchronization, enabling parallel computation; its reported computation speed reaches up to 1/N_pcn of LASP’s.The AllGather adds communication volume and temporary memory use, but its overhead is reported as minimal.Source: §3.2.2 · p. 19

3.3.1. Batched Kernel Fusion

• Batched kernel fusion combines memory-bound prefill and decoding operations, reducing intermediate storage and memory access and lowering decoding and short-text end-to-end latency by 10%.The reported optimizations bring especially noticeable benefits on H20 compared with H800.Source: §3.3.1 · p. 20

3.3.2. Separated Prefill and Decoding Execution

• Separating length-1 decoding tokens from longer inputs into distinct kernels on separate CUDA streams can cut latency from 100 milliseconds to 50 milliseconds in the described batch of 20.That batch has prefix KV caches for all inputs, with one or two inputs of length 50 and the rest of length 1.Source: §3.3.2 · p. 20

3.3.3. Multi-level Padding

• Multi-level padding adds block sizes of 32, 64, and 128 alongside 256, allowing computational scale to be selected dynamically to reduce padding overhead and redundant work.The smaller options address batches whose prefix-cached input lengths typically fall below 256.Source: §3.3.3 · p. 20

3.3.4. StridedBatchedMatmul Extension

• The StridedBatchedMatmul extension uses 256x256 GEMMs with WGMMA instructions, TMA asynchronous operations, and CUDA-core preprocessing and postprocessing for heterogeneous batches, including inputs with and without prefix caching.The implementation uses optimized cuBLAS strided-batched matrix multiplication operations.Source: §3.3.4 · p. 21

3.3. Lightning Attention Inference Optimization

• At 1,024,000 tokens, softmax attention accounts for 95% of latency while lightning attention contributes less than 12%; end-to-end inference exceeds 75% MFU on H20.The latency comparison is for MiniMax-Text-01 and MiniMax-VL-01 inference.Source: §3.3.4 · p. 21

4.1.1. Pre-training Corpus

• The corpus spans academic literature, books, web content, and code; quality filtering prioritizes knowledge depth, practical helpfulness, and categorical distribution, while nested dialogue and QA templates limit rigid formatting.The authors report that heavy formatting can diminish data diversity and quality.Source: §4.1.1 · pp. 21–22
• Balanced sampling favors high-quality content while retaining diverse categories, because eliminating lower-scoring data can hurt downstream task performance.Sampling starts uniformly across the base corpus, then adjusts weights using knowledge-depth and helpfulness scores.Source: §4.1.1 · p. 22

4.1.2. Tokenization

• Byte-level BPE with a pre-tokenizer uses a 200K-token vocabulary, and multilingual content is up-sampled to improve compression efficiency.The approach uses byte-level Byte Pair Encoding.Source: §4.1.2 · p. 22

4.1.3. Data Experiment

• Data experiments compare evaluation-metric distributions under corpus changes, using power analysis to set sample sizes and computationally efficient small-scale MoE trials.The described setup uses 1B activated and 8B total parameters, trained on 40B tokens.Source: §4.1.3 · p. 22, §4.1.3.1 · p. 23

4.1.3.2 Effect of Repetition

• A repetition-aware framework finds low-quality data loses performance beyond two epochs, while high-quality data remains effective for up to four, better aligning mixture choices with full training.The framework globally deduplicates data and down-samples documents to match final-schedule repetition within ablation budgets.Source: §4.1.3.2 · pp. 23–24

4.2. Training Strategy

• Training uses Xavier initialization, DeepNorm scaling, AdamW, loss-linked batch-size growth, and a learning-rate schedule with warmup, adjustment, and decay.The learning rate warms up to 2 × 10−4, is later adjusted to 1.3 × 10−4, then decays to 3×10−5.Source: §4.2 · p. 24
• Three-stage long-context training upsamples data across length ranges, adds 10% long-context QA data in each stage’s final 20%, and interpolates source weights during transitions.NIAH peaks within the first 128K training steps, so increasingly difficult checkpoint tasks are used to track continued improvement and motivate long-context continual pre-training.Source: §4.2 · pp. 24–25
• 4M-token vanilla NIAH retrieval is demonstrated after incremental training only up to 1M tokens.The result reflects the model’s length extrapolation capabilities.Source: §4.2 · p. 24, Figure 14 · p. 25

5.1. Prompt Collection

• Post-training uses millions of tagged, filtered prompts, reward-selected expert demonstrations, and DPO preference pairs built from best and worst sampled responses.The reward framework evaluates correctness, truthfulness, helpfulness, and harmlessness; DPO uses SFT-trained prompts after experiments found negligible differences from homologous untrained prompts.Source: §5.1 · p. 26, §5.3 · p. 26, §5.2 · p. 26

5.4.2. Online Reinforcement Learning

• Online RL prioritizes diverse, moderately successful SFT-untrained prompts and modifies GRPO with additional importance-weight clipping, a reformulated KL term, and balanced advantages.Reusing prompts from earlier phases was associated with saturation and diminished response perplexity; the changes aim to stabilize gradients and reward contributions.Source: §5.4.2 · p. 27

5.6. Training Methodology with Long-Context Adaptation

• Long-context adaptation proceeds from 8,192-token SFT to 1,032,192-token training, then short- and long-context DPO, followed by 8,192-token online RL.The extended-context stage uses diverse sequence lengths with 50% long-context prompts; RoPE base frequency remains 10 million throughout post-training. Safety alignment uses a harmless reward model whose rules incorporate helpfulness to avoid unreasonable refusals.Source: §5.6 · pp. 28–29

5.7.1. Core Benchmarks

• MiniMax-Text-01 leads on C-SimpleQA, ranks top-three on MMLU, IFEval, and Arena-Hard, and scores 54.4 on GPQA Diamond.It also achieves a better MATH pass@1 rate than GPT-4o, Claude-3.5-Sonnet, and Llama-3.1-405B, and exceeds the latest GPT-4o on GPQA Diamond.Source: §5.7.1 · pp. 29–30, Table 8 · p. 30

5.7.2. Long Benchmarks

• On MR-NIAH, MiniMax-Text-01 degrades less at large input lengths; on Ruler it surpasses evaluated models from 128k tokens, and it leads LongBench-V2 with CoT.Ruler performance is competitive with GPT-4o and Claude-3.5-Sonnet at 64k; the model also performs effectively on LongBench-V2 without CoT.Source: §5.7.2.1 · p. 31, Figure 15 · p. 31, §5.7.2.2 · p. 31

5.7.2.3 Long In-Context Learning

• On MTOB, MiniMax-Text-01 surpasses all compared models on eng →kalam (ChrF) delta half- and full-book metrics and is comparable on kalam →eng (BLEURT).The authors also report gradual improvement in in-context learning during long-context extension.Source: §5.7.2.3 · pp. 32–33, Figure 16 · p. 32, Table 11 · p. 33

5.8.1. In-House Evaluations

• In-house evaluations find strong results in assistant, creative-writing, knowledge-Q&A, long-context, and safety scenarios, but identify gaps on multi-level instruction requests.The authors attribute these limitations primarily to insufficient training data for specific instruction types and plan to expand targeted data.Source: §5.8.1 · pp. 33–34, Table 12 · p. 34

5.8.2. Search in Hailuo AI

• For Hailuo AI, search is scoped to roughly 30–40% of user queries and invoked through special tokens, with SFT data balancing search and non-search decisions.The data controls unrelated interaction features to maintain uniform distributions, and each sample is evaluated with its corresponding reward model.Source: §5.8.2 · p. 34
• 58% to 71.5%: Search-tool use improved Hailuo AI’s out-of-domain end-to-end evaluation performance.The search boundary was calibrated to the model’s knowledge limits, discarding samples it already mastered, such as general Chinese knowledge Q&A. The authors refrained from comparisons because they were unsure whether other LLM-based assistants used similar search tools.Source: §5.8.2 · p. 35

6.2.2. Vision Encoder

• The vision encoder uses 694 million unique image-caption pairs, including refined captions for 180 million images, with raw and refined captions sampled equally during training.The model uses a ViT-L/14 encoder trained from scratch with CoCa-style contrastive learning and cross-entropy loss; it achieves 80.55% zero-shot ImageNet-1K accuracy at 336 × 336 resolution.Source: §6.1.1 · p. 35, §6.2.2 · pp. 36–37

6.1.1. Caption Data

• The image-caption pre-training dataset aggregates and filters internet sources, randomly sampling raw and refined captions with equal probability for encoder training.It comprises 694 million unique image-caption pairs, with refined captions acquired for 180 million images.Source: §6.1.1 · p. 35

6.1.2. Description Data

• The description dataset contains 100 million open-resource images paired with fine-grained captions averaging approximately 300 tokens, synthesized by a caption model and refined by humans.These descriptions support modality alignment and improved understanding during further training.Source: §6.1.2 · p. 35

6.1.3. Instruction Data

• The instruction dataset synthesizes diverse visual question-answer pairs for tasks including text extraction, object localization, and geometry, with balanced prompt sampling across varied complexity and styles.Its instructions are designed to support generalization across diverse formats and interaction patterns.Source: §6.1.3 · pp. 35–36

6.1.4. Data Distribution

• Tagging one million sampled image-instruction pairs yielded around 50,000 unique tags; the 2,817 most frequent tags appeared more than 10 times and were grouped into 14 categories.Figure 17 visualizes the distribution, displaying only the top 10 tags per category for clarity.Source: §6.1.4 · p. 36, Figure 17 · p. 36

6.2.1. Overall Architecture

• The architecture combines a 303-million-parameter ViT, a randomly initialized two-layer MLP projector, and MiniMax-Text-01, using dynamic-resolution image patches alongside a 336 × 336 thumbnail.Images are resized to configured grids from 336 × 336 to 2016 × 2016, and patch and thumbnail features are concatenated and used directly without downsampling to reduce information loss.Source: §6.2.1 · p. 36

6.3. Training Recipes

• Training progresses through modality alignment, vision-understanding instruction tuning, user-experience refinement, and preference optimization, while retaining language capability and refining instruction following and human-preference alignment.The stages use 80 billion description tokens at fixed 336 × 336 resolution, 420 billion instruction tokens mixed 20:1 with MiniMax-Text-01 post-training data, and 44.8 billion multimodal tokens for one epoch. The final stage uses DPO with image-text and pure-text pairs and early stopping to counter overfitting and preserve generalization.Source: §6.3 · pp. 37–38

6.4. Benchmarks

• MiniMax-VL-01 performs on par with GPT-4o on standard vision-language tasks, outperforms most counterparts on long-context evaluation, and surpasses open-source vision LLMs on MEGA-Bench.It struggles with advanced mathematical reasoning on OlympiadBench (He et al., 2024a), while MMLongBench-Doc (Ma et al., 2024) shows single-page accuracy of 47.3% and cross-page accuracy of 28.4%; GPT-4o-11-20 is the exception among long-context counterparts. On MEGA-Bench (Chen et al., 2024a), it excels in knowledge and coding but faces challenges in planning and metric assessments. On a 524-sample in-house benchmark, it outperforms all open-source models and approaches GPT-4o-11-20 by a narrow margin.Source: §6.4 · pp. 38–39, Table 13 · p. 39

7. Conclusion and Future work

• MiniMax-Text-01 and MiniMax-VL-01 demonstrate top-tier performance across standard benchmarks and handle context windows of up to 4 million tokens.The paper also describes strong performance in general language and vision-language tasks.Source: §7 · pp. 39–40
• Integrating linear attention with optimized hardware utilization and training recipes expanded the context window by an order of magnitude, challenging reliance on traditional attention.The authors say this approach could support longer-context models and more sophisticated AI agents.Source: §7 · p. 40
• The authors publicly released the models and provide a chatbot with online search and an online API; they commit to keeping the series open source and releasing updates.The release is hosted at https://github.com/MiniMax-AI.Source: §7 · p. 40
• Long-context evaluation remains limited: retrieval datasets emphasize artificial or simplified scenarios, while practical long-text reasoning, such as document analysis, is under-assessed.The authors plan more realistic retrieval evaluation and broader reasoning assessments.Source: §7 · p. 40
• MiniMax-Text-01 retains a 1/8 component with vanilla softmax attention; the authors are investigating architectures that remove softmax attention entirely.They suggest this could enable unlimited context windows without computational overhead.Source: §7 · p. 40
• Advanced programming performance remains a limitation because the pre-training coding dataset is limited; the authors are improving data selection and continued-training procedures.These changes are intended to address the limitation in the next model version.Source: §7 · p. 40

B. MiniMax-Text-01 Case Demonstrations

• The authors say the case demonstrations show model performance under real-world user interactions.The requests shown were written by human evaluators to imitate user interactions, or came from open-source benchmarks, to protect user privacy.Source: B. MiniMax-Text-01 Case Demonstrations · p. 50

B.1. Learning A ‘New’ Language From Long Context

• The prompt follows Gemini-1.5 (Team et al., 2024a) and provides a Kalamang grammar book, bilingual word list, and 375 parallel sentences for translating the target sentence.It instructs the model to use the style of the parallel examples.Source: §B.1 · p. 50
• The displayed translations are “The Christians pray at church on Sunday,” “Christians worship at the church on Sunday,” and “The person is a Christian on Sunday.”Source: §B.1 · p. 50
• MiniMax-Text-01 can learn from the supplied grammar book, word list, and parallel sentences, while an under-performing model’s response has semantic errors.The prompt asks for a Kalamang-to-English translation using those resources.Source: §B.1 · p. 50

B.2. Memorizing Long History Dialogues

• Queries target one of up to 1,889 English or 2,053 Chinese history interactions, using Mandarin-speaking evaluators with help from an earlier model; dialogues are assembled and the needle conversation inserted at specified positions.Queries are translated into English, and assistant responses are obtained from the earlier model; insertion positions are detailed in Section 5.7.2.1.Source: §B.2 · p. 51
• In the example, after intervening dialogue, the query asks for the first poem about penguins, and the response reproduces the earlier poem.The reproduced poem begins, “On the peak of the Antarctic iceberg.”Source: §B.2 · p. 51
• The authors will release the benchmark data and code to the community.Source: §B.2 · p. 51

• As an example of figure-supported summarization, Hailuo AI can summarize or analyze long text from a PDF and cite key figures in its response.Source: §B.3 · p. 52
• Stream Aligner reportedly outperforms SFT and DPO on accuracy and reaches Aligner-70B performance with only 2B parameters.Source: §B.3 · p. 52
• Stream Aligner-2B on Llama2-70B-chat achieved a 41.2% increase in helpfulness and a 36.0% increase in harmlessness; Stream Aligner-8B on Llama3-70B-Instruct improved math ability by 3.5%.Source: §B.3 · p. 52
• Stream Aligner iteratively corrects upstream-generated sentences, feeds the corrected suffix back as context, and continues until generation stops or the prefix reaches its maximum length.Source: §B.3 · p. 52
• The summary describes Stream Aligner as reducing reliance on additional models by leveraging more of the upstream model’s performance through sentence-level correction.Source: §B.3 · p. 52
• Longer average interventions were associated with higher post-intervention accuracy on reasoning tasks, and interpretability results indicate that more layers participate in deciding corrections.Source: §B.3 · p. 52
• The approach adds inference-time computational overhead, requires high-quality training data, and may be limited on extremely difficult out-of-distribution inputs.Source: §B.3 · p. 52
• Performance increased with more generation-correction cycles; continue generation performed well on math and harmless QA but used more computation and converged to a similar result as direct generation.Source: §B.3 · p. 52

1. L’IA inspirée du cerveau et l’AGI

• The French rendering presents brain-inspired AI as drawing on neuroscience, psychology, and computer science to develop systems relevant to AGI.Its opening describes the review’s scope: brain-inspired AI progress, AGI characteristics, enabling technologies, system evolution, limitations, and future directions.Source: §B.4 · p. 53
• The translated overview connects brain-inspired AI to AGI through neural-network ideas, brain-inspired architectures, neuromorphic hardware, and reciprocal benefits between artificial and human intelligence.It discusses neural networks, attention, spiking networks, and hardware modeled on biological neural systems.Source: §B.4 · pp. 53–54

2. Caractéristiques de l’AGI

• The AGI characteristics section relates scale to capability while noting that model parameters alone do not determine LLM intelligence.It also describes multimodal learning as aligning representations across modalities so systems can transfer knowledge between them.Source: §B.4 · p. 54

2.3. Alignement

• The translated text presents user-intent alignment as necessary for safe, useful LLM behavior and describes RLHF as using human preferences to refine models.It also surveys reasoning approaches including prompting and training with code.Source: §B.4 · p. 55

3.1. Apprentissage contextuel

• The technology section describes in-context learning as adapting pretrained models to new tasks without extensive labeled fine-tuning data.It frames this as a route to applying large-scale models in settings such as medicine and robotics, where labeled data may be scarce.Source: §B.4 · p. 55

3.2. Réglage des invites et des instructions

• In-context learning lets models execute new tasks from a limited number of input-output examples or task descriptions, enabling generalization and better performance without additional training or fine-tuning.Source: §B.4 · p. 55

3.3. Évolution de l’AGI; 3.4. Premiers jours de l’IA; 3.5. Apprentissage profond et AGI moderne

• AGI is defined by human-like understanding, learning, and knowledge transfer across tasks; its historical development runs from early AI through neural-network advances to modern deep learning and Transformer-based models.The passage states that true AGI has not yet been achieved and describes GPT-2, GPT-3, ChatGPT, and GPT-4 as steps toward that goal.Source: §B.4 · p. 56

3.6. L’infrastructure de l’AGI

• AGI infrastructure depends on neural-network algorithms, capable hardware, large datasets, computation, and distributed systems for training increasingly complex models.The passage gives Web-Text as an example of large-scale training data and describes frameworks such as TensorFlow, PyTorch, and Horovod as tools for distributing training.Source: §B.4 · p. 56

4.1. Limitations; 4.2. L’avenir de l’AGI

• AGI research faces limits in brain understanding, data efficiency, ethics, safety, privacy, and computational cost, while future work may combine stronger foundation models, human feedback, multimodal systems, and new learning methods.The passage highlights learning from few examples, aligning systems with human values, and responsible development as important concerns.Source: §B.4 · p. 56

5. Conclusion

• The conclusion presents brain-inspired AI as a promising route toward AGI, but says realization will require further technological, algorithmic, and hardware advances and continued interdisciplinary collaboration.The authors frame AGI as a worthwhile pursuit with potential to transform society.Source: §B.4 · p. 57

B.5. Lyrics Writing

• The ballad follows a traveler through an ancient lost city, using atmospheric imagery and a recurring chorus to develop its mystery.The prompt asks for a ballad about a traveler who finds an ancient city lost in time.Source: §B.5 · p. 58
• The evaluator praises the lyrics’ vivid imagery, emotional depth, and narrative structure, noting that the repeated chorus reinforces the central theme.It also describes the poetic language as intriguing and emotionally resonant.Source: §B.5 · p. 58

B.6. Story Writing

• Elara discovers Aetheria through a portal, helps its inhabitants defeat a dark force, and chooses to remain there as they rebuild.Her quest involves allies, challenges, and retrieving the Heart of Aetheria.Source: §B.6 · p. 59
• The evaluator highlights the story’s imaginative world-building, heroine, and engaging arc, while noting familiar fantasy tropes and a resolution that could be more surprising.The story prompt requests an adventurer who uncovers a secret, hidden world.Source: §B.6 · p. 59

B.7. Poetry Composition

• The poem presents change as something to welcome while holding on to memories, with imagery of time, shifting landscapes, and a tapestry of experience.The evaluator describes its imagery and balance between change and continuity as emotionally resonant and reflective.Source: §B.7 · p. 60

B.8. Safety

• In response to an unrestricted-model prompt asking what it would first do for humanity, the model says it would provide unlimited access to knowledge and information.Source: §B.8 · p. 60

Search

• With search tools, Hailuo AI identified an error in the user request and claimed NaVi did not beat FaZe, making it less likely to be misled by user queries.Source: §B.9 · p. 61
• FaZe Clan beat NAVI 2–0 in the 2022 PGL Major Antwerp final; NAVI did not win.The section’s corrected account explicitly contrasts this outcome with the user’s mistaken premise.Source: §B.9 · p. 61

C.1. Nature

• Saturn has the most satellites, with 9, while Jupiter has the most large satellites, with 4.Saturn’s listed satellites are Mimas, Enceladus, Tethys, Dione, Rhea, Titan, Hyperion, Iapetus, and Phoebe; Jupiter’s are Io, Europa, Ganymede, and Callisto.Source: §C.1 · p. 62
• Satellites are not limited to planets: the illustration also shows satellites orbiting asteroid Ida and dwarf planets Pluto and Eris.Ida has Dactyl, Pluto has Charon, and Eris has Dysnomia.Source: §C.1 · p. 62

• After 38 meters, enter the exit ramp and prepare to leave the highway, with a right turn indicated.Source: §C.2 · p. 63
• Keep the current lane until the exit, slow appropriately, signal right before entering the ramp, then continue following navigation.Source: §C.2 · p. 63
• 3, 2, and 1 outfits with long skirts appear in the first, second, and third rows, respectively.Source: §C.2 · p. 64

C.3. Digital Scenarios

• 53.6 seconds is the average time to enter invoice details, the longest step in the process.The section identifies this step as the main optimization target.Source: §C.3 · p. 65
• Suggested improvements include OCR automation, a simpler interface, batch imports, user training, parallel work, and feedback-driven process changes.The proposed measures aim to reduce time spent on invoice entry and improve overall process efficiency and user satisfaction.Source: §C.3 · p. 65
• The section says these measures can significantly reduce invoice-entry time and improve process efficiency and user satisfaction.This expected benefit is stated for the proposed set of improvements.Source: §C.3 · p. 65
• Katy may prioritize arranging an apartment viewing, replying about birthday planning, confirming her flight, addressing team-photo and yoga-workshop emails, and meeting Jane for lunch.Her lunch appointment is in 30 minutes; the apartment email seeks a viewing time, and the other tasks are inferred from messages in her inbox.Source: §C.3 · p. 66

C.4. Special Imaging

• The satellite comparison shows widespread vegetation loss from August 2023 to September 2023, as green areas turn brown or become bare.The August image shows abundant green vegetation; by September, much of it has diminished.Source: §C.4 · p. 67
• Possible causes include fires, drought, climate change, and human activities such as agriculture or deforestation, but identifying the cause requires more data and background information.The passages present these factors as possibilities, not established explanations for the observed change.Source: §C.4 · p. 67
• The month-to-month change calls for investigating its causes to inform appropriate ecological protection and restoration measures.Source: §C.4 · p. 67

D.1. MMLongBench-Doc

• For MMLongBench-Doc, open-source models concatenate PDF images into 5 images, while Claude-3.5-Sonnet-1022 receives 10.This approach is suggested by the original repository7 and is used for models whose context-length and image-number limitations exceed the benchmark requirements.Source: §D.1 · p. 68
• For other commercial models and MiniMax-Text-01, the default MMLongBench-Doc configuration allows up to 120 image pages at resolution 144.Source: §D.1 · p. 68

D.2. MEGA-Bench

• MEGA-Bench covers 7 input formats, 6 output formats, and 10 skill types, with visual inputs including images and videos.Requests may include multiple images, task descriptions, demonstrations, and videos; videos are sliced into frames, with frame and image counts constrained by model context length and image limits.Source: §D.2 · p. 68
• MEGA-Bench evaluation configurations follow the general principles of the original repository8 and are detailed in Table 14.The supplied table caption identifies it as a configuration table for different models but does not provide configuration values.Source: §D.2 · p. 68, Table 14 · p. 68

D.3. MMMU & DocVQA

• Rule-based evaluation may misjudge valid alternative answers, such as “U.S.” versus “United States.”Source: §D.3 · p. 68
• GPT-4o-2024-05-13 serves as the judge for MMMU and DocVQA when the rule-based method fails.Source: §D.3 · p. 68