PaperlayerView exact revision on arXiv

Source-linked AI summary

BabyLM Turns 4 and Goes Multilingual: Call for Papers for the 2026 BabyLM Workshop

Leshem Choshen, Ryan Cotterell, Mustafa Omer Gul, Jaap Jumelet, Tal Linzen, Aaron Mueller, Suchir Salhan, Raj Sanjay Shah, Alex Warstadt, Ethan Gotlieb Wilcox

arXiv:2602.20092v2cs.CL

TL;DR

BabyLM brings cognitive modeling and language-model pretraining together, while previous challenge editions primarily focused on English. This call combines the fourth challenge with a workshop and adds a multilingual track in English, Dutch, and Chinese, alongside workshop themes spanning language diversity and efficient modeling.

  • Problem

    BabyLM addresses how computational systems can learn language from limited input and seeks connections between cognitive scientists studying childhood language learning and computer scientists building efficient systems.

    Source: §1 · p. 1

  • Method

    The fourth BabyLM Challenge retains STRICT and STRICT-SMALL and introduces a MULTILINGUAL track, while the workshop invites research on BabyLM-related topics.

    Source: §1 · p. 1, §4.1 · p. 3

  • Results

    The new track focuses on English, Dutch, and Chinese, and the workshop theme encourages BabyLM research beyond English.

    Source: §3.1 · p. 2, §4.1 · p. 3

  • Takeaways & Limitations

    The workshop welcomes work on cross-lingual transfer, low-resource languages, typological diversity, and data-efficient language modeling across diverse linguistic settings.

    Source: §3.1 · p. 2

    The challenge does not restrict compute, citing uncertainty about whether submissions exceed children’s available computation and the technical burden of measuring FLOPs.

    Source: §4.2 · p. 5

Abstract

from arXiv · show

The goal of the BabyLM is to stimulate new research connections between cognitive modeling and language model pretraining. We invite contributions in this vein to the BabyLM Workshop, which will also include the 4th iteration of the BabyLM Challenge. As in previous years, the challenge features two ``standard'' tracks (Strict and Strict-Small), in which participants must train language models on under 100M or 10M words of data, respectively. This year, we move beyond our previous English-only pretraining datasets with a new Multilingual track, focusing on English, Dutch, and Chinese. For the workshop, we call for papers related to the overall theme of BabyLM, which includes training efficiency, small-scale training datasets, cognitive modeling, model evaluation, and architecture innovation.

1 Introduction: BabyLM

BabyLM connects cognitive science and language modeling around how systems can learn language from limited input. The fourth challenge retains constrained-data tracks while adding multilingual participation and expanding what existing tracks permit.

1 Introduction: BabyLM

  • BabyLM brings cognitive scientists studying childhood language learning together with computer scientists building efficient systems, aiming for exchange in both directions.Cognitive-science insights are intended to inform sample-efficient model design, while language-modeling advances may generate cognitive-science hypotheses and experimental paradigms.Source: §1 · p. 1
  • The fourth-year workshop invites research at the intersection of cognitive science and language modeling under the theme “Going beyond English,” including multilingual acquisition and typological diversity.The workshop call follows the release of BABYBABELLM (Jumelet et al., 2025).Source: §1 · p. 1
  • The challenge retains STRICT and STRICT-SMALL at 100M and 10M words, respectively, and adds a MULTILINGUAL track using BABYBABELLM data.The MULTIMODAL and INTERACTION tracks are folded into STRICT and STRICT-SMALL, where participants may use paired image-text data or teacher-model feedback.Source: §1 · pp. 1–2

2 Key Dates

The tentative timeline runs from the February call and training-data release to the October workshop in Budapest, with submission and review milestones in between.

2 Key Dates

  • Workshop papers may be submitted through ARR or directly through OpenReview, and can ignore competition-entry deadlines.The direct-submission deadline is mid-July.Source: §2 · p. 2
  • Milestones include a mid-February CfP and training data release, early-April evaluation pipeline and baselines release, May 25 ARR submission deadline, mid-July direct submissions deadline, early-August reviews due and ARR commitment deadline, and mid-August decisions released.Source: §2 · p. 2

3 Non-competition Workshop Submissions

The workshop welcomes interdisciplinary work on BabyLM themes, with a special focus on moving beyond English. Submissions undergo double-blind review, with archival or nonarchival publication options.

3.1 Topics

  • Topics include data-efficient architectures and curation, cognitively or linguistically inspired modeling and evaluation, small models, multimodality, teacher interaction, and second-language acquisition, bilingualism, or multilingualism.The “Going beyond English” theme encourages work on underrepresented and low-resource languages, cross-lingual transfer, language mixtures, typological diversity, morphology, script variation, inclusive benchmarks, and ethical data practices.Source: §3.1 · p. 2

3.2 Paper submission

  • Submissions may be full archival papers or nonarchival upon request, up to eight pages, and allow dual submission but not dual publication.Submission is through OpenReview, with EMNLP 2026 workshop formatting, anonymity, and length requirements.Source: §3.2 · p. 2

3.3 Review & Publication

  • Workshop papers receive double-blind review for merit and relevance; competition submissions are judged primarily on technical soundness.Competition entries may be rejected for unjustified claims, significant technical problems, insufficient replication detail, or minimal time investment.Source: §3.3 · p. 2

4 Competition Details

The challenge offers STRICT, STRICT-SMALL, and a new multilingual track, with rules for data exposure, checkpoints, datasets, evaluation, baselines, and submissions. Workshop participation does not require entering a competition track.

4.1 Track Rules

  • The new MULTILINGUAL track uses cognitively plausible BABYBABELLM data and focuses on English, Dutch, and Chinese; participants can choose a custom mixture totaling 100M tokens, with language amounts adjusted for Byte Premium.English enables comparison with STRICT tracks; Dutch and Chinese were selected to examine cross-lingual transfer across typologically similar and distant languages and different scripts.Source: §4.1 · p. 3

4.4 Evaluation

  • Evaluation combines functional and formal linguistic competence tasks using fine-tuning or zero-shot prompting, with hidden tasks intended to assess generalization beyond released tasks.The multilingual track evaluates English, Dutch, and Chinese; evaluation details are to be released later.Source: §4.1 · p. 3, §4.4 · p. 5

4.1 Track Rules

  • STRICT and STRICT-SMALL retain 100M- and 10M-word corpus caps; MULTIMODAL and INTERACTION are merged into those tracks, which may still use multimodal data or interactive training under their rules.STRICT and STRICT-SMALL are language-only evaluated; external-model interaction is permitted under specified limits, and distillation is not allowed.Source: §4.1 · pp. 3–4

4.2 Training Requirements

  • STRICT-SMALL checkpoints are limited to 100M words seen, while other tracks are limited to 1B; intermediate checkpoints are required at increasing input-word intervals to assess learning efficiency and dynamics.Counts include repeated exposures; for STRICT and STRICT-SMALL, generated tokens used as input also count, and workshop papers are not subject to the competition’s epoch limits.Source: §4.2 · pp. 4–5

4.3 Updated Dataset

  • The updated detoxified datasets include 100M-word STRICT and 10M-word STRICT-SMALL corpora, plus a 100M-word multimodal corpus; participants may swap in other data within the word constraint.The update responds to reported toxicity and hateful content in the earlier corpus.Source: §4.3 · p. 5

4.5 Baselines

  • Baselines include GPT-BERT, a SimPO-based preference-optimization baseline, and GPT-2 Small for STRICT tracks; multilingual GPT-BERT and GPT-2 Small use a naive equal-language sampling approach adjusted for Byte Premium and BPE tokenization.The baselines are intended to encourage innovation and improvement beyond existing approaches.Source: §4.5 · pp. 5–6

4.6 Competition Submission

  • Competition entries go through OpenReview and must provide models and checkpoints, predictions, and—when applicable—custom-dataset information and external-model training data.Predictions must also be uploaded to the Hugging Face leaderboard by the deadline to count as competition entries.Source: §4.6 · p. 6

5 FAQs

The FAQs encourage innovative work, clarify submission and training rules, and welcome evaluation contributions suited to small models. They also explain how participants can handle modalities and external tools within the challenge’s data restrictions.

5 FAQs

  • Innovative ideas are encouraged; participants can contact organizers if competition rules constrain an idea, which may still fit as a workshop paper.Source: §5 · p. 6
  • A single paper may cover models submitted separately to multiple tracks, such as STRICT and MULTILINGUAL.Source: §5 · p. 6
  • Competition submissions require an accompanying paper of up to eight pages, published archivally, and papers may report experiments and analyses beyond the competition’s scope.Source: §5 · p. 6

Can I submit additional evaluation metrics?

  • Additional evaluation metrics are permitted alongside competition models, and the workshop especially welcomes them and evaluations of meaningful aspects suited to small models.The organizers also welcome analysis of previous submissions or related topics.Source: Can I submit additional evaluation… · p. 7
  • Any training objective or regime is allowed if data restrictions are followed, with ancillary models’ training data counted toward the budget; evaluation requires scoring word sequences without additional fine-tuning.Source: Can I submit additional evaluation… · p. 7
  • Hyperparameters are unrestricted, and participants are asked to share what they learn; epoch limits apply as of the previous challenge, with details referred to Section 4.2.Source: Can I submit additional evaluation… · p. 7
  • Language-trained external tools count toward the word limit, while synthetic data is allowed when the training-data budget remains within a closed system.The FAQ gives training two language models on half the words, generating additional text, and training on both original and generated data as an example.Source: Can I submit additional evaluation… · p. 7
  • Additional modalities are encouraged as research questions; non-linguistic data requires justification, while linguistic audio counts toward the learned-word total.Source: Can I submit additional evaluation… · p. 7

Related papers

Loading 2602.20092v2…