Source-linked AI summary
Births are difficult to predict even with rich survey and full-population register data
Elizaveta Sivak, Emily M. Cantrell, Thomas Emery, Javier Garcia-Bernardo, Flavio Hafner, Kasia Karpinska, Malte Lüken, Adrienne Mendrik, Joris Mulder, Hanzhang Ren, Varun Satish, Mark Verhagen, Angelica M. Maineri, Paulina Pankowska, Jasmin Abdel Ghany, Bruno Arpino, Giovanni Cassani, Julia Hellstrand, Katya Ivanova, Sanni Kuikka, Ana Macanovic, Charles Rahal, Felix C. Tropf, Roland J. Veen, Nicole Walasek, Daniël van Wijk, Kelsey Q. Wright, Emilio Zagheni, Henry Abbink, Emanuele Aliverti, Matteo Amestoy, Tilbe Atav, Nicola Barban, Sunnee Billingsley, Goan J. Booij, Louis Boucherie, Yael Broos, Li Ya Chang, Jamie C. Chiu, Chiara Ludovica Comolli, Boris Cule, Qixiang Fang, Dennis M. Feehan, Rachel Ganly, Erwin Gielens, Rolando M. Gonzales Martinez, Andrea Gradassi, Rosember Guerra-Urzola, Mario Guerra-Urzola, Stéphane Guerrier, Enamul Hassan, Vincent A. Haverhoek, Andrew T. Hendrickson, Amber Howard, Yuxuan Jin, Sayash Kapoor, Erik-Jan van Kesteren, Iris ten Klooster, Marie Labussiere, Lydia T. Liu, Tiffany Liu, Adam Maghout, Simone Meneghello, Lasse Mohr, Clara H. Mulder, Saul J. Newman, Jessica Nisén, Janis Norden, Mikkel Odgaard, Riccardo Omenti, Ozancan Ozdemir, Christina Pao, Paige Park, Gaia Penta, Juan C. Perdomo, Tanzir Pial, Alessio Piraccini, Federica Querin, Ziwei Rao, Christian Rellama, Adrien Remund, Frederieke Richert, Arnout van de Rijt, Mojtaba Rostami Kandroodi, Stijn J. Rotman, Lucas Sage, Germans Savcisens, Katrin Schwanitz, Steven Skiena, Alessandro Spata, Yannick Stadtfeld, Benedikt Stroebl, Gaetano Tedesco, Mathilde Theelen, Gianluca Tori, Abigail Tun-Mendicuti, Rishabh Tyagi, Keyon Vafa, Luiz Felipe Vecchietti, Linda Vecgaile, Willem R. J. Vermeulen, Maria-Pia Victoria Feser, Lionel A. Voirol, Thom B. Volker, Xinran Wang, Jiani Yan, Xinyi Zhao, Flora Zhou, Zuzana Zilincikova, Malvina Nissim, Matthew J. Salganik, Gert Stulp
TL;DR
Individual fertility outcomes remain underexplored as prediction targets, despite fertility’s importance to individual lives and societies. In a Dutch data challenge using survey and register data alongside simulations of reproductive biology, the authors find births within three years only moderately predictable even under favorable conditions, with chance in conception and pregnancy imposing a non-trivial ceiling.
Problem
Individual-level fertility outcomes remain underexplored as prediction targets, despite fertility’s importance to individual lives and societies.
Source: Introduction · p. 2Method
The authors combined a large Dutch prediction challenge using survey and register data with simulations of conception and pregnancy stochasticity to estimate a predictive ceiling.
Source: Introduction · p. 2, Discussion · p. 10Results
Even under highly favorable conditions, having a child within three years was only moderately predictable, while simulations showed chance in conception and pregnancy imposes a non-trivial ceiling.
Source: Discussion · pp. 10–11Takeaways & Limitations
The findings set realistic expectations for individual-level fertility predictions and possibly similar life-course outcomes.
Source: Discussion · p. 10The estimated stochastic variation may partly reflect epistemic uncertainty reducible by better data and methods, although some required biological data may remain unavailable.
Source: Discussion · p. 10
Abstract
from arXiv · showhide
Major life events have proven difficult to predict. Does this reflect limits of theory, data, and algorithms, or the large role of chance? We examine one outcome - having a child within three years - through a near-ideal setting for prediction: a data challenge where 147 researchers predicted births for Dutch residents aged 18-45, using survey data and full-population registers. Methods ranged from logistic regression to a large language model and transformers. Predictions were moderately accurate (best F1: register 0.59, survey 0.76); advanced models did not outperform classical ones; and the larger registers did not beat the survey. Simulating the stochastic biology of conception and pregnancy, we estimated a predictive ceiling (survey F1 ~ 0.86-0.94, register 0.88-0.96). Observed performance falls short of this ceiling, implicating imperfect data, methods, and unmodelled chance, while the ceiling itself shows that chance in reproduction alone sets a non-trivial limit on predicting individual lives.
Introduction
PreFer tests individual fertility prediction under unusually favorable conditions, combining Dutch register and survey data with a diverse research challenge. It uses fertility’s measurable outcomes and simulated reproductive chance to assess the limits of prediction.
Introduction
- PreFer recruited over 140 researchers to predict having a child within three years, testing current fertility prediction with full-population registers and high-quality nationally representative survey data.The challenge’s diverse approaches and competitive format were intended to reflect the best predictions possible with the available datasets, timeframe, and computational resources.Source: Introduction · p. 2
- Fertility is a useful test case because individual-level prediction remains unexplored, births are readily measured, and reproductive chance can be simulated to set an upper bound on accuracy.Comparing predictive performance with that bound can indicate how much room for improvement remains.Source: Introduction · p. 2
- Registers provide population-scale longitudinal detail, while surveys capture fertility intentions and other subjective factors; linked data lets PreFer test their relative predictive advantages.Registers may lack self-reported measures, while surveys face smaller samples, selective non-response, and attrition.Source: Introduction · pp. 2–3
- The task was to predict births in 2021–2023 for Dutch residents aged 18–45, using data through 2020 and household-level training and holdout splits.The survey sample comprised 1,382 people with known outcomes; registers covered about 6 million people.Source: Introduction · p. 3
- The challenge received 69 valid survey submissions from 41 teams and 11 register submissions from 12 teams, including a symbolic-life-course transformer and a fine-tuned life-course language model [48] [49,50].Register-phase teams used a Dutch national supercomputer to work with restricted data.Source: Introduction · p. 3
- The simulated ceiling treats time to conception, fecundability, age at sterility, fetal survival, and contraceptive failure as random, with age, partnership status, and fertility intentions as the only other outcome determinants.A model evaluated on the simulated outcomes uses those three variables; its F1 score represents error from chance alone under these assumptions.Source: Introduction · pp. 3–4
Results
The best models achieved moderate birth-prediction performance, and register data did not outperform survey data in a shared-holdout comparison. Simulated randomness in conception and pregnancy imposes a non-trivial predictive limit, while model combinations and larger samples alone offered limited gains.
Predictability is moderate even with very rich and large data
- Register models identified 64% of births and 91% of non-births, while survey models identified 66% and 98%, respectively; survey-model precision was 91%.Only 38% of survey models exceeded the fertility-intentions baseline; the best model’s F1 = 0.76 was 19% higher than the baseline’s F1 = 0.64, and removing intentions substantially reduced performance.Source: Predictability is moderate even with… · pp. 4–5
- Best register- and survey-based models achieved F1 0.59 and 0.76, respectively; gradient-boosted trees won both, while the transformer and fine-tuned large language model each scored F1 = 0.57.The best models’ F1 scores were about three times those of the demographic baselines: register F1 = 0.19 and survey F1 = 0.25.Source: Predictability is moderate even with… · p. 4, Figure 2 · p. 5
- After prevalence adjustments, survey F1 was 0.68 (95% CI [0.59, 0.76]) and 0.63 (95% CI [0.56, 0.69]), both above register F1 = 0.59; on a shared holdout, register predictions had higher MSE by 0.03.When trained on a survey-sized sample, the register model’s F1 fell to 0.50.Source: Predictability is moderate even with… · p. 6
Predictive gap may in part reflect randomness in conception and pregnancy progression
- With assumed contraceptive-failure rates of 1% and 17%, predictive upper-bound F1 was 0.963 and 0.881 for registers, and 0.94 and 0.86 for surveys, respectively.These randomness assumptions constrain prediction but do not explain the full gap: closing the survey-model gap would require recall to rise from 66% to 83–99% without losing precision, versus an 8-point gain across challenge rounds.Source: Predictive gap may in part reflect… · pp. 6–7
Combining models does not improve performance
- Combining models yielded no performance gains, and 21% of holdout cases with a birth were misclassified by all top-five register models.For survey data, top models also converged on similar performance and poorly predicted cases; aggregation did not improve performance.Source: Combining models does not improve… · p. 7, Figure 3 · p. 8
Larger sample sizes alone are unlikely to substantially improve performance
- Register-model gains became marginal beyond approximately 100,000 training observations, and survey-model gains beyond approximately 1,000 observations, making sample-size increases alone unlikely to yield major improvements.The learning curves show F1 scores as training samples grow.Source: Larger sample sizes alone are unlikely… · p. 8, Figure 4 · p. 9
Improvement from combining register and survey
- Adding register-based predictions to the survey model raised F1 from 0.63 to 0.68; experiments pointed to additional variables and larger-sample pattern learning as sources of improvement.Source: Improvement from combining register… · p. 9
Discussion
Even with rich survey and register data, state-of-the-art methods, and 147 researchers, births were only moderately predictable. The findings identify missing information and chance as important constraints, while showing that reproductive chance alone sets a meaningful ceiling on prediction.
Discussion
- Increasing survey sample size through register linkage yielded only minor gains, while diverse modelling approaches and ensembling provided no evidence that sample size or algorithms were the primary constraints.The authors caution that richer features, other modelling approaches, or greater computational resources could change this assessment.Source: Discussion · p. 10
- Missing or underused information constrained both datasets, although adding health, genetics, or high-resolution longitudinal data may yield limited gains given typically small effect sizes.Social interactions and some health and genetic data were absent or inaccessible in PreFer; transfer learning could help combine data sources.Source: Discussion · p. 10
- Simulated stochastic processes in conception, successful pregnancy, and unplanned births impose a meaningful predictive constraint, though the authors caution that some attributed chance may be reducible in principle but irreducible in practice.They may underestimate chance because other life domains also affect fertility, and uncertainty about contraceptive-failure births substantially varies their upper-bound estimates.Source: Discussion · pp. 10–11
- The best survey-based model outperformed the best register-based model despite registers’ larger samples and longitudinal coverage, with fertility-intention variables accounting for much of the survey advantage.The authors suggest that intentions may summarise latent factors, such as perceived costs and rewards of childbearing, that neither data source captures directly.Source: Discussion · p. 11
- PreFer offers a reusable benchmark that can be rerun on updated cohorts to track whether fertility predictability and its underlying mechanisms change over time.Repeated challenges could help assess whether births become more or less predictable as data, methods, and social conditions evolve.Source: Discussion · p. 11
- Despite unusually favourable data, methods, and researcher effort, having a child within three years proved only moderately predictable, with chance in conception and pregnancy setting a non-trivial ceiling.The study argues that the gap between observed performance and this ceiling also leaves room for improvement from better data and methods.Source: Discussion · pp. 10–11
Methods
The study prepared register and LISS survey datasets and split the target populations into household-level training and holdout sets to limit data leakage. Participants could use data through 2020, while outcomes were withheld outside the training sets.
Register data and sample
- The selected register datasets covered population, taxes, education, employment, benefits, housing, and neighbourhoods, generally from 1994–1995 onward, excluding highly sensitive health records.The datasets were managed by Statistics Netherlands.Source: Methods · p. 12
- The register population comprised 5,878,530 Dutch residents aged 18–45, split 70%/30% by household into training and holdout sets.Married or registered partners living in separate households were also assigned to the same set.Source: Methods · p. 12
Data access and leakage
- Participants could use data through 2020 for holdout and out-of-target individuals to construct network characteristics, but outcomes were withheld for everyone outside training.The provided register datasets were supplemented with a 2020 PreFer Base dataset containing selected variables for individuals and their married or registered partners; residual leakage risk related to identifying unregistered cohabiting partners was evaluated.Source: Methods · p. 12
Survey data and sample
- The LISS sample included 6,877 members aged 18–45 who participated in at least one Core module during 2007–2020, but 2021–2023 outcomes were available for only 1,382.The study used ten annually or biennially fielded Core modules and the monthly Background survey of household demographics.Source: Methods · p. 12
- The LISS sample was split by household and stratified on whether any member had a child, yielding 6,418 training members and 395 holdout members with known outcomes.Training outcomes were known for 987 members.Source: Methods · p. 12
- Participants received three LISS-based datasets: a main training dataset, a supplementary dataset for out-of-target participants, and a monthly household-background dataset.The main dataset combined 2007–2020 Core studies with selected Background survey variables.Source: Methods · p. 13
Calculating the outcome
The outcome was having at least one new biological or adopted child in 2021–2023, identified using administrative registers or survey responses.
Calculating the outcome
- The target outcome was having at least one new biological or adopted child in 2021–2023.Source: Calculating the outcome · p. 13
- Register outcomes were identified by combining Kindoudertab legal-parent records with Persoontab birthdates to detect children born in 2021–2023.Source: Calculating the outcome · p. 13
- Survey outcomes used 2020–2023 reports of biological and adopted children, supplemented with Background survey birthday data when responses were missing.Source: Calculating the outcome · p. 13
Performance metric
The section defines F1 as the evaluation measure and estimates a predictive upper bound by assuming age, partnership, and fertility intentions are fully known, leaving simulated reproductive randomness as the source of prediction error.
Performance metric
- F1 for the positive class is the harmonic mean of precision and recall, and teams could tune submission thresholds while baselines and the upper bound used 0.5.F1 rewards identifying people who had a child while penalising false positives, which matters for imbalanced outcomes.Source: Performance metric · p. 13
- The survey phase had 41 teams and 69 valid submissions, while the register phase had 12 teams and 11 valid submissions, with teams submitting only once.Survey participants submitted Docker code and could submit up to five times; register participants submitted predictions with their trained model and preprocessing and training code.Source: Performance metric · pp. 13–14
- The simulation includes stochastic conception timing, age-varying fecundability and fetal survival, plus a 1% assumed share of births due to contraceptive failure.Conception timing determines whether a birth occurs within the three-year window; the authors also simulated alternative contraceptive-failure proportions.Source: Performance metric · p. 14
- The predictive upper bound assumes age, partnership, and stable fertility intentions fully determine attempts, so only simulated reproductive randomness can cause prediction errors.The simulation sampled real respondents to create synthetic populations and evaluated a logistic regression using age, age², partnership status, and fertility intention; a random forest produced the same results.Source: Performance metric · pp. 14–15
- The ceiling assumes binary, stable intentions and is conservative under high fecundability, uninterrupted trying, and independent reproductive factors.Some variation treated as random may reflect genetic or environmental factors, and the ceiling reflects current biological knowledge and contraceptive technology; real reproductive decision-making can be ambivalent.Source: Performance metric · pp. 15–16
Data Availability
LISS panel data and prepared PreFer survey datasets are available through designated platforms, while register data access is restricted to vetted researchers at authorised institutions.
Data Availability
- Register data is not publicly available; scientific access is limited to vetted researchers affiliated with authorised institutions, which can apply for authorisation through Statistics Netherlands.The PreFer Base dataset is available via the CBS Data Storage.Source: Data Availability · p. 16
- LISS panel data is available through the LISS archive, and prepared PreFer survey datasets are available via SANE.The LISS archive is at https://www.lissdata.nl/how-it-works-archive; SANE is at https://odissei-data.nl/facility/secureanalysis-environment-sane/.Source: Data Availability · p. 16
Supplementary Information · 1. PreFer design · 1.1 Data
PreFer participants received datasets drawn from Dutch registers; Table S1 lists them, and codebooks and variable descriptions were available through CBS and ODISSEI.
1.1.1 Register data
- Table S1 lists the Dutch-register datasets available to participants, with codebooks linked through CBS and descriptions and variable lists through ODISSEI.Source: §1.1.1 · p. 24, Table S1 · p. 24
- Partner IDs linked partner characteristics, while migration dates and household identifiers, composition, and start/end dates were available from 1995.Source: §1.1.1 · p. 25
- Education levels and personal socio-economic categories were covered from 1999; employment and job characteristics were covered from 2010.Source: §1.1.1 · p. 25
- Personal and household income data were covered from 2011, while household wealth data were covered from 2006.Source: §1.1.1 · p. 25
- Register data included childcare proximity from 2012 and address histories from 1994, with housing type available from 2020.Source: §1.1.1 · p. 25
- Neighbour, colleague, family, household, and classmate network data were covered from 2009.Source: §1.1.1 · p. 26
- External election data reported Dutch general, provincial, and municipal results by municipality for 2017, 2019, and 2020, respectively.Source: §1.1.1 · p. 26
Target population and train-holdout splits
The study targeted Dutch residents aged 18–45 at the end of 2020 and excluded those no longer resident by the end of 2023. A 70/30 training-holdout split supported intermediate and final evaluations while restricting access to holdout outcomes.
Target population and train-holdout splits
- Of 6,092,379 people aged 18–45 registered at a Dutch residential address at the end of 2020, 213,849 (3.5%) were excluded because they were no longer residents at the end of 2023.Residents were defined as people registered at a Dutch address who had stayed in the Netherlands for at least four months; temporary moves abroad did not trigger exclusion if a person was registered again at the end of 2023.Source: Target population and train-holdout… · p. 26
- After the 70/30 training-holdout split, one-third of the holdout was used for intermediate submissions and two-thirds for final submissions.Ten percent of the training set was flagged as an evaluation set for participants to test models without reproducing the split; holdout and supplementary-group outcomes were withheld to reduce data leakage, though their features were available.Source: Target population and train-holdout… · pp. 26–27, Figure S1 · p. 27
PreFer Base dataset
The PreFer Base dataset provided 108 variables for target-population individuals and other Dutch residents, while the LISS survey data were split by household into training and holdout sets that were similar on measured characteristics.
PreFer Base dataset
- The Base dataset comprised 108 variables from most selected datasets, including constructed measures, and covered target-population individuals plus Dutch residents younger than 18 or older than 45 in 2020.Constructed variables included total number of children, age of the youngest child, and linked partner information for married or registered partners.Source: PreFer Base dataset · p. 27
1.1.2 Survey data
- Households were randomly split 70/30 to limit leakage; the holdout contained 395 participants, and training contained 6,418, with outcomes known for 987.The 30% holdout selection was stratified by whether any household member had a new child; 64 individuals with missing outcomes were excluded from holdout.Source: §1.1.2 · p. 29
- Training and holdout were similar: outcome rates were 21.5% vs 22.3%, mean ages were 32.7 vs 31.9 years, and recent module participation distributions were similar.The number of Core Study module waves was also similar between sets over the last five years and the last three years.Source: §1.1.2 · p. 29
LISS attrition rates
The LISS panel used biennial register-based refreshment samples to counter annual attrition and maintain representativeness. Nevertheless, most target members had dropped out by 2021–2023, and outcomes could be calculated for only 1,382 respondents.
LISS attrition rates
- Most of the 6,877 target members had dropped out by 2021–2023, leaving outcomes calculable for only 1,382 respondents.The target group comprised LISS members who had participated in a Core study before 2020 and were aged 18–45 in 2020.Source: LISS attrition rates · p. 29
- Annual attrition was approximately 10%; biennial register-based refreshment samples were used to maintain panel representativeness.The panel began in 2007 with approximately 5,000 households comprising 8,000 individuals aged 16 and older.Source: LISS attrition rates · p. 29
LISS-based datasets prepared for PreFer
PreFer prepared separate LISS training datasets for challenge participants and organisers, while restrictions on secure register-data computing affected teams’ methods and submissions. The paper’s results were generated using R and Python.
LISS-based datasets prepared for PreFer
- The main LISS training dataset covered 6,418 target-sample members, with outcome data for 987, while supplementary data covered 10,644 people outside the target age range.The outcome files were separate; the supplementary group excluded 161 holdout-household members, and pseudonymised IDs prevented linking to outcome-window data in the LISS archive.Source: LISS-based datasets prepared for PreFer · p. 30
- In phase one, 147 participants from 41 teams made 69 valid submissions; in phase two, 12 teams made 11 valid submissions using restricted register data.Only 26 of 45 phase-two team members had access to the register data.Source: LISS-based datasets prepared for PreFer · pp. 31–32
- Survey-phase submissions used containerised code, automated checks, and an anonymised leaderboard to support reproducible evaluation on organiser-held holdout data.The holdout training, outcome, and background files were available only to organisers; participants could view their anonymised leaderboard scores.Source: LISS-based datasets prepared for PreFer · pp. 30, 32, Figure S2 · p. 31
- OSSC scheduling and GPU constraints affected teams’ methodological choices, and one team could not produce a valid submission.Limited advance-request GPU time, scheduled rather than interactive jobs, and extra configuration for multiple GPUs led one team to extract network features manually instead of developing a graph neural network; another reduced its data use.Source: LISS-based datasets prepared for PreFer · p. 33
1.3.2 For producing this paper
- The paper’s results were generated using R 4.3.3 and Python 3.10.The packages used are listed in the replication-package scripts.Source: §1.3.2 · p. 34
2. Analysis of the predictive performance
The analyses examine what drives fertility-prediction performance, how survey and register predictions compare, and how reproductive randomness and case difficulty shape errors. They also assess the added value of register data and the impact of potential leakage from future births into household predictors.
2.1 Calculating confidence intervals; 2.2 Alternative train-test splits
- Confidence intervals used 1,000 bootstrap samples of the holdout set, and the original survey split’s performance fell near the middle of results across 25 alternative splits.The split-robustness check was not conducted for the register model because its holdout exceeded one million observations.Source: §2.1 · p. 34, §2.2 · p. 34, Figure S3 · p. 34
2.3 Baselines
- The nine-category fertility-intentions baseline learned that three-year hopes had lower realization rates than one-to-two-year timelines, explaining its slightly better performance than the literal baseline.The demographic baseline used logistic regression with age and age², gender, partnership status, education, and a composite measure of children and youngest-child age.Source: §2.3 · p. 35, Table S3 · p. 36
2.4 Excluding fertility intentions from survey-based models
- Removing fertility intentions reduced the best survey model’s F1 from 0.76 to 0.65 and the data-driven model’s from 0.71 to 0.54.The literal baseline for hoping for a child within three years performed slightly worse than the nine-category intentions baseline, because realization rates were lower for three-year timelines than for one-to-two-year timelines.Source: §2.3 · p. 35, §2.4 · p. 37, Figure S4 · p. 37
2.5 Accounting for different base rates in the survey and register data
- At 15% prevalence, survey-model F1 was 0.63 (95% CI [0.56, 0.69]); register-minus-survey MSE remained positive at 0.02 (95% CI [0.01, 0.03]).On the original survey holdout, the retrained survey model had F1 = 0.68, 95% CI [0.59, 0.76], and the register model had higher MSE (0.12 vs. 0.09; difference = 0.03, 95% CI [0.01, 0.05]).Source: §2.5 · pp. 38–39, Figure S5 · p. 39
2.6 Predictive upper bound: estimates for different assumptions
- Survey upper-bound precision was 0.93 in the first two conditions and 0.91 thereafter, while recall fell from 0.99 without contraceptive-failure births to 0.83 at a 17% share.Unpredictable births from contraceptive failure add false negatives without false positives; logistic regression and random-forest upper-bound results were almost the same.Source: §2.6 · pp. 40–41, Table S5 · p. 41, Figure S6 · p. 41
2.7. Hard-to-predict cases
- Among register-holdout births, 21% were misclassified by all top-five models, versus 1% of non-birth cases; recall declined for births later in the 2021–2023 window.The steepest decline in recall occurred in the first months, indicating that later births were harder to anticipate using pre-2021 data.Source: §2.7 · p. 43, Figure S8 · p. 43, Figure S9 · p. 44
2.8 Combining submissions
- Aggregating model predictions, using predicted classes or probabilities as features, and adding features from other top models did not improve performance.The analyses tested model averaging, prediction-based ensembles, and cumulative feature additions or removals.Source: §2.7 · pp. 44, 46, 47, Figure S10 · p. 45, Table S6 · p. 46
2.10 Combining survey and register data
- Adding the best register model’s predicted probabilities to the updated survey model raised F1 from 0.63 to 0.68; those probabilities were available for 94% of survey cases with known outcomes.The baseline and enhanced-model comparison excluded time-shifted data.Source: §2.7 · p. 50
2.7. Hard-to-predict cases
- F1 rose to 0.65 with the full register sample and 0.68 with the full sample and complete features; smaller samples did not improve performance.The improvement came from additional register variables and the very large sample enabling more accurate learning of patterns shared across datasets.Source: §2.7 · p. 50, Figure S14 · p. 51
- Future births could influence earlier household-type predictors through partner identification, affecting 9 of 11 models that used household type.Pairs who had a child during 2021–2023 were more likely to be identified as unregistered cohabiting partners in prior-year data.Source: §2.7 · p. 51
- Only 7,699 holdout observations—0.6% of all holdout observations and 4% of positive outcomes—were potentially affected by leakage.The household dataset was created in April 2021, limiting potential effects to births in the first few months of that year among unmarried partners.Source: §2.7 · pp. 51–52
- Excluding the potentially affected group lowered the best model’s F1 from 0.59 to 0.58, and the authors concluded leakage did not substantially affect results.Drops were also small for most other household-type models, though two fell from 0.50 to 0.36 and from 0.57 to 0.54; interpretation is uncertain because partnered individuals may be inherently more predictable.Source: §2.7 · p. 52, Table S7 · p. 52
3. Submissions: scores and brief description
The submissions section surveys a range of prediction methods, feature choices, and preprocessing approaches, and reports the scores achieved by participating teams.
Register-based submissions
- Stork Oracle’s 0.59 register submission used CatBoost with about 100 manually selected variables based on ideas about fertility prediction.The model used the PreFer Base file plus register features covering family networks and cohabiting partners.Source: §3 · pp. 53–54
- Cruijff’s register-based LLM scored 0.57, and its team attributed performance mainly to an embedded gradient-boosted prediction rather than the language model.The model fine-tuned Llama 3.1 8B-Instruct on enriched natural-language life histories, limited to about 1,000 tokens.Source: §3 · p. 54
- SBU EUI’s transformer scored 0.57 after pre-training from scratch on about 19 million life sequences and fine-tuning for fertility prediction.Its sequences represented demographic, education, household, income, employment and partnership information, among other domains.Source: §3 · p. 55
Scores and submissions
- 0.76 for survey Round 5 XGBoost and 0.59 for register-based Stork Oracle were the highest listed scores in this part.The submissions are described as ranked by score or team ranking in the supplied passages.Source: §3 · pp. 53, 58
Survey-based submissions
- Round 5 survey XGBoost tripled its training sample with time-shifted data, merged partner survey variables when available, and averaged related features into scales.The scales included attitudes toward fertility, motherhood, fatherhood and marriage, among other topics.Source: §3 · p. 58
Other submissions
- A LightGBM team selected its final model for robustly better performance than logistic regression, random forests and Naive Bayes; another team said 1,000-plus features likely degraded holdout performance.The latter team also reported that self-training and tri-training did not improve F1, while time-shifting worked better but conflicted with its fully data-driven aim.Source: §3 · pp. 61–62
Predictive features
- Fertility intentions predicted fertility outcomes, and household income mattered more than individual income; marital status was also consistently important.Source: §3 · p. 64
XGBoost and sequential models
- 0.71 F1-score on the round-two hidden validation set was achieved by XGBoost, used as a baseline; the team’s best-performing approach combined ExcelFormer with an RNN.ExcelFormer encoded questionnaire answers, and the RNN modeled temporal interactions between yearly questionnaires.Source: §3 · p. 64
Feature selection
- XGBoost performed best with about 200 questions, with performance plateauing or declining as features were added; the strongest question asked when respondents hoped to have a child.Important questions were mainly from 2020 and concerned family and household topics.Source: §3 · pp. 64–65
ExcelFormer–RNN
- 0.771 F1-Score, 0.720 MCC, 0.895 Precision, and 0.679 Recall were reported for the ExcelFormer–RNN model on holdout data.The reported 95% confidence intervals were [0.667, 0.861], [0.600, 0.828], [0.786, 0.976], and [0.545, 0.808], respectively; mean Average Precision was 0.872 [0.789, 0.937].Source: §3 · p. 66
Predictive features
- Only a few features had notable predictive power in most surveys; relevant variables included desire for a child, income, age, civil status, and gynecologist visits.Source: §3 · p. 69
Missing values
- Longitudinal imputation improved the predictive power of many features, but the strongest fertility-intention variable scored lower afterward, suggesting nonresponse itself carried predictive information.For missing answers, earlier responses to analogous questions were used; nonrespondents to the child-timing question appeared more likely not to have a child.Source: §3 · p. 69
Validation
- Five-fold cross-validation was too optimistic; repeated stratified shuffle splits with a 30% validation set produced internal scores closer to test-set performance.The team used ten repetitions of the shuffle split.Source: §3 · p. 72
Tree-based ensemble
- The final model was a weighted vote of CatBoost, LightGBM, and extra trees, weighting CatBoost at 0.5; standalone CatBoost performed closely to the combination.Source: §3 · p. 72
Submissions: scores and brief description
- One submission used a hard-voting ensemble of five classifiers, weighting their votes by individual F1 scores and oversampling the minority class to match the majority.Its features spanned fertility intentions, health, religiosity, children, mortality beliefs, and 2020 background-survey variables.Source: §3 · p. 74
- A theory-driven model focused on fertility intentions, age, partnership status, and religious attendance; adding parity, gender, education, employment, and income did not improve fit or yield expected results.Another team’s cross-validated feature comparisons found that fertility-intention questions tended to dominate feature importance for both broad and hand-crafted feature sets.Source: §3 · pp. 74–75
- A random forest on indiscriminate features scored ~0.93 F1 on the test set but ~0.61 on the holdout, so the team submitted its hand-crafted-feature model instead.Its test-set F1 was only around 0.002 lower, motivating the choice of the simpler feature set.Source: §3 · p. 76
- XGBoost performed best among one team’s tested approaches, with missing-value handling and insufficient observations for deep learning cited as relevant factors.The team reported that attempted imputation methods degraded results, while regression methods could not capture nonlinear relationships.Source: §3 · p. 76
- A LightGBM submission refined feature selection after recognizing possible selection bias, comparing feature importance against two noise variables across simulations.Frequently selected features came from the Family & Household, Economic Situation Assets, and Health surveys, in that order.Source: §3 · pp. 77–78
- Adding job status, health conditions, cohabitation, and gender attitudes did not significantly improve one logit model.Source: §3 · p. 83
3. Submissions: scores and brief description
- Two Transformer submissions scored 0.39: one used concatenated tabular variables, while the other used fewer variables and shorter sequences.The full encoder-decoder approach used demographic, household, employment, and socioeconomic variables.Source: §3 · p. 83
- A five-variable logistic regression scored 0.37 after rows missing any of those variables were removed.Its predictors included partnership status, fertility intentions, and satisfaction with relationships and finances.Source: §3 · p. 83
- A logistic regression with oversampling and a 0.25 decision threshold scored 0.29.The approach used age, partnership status, religiousness, household income, and urbanicity to increase positive-class recall.Source: §3 · p. 83
- Other logistic-regression submissions used 20 manually selected imputed binary variables or a broad set of variables from nine LISS modules, and scored 0.19 and 0, respectively.Source: §3 · p. 83
- A neural-network submission scored 0 after normalizing the ID column, while a one-feature logistic regression using financial satisfaction also scored 0.Source: §3 · p. 83
4. Additional funding
The acknowledgments list support from French and Finnish research programs, alongside grants and institutional funding for other contributors and teams.
- Lucas Sage acknowledged French Agence Nationale de la Recherche funding under the Investissement d’Avenir programme (ANR-17-EURE-0010).Source: §4 · p. 84
- Julia Hellstrand, Kelsey Wright, and Ziwei Rao received Strategic Research Council support through FLUX, with decision numbers 364374, 364375, 345130, and 345131.Jessica Nisén also cited FLUX support (nr. 364374) and Academy of Finland INVEST support (nr. 320162).Source: §4 · p. 84
- Other acknowledgments cite UKRI FINDME, Italian Science Fund support for SO-UNFER, Dutch Research Council support, NSF and French ANR support for Team SBU EUI, and Princeton initiatives.The Princeton team cited Princeton Precision Health, the Princeton AI Lab, and the Princeton Catalysis Initiative.Source: §4 · p. 84