PaperlayerView exact revision on arXiv

Source-linked AI summary

Probable but Not Valid: A Psychometric Audit of Decision Models as Synthetic Survey Respondents

Mantas Lukauskas, Viktorija Šarkauskaitė

arXiv:2610.14912v1cs.AIcs.CYstat.AP

TL;DR

This paper asks whether decision models or newer generative LLMs can simulate human survey respondents on a psychometric benchmark. It compares their sample similarity and individual forecasts with human and statistical references, finding that sample-level fit and person-level forecasting differ.

  • Problem

    LLMs can match human survey averages yet differ in variance and regression estimates, motivating a psychometric comparison of decision models and newer generative models with human data.

    Source: §1 · p. 2

  • Method

    The study compares models on a psychometric survey using a five-component Psychometric Similarity Score and proper forecast scores against answer-frequency and profile-regression references.

    Source: §3 · p. 5, §4 · p. 7

  • Results

    The best language models are level with a Gaussian copula fit on other respondents and none exceeds it; decision vectors match item marginals better than rounded answers but forecast individuals worse than other respondents’ frequencies.

    Source: §1 · p. 2

  • Takeaways & Limitations

    At n=100, PSS cannot separate the best language models from real respondents, and profile-based synthetic respondents should aim at sample-level fidelity.

    Source: §8 · p. 25

    Mediation is the least stable PSS component, so single mediation values at n=100 should not be read as model properties.

    Source: §5.2 · p. 9

Abstract

from arXiv · show

Language models are increasingly used to create synthetic survey samples, also called silicon samples. Decision models are a new class of language models that return a probability for each point of a rating scale instead of text. We ask whether these models, or the newest generative LLMs, can stand in for human respondents on a psychometric survey. We test fourteen decision models from eleven providers and 47 generative LLMs on an open Lithuanian organisational-psychology survey (n=263, 68 items), with 100 real respondent profiles, the Psychometric Similarity Score (PSS) and statistical baselines. The best generative model, MiniMax M3, reaches PSS 0.813. This is level with a Gaussian copula fit only on other respondents (0.824) and below a copula fit in sample (0.927). A second sample of 100 real people scores below the best models, so at this size PSS cannot rank them against real respondents. In 5 of 10 product lines the newest generative release scores below an earlier release of the same line beyond resampling noise, and none gains beyond it. With the default rounded readout, 13 of 14 decision models score below the generative reference gpt-5.4-mini. Of the 47 generative models, 33 reproduce the pattern of item correlations better than any decision model. Averaged over respondents, the probability vectors are much closer to the human item marginals than rounded answers. They also rank respondents about as well as a profile regression fit on 163 real people. As forecasts of each person's own answers, however, every decision model is worse than the answer frequencies of other respondents, mainly because its average vector differs from them. The profile regression improves on these frequencies by only 1.0%, so the profile fields say little about how a person answers. Synthetic respondents built on such profiles should therefore aim at sample-level fidelity.

1 Introduction

This paper tests whether decision models or newer generative LLMs can better simulate respondents on a psychometric survey. Its findings distinguish sample-level similarity from forecasting individual answers.

1 Introduction

  • The study evaluates whether decision models outperform generative LLMs and whether newer generative releases outperform earlier ones, using the same respondents, profiles, scoring, and baselines.It also tests whether probability vectors forecast each person’s answers better than other respondents’ answer frequencies.Source: §1 · p. 2
  • Here, validity means that synthetic answers support the same interpretations as human answers; the audit examines internal structure and relations to other variables, not content validity.The paper lists an evaluation of 14 decision models, 47 generative LLMs, and their probability readouts among its contributions.Source: §1 · pp. 2–3
  • The best language models are level with a Gaussian copula fit on other respondents, and none is above it; in 5 of 10 product lines, the newest release scores lower beyond resampling noise.Newer generative releases do not score higher overall.Source: §1 · p. 2
  • Decision-model probability vectors come closer to human item marginals than rounded answers, but reproduce item correlations and factor structure less well than most generative LLMs.As individual forecasts, the vectors are worse than other respondents’ answer frequencies; profile regression improves on those frequencies by only 1.0%.Source: §1 · p. 2

2 Related work

Prior work has studied LLM-generated survey responses across social-science settings, often comparing group-level distributions. This paper extends that work to psychometric structure, individual forecasts, and model releases.

LLMs as survey respondents

  • Earlier studies used persona-conditioned LLMs for public-opinion splits, economic choices, and classic experiments, while other work found variance, regression, and identity-group mismatches.Research also reports that LLM simulations predict some survey-experiment treatment effects about as well as human forecasters.Source: §2 · p. 3

Predicting one person

  • Individual-answer studies have used interviews, demographics, fine-tuning, or trial-level data, while this paper benchmarks forecasts against answer frequencies from other respondents [38] [28].The earlier studies report stronger performance with interviews than demographics and weaker prediction for never-asked questions.Source: §2 · p. 3

How the answer is elicited

  • Survey elicitation choices can shift LLM responses, and prior probability-readout work compares groups; this paper asks what a probability vector says about one person.Other approaches include fine-tuning on survey data and mapping free-text answers to scale anchors [34].Source: §2 · pp. 3–4

What this paper adds

  • The paper evaluates probability-only models alongside 47 generative LLMs across psychometric measures, scores individual forecasts, and compares successive releases within 10 product lines.It notes that model behavior can change between versions and uses a Gaussian copula as a statistical anchor.Source: §2 · p. 4

3 Benchmark and data

The benchmark uses an open sample of Lithuanian employees and four organisational-psychology scales. The Psychometric Similarity Score combines five components to compare synthetic and human samples.

Data

  • The human reference comprises 263 Lithuanian employees who answered four scales totaling 68 items in 12 subscales.The change-engagement scale is less well validated than the other three, and 13 items are reverse-keyed.Source: §3 · p. 4

Profiles and sample

  • The headline sample uses 100 respondent IDs stratified by gender and position, with the same IDs as the gpt-5.4-mini C3 cell.The C3 profile includes eleven fields, including demographics, employment characteristics, and two ratings about recent organisational change.Source: §3 · pp. 4–5

PSS

  • PSS weights item-distribution similarity, item-correlation similarity, reliability, mediation, and demographic-contrast agreement into a score from 0 to 1.Its weights are those of the original audit; the item-correlation component compares matrix patterns and does not penalise a uniform rise in correlations.Source: §3 · p. 5

4 Querying and scoring the models

Models receive respondent profiles and survey items through decision or generative interfaces, with synthetic samples evaluated against human data and statistical references. The study also examines forecast quality, sensitivity, and uncertainty.

Decision models

  • Fourteen decision models from eleven providers were retained after applying rubric, request-capacity, and completion criteria; all 1,400 respondent-model cells completed.The Solar Decide endpoint was excluded after only 46 respondents completed in eight hours, below the 90-respondent threshold.Source: §4 · p. 5

Readouts and generative models

  • Decision models retain full probability vectors and produce rounded expected-score, most-probable-category, and sampled-category readouts; rounded scores are the default.The generative reference includes 36 original-audit cells and 11 newer models, with some provider and completion exclusions.Source: §4 · pp. 5–6

Requests and model classes

  • Decision requests send profile data and item questions as JSON, while generative models answer all 68 items in one text; interface, language, provider, and sampling also differ.The design cannot separate these factors, though the paper compares two interfaces for one model name.Source: §4 · pp. 5–6

Uncertainty

  • Paired bootstrap draws estimate uncertainty across respondent and human samples, but single mediation values at n=100 should not be read as model properties.Mediation is the least stable PSS component in the reported bootstrap analysis.Source: §4 · p. 6

Further analyses

  • The analysis adds control grids, profile-field swaps, mediation and structure analyses, and release comparisons; several analyses were added after initial results and are exploratory.The study was not preregistered, the hypotheses were not registered, and p-values are not corrected for multiple testing unless stated.Source: §4 · pp. 6–7

Forecast scores

  • Forecasts are scored against the same respondents’ answers using ranked probability and log scores, with population frequencies, profile regression, and uniform forecasts as references.Forecast calibration is also tested by mixing each vector with the population distribution; the profile regression is fit on 163 other respondents.Source: §4 · p. 7

5 Results

The results assess how generative and decision models reproduce human sample-level patterns across readouts and how well their probability vectors and profiles support forecasting individual answers. They also examine scale compression and whether newer generative releases score higher.

5.1 Where the language models stand

  • The best language models are level with the out-of-sample copula fit on other respondents (0.824), but none reaches the in-sample copula (0.927) or oracle, the real answers of the same 100 people (0.886).The in-sample copula is fit on the 263 people it is scored against; the 100 real respondents are a subsample with their own sampling noise.Source: §5.1 · p. 8, Figure 1 · p. 10

5.2 Decision models against the generative field

  • 13 of 14 decision models score below gpt-5.4-mini under the rounded readout, while every decision model scores below MiniMax M3 with no interval containing zero.Jev 1.13 is the strongest rounded decision model at 0.681, with a difference of −0.13 [−0.21, −0.05] to MiniMax M3. GPT-6 Luna’s decision interface also has lower correlation than its generator interface (+0.23 [+0.12, +0.33] in favour of the generator for the rounded readout), though the runs differ in route, prompt, and sampling.Source: §5.2 · p. 8, §5.3 · pp. 9–10

5.4 The readout decides which component is sacrificed

  • 0.83–0.96 is the distribution-component range after sampling decision-model vectors, while sampling lowers every model’s correlation component.Mercury Decide loses the least correlation, retaining 0.463, and reaches the highest sampled PSS (0.761), above the generative median. Under rounded readout, 12 of 14 decision models exceed the profile regression’s correlation component, but remain below most generative models.Source: §5.4 · pp. 10–11

5.5 What the probability vectors contain

  • 0.037–0.156 Jensen-Shannon divergence from human item marginals for average probability vectors is lower than the 0.218–0.411 divergence for rounded answers.Only 0.024–0.304 of answer variance depends on the respondent, yet decision vectors rank respondents about as well as profile regression: mean item-level Spearman correlations are 0.10–0.19 for decision models and 0.15 [0.08, 0.22] for regression. The person signal is stable across repeated runs, with at least 0.82 of between-respondent variance repeating and composite-score correlations of 0.95 or more.Source: §5.5 · pp. 11–13, §5.7 · p. 14

5.6 Correlation level and factor structure

  • 0.36 is the generative-model median mean inter-item correlation, above 0.22 for all 263 humans and 0.25 for the same 100 people; 42 of 47 generative cells exceed the latter.Decision models have a median mean correlation of 0.28, closer to the human level, but 13 of 14 separate subscales less well than humans; every decision model’s Tucker’s φ is below 0.85. Their expected scores concentrate 0.52–0.85 of variance in the first eigenvalue, indicating a general direction rather than subscale-specific structure.Source: §5.6 · p. 13

5.7 The person signal is stable and comes from the profile

  • 96% and 93% of decision-model composite swaps move in the human direction for role and education, respectively, but their median shifts are smaller than the observed human role contrast.Decision-model median shifts are 0.22 human SD for both role and education; the human role contrast is 0.46 [0.31, 0.62]. Gender shifts are 0.01–0.16 human SD (median 0.03) for decision models, and neither model class adds a gender effect of practical size. Repeated requests show the person signal is stable, while no-profile models generally return the same answers for everyone.Source: §5.7 · p. 14, §5.8 · p. 15

5.9 The vectors as forecasts of each person’s answers

  • 1.0% [0.1, 1.8] is the profile-regression improvement over the population distribution, while no decision model reaches that population baseline as a forecast of individuals’ answers.Decision vectors are 6–55% worse than the population distribution (median 20%), largely because their average vectors are 7–51% worse; rounding makes forecasts 48–105% worse. Generative repeat distributions are 41–104% worse, but after recalibration neither class does much better than profile regression.Source: §5.9 · pp. 16–17

5.10 Newer generative models do not score higher

  • 0.641 is the median PSS of 11 generative models released after the original audit, below 0.701 for 36 older cells (Mann-Whitney p=0.06).Without the mediation component, the medians are 0.734 for newer models and 0.765 for older cells (p=0.09); the data do not support the hypothesis that newer releases score higher.Source: §5.10 · p. 17
  • The newest release scores lower in 8 of 10 product lines, is level in 1 and gains in 1; the median change is −0.08, and no line’s paired interval favors the new release.Five paired intervals lie below zero, but none of the release-wave comparisons survives Holm correction (smallest adjusted p=0.14); the result’s direction also holds without the mediation component.Source: §5.10 · pp. 18–19, Table 9 · p. 20

5.11 Response style and downstream use

  • No language model forecasts individuals’ answers better than other respondents’ answer frequencies; performance-prediction R2 is 0.29 for copula-trained regression and 0.28 for human-trained regression.Only Jev among decision models matches that level at 0.30; generative LLMs have median R2 0.10, and only 25 of 47 beat predicting the mean.Source: Figure 4 · p. 19, §5.11 · p. 19, Table 11 · p. 22
  • All models compress the scale: extreme answers are 0.270 for humans, 0.024 for gpt-5.4-mini, and at most 0.196 for decision models.Every model uses the midpoint at least as often as humans; several decision models’ shifts are concentrated in reverse-keyed items, a pattern the passage says fits misresponse to reversed and negated items better than general acquiescence.Source: §5.11 · p. 19, Table 10 · p. 21
  • In the null-mediation test, generative LLMs fabricate 0–6 of 7 false paths (median 2), gpt-5.4-mini fabricates 6, and decision models fabricate 0–6.At n=100, a single triple can change the count, so the passage cautions against over-reading model rankings.Source: §5.11 · p. 20, Table 11 · p. 22

5.12 The four hypotheses

  • Sampled decision-model answers reach distribution components of 0.83–0.96, while only 2 of 14 decision models with rounded readouts exceed the generative median of 0.749.Averaged probability vectors come much closer to human item marginals than rounded answers.Source: §5.12 · p. 20
  • No deterministic decision model exceeds a correlation component of 0.667, while 33 of 47 generative cells are above that value.The passage attributes this partly to model format and to decision models’ nearly one-directional person signal, which also weakens their reproduction of factor structure.Source: §5.12 · p. 20
  • Decision-model vectors contain more respondent-specific signal than profiles support and rank respondents about as well as profile regression, but without profiles their sampled PSS falls by 0.06 to 0.33.The passage identifies calibration, rather than weak use of profiles, as the problem.Source: §5.12 · p. 21

6 Discussion

Decision-model probability vectors improve item-marginal estimates, but most generative models better reproduce item-correlation patterns, and decision models forecast individuals worse than other respondents’ answer frequencies. The discussion therefore recommends targeting sample-level fidelity and treating synthetic respondents as no substitute for human data in several survey uses.

Marginals and cost

  • Only 6 of 47 generative models come close to Jev’s synthetic-sample regression, which predicts held-out humans about as well as a regression trained on humans.Decision-model vectors averaged over respondents estimate item marginals better than any model’s rounded answers.Source: §6 · p. 22

Correlation structure and calibration

  • Decision models reproduce item-correlation patterns and factor structure less well than most generative LLMs, and their individual forecasts are worse than other respondents’ answer frequencies.Their average vector differs from real respondents’ answer frequencies; shrinking forecasts toward those frequencies improves them.Source: §6 · p. 22

How much a profile can tell

  • A profile regression using 163 other people’s answers improves forecasts of an individual’s answers by only 1.0%.The discussion argues that profile fields leave much human response variation unexplained, while generative models may overdo the unobserved differences they simulate.Source: §6 · pp. 22–23

Newer releases

  • The newest generative release scores lower in 8 of 10 product lines, with drops beyond resampling noise in 5; the cause was not tested.The proposed post-training explanation is explicitly tentative, and the release comparisons are exploratory.Source: §6 · p. 23

Recommendations for evaluating synthetic respondents

  • The nine evaluation recommendations address reverse-keying, source-row selection, uncertainty, readout, versioning, counterfactuals, individual forecasts, human comparisons, and correlation structure.They include reporting component standard errors, scoring probability forecasts against each person’s answers, and comparing a second human sample of the same size.Source: §6 · p. 23

Advice for survey practice

  • Averaged decision-model vectors over real target-population profiles offer a cheap rough estimate of item marginals, but cannot replace pilots, pretests, sample-size planning, reliability checks, or human samples for construct relations.The fidelity results refer to a non-probability online sample; where similar-population data exist, a copula fit is better.Source: §6 · p. 23

7 Limitations and future work

The audit’s conclusions are bounded by one Lithuanian sample, a 100-person model evaluation, and several noisy or changing measurement conditions. The authors call for broader replications, richer person-level information, and more robust comparisons.

7 Limitations and future work

  • Results come from one Lithuanian self-report sample, one profile, and four instruments; the education contrast is too noisy to assess model exaggeration.The observed contrast rests on 15 secondary-educated respondents.Source: §7 · p. 24
  • At n=100, PSS cannot rank the best language models against real respondents, and rankings within the top group are unstable.A second human sample of 100 scored below the best models; confirming leading-model order requires another respondent sample.Source: §7 · p. 24
  • Minimal reasoning, different query dates, and different providers limit comparisons across model releases; reasoning effort can move scores in either direction.The older cells were not re-queried, so subsequent vendor-side changes are not captured.Source: §7 · p. 24
  • Mixed English-profile/Lithuanian-item requests and split item batches may affect responses, while the alpha decisions endpoint exposes no seed and was retested only on one day.The audit therefore does not measure endpoint drift over weeks.Source: §7 · p. 24
  • Structure tests use exploratory factor alignment without measurement-invariance tests, and some decision samples have zero-variance items and are unstable at n=100.The human data have been public since 2020, and memorisation was not tested, although 12 of 13 models with C0 grids were farther from human marginals without profiles.Source: §7 · p. 24
  • Forecast comparisons use population and profile-regression references fit on 163 survey participants, so they do not establish distance from the best possible forecast.Generative forecasts use repeated runs at temperature 0.7; higher temperature would widen them.Source: §7 · pp. 24–25
  • The scoring update recomputes headline results and selected analyses at C3; other original-audit analyses are recomputed in its revised version.Source: §7 · p. 25
  • Future work should test richer person information, small-sample recalibration, joint or sequential item requests, and hybrids across languages, instruments, and probability samples.The authors also call for testing how many respondents calibration needs and whether a same-pilot copula performs better.Source: §7 · p. 25

8 Conclusion

Neither decision models nor newer generative models outperform the statistical baselines on this benchmark. Decision-model vectors match population answers closely but forecast individuals poorly, supporting sample-level rather than person-level fidelity as the practical target.

8 Conclusion

  • Decision-model vectors closely match population answers but recover less item dependence than most generative LLMs and forecast individuals worse than other respondents’ answer frequencies.They rank people about as well as profile regression, but spread respondents too far apart.Source: §8 · p. 25
  • Newer generative models do not close the gap, and in most product lines the newest release scores below its predecessor.Source: §8 · p. 25
  • The in-sample copula beats every language model, while the out-of-sample copula matches the best of 47; at n=100, PSS cannot rank the best models against real respondents.Source: §8 · p. 25
  • The study reuses de-identified data collected in 2020 under an informed-consent protocol and collects no new human data.Code, raw model outputs, and analysis tables are available from the corresponding author on reasonable request.Source: §8 · pp. 25–26

A Example request and response

The example shows how one respondent profile and a Lithuanian survey item are submitted to a decision model. The returned probability vector yields an expected score and a rounded category.

A Example request and response

B Scoring update relative to the published version of the original audit

The revised scoring reverses model answers on reverse-keyed items and restores a missing baseline component, changing model scores and rankings. Under the revision, the in-sample copula leads all language models, while the out-of-sample copula is level with the best.

B Scoring update relative to the published version of the original audit

  • Reversing the 13 reverse-keyed items raises PSS for all 36 C3 cells, by a median 0.093 and at least 0.016.For gpt-5.4-mini, aligned PSS is 0.743; the change also affects correlations, reliability, mediation, and demographic contrasts.Source: B Scoring update relative to the… · p. 30
  • Figure 5 compares raw and aligned PSS for the 36 C3 cells; the revised ranking has Spearman correlation 0.79 with the published order.gpt-5.4-mini moves from first to 10th, while MiniMax M3 (0.813), MiniMax M2.7 (0.809), and Kimi K2.7 Code (0.802) lead.Source: Figure 5 · p. 31, B Scoring update relative to the… · pp. 31–32
  • Including the previously omitted distribution component brings the original-audit copula total to 0.927 in sample and 0.824 out of sample.The out-of-sample copula is fit on the 163 people outside the headline sample.Source: B Scoring update relative to the… · p. 31
  • The revised keying gap measures content-following rather than agreement, while balanced ARS is computed from forward- and reverse-item means on a 0–1 scale.For gpt-5.4-mini, ARS shifts by +0.21 human SD; the human keying gap is 2.001 versus 1.512 for the model.Source: B Scoring update relative to the… · p. 31
  • The mediation component falls for 30 of 36 cells, with median −0.18; reliability and mediation changes vary more between models than correlation changes.Their between-model SDs are 0.18 and 0.19, versus 0.07 for correlation.Source: B Scoring update relative to the… · p. 32
  • The in-sample copula exceeds the best LLM by +0.11 [+0.06, +0.17], while the out-of-sample copula is level with it at +0.01 [−0.07, +0.09].Source: B Scoring update relative to the… · p. 32
  • The in-sample copula’s mediation component is 0.968 versus 0.929 for gpt-5.4-mini, and its demographic component is 0.663 versus 0.489.Against the out-of-sample copula, gpt-5.4-mini’s mediation difference is −0.21 [−0.39, −0.03], while its demographic difference is within noise.Source: B Scoring update relative to the… · p. 32
  • Inter-LLM similarity and test-retest reliability are unaffected by reverse-item direction, while paired counterfactual swaps are now signed by direction and expressed in human SD units.Without signs, swaps in opposite directions canceled and made the role effect appear small.Source: B Scoring update relative to the… · p. 32

C Models, costs and item keying

The appendices document the decision-model run and reverse-keying comparison, then describe the revised-scoring generative leaderboard.

C Models, costs and item keying

  • Table 13 records decision-model versions, context limits, prices, and per-respondent costs; the scored C3 grid cost $1.27, with another $0.08 for the incomplete Solar Decide run.The three Clef models and Drex received the profile twice because they accept at most 64 questions.Source: C Models, costs and item keying · pp. 32–33, Table 13 · p. 33
  • Table 14 compares mean item Jensen-Shannon divergence for reverse-keyed and forward-keyed items under raw and aligned scoring.Raw scoring leaves model answers unflipped while human answers are reverse-coded for reverse-keyed items.Source: C Models, costs and item keying · p. 32, Table 14 · p. 33

D All generative cells

  • Table 15 lists all 47 generative cells at C3, ordered by aligned PSS, and gives raw PSS and rank for original-audit cells.It omits Qwen 3.7 Plus, which had 75 complete respondents, and six cells run with high reasoning effort.Source: D All generative cells · p. 33, Table 15 · p. 34

E Reference sample

The reference-sample appendix compares sources against the full human sample and the 100 people whose profiles were simulated.

E Reference sample

  • Table 16 reports PSS against all 263 people and against the 100 headline people whose profiles were simulated.The second human sample comprises 100 of the 163 people outside the headline sample.Source: Table 16 · p. 35, E Reference sample · p. 35
  • The out-of-sample copula is trained on 163 people outside the headline reference, while the in-sample copula is fit on all 263 and overlaps it.The table also includes a 2000-row copula that repeats each headline profile 20 times to show the copula without its own sampling noise.Source: Table 16 · p. 35

F Agreement between models and sensitivity to the weights

Decision models show little shared bias in item-correlation structure, and their ranking is generally stable under changes to PSS weights.

F Agreement between models and sensitivity to the weights

  • 0.95 mean Spearman correlation: across 500 Dirichlet draws, decision-model and baseline rankings stayed close to the canonical ranking, and the in-sample copula ranked first in every draw.The minimum correlation was 0.71, and 0.60 of draws exceeded 0.95.Source: F Agreement between models and… · p. 35
  • 0.48 of draws placed independent marginals, Tev1 4B, and Solar Decide Flash among the bottom three; ranking changes occurred within the close middle group.Source: F Agreement between models and… · p. 35
  • 0.384 versus 0.422: decision models agree with each other slightly less than with humans on the item-correlation structure, showing little shared bias on this measure.Figure 6 compares the upper triangles of aligned, rounded 68 × 68 item-correlation matrices; the human matrix uses 263 respondents and other sources use the same 100 profiles.Source: F Agreement between models and… · p. 35, Figure 6 · p. 36

G Reproducibility

The reproducibility appendix identifies scripts for generation, scoring, analyses, and building paper assets from the resulting data.

G Reproducibility

  • run_decision_c3.py stores each respondent-model response with full probability vectors, resolved model version, token count, and cost, and can resume interrupted runs.The appendix also names the original-audit generator and the command settings used for new generative models.Source: G Reproducibility · p. 36
  • Separate scripts score decision and generative models, run control grids, analyze forecasts and references, and assess response structure.Source: G Reproducibility · p. 36
  • make_decision_paper_assets.py builds every table, figure, and number from the resulting CSV files; no API call is needed after generation.Source: G Reproducibility · p. 36

Related papers

Loading 2610.14912v1…