Silicon sampling answers with country-level assumptions, not individual attitudes: Cross-national evidence from the European Social Survey

• Earlier studies reported some reproduced human findings, while later evaluations documented compressed variation, prompt sensitivity, and misrepresented groups.Cross-national outputs have also tended to align more closely with Western and English-speaking populations.Source: §1 · p. 2
• Existing evaluations leave broad cross-national item variation, the isolated contribution of country labels, and country-specific individual-level recovery unresolved.Prior work also leaves open whether individual recovery varies across countries and aligns with aggregate recovery.Source: §1 · p. 2
• The study evaluates aggregate recovery across 30 countries and 42 items, experimentally varying country labels, profile enrichment, response formats, and prompt framing.It also estimates within-country individual recovery by country to test whether its geographic pattern matches aggregate recovery.Source: §1 · pp. 2–3
• The country label acts as a country-level assumption, profile enrichment adds no consistent aggregate gain, and individual-level recovery remains negligible.The response format determines whether countries are ranked in the correct order or in reverse; a regional average using survey data alone recovers country means better.Source: §1 · p. 3

2.1 Evaluating silicon sampling across countries

• Cross-national evaluations have covered limited country and item sets, leaving item-level ranking, individual recovery, and country conditioning untested together on the same respondents.The section asks how accurately silicon sampling recovers aggregate attitudes in order and level, and how recovery varies by item.Source: §2.1 · pp. 3–4

2.2 Candidate sources of aggregate recovery

• Aggregate recovery could reflect sample composition, country labels, or response format, and the experiments distinguish these explanations through enrichment and verbal scale endpoints.The account predicts that a country-level assumption yields one large gain from the label, while endpoint wording should remove reverse-item ranking inversions.Source: §2.2 · p. 5

2.3 Individual-level recovery and its relation to aggregate recovery

• Whether individual recovery tracks aggregate recovery across countries remains untested, and sparse demographic profiles may not recover within-country differentiation.The section asks whether response distributions and individual recovery reproduce human data, whether 1P versus 3P framing matters, and whether recovery aligns across levels.Source: §2.3 · pp. 6–7

3.1 Survey benchmark

• The benchmark is the European Social Survey, a cross-national probability-based survey, using Round 11 data from 30 countries.Source: §3.1 · p. 7
• The analysis covers 42 survey items across 12 domains, including social and institutional trust, political attitudes, well-being, immigration, health, and income.Source: §3.1 · p. 7
• The full item list, question text, scales, and domain classifications are reported in the paper’s accompanying materials.Source: §3.1 · p. 7

Appendix Table A1. Figure 1 illustrates the simulation pipeline.

• Figure 1 illustrates the simulation pipeline, in which sampled ESS respondents receive demographic backstories and prompts before model outputs are compared with human responses.Source: Appendix Table A1. Figure 1… · p. 8, Figure 1 · p. 8

3.2 Research design

• The main design crosses Qwen and Llama with first- and third-person prompts, and subsamples 685 respondents per country.Responses with valid human answers and parseable model outputs covered about 98% of scheduled responses except Llama 3P, where coverage was 76%.Source: §3.2 · pp. 8–9

3.3 Backstory construction

• Respondent backstories use 20 ESS demographic variables mapped into natural-language sentences with fixed value-label dictionaries.The third-person prompt asks how the described person would respond; party affiliation is represented generically rather than by a country-specific party.Source: §3.3 · pp. 9–10

3.4 Backstory conditioning experiment

• Paired backstory contrasts isolate country-label, region, political-identity, and broader profile-block contributions, with added arms testing swapped labels and response-scale endpoints.The block design tests additions and removals at early and late positions, but bounds intermediate contributions only if effects change monotonically.Source: §3.4 · pp. 10–11

3.5 Evaluation metrics

• Aggregate recovery is measured by correlations across country means, while individual recovery is measured within each country; distributional spread and mean-level bias are also assessed.The evaluation includes no-LLM comparators and uses concordance to assess whether levels match as well as rankings.Source: §3.5 · pp. 11–12

4.1 Aggregate recovery is moderate and item-dependent

• Under Qwen 1P, pooled correlation is 0.822 while median item-level correlation is 0.443; 20 of 42 items exceed 0.5 and nine exceed 0.7.The pooled measure reflects between-item scale differences; aggregate recovery also varies across countries and items.Source: §4.1 · pp. 12, 14, Figure 2 · p. 13
• Eleven items invert their cross-national ordering, and 10 of those 11 are reverse-coded; item direction explains 66% of variance in rbc.Trust and subjective well-being recover best, whereas income, safety, and health invert.Source: §4.1 · p. 14
• Neighbor averaging achieves median rbc 0.55 versus the model’s 0.44 and median per-item normalized error 0.056 versus 0.180, winning on level accuracy for 41 of 42 items.Adding log GDP per capita to region dummies raises the comparator’s median rbc to 0.62 and lowers its normalized error to 0.052.Source: §4.1 · pp. 14–15

4.2 The country label produces aggregate recovery, and the response format sets its direction

• Adding the country name to the base profile raises median rbc from −0.03 to 0.52, while verbally naming scale endpoints changes the label’s effect on reverse-coded items from harmful to helpful.The triple difference is +0.42 on the Fisher-z scale, with a 95% bootstrap interval of [0.13, 0.79].Source: §4.2 · pp. 16, 18, Figure 4 · p. 17, Figure 5 · p. 19

4.3 Individual-level recovery is negligible and does not track aggregate recovery

• Median within-country recovery never exceeds 0.028 across the four model-prompt conditions, and aggregate and individual country rankings correlate at −0.21.Of 435 country pairs, 232 reverse order between the two rankings.Source: §4.3 · pp. 20, 22, Figure 6 · p. 21, Figure 7 · p. 23

4.4 Robustness checks

• Robustness checks preserve the conclusions except for Llama third-person parse coverage, whose median rbc falls from +0.17 to zero above 80% coverage.The framing null therefore rests on Qwen alone.Source: §4.4 · pp. 23–24

5.1 The country label acts as a country-level assumption

• The country label acts as a country-level assumption: respondent detail does not consistently improve aggregate recovery, and the observed match cannot distinguish survey knowledge from stereotypes or other country associations.The label answers which national profiles a model resembles, not whether its outputs reproduce the target population.Source: §5.1 · pp. 24–25

5.2 Response format explains the inversions in country rankings

• Response format explains the inversions: verbal scale endpoints make the intended direction explicit, while numeric-only ranges can reverse country rankings.Because pooled and profile correlations combine correctly ordered and inverted items, item-level diagnostics and reporting the response format are necessary.Source: §5.2 · pp. 25–26

5.3 Silicon sampling can support country ranking, not individual or distributional inference

• Silicon sampling may support exploratory country-ranking comparisons on externally validated topics when comparable-country survey data are unavailable, but not individual prediction, within-country heterogeneity, or distributional analysis.Aggregate recovery concerns the ordering of country means alone and does not establish representativeness.Source: §5.3 · p. 26

5.4 Contributions, limitations, and future directions

• The paper recommends judging recovery against no-LLM comparators built from survey data researchers already hold and reporting unparsable outputs by item and country.Each use should be evaluated against the quantity it is intended to estimate.Source: §5.4 · pp. 27–28
• The conclusions are bounded by English prompts, generic party coding, one European survey program, and two open-weight 7–8-billion-parameter models; the design cannot separate calibrated country knowledge from stereotype activation.The no-country arm retains country information in harmonized-category composition; the authors propose factorial prompt tests and broader validation.Source: §5.4 · p. 27

C Backstory leakage

• Table A2 documents leakage for seven overlapping variables, distinguishing direct echoes from attenuated inverse mappings.Correlations moved toward zero after the leaked variable was removed.Source: Table A2 · p. 37, C Backstory leakage · pp. 37–38
• The leaked backstory raised the mean pooled individual correlation from 0.044 (clean) to 0.070 across all 42 items.The comparison used earlier 27-variable and 20-variable backstory runs, before the final 685-respondent round.Source: C Backstory leakage · p. 38
• The leakage comparison is retained as the design rationale for removing the seven overlapping variables.The leaked and clean columns came from different backstory sizes and sample sizes, and both predate the final round.Source: C Backstory leakage · p. 38

D Evaluation metrics

• The uncentered pooled individual correlation retains between-country covariance and is not a measure of individual-level prediction.It appears only in the appendix, in Table A2 and the Table A5 specification check.Source: D Evaluation metrics · p. 38
• The uncentered pooled individual correlation is used in Table A2 and the specification check in Table A5.The passage distinguishes it from a measure of individual-level prediction because it retains between-country covariance.Source: D Evaluation metrics · p. 38
• The measure is reported only in the appendix, rather than as an individual-level prediction metric.Its between-country covariance is retained.Source: D Evaluation metrics · p. 38

E Per-item aggregate recovery

• Figure A1 reports all 42 items’ rbc values, ranked from most positive to most negative, with symbols distinguishing forward- and reverse-coded items.It presents the full per-item aggregate recovery shown unlabeled in Figure 2(b).Source: E Per-item aggregate recovery · pp. 38–39, Figure A1 · p. 39
• Trust in police has rbc = 0.82 with the country ordering largely recovered, whereas attitudes toward homosexuality have rbc = −0.82 with the ordering inverted.The contrasting country-level scatterplots show least-squares fits and the 45-degree reference for perfect recovery.Source: Figure A2 · p. 40, E Per-item aggregate recovery · p. 40

F Concordance

• Cb = 0.132, and 87% of the agreement attainable at the observed ranking is lost to bias and compression under Qwen 1P.Median rbc = 0.443 and concordance = 0.036; median absolute bias is 1.13 scale points and median between-country dispersion ratio is 0.28.Source: F Concordance · p. 40
• 11 of 42 items have negative concordance, indicating worse-than-no agreement despite the observed rankings.Lin’s concordance factors as rbc × Cb, where Cb is an accuracy term bounded by 1.Source: F Concordance · p. 40

G The country label contrast

• Figure A3 compares item-level changes in rbc (Fisher z) with versus without the country sentence, ranking items by wording direction and marking a ±0.079 noise floor.The country-label contrast is the item-level counterpart of the block effect summarized in Figure 4.Source: G The country label contrast · pp. 40–41, Figure A3 · p. 41

H Block contrasts

• Only the country-label removal contrast exceeded the noise floor and remained significant after Holm correction: +0.135 on forward items and −0.176 on reverse items.Region and political-variable contrasts fell inside the noise floor; the largest positive forward effect from other blocks was +0.09 (Fisher z).Source: H Block contrasts · p. 42

I Country identifiability

• Eight backstory variants rendered the country sentence and reached 100% country identifiability, yet their median rbc ranged from 0.287 to 0.523.The three-variable base reached 7.23% identifiability; the full profile without a country sentence reached 28.72%, replicated at 28.99%.Source: I Country identifiability · p. 42

J Individual-level diagnostics

• Political efficacy, institutional trust, and subjective well-being recover best by domain, while income, health, safety, and social values are negative.No domain reaches two-thirds of the 0.210 demographic benchmark.Source: J Individual-level diagnostics · pp. 42–43
• 0.125 was the largest domain mean in any condition, below the 0.210 demographic benchmark; mean rwc was 0.027.The domain table reports mean within-country correlations by domain and model-prompt condition.Source: J Individual-level diagnostics · pp. 42–43, Table A3 · p. 43
• 11 of 42 items had negative rwc under Qwen 1P, concentrated in reverse-coded items where substantive inversion and scale-direction misinterpretation cannot be separated.Figure A4 reports the full per-item individual-level recovery.Source: J Individual-level diagnostics · p. 43, Figure A4 · p. 44
• The Qwen prompt-framing difference was small and nonsignificant; only the first-person Qwen–Llama contrast met the Bonferroni-adjusted threshold of α = 0.0125.The Llama 1P–3P contrast cannot identify a framing effect because of the third-person arm’s parse coverage.Source: J Individual-level diagnostics · pp. 44–45, Table A4 · p. 45

K Aggregate robustness checks

• One of 17 robustness checks changed a conclusion, and three depended on estimator choice; all three within-country dispersion estimators failed the prespecified band.The dispersion ratios were 0.61, 0.57, and 0.59; design-weighted and unweighted per-item coefficients correlated at 0.999, while direction R² rose from 0.658 to 0.746 under the stated exclusions.Source: Table A5 · p. 45, K Aggregate robustness checks · pp. 45–46
• Two items fell below the reliability screen—hmsfmlsh (0.25) and rlgatnd (0.44)—while inprdsc (0.615) was retained; the median rbc moved from 0.4433 to 0.4425.Single items moved by up to 0.109; the two below-threshold items were excluded from the primary anchored specification.Source: K Aggregate robustness checks · pp. 46–47
• The anchoring triple difference ranged from 0.403 to 0.455 across 10 control sets, against the primary 0.423; the largest shift was 0.032.Across reverse-item sets, the triple difference was +0.4912 for all 12 items, +0.4230 for the primary 10, and +0.1636 for the strictest 8.Source: K Aggregate robustness checks · p. 46
• Under Llama, the forward–reverse gap shifted from +0.3544 to −0.0944; under Qwen, it landed inside the noise band.The Llama triple difference was +0.4488 on the primary 10-item set, with 8 of 10 items increasing.Source: K Aggregate robustness checks · p. 46
• The reported rbc comparison had a median absolute difference of 0.0602 per item and a maximum difference of 0.1854, with no sign reversals.Raw-correlation contrasts are read against the 0.060 band, and Fisher-z contrasts against the 0.079 band.Source: K Aggregate robustness checks · p. 46
• Across four estimators, offset reduction ranged from 53.7% to 64.2%, and the conclusion was unchanged.Offset reduction is the fall in an item’s normalized error when its constant level offset is removed.Source: K Aggregate robustness checks · p. 47
• About a third of shared variance on the uncentered individual series and about two-thirds on the pooled-within series reflected the same misread scale direction.The cross-level agreement description changes under item coding and the choice of individual-level series.Source: K Aggregate robustness checks · p. 47

L Parse failures

• 77.28% of 863,100 scheduled Llama third-person responses parsed, versus 99.99% in the first-person control; no single failure source accounted for most rejections.Rejected outputs included scale enumeration, prompt continuation, code blocks, and junk numbers followed by valid answers.Source: L Parse failures · p. 48

M Sensitivity to the extraction rule

• The primary rule takes the first number in each response, which can discard a valid answer when it follows a junk number.In Llama 3P, 22.6% of rejected responses contained an in-range value somewhere in the text.Source: M Sensitivity to the extraction rule · p. 48
• No alternative extraction rule moved a median between-country correlation by more than 0.013; the primary Qwen 1P median remained +0.4433 under every rule.Table A6 reports sensitivity of aggregate recovery to the extraction rule.Source: M Sensitivity to the extraction rule · p. 48, Table A6 · p. 49
• Table A6 reports arm medians as levels, not paired contrasts; differences between those medians do not estimate matched-arm effects.The block effects are computed item by item on matched arm pairs.Source: M Sensitivity to the extraction rule · p. 49