Adaptive Resource Allocation for Effective and Efficient LLM Social Survey Simulation

1 INTRODUCTION

• LLM survey simulation predicts an unanswered response from a respondent’s persona and relevant past answers, offering a scalable complement to fielding surveys.Retrieved answers serve as personal references for grounding predictions in the respondent’s expressed views.Source: §1 · p. 1
• Fixed, often large model-and-history configurations overlook request-dependent needs: stronger models and additional history can help some requests but add cost or distract on others.When respondent patterns are clear, a smaller model may suffice; additional history can have diminishing or negative returns once evidence is sufficient.Source: §1 · p. 2
• Changing history budget changes the better-performing model for 20.8% of requests, while switching models changes the better-performing history budget for 30.7%.These findings motivate jointly selecting model and history budget at the request level.Source: §1 · p. 2
• E2Sim learns a lightweight policy from offline configuration evaluations to predict Accuracy–Cost Trade-off and jointly select model and history budget per request.Its ACT score is Accuracy − α · Cost, with α controlling the relative cost penalty.Source: §1 · p. 2
• E2Sim perturbs retrieved histories by dropping and swapping responses, then trains from large ACT-margin decisions toward ambiguous ones.This combines robustness-oriented history augmentation with a margin-based curriculum.Source: §1 · pp. 2–3
• The paper evaluates the framework across four surveys, varied model pools, and history budgets.The authors characterize these as evaluations of effectiveness and robustness.Source: §1 · p. 3

2 PROBLEM FORMULATION AND DEFINITION

• A respondent u has a textual persona P_u and observed responses S(u); the target question q includes its response options.Together these define the inputs to a simulation request.Source: §2 · p. 3
• A history budget k determines H_k, the k historical responses most relevant to the target question, which are combined with the persona and question for model prediction.The context is ranked by relevance before the model predicts an answer.Source: §2 · p. 3
• The candidate model pool M and available history budgets K form a joint action space, and an allocation policy maps each request to one joint action.This formulation captures interdependence: model performance can vary with k, and history’s predictive value can vary with m.Source: §2 · p. 3

3.1 SURVEY-GROUNDED OFFLINE SUPERVISION

• Offline supervision scores each model–history action using accuracy and normalized price-weighted input-token cost, preserving the full ACT outcomes rather than only a best-action label.Cost reflects model-specific token prices and prompt length; ACT combines accuracy and cost through α.Source: §3.1 · p. 4

3.2 JOINT MODEL–HISTORY ALLOCATION POLICY

• The joint scorer uses persona, respondent history, and target question with maximum-budget history to predict scores for every model–history configuration.Joint scoring preserves the interaction between model choice and history budget within one allocation decision.Source: §3.2 · pp. 4–5
• Drop/Swap augmentation varies history content and order while keeping persona and target question fixed, encouraging stable allocation under small history changes.It addresses incomplete histories and changes in the ordering of similarly relevant responses.Source: §3.2 · p. 5
• The ACT-margin curriculum starts with requests whose best and second-best actions are clearly separated, then progressively adds more ambiguous allocation decisions.The admitted training subset expands over epochs from requests with the largest margins.Source: §3.2 · p. 5

3.3 REQUEST-LEVEL ADAPTIVE INFERENCE

• At deployment, the policy selects a model and history budget for each request, and the selected model predicts using only the chosen top-k responses.The policy examines up to kmax responses, while each request requires one policy evaluation and one call to the selected survey model.Source: §3.3 · p. 5

4.1 EXPERIMENTAL SETUP

• Experiments cover ANES, BSA, GSS, and WVS, using three candidate models and history budgets K = {0, 1, 3, 5}.The model pool is Qwen3-8B, Qwen3-32B, and DeepSeek-V4-Flash; k=0 denotes no history.Source: §4.1 · p. 6
• E2Sim is compared with WCR, Random, LLM-S, and oracle BF using Accuracy, normalized Cost, and ACT.The policy uses a ModernBERT-base encoder and linear scoring head; its cost and LLM-S’s Qwen3-32B inference overhead are included in reported Cost.Source: §4.1 · pp. 6–7

4.2 MAIN RESULTS

• Across four datasets and both model configurations, E2Sim improves on oracle Best Fixed (BF) by 1.5–3.3 accuracy points and cuts normalized Cost by 10.4–55.3% for Qwen3-8B + Qwen3-32B.For Qwen3-8B + DeepSeek-V4-Flash, it raises Accuracy by 0.1–2.1 percentage points, reduces Cost by 25.6–37.1%, and raises ACT by 0.9–3.2 percentage points relative to BF.Source: §4.2 · p. 7
• Offline supervision breaks even after 57K–78K requests across datasets, or 3.6–4.9 times the training volume.At original survey scales, the other three datasets also exceed their break-even points by factors of 2.9–21.9.Source: §4.2 · p. 7
• Across evaluated α values, E2Sim has a better Cost–Accuracy trade-off than either fixed-model curve, while larger α prioritizes lower Cost and smaller α prioritizes Accuracy.Figure 3 compares Qwen3-8B and Qwen3-32B at different history budgets with E2Sim across α values.Source: §4.2 · p. 7, Figure 3 · p. 7

4.3 ROBUSTNESS ANALYSIS

• Across four history-budget spaces and two datasets, E2Sim exceeds BF in Accuracy and ACT and lowers normalized Cost, with ACT gains of 1.7–4.3 percentage points.It also outperforms BF with K={0}, where only model choice is adaptive.Source: §4.3 · p. 8, Table 2 · p. 8
• With one or three candidate models, E2Sim improves over BF on ANES and BSA; the ACT gains are 6.6 and 3.5 points with one model, and 3.1 and 1.9 with three.The three-model pool enables joint model and history-budget selection; the one-model pool permits history-budget adaptation only.Source: §4.3 · p. 8, Table 3 · p. 8

4.4 ABLATION STUDY

• Across ANES, GSS, and WVS, full E2Sim beats all ablations; curriculum removal hurts most on GSS, joint-selection replacement on ANES and WVS, and augmentation removal lowers Accuracy and raises Cost everywhere.The ablations remove the ACT-margin curriculum, respondent-history deletion and permutation during training, or joint model–history selection.Source: §4.4 · pp. 8–9, Figure 4 · p. 9

LLM-based Survey Simulation

• Survey-simulation research ranges from persona-conditioned synthetic respondents to specialized and respondent-level approaches, while E2Sim allocates model and historical-response context per request.Prior approaches generally treat model and respondent context as study-level choices.Source: §5 · p. 9

LLM Routing and Adaptive Inference

• LLM routing assigns queries to models with suitable capability–cost trade-offs, whereas E2Sim jointly allocates model capability and respondent-history budget for each survey request.The reviewed routing approaches use signals including preference feedback, query representations, contrastive supervision, and model–query relations.Source: §5 · p. 9

6 CONCLUSION

• E2Sim jointly selects a model and history budget per request using respondent-history drop-and-swap augmentation and an ACT-margin curriculum; it generally raises ACT and improves Accuracy and reduces Cost in most evaluated settings.Future work will extend the action space and adapt model-cost weights to deployment-specific pricing.Source: §6 · p. 10

AI USE STATEMENT

• Candidate LLMs generated survey responses and Qwen3-32B served as the configuration-selection baseline; generative AI assistance was limited to editing, translation, formatting, and troubleshooting.The authors state that these tools did not originate the research idea, formulate the method, design experiments, alter data or results, or interpret findings.Source: AI USE STATEMENT · p. 10

• The study uses publicly released social-survey datasets and publicly available language models, and collects no new data from human participants.Source: ETHICS STATEMENT · p. 10
• Simulated responses are research artifacts, not substitutes for informed responses from real individuals or authoritative statements about demographic groups.Source: ETHICS STATEMENT · p. 10
• Language-model outputs may reproduce biases or errors from their training data and prompts, so reported simulations warrant caution.Source: ETHICS STATEMENT · p. 10

• The paper provides formal problem and training details, experimental settings and evaluation protocol, and evaluation prompt templates.Source: REPRODUCIBILITY STATEMENT · p. 10
• Code, experiment launchers, figures, and table-generation scripts are released, while exact reproduction requires the specified survey data, model endpoints or checkpoints, embedding model, and model-price settings.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 DATASET STATISTICS

• The experimental pool contains 20,000 observed respondent–question pairs per survey; 20% are randomly held out, and histories for both splits come only from training responses, excluding the target pair.Source: §A.1 · p. 13, Table 4 · p. 13

B.1 SURVEY-RESPONSE GENERATION PROMPT

• The response-generation prompt supplies persona, retrieved historical answers, and the target question, instructing the model to use past answers as behavioral evidence and return one listed option as JSON.Source: §B.1 · pp. 13–14

B.2 LLM-BASED SELECTOR PROMPT

• The LLM-based Selector receives the persona, question, available histories, configurations, and normalized costs, then estimates accuracy and chooses the action maximizing Accuracy − 0.05 × Cost.Source: §B.2 · pp. 14–15

C COST COMPUTATION

• Normalized Cost uses a shared setting-specific maximum candidate execution cost, and reported selector and E2Sim costs include their respective selection-model costs.Source: C COST COMPUTATION · p. 16

D OFFLINE-SUPERVISION AMORTIZATION DETAILS

• The amortization analysis counts the one-time offline-supervision penalty against per-request deployment gains over Best Fixed, while assigning that baseline no offline selection cost.Source: D OFFLINE-SUPERVISION AMORTIZATION… · p. 16

E TWO-STAGE ABLATION DETAILS

• The No Joint Learning ablation first selects a model from no-history ACT predictions, then uses that model’s history selector to choose the budget; only Stage 2 receives history augmentation.Source: E TWO-STAGE ABLATION DETAILS · p. 17, Figure 5 · p. 17

• E2Sim operates over a fixed set of candidate models and history budgets; adding one requires evaluating the added configurations and updating the allocation policy.Source: F LIMITATIONS · p. 17
• Empirical relative model-cost weights support controlled comparisons but do not universally estimate latency or monetary cost; deployment should use current API prices or target-hardware measurements.Source: F LIMITATIONS · p. 17