PaperlayerView exact revision on arXiv

Source-linked AI summary

VISTA: Steering LLMs with Latent User Values Inferred from a Few Survey Answers

Yinuo Xu, Jadie Adams, Brian Hu, Shivani Kumar, David Jurgens, Arslan Basharat

arXiv:2610.23702v1cs.CL

TL;DR

Personalizing language models requires representing what individuals value beyond task-specific preferences or prompt profiles. VISTA infers latent values from survey answers, maps them to interventions in a frozen LLM, and adaptively selects questions. Across three datasets it matches or beats prompting the same evidence, with eight WVS questions recovering 94% of the full-history gain.

  • Problem

    Many personalization methods learn task-specific preferences or rely on user profiles in prompts, even though values are shared across situations and individuals differ in their priorities.

    Source: §1 · p. 1

  • Method

    VISTA infers a posterior over a user's values from survey answers, maps it through a hypernetwork to an intervention in a frozen LLM, and uses the posterior to select questions and stopping time.

    Source: §1 · p. 2, §4.2 · p. 4

  • Results

    Across Food2D, WVS, and PrimeX, VISTA matches or beats prompting the same answers; on WVS, eight questions recover 94% of the gain from a respondent's full history.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The results suggest that a compact value representation, elicited once and reused, is a practical route to aligning a model with an individual.

    Source: §6 · p. 8

    The study does not test whether steering weakens the base model's safety behavior and says personalization should be bounded away from factual or safety-relevant content.

    Source: §7 · p. 9

Abstract

from arXiv · show

Personalizing a language model requires knowing what a person values, which stays largely consistent across tasks. Yet most methods either learn task-specific preferences or place the user's profile in the prompt, where models struggle to act on it. We introduce VISTA (Value-Informed Steering and Targeted Acquisition), which infers a latent profile from users' (full or partial) survey answers, and maps the profile through a hypernetwork to a user-specific steering intervention in a frozen LLM to select answer choices or generate free-text responses. Given the same evidence, VISTA matches or beats in-context profiles on synthetic preferences (Food2D), the World Values Survey (WVS), and PrimeX, without adding prompt tokens. To reduce the cognitive burden of completing full surveys, VISTA also chooses which questions to ask and when to stop: on WVS it stops after 6% of the questionnaire while retaining 94% of the full-history gain, where standard adaptive testing retains at most 42%. The dominant steering direction is an interpretable value axis on which VISTA places unseen users accurately. Our results point toward pluralistic alignment that adapts to individuals at deployable cost.

1 Introduction

Personalization based on task-specific judgments or prompt profiles can miss cross-situation values, while long surveys burden respondents. VISTA infers values from survey answers, steers a frozen LLM, and adaptively selects questions; it reports gains across datasets and retains most WVS full-history gain after a short survey.

1 Introduction

  • Task-specific preference learning ties user representations to the task of the pairwise judgments, whereas values reflect priorities across situations.Demographic personas capture group tendencies rather than individuals; prompt profiles can be ignored by models and grow with each answer they carry.Source: §1 · pp. 1–2
  • VISTA infers a posterior over values from full or partial survey answers and maps it through a hypernetwork to a user-specific intervention in a frozen LLM.The same posterior is used to select questions and decide when to stop.Source: §1 · pp. 1–2, Figure 1 · p. 1
  • VISTA infers a posterior over user values from a few survey answers, generates an activation-level intervention, and selects questions and stopping time.The value encoder transfers from WVS to PrimeX without retraining on the new population and questionnaire.Source: §1 · p. 2
  • Across Food2D, WVS, and PrimeX, VISTA matches or beats prompting the same user evidence; on PrimeX it is the only system with a significant net gain in personalized free text.Each query adds one steering vector or 16 soft tokens to a frozen model, rather than re-reading the user's answers in a prompt.Source: §1 · p. 2
  • VISTA achieves 94% of the full-history gain; its dominant steering direction is a human-readable value axis, positioning unseen respondents at their own position on it.The reported correlations are r = 0.73 on WVS and 0.59 on PrimeX.Source: §1 · p. 2

2 Related Work

Prior work addresses preference modeling, user-specific steering, value representation, and adaptive elicitation through separate approaches. VISTA combines latent value inference with interventions and question selection and stopping.

2 Related Work

  • Preference models infer shared structures from pairwise comparisons, but typically personalize through reranking or reward-guided decoding that requires multiple candidates or per-step evaluation.The surveyed approaches include weighted basis rewards, Gaussian bases, prototype mixtures, variational latents, and sociolinguistic or value dimensions.Source: §2 · p. 2
  • Prior user representations include demographic personas, narratives, rationales, and compressed value profiles, but the cited work reports poor individual representation or limited free-text gains.Structured moral profiles can yield near-chance performance; Sorensen et al. (2025) report no significant gains in free-form text generation from their lossy value profiles.Source: §2 · p. 2
  • Adaptive elicitation work selects informative items or questions, while many LLM-based methods use fixed budgets and some systems re-elicit for each request.VISTA's stated distinction is to infer a value posterior, map it to reusable interventions, and adaptively select questions and stopping time.Source: §2 · pp. 2–3

3 Data

The evaluation covers synthetic food preferences and real-world social values and beliefs, including a transfer test to a new population and survey instrument.

3 Data

  • Food2D is a synthetic food-preference dataset with 10 questions, 20 candidate responses per question, and two annotated features: spiciness and vegetarianness.It contains 1,900 response pairs; simulated users choose noisily according to a two-dimensional taste vector, capping accuracy at 66% even with the true vector.Source: §3 · p. 3
  • WVS uses US and UK respondents answering value statements across 15 topics, yielding 73,288 held-out person-item pairs.The data include 5,652 people and about 130 of 226 value statements per respondent; items are converted to binary choice questions.Source: §3 · p. 3
  • PrimeX tests transfer to a new population, item bank, and domain, using 858 US respondents and 48 opinion and world-belief items.The items cover medicine, science, risk, and primals; responses are converted into pairs of first-person statements.Source: §3 · p. 3

4 Methods

VISTA infers uncertain latent values from survey responses, maps the representation to interventions in a frozen LLM, and uses predictive information gain to select questions and stop enrollment.

4.1 Inferring Latent Values

  • VISTA-VI learns probabilistic latent representations of user values from enrollment surveys, using shared population structure and swap-guided regularization.It builds on Variational Preference Learning and uses semi-amortized inference to refine user-specific representations.Source: §4.1 · p. 3
  • The encoder estimates a diagonal Gaussian posterior over a 32-dimensional user latent from selected-response text embeddings pooled with learned-query attention.The posterior mean represents the estimated preference vector and its variance captures uncertainty; evaluation uses the mean, while training samples the latent via the reparameterization trick.Source: §4.1 · p. 3

4.2 Activation Steering from Inferred Values

  • A hypernetwork maps the inferred value representation to a user-specific intervention in frozen Llama-3-8B-Instruct.For forced choice it outputs a unit-norm steering vector added at layer 10; for free text it outputs a 16-token soft prefix, trained by distillation and personalized DPO.Source: §4.2 · p. 4

4.3 Adaptive Elicitation Framework

  • Question selection maximizes expected predictive information about target-item responses rather than simply reducing uncertainty in the latent representation.The method estimates mutual information for each unasked question using posterior samples and question-specific logistic regression models.Source: §4.3 · p. 4
  • Elicitation stops when the highest remaining EPIG score falls below a fraction of the user's initial maximum, allowing different users to receive different question counts.The comparison with fixed-budget elicitation uses the same users and frozen hypernetwork to measure the tradeoff between enrollment burden and personalized prediction accuracy.Source: §4.3 · p. 4

5 Experiments

The experiments test whether survey-inferred value profiles can personalize choices and free text, and guide question selection and stopping. They compare VISTA with prompt-based profiles and adaptive-testing baselines.

5 Experiments

  • The experiments cover inferred profiles and answer-choice steering, free-text opinions, and adaptive selection of questions and stopping.The evaluation uses Food2D, WVS, and PrimeX; PrimeX tests transfer to a new population and survey instrument.Source: §5 · pp. 4–5, §5.1 · p. 5

5.1 Latent value inference and steering (RQ1)

  • VISTA ties in-context conditioning on PrimeX (+0.41 points) and contested WVS (+0.32), but wins on WVS overall (+2.61) and by nearly ten points on Food2D.These comparisons use the same observed answers for VISTA and the prompt-based profile.Source: §5.1 · pp. 5–6
  • The dominant steering direction tracks religiosity, traditional morality, and immigration restriction, and places unseen respondents along the value axis (r = 0.73 on WVS; r = 0.59 on PrimeX).The WVS axis also tracks the unseen moral-foundations binding factor (r = 0.49).Source: §5.1 · p. 6, Figure 2 · p. 7

5.2 Steering to Generate Free Text (RQ2)

  • On PrimeX, VISTA beats unsteered and all prompting baselines by +2.7 to +4.3, while only its user-control gain is significant (+1.2).The free-text evaluation covers 686 unseen users and uses an LLM judge.Source: §5.2 · p. 6, Figure 3 · p. 7

5.3 Adaptive Elicitation (RQ3)

  • On WVS, VISTA stops after 6.2% of the questionnaire with 93.5% of the gain, versus CAT’s best-threshold 42.2%.On PrimeX, VISTA stops at 25.4% of the pool with 62.5% of the gain, while CAT’s stopping rule cannot fire.Source: Figure 4 · p. 8, §5.3 · p. 8
  • On Food2D, VISTA reaches 85.0% of the available range, versus CAT’s 40.4% and Prompt-Only’s 25.7%.At zero questions, systems do not differ, so the separation comes from question selection.Source: §5.3 · p. 8

6 Conclusion

VISTA infers users’ values from survey answers and uses the resulting profile to steer a frozen LLM and guide question selection. Across three datasets, it matches or beats prompting the same answers, while a small number of questions recovers most of the full-history gain.

6 Conclusion

  • Across Food2D, WVS, and PrimeX, VISTA matches or beats prompting the same answers, at a fixed cost per query.The approach infers a value posterior, maps it to a steering intervention, and uses the posterior to select questions and decide when to stop.Source: §6 · p. 8
  • Eight WVS questions recover 94% of the gain from a respondent’s full history.Source: §6 · p. 8
  • The dominant steering direction is a recognizable value axis that places unseen respondents accurately.Source: §6 · p. 8
  • A compact value representation, elicited once and reused, is presented as a practical route to aligning a model with an individual.Source: §6 · p. 8

7 Limitations

The evaluation and method have boundaries: adaptive elicitation is tested only for forced-choice tasks, and experiments use one frozen base model. The paper also flags unresolved risks around behavior transfer and safety.

7 Limitations

  • Question selection and stopping are evaluated for forced-choice tasks only; which questions inform free-text generation and when to stop remain open.Source: §7 · p. 9
  • All steering experiments use Llama-3-8B-Instruct, and the model-specific hypernetwork must be retrained for each base model.Value inference and adaptive elicitation do not depend on the LLM, but the hypernetwork output is tied to one model’s residual stream.Source: §7 · p. 9
  • The steering intervention uses the posterior mean alone, while posterior variance and answer coherence are not fully used.The authors suggest future work could use these uncertainty signals to scale steering or ask more questions of users poorly represented by the learned structure.Source: §7 · p. 9
  • On majority-view rows, accuracy falls by 3.7 points on PrimeX forced choice and 2.1 points on free text, despite a positive net effect.Source: §7 · p. 9
  • Whether survey-inferred values steer models well on concrete dilemmas remains open, given modest and uneven value–behavior correlations.Source: §7 · p. 9
  • The authors do not test whether steering weakens base-model safety behavior and warn that it may reinforce existing views, including harmful ones.They state personalization should be bounded away from factual or safety-relevant content.Source: §7 · p. 9

9 Appendix

The appendix documents VISTA’s model and steering choices, free-text evaluation setup, and additional personalization analyses. It also reports that K = 32 was the only latent size at which every search trial converged.

9.2 Value-model search: latent size and training hyperparameters

  • The value-model search tunes latent size, KL regularization, free bits, encoder and decoder learning rates, and related settings to predict held-out survey responses while guarding against posterior collapse.K denotes latent dimension; βtarget weights the KL term, βσ separately scales its width component, and free bits apply to the mean component.Source: §9.2 · p. 12
  • K = 32 was the only latent size at which every trial converged, so the authors selected it as the robust choice despite the best single trial occurring at K = 16.The appendix notes that trial counts were small and adaptively allocated; four of six K = 16 trials collapsed.Source: Table 1 · p. 12, §9.2 · p. 12

9.4 Value-model training and refinement

  • The appendix describes swap-guided regularization and semi-amortized user refinement, including 100 user-specific optimization steps initialized by the encoder.The refinement balances observed enrollment evidence against the population-informed initialization; an alternating decoder-update procedure is also used to improve consistency.Source: §9.4 · pp. 12–13

9.5 Steering details

  • For forced-choice steering, the hypernetwork outputs a fixed-norm vector whose direction varies by user, injected at layer 10 with strength α = 1.The base LLM remains frozen, while gradients propagate through its injection site to train the hypernetwork end-to-end.Source: §9.5 · pp. 13–14
  • For free-text generation, VISTA distills value-conditioned targets into a soft-prefix hypernetwork, then applies personalized DPO to length-matched explanations taking opposing positions.Food2D targets are selected from 16 candidates using the value model; PrimeX targets are explanations from respondents choosing the predicted option, with random-target controls.Source: §9.5 · p. 14

9.6 Steering to generate free text experiment details (RQ2)

  • The free-text evaluation excludes WVS because it has no user-written responses and uses a separate-family judge for Food2D and PrimeX.The judge is validated on both datasets; PrimeX evaluation covers 686 unseen respondents, and the Food2D baselines use oracle taste profiles while VISTA never sees the true preference vector.Source: §9.6 · pp. 14–15

9.7 Active elicitation experiment details (RQ3)

  • VISTA’s Food2D steering arm leads the in-context arm by +9.57 points [+7.83, +11.33] on byte-identical held-out pairs.The value-inference analysis also reports the largest user-specific lift for VISTA-VI on Food2D (+11.4 vs PAL’s +9.3) and WVS (+8.7 vs +6.6); PAL leads on PrimeX AB (+3.5 vs +2.9).Source: Table 4 · p. 17, Table 5 · p. 18
  • Additional personalization analyses separate minority-view from majority-view rows, report share-weighted net gains, and show illustrative PrimeX texts for unseen minority-view users.The PrimeX analysis covers 858 unseen users; the WVS analysis covers 848 unseen respondents, and the examples are drawn from rows where text takes the user’s side only under their own profile.Source: Table 6 · p. 18, Table 7 · p. 19, Table 8 · p. 19
Loading 2610.23702v1…