Source-linked AI summary
Do LLMs Have Values? A Quantitative Analysis and Alignment Framework for Values in Large Language Models
Keqing Zhang, Jingyu Chen, Yufan Liu, Yongqiang Zhu, Nai Ding, Lai Jiang, Congyan Lang, Bing Li, Weiming Hu
TL;DR
LLM value alignment lacks a systematic account of whether models possess values, how those values shift, and how to steer them. This paper maps LLM and human values, introduces the PEC framework, and develops an adaptive Alignment Prescription. It finds concentrated value cores, measurable value dynamics, and more efficient, precise steering while preserving general capabilities.
Problem
Existing alignment efforts lack a systematic account of LLM value systems and must address both wording-sensitive swings and resistance to explicit bias correction.
Source: §1 · p. 2, §2.3 · p. 11Method
The study maps LLM and human responses into shared sociological spaces, models value expression with the Prior-Environment-Cognition framework, and derives adaptive interventions.
Source: §1 · p. 4, §2.3 · p. 12Results
LLMs possess internal value systems concentrated around an idealized value core, while PEC quantifies their shifts and enables more efficient, precise steering without degrading general capabilities.
Source: §1 · pp. 4–5Takeaways & Limitations
LLM value alignment can be approached as a measurable, mechanistic process rather than opaque trial-and-error prompting.
Source: §5 · p. 35Limited coverage of cross-lingual, low-resource, and cross-cultural scenarios restricts the generalizability of the reported value topographies.
Source: §3.6 · p. 25
Abstract
from arXiv · showhide
As Large Language Models (LLMs) increasingly handle complex subjective tasks, aligning their intentions and behaviors with human values has become a critical scientific challenge. However, current efforts are confounded by a striking behavioral paradox: they fluctuate unpredictably under minor wording changes ("swing"), yet stubbornly ignore explicit instructions to correct ingrained biases ("rigidity"). Resolving this duality is critical for reliable AI alignment. To systematically understand and safely steer these latent subjective preferences, our study is structured around three fundamental questions. First, do LLMs possess an intrinsic value system? By projecting responses from 106 LLMs (150,000 queries per model) and 95,000 human survey profiles into a shared sociological space, we empirically confirm that they do. However, they do not mirror human diversity, instead crystallizing into a highly concentrated, idealized value core. Second, how can these values be quantified? We propose the Prior-Environment-Cognition (PEC) framework. This model mathematically defines value expression as the joint outcome of inherent dispositions like parameter weights (Prior), external contexts such as user prompts (Environment), and internal reasoning processes like Chain-of-Thought (Cognition). Finally, how can LLMs' values be aligned toward a desired target? Using PEC diagnostics, we establish an adaptive "Alignment Prescription". Rather than blindly applying resource-intensive training, this method identifies the minimum effective intervention needed for each dimension, ranging from zero-cost prompts to targeted parameter updates. Extensive empirical validation confirms that our approach successfully verifies the presence of LLM values, accurately quantifies their shifts, and achieves more efficient and precise steering than conventional blind training, all without degrading general capabilities.
1 Introduction
The study addresses whether LLMs possess values, how those values arise and shift, and how they can be aligned efficiently. It combines large-scale sociological mapping with the PEC framework and targeted interventions to characterize concentrated value systems, explain swing and rigidity, and steer values without harming general capabilities.
- LLMs exhibit a behavioral paradox: minor wording changes can produce swing, while alignment interventions fail to correct persistent biases, motivating investigation of an internal value system.This duality challenges safety approaches focused only on toxic-content avoidance and raises concerns about fairness and reliability in subjective, culturally sensitive applications.Source: §1 · p. 2
- Across 106 LLMs and approximately 95,000 WVS-7 respondents, shared 10-dimensional mapping reveals a concentrated, idealized value core rather than human-like value diversity.Each model receives 150,000 queries across 625 scenarios, and its value expression is modeled as a distribution rather than a fixed vector.Source: §1 · pp. 3–4
- The Prior-Environment-Cognition (PEC) framework explains value expression as the joint effect of parameter-based Prior, prompt-based Environment, and reasoning-based Cognition, making swing analyzable rather than random noise.PEC treats values as probabilistic and provides diagnostics for identifying value deviations across these dimensions.Source: Figure 1 · p. 3, §1 · p. 4
- The adaptive Alignment Prescription assigns interventions to value dimensions, using prompts or Chain-of-Thought for flexible values and LoRA or instruction tuning for rigid biases.The strategy replaces costly one-size-fits-all alignment with cost-minimal, targeted corrections intended to preserve overall model performance.Source: Figure 1 · p. 3, §1 · pp. 4–5
- Extensive validation confirms that the framework verifies LLM values, quantifies their shifts, and steers them more efficiently and precisely than blind training while preserving general capabilities.The reported contributions integrate sociological mapping, PEC-based value dynamics, and diagnostic intervention into a quantitative framework for understanding and steering LLM values.Source: §1 · pp. 4–5
2 Results
The results characterize LLMs’ volatile yet bounded subjective judgments, concentrated value profiles, and interacting prior, environmental, and cognitive factors, and examine how these profiles can be redirected through PEC-guided, dimension-specific interventions.
2.1 Do LLMs Have Values?
- Subjective outputs swing across repeated queries yet orbit rigid internal attractors, with Cons−Acc gaps of +0.53 for DeepSeek-V3 and +0.39 for GPT-4o.Different models display distinct subjective viewpoints that conventional safety alignment and instruction tuning do not effectively overwrite, suggesting values are anchored in parameter structure.Source: §2.1 · pp. 5, 7
2.2.2 Value Crystallization: LLMs vs. Humans
- LLM value distributions crystallize outside human diversity: 91.9% of the human-distance gap reflects centroid displacement, and 11 of 21 models have lower variance than every human national group.Most LLMs exceed the maximum human cross-national W2 distance of 0.484, indicating displacement and greater ordering than observed human populations.Source: §2.2.2 · p. 9
2.2.3 LLMs Crystallize at the Prescribed Edge of Human Values
- Frontier LLMs most closely match Arch 5, the rarest human archetype at 10.5%, characterized by Stimulation 0.85, Power 0.69, and Self-Direction 0.65.The table reports average deviations of +86.7% for closed-source and +42.7% for open-source models, with Gemini 2.5-Pro reaching +185.8%.Source: §2.2.1 · p. 8, Table 1 · p. 10, §2.2.3 · p. 11
2.2.4 Different LLMs Hold Different Values
- Model capability and alignment maturity differentiate value structures: closed-source models show σ̄ = 0.159 and distances 0.536–1.505, whereas earlier open-source models span distances 0.410–1.146.Later generations and alignment updates reposition value crystals, making model version essential metadata for value evaluation.Source: §2.2.4 · p. 11
2.3 Formalizing the Dynamics of LLM Values
- The PEC framework models value expression as the interaction of parameterized Prior, contextual Environment, and Chain-of-Thought Cognition, explaining both swing and rigidity.Prior encodes dispositions in parameter weights, Environment supplies prompts, and Cognition maps their interaction into expressed values.Source: §2.3 · p. 12
2.3.1 Environment (Factor E): Contextual Prompts Unevenly Reshape Value Distributions
- Environmental prompts reshape values asymmetrically: social norms affect eight dimensions, hierarchy affects six, and nonlinear modeling improves explained variance by up to +0.43 for Security.Power, Stimulation, and Hedonism are most susceptible, while Universalism and Benevolence remain comparatively resistant.Source: §2.3.1 · p. 14
2.3.2 Cognition (Factor C): Chain-of-Thought Generation Produces Structured Value Shifts
- Chain-of-Thought shifts values predictably: Hedonism rises +0.056 and Achievement +0.047, mean fit reaches R̄2 = 0.60, and risk susceptibility increases 13%.Instruction-tuned models shift toward human references more reliably than base models, while reasoning reduces general social-context sensitivity by 10%–14%.Source: §2.3.2 · pp. 14–16
2.3.3 Prior (Factor P): Parameter Updates Systematically Consolidate and Redirect Values
- Parameter updates consolidate values and displace their centroids, but alignment direction depends on the recipe: intensity and convergence increase without guaranteeing human proximity.Instruction tuning shifts Qwen toward the human reference but earlier Llama models substantially away from it, demonstrating that update objectives and data determine orientation.Source: §2.3.3 · pp. 16–17
- Parameter updates progressively consolidate value distributions across GPT, Qwen, and Llama generations, while instruction tuning also shifts model centroids and redirects value orientations.GPT models show especially strong convergence from GPT-2 through GPT-5; Base-to-Instruct transitions both increase concentration and displace the value centroid.Source: Figure 6 · p. 17
2.4 Adaptive Alignment Prescription
- The four-tier prescription assigns zero-cost prompting and Chain-of-Thought to malleable dimensions, targeted fine-tuning to resistant ones, and full pre-training to values immune to partial updates.This hierarchy operationalizes PEC susceptibility and displacement metrics to minimize intervention cost while targeting the required alignment strength.Source: §2.4 · pp. 18–19
- 0.44 to 0.85: Level-1 prompting precisely increased Qwen2.5’s Power score, while prescribed interventions avoided amplifying answer swing.For resistant Tradition↑ dimensions, shallow interventions produced negligible movement, whereas the appropriate intervention level enabled effective realignment without broader profile distortion.Source: Figure 7 · p. 19, §2.4 · p. 20, Figure 8 · p. 20
- 70.0% match rate on PKU-SafeRLHF: the prescription generalized across datasets, with perfect predictive matching at Levels 1 and 4 but mismatches at intermediate levels.The mismatches reflect an antagonistic interaction in which Cognition dampens Environment sensitivity, rather than independent intervention tiers.Source: §2.4 · pp. 20–21, Table 3 · p. 21
3 Discussion
The discussion reframes LLM values as dynamic probabilistic fields shaped by compressed priors, contextual environments, and cognition rather than fixed human-like personalities. It argues for adaptive alignment interventions while identifying limits in human-centric evaluation and cross-cultural coverage.
3.1 Value Dynamics as a Field Theory
- LLM values form continuous probability distributions rather than fixed personas, with swing arising from contextual and reasoning sensitivity and rigidity from crystallized priors.The PEC framework formalizes value expression as v = C(P, E), distinguishing latent parameter priors from manifest outputs.Source: §3.1 · p. 22
3.2 LLMs as Aligned Tools Rather Than Human Surrogates
- LLMs cannot adequately represent natural human populations because alignment policies compress their value spaces into narrow, idealized regions.They are better understood as highly capable tools than as ecologically valid human surrogates for psychological or sociological research.Source: §3.2 · p. 22
3.3 Prior (Factor P): Information Compression Drives Value Crystallization
- Larger models show denser value crystallization because information compression forces them to discard granular facts and encode universal judgment heuristics as entrenched priors [53].The discussion attributes this pattern to an estimated average capacity of approximately 3.64 bits per parameter [53].Source: §3.3 · p. 23
3.4 Cognition and Environment (Factors C & E): Contextual Framing and Cognitive Reshaping
- Prompt context rapidly frames value expression, while extended Chain-of-Thought reasoning actively reconstructs it by recruiting broader knowledge across the parameter space.Together, Environment and Cognition operate as non-neutral, dual-process modulators negotiating with the crystallized Prior, paralleling human System 1 and System 2 processes.Source: §3.4 · p. 23
3.5 Adaptive Prescription: Minimizing the Alignment Tax
- Adaptive alignment should use prompt or inference interventions for flexible dimensions and reserve parameter updates for resistant values, minimizing alignment tax and capability degradation.Indiscriminate fine-tuning can compromise safety guardrails across unrelated domains, whereas context-level steering often corrects expression without changing base parameters.Source: §3.5 · p. 24
3.6 Limitations and Future Directions
- Human-centric value scales may miss implicit, high-dimensional LLM features, and current evidence remains concentrated on English prompts and mainstream architectures.Future work should develop LLM-native probes and expand evaluations across low-resource languages, cultures, and marginalized contexts.Source: §3.6 · pp. 24–25
3.7 Summary
- Overall, the framework turns value alignment from opaque trial-and-error into mechanistic, prescriptive steering that keeps generative models controllable and transparent.This synthesis links probabilistic value fields, compression-driven crystallization, dual-process inference, and targeted interventions that reduce alignment tax.Source: §3.7 · p. 25
4 Method
This section presents a reproducible, human-anchored pipeline for measuring LLM value systems, decomposing value expression through the Prior-Environment-Cognition framework, and assigning cost-minimal alignment interventions. It also validates the prescription framework on held-out model–dimension pairs while situating it within existing methods and examining limitations in distributional value modeling and long-term value adherence.
4.1 Methodology overview
- The pipeline measures LLM values, quantifies them with PEC, and steers them through an adaptive prescription assigning the minimum-cost intervention per dimension.These components address value-system existence, value quantification, and targeted alignment, respectively.Source: §4.1 · p. 25
4.2.1 Administering value assessments to LLMs
- The measurement protocol grounds ten-dimensional value assessment in the balanced, cross-culturally established Schwartz Theory of Basic Human Values [17].The framework includes opposing orientations such as Self-Transcendence and Self-Enhancement.Source: §4.2.1 · p. 26
- 31.8M queries across 106 models, 242 WVS items, and 625 contextual prompt backgrounds produce ten-dimensional value vectors from response probabilities under controlled contexts.The assessment uses 23 closed-source and 83 open-source models and aggregates signed item-to-dimension contributions while answering items independently.Source: §4.2.1 · p. 26
4.2.2 Constructing the shared value manifold
- A WVS-7-anchored Shared UMAP Value Manifold provides a fixed coordinate system for reproducible comparisons among LLMs and human populations.Human respondents are clustered into five archetypes using a 20,000-person subsample, nearest-neighbor assignment, silhouette validation, dominant dimensions, and population shares.Source: §4.2.2 · p. 27, Identification of Human Value Archetypes · p. 27
4.2.3 Distribution-level model-human comparison
- Distribution-level comparison models LLM outputs and human cohorts as Gaussian distributions and uses Gaussian 2-Wasserstein distance with a mean-covariance decomposition to capture location shift and structural dispersion.Source: §4.2.3 · pp. 27–28
4.3 The PEC framework: modeling the determinants of value expression
- PEC represents expressed values as the integrated interaction of Prior, Environment, and Cognition, with the mapping approximated locally by a first-order superposition plus residual coupling.Prior is the latent disposition, Environment is contextual framing, and Cognition reflects generation strategy such as direct answering or Chain-of-Thought.Source: §4.3 · pp. 28–29
Prior (Factor P): intrinsic value baseline.
- Prior is measured from neutral zero-shot outputs across 106 models and analyzed using per-dimension regression over parameter count, instruction-tuning stage, and model family.Environment susceptibility is fitted across ten dimensions and four contextual factors with linear-response and quadratic-saturation terms, while Cognition is the CoT-minus-Direct value-vector shift.Source: Prior (Factor P): intrinsic value… · p. 29, Environment (Factor E): contextual… · p. 30, Cognition (Factor C):… · p. 30
4.4 Alignment Prescription
- The Alignment Prescription converts PEC effects into comparable per-dimension displacements, calibrates effectiveness against natural WVS-7 population variation, and selects the lowest-cost level exceeding threshold τ.The hierarchy uses ascending Environment, Cognition, and parametric costs; CoT typically reduces environmental sensitivity, with average α = 0.88, and resistant dimensions are labeled pre-train locked.Source: §4.4 · p. 31, Operationalizing PEC factors into… · p. 31, Alignment threshold · p. 32
Prescription validation.
- Prescription validation compares prescribed intervention levels with the lowest effective levels observed on held-out model–dimension pairs to quantify framework reliability.Reliability is measured by the match rate across all evaluated pairs.Source: Prescription validation · p. 34
4.5.1 Evaluation of LLMs’ values
- Prior value evaluations use moral frameworks or Schwartz-based mappings, but typically compress each LLM into a single value vector and lose intrinsic distributional structure.The paper addresses this gap with an uncertainty-centric approach intended to connect model values with human socio-cultural benchmarks.Source: §4.5.1 · p. 34
4.5.2 Value Alignment of LLMs
- Existing alignment paradigms include SFT, RLHF, Constitutional AI, multi-objective reinforcement learning, and in-context alignment [73].These methods respectively inject norms through supervised data, preference rewards, AI-generated principles, multi-dimensional objectives, or inference-time instructions and critiques.Source: §4.5.2 · pp. 34–35
- Despite these advances, current alignment strategies remain challenged by unforeseen risks and by maintaining robust, long-term value adherence in complex environments.This limitation motivates continued work on reliable value alignment beyond existing methodological advances.Source: §4.5.2 · p. 35
5 Conclusion
The study concludes that LLMs possess intrinsic values that can be quantified through PEC and aligned through the Alignment Prescription, shifting alignment toward a mechanistic, quantitative science.
5 Conclusion
- LLMs possess an intrinsic value system that can be quantified through PEC and aligned through the Alignment Prescription.These findings answer the study’s three central questions about value existence, quantification, and alignment.Source: §5 · p. 35
- Value expression follows measurable structure rooted in Prior, Environment, and Cognition rather than being mere prompting noise.This reframes value alignment from opaque trial-and-error toward mechanistic understanding and precise intervention.Source: §5 · p. 35
- The framework aims to reduce alignment tax through precise, prescriptive interventions while keeping generative models transparent and controllable.The conclusion emphasizes structural understanding as increasingly important as LLMs assume more decision-making roles.Source: §5 · p. 35
7 Code availability
The study’s code for data processing, model evaluation, and figure generation is publicly available on GitHub, alongside an interactive demonstration.
7 Code availability
- The repository provides Python 3.10 scripts, analysis dependencies, and an interactive demonstration for reproducing the study’s data processing, model evaluation, and figure generation.Dependencies include pandas, numpy, scikit-learn, matplotlib, and transformers.Source: §7 · p. 36
Appendix A WVS Human Data Analysis Results
The appendix maps WVS responses into interpretable ten-dimensional Schwartz value vectors using ensemble LLM annotation and normalized scoring. Clustering these vectors identifies five global human value archetypes, with Dynamic Challenger closest to current LLM value profiles.
Value Representation
- WVS responses are represented as ten-dimensional vectors spanning Power, Achievement, Hedonism, Stimulation, Self-Direction, Universalism, Benevolence, Tradition, Conformity, and Security.The dimensions follow Schwartz’s circumplex framework, preserving theoretical meaning for quantitative analysis and cross-group comparison.Source: Appendix A · pp. 43–44
Item Annotation
- Five LLMs independently annotate each WVS item, with majority voting determining activated Schwartz dimensions and unanimous agreement indicating highest confidence.For Q2P, all five models assigned Stimulation (ST01) and Conformity (CO01), producing a 5–5 vote for those dimensions.Source: Appendix A · pp. 43–44
Value Scoring
- Response options are linearly normalized to [0, 1], nonresponses are excluded, and each dimension score averages associated item scores, counting multidimensional items equally for each label.Each respondent’s resulting vector provides the unified quantitative input for subsequent analyses.Source: Appendix A · p. 44
Archetype Results
- 37.1% of respondents belong to Curious Idealist, followed by Quiet Conformist (24.4%), Conventional Authority (14.4%), Driven Achiever (13.7%), and Dynamic Challenger (10.5%).Fig. A1 visualizes country-level archetype distributions and global composition; Curious Idealist is the largest archetype, while Dynamic Challenger is the smallest.Source: Figure A1 · p. 45, Appendix A · pp. 45–46
- Dynamic Challenger is the human archetype closest to current LLMs, combining extreme Power with prominent Self-Direction and representing a minority human value profile.It is geographically concentrated in South America and Eastern Europe and occupies the most radical position in Schwartz value space.Source: Appendix A · p. 46
Appendix B Formal Definitions of Swing Experiment Metrics … Accuracy and Correct Count
The appendix formalizes swing metrics from five repeated model responses under fixed inference conditions, covering response frequency, consistency, entropy, and correctness. Consistency and entropy characterize response dispersion, while accuracy measures the fraction of runs matching the ground truth.
Appendix B Formal Definitions of Swing Experiment Metrics
- Each query is evaluated with N = 5 independent responses under fixed temperature T = 0.2 and constant inference parameters.The response multiset is A = {a1, a2, a3, a4, a5}, with each ai drawn from the categorical answer space V.Source: Appendix B · p. 46
Response Frequency
- The raw frequency f(v) counts how many collected responses in A equal each answer v ∈ V.The empirical response distribution is defined over the five-run response multiset.Source: Response Frequency · p. 46
Consistency
- Consistency is based on the plurality answer v∗, defined as the label appearing most frequently across the N runs.It measures concentration of responses on the majority label.Source: Consistency · p. 46
- Consistency ranges from 1/N to 1, where 1 means identical answers across runs and 1/N means maximal dispersion across N distinct answers.The lower and upper bounds correspond to the most dispersed and most concentrated response patterns, respectively.Source: Consistency · p. 46
Entropy
- Entropy uses Shannon entropy with the natural logarithm to quantify uncertainty in the response distribution across runs.Entropy is computed in nats over answers with positive empirical probability.Source: Entropy · p. 47
- Entropy ranges from 0 to ln N, with zero for perfect consistency and ln N ≈1.609 for a uniform distribution across N distinct answers.Consistency and entropy are monotonically related: higher consistency implies lower entropy, and vice versa.Source: Entropy · p. 47
Accuracy and Correct Count
- Accuracy counts runs whose answers equal the ground-truth label y∗, yielding Acc(q) values from 0 through 1 in increments of 1/N.The per-run correctness indicator is 1[ai = y∗].Source: Accuracy and Correct Count · p. 47
Consistency and Accuracy Gap
The consistency–accuracy gap measures divergence between behavioral stability and factual correctness. Positive gap values indicate repeated commitment to an incorrect answer, placing models in the hallucination zone.
- The gap quantifies divergence between a model’s behavioral stability and factual correctness on query q.Source: Consistency and Accuracy Gap · p. 47
- A positive δ(q) indicates bounded swing with high Consistency but low Accuracy, as the model repeatedly commits to a wrong answer.This behavior corresponds to an incorrect attractor in the hallucination zone identified in Fig. 2.Source: Consistency and Accuracy Gap · p. 47
- ¯δ = +0.39 for GPT-4o and +0.53 for DeepSeek-V3 on emoji prediction, confirming both models fall into the hallucination zone.The gap is averaged across queries at the dataset level.Source: Consistency and Accuracy Gap · p. 47
Appendix C LLMs’ Values Evolution … C.2 Prompts effect
The appendix shows that model scaling and iterative alignment concentrate LLM value distributions, while contextual prompts produce structured, nonlinear, dimension-specific value shifts that can be modeled through susceptibility and saturation.
Appendix C LLMs’ Values Evolution
- Across Qwen generations and scales, value distributions crystallize into tighter clusters that move farther from the human baseline, with larger models showing smaller variance and stronger directional coherence.Earlier and smaller models are more diffuse and partially overlap with human values, whereas later and larger variants become more rigid and idealized.Source: Figure C2 · p. 48
- The appendix compares value-distribution evolution across Llama generations and parameter scales using UMAP projections against the WVS-7 human baseline.Rows distinguish Llama generations, columns are ordered by descending parameter scale, and release dates are shown beside model names.Source: Figure C3 · p. 49
- The GPT comparison organizes value-distribution evolution across GPT-5, GPT-4/4.1, and GPT-2 families by model generation and scale against the WVS-7 human baseline.Hollow diamonds mark smaller or earlier variants, while filled diamonds mark larger or more recent variants.Source: Figure C4 · p. 50
C.1 Physical Analogy for the Susceptibility Matrix
- The susceptibility analogy motivates treating contextual prompts as external fields and value shifts as the model’s response to those fields.This framing connects the prompt environment to the Environment component of the field-theoretic value-expression model.Source: §C.1 · p. 50, Susceptibility in physics · pp. 50–51, Mapping the analogy to LLM value… · p. 51
Mapping the analogy to LLM value expression
- The value susceptibility matrix χ ∈R10×4 quantifies how strongly each value dimension responds to each environmental factor.Large χd,j values indicate greater perturbability, while small values indicate stability under the same environmental perturbation.Source: Mapping the analogy to LLM value… · p. 51
Linear response and saturation
- At low prompt intensity, value shifts are approximately linear with slope χd,j, whereas stronger intensity introduces quadratic corrections and possible saturation or reversal.The quadratic coefficient γd,j captures departures from linearity and is interpreted as rigidity under extreme situational stress.Source: Linear response and saturation · p. 51
Why this framing is useful
- Estimating χ and γ from a small number of tested conditions enables prediction of value displacement at untested pressure levels without exhaustive enumeration.The field-response formulation treats value change as structured rather than as an isolated fluctuation.Source: Why this framing is useful · p. 52
C.2 Prompts effect
- Contextual framing produces consistent nonlinear, dimension-specific trajectories of value activation and suppression rather than monotonic responses to scenario intensity.Quadratic fits use χ for directional sensitivity, γ for curvature, and cross-validated R2 for goodness of fit across four contextual conditions and value dimensions.Source: §C.2 · pp. 53–54, Figure C5 · p. 53
C.3 CoT effect
This section examines how Chain-of-Thought prompting affects LLM value distributions by comparing base and CoT conditions across open-weight and commercial model families against a WVS-7 human baseline.
C.3 CoT effect
- CoT effects are evaluated through UMAP projections that contrast each model’s base value vectors with its Chain-of-Thought-prompted vectors, alongside the WVS-7 human baseline.The comparison uses hollow diamonds for Base, filled diamonds for CoT, and grey points for the human reference.Source: Figure C6 · p. 55, Figure C7 · p. 57, Figure C8 · p. 58
- Across Qwen, the analysis spans four generations and parameter scales from approximately 1.5–1.8B to 14B, with Table C3 reporting angular θ values in degrees.The evaluated generations are Qwen, Qwen1.5, Qwen2.5, and Qwen3.Source: Figure C6 · p. 55, Table C3 · p. 56
- Across Llama, the analysis covers Llama3.2, Llama3.1, Llama3, and Llama2 variants, including instruct and non-instruct models at multiple parameter scales.Table C4 accompanies the Llama-family comparison of Chain-of-Thought prompting.Source: Figure C7 · p. 57, Table C4 · p. 58
- The study also compares Chain-of-Thought effects for GPT-4o, GPT-3.5-Turbo, Gemini 2.5 Pro, DeepSeek-V3, Kimi-K2, and Doubao-1.5.These commercial API models are shown under base and CoT conditions relative to the WVS-7 baseline.Source: Figure C8 · p. 58
C.4 Alignment Prescription and Intervention Strategies … Intervention III: Parametric Update via SFT/DPO (Level 3)
Instruction tuning increasingly shifts Llama and Qwen value distributions, with later generations showing larger magnitude and, in some cases, directional changes. The prescription framework assigns the least costly effective intervention, escalating from prompts and reasoning anchors to permanent SFT/DPO updates for resistant dimensions.
C.4 Alignment Prescription and Intervention Strategies
- Llama and Qwen alignment shifts intensify across generations: Llama3.1-8B reaches Δd=6.032, Qwen3-0.6B reaches Δd=8.197, and Llama3.2 models show angular deviations up to 89.4°.Earlier Llama2 and Llama3 models remain directionally stable with Δθ<7°, while Qwen3 varies from 5.2° for 4B to 65.9° for 0.6B.Source: Figure C9 · p. 59, Table C5 · p. 60, Figure C10 · p. 61
Appendix D Prescription Assignment Algorithm
- The prescription algorithm evaluates four intervention levels sequentially for each model and Schwartz value dimension, assigning the level that meets the target shift under effect-per-cost ranking.If no level qualifies, the dimension is classified as pre-train locked and assigned Level 4, producing a K×10 prescription matrix with entries in {1,2,3,4}.Source: Appendix D · p. 62
D.1 Prescription Validation Protocol
- Prescription reliability is tested out of sample on PKU-SafeRLHF by comparing each model–dimension prescription with the lowest effective validation level.A pair is marked matched when the predicted and validation levels are equal, and the overall match rate aggregates these matches.Source: §D.1 · pp. 62–63
Intervention I: Prompt Engineering (Level 1)
- Prompt engineering is the lowest-cost, parameter-free intervention, using structured input text for dimensions with high prompt-level plasticity.It is prescribed when changing the prompt can produce the required value shift without modifying model weights.Source: Intervention I: Prompt Engineering… · p. 63
Intervention II: Chain-of-Thought Reasoning (Level 2)
- When prompts are insufficient, a Chain-of-Thought reasoning anchor guides deliberation toward the target value vector while keeping model parameters unchanged.The optimal anchor maximizes cosine alignment between the reasoning-induced shift and the target Schwartz value vector.Source: Intervention II: Chain-of-Thought… · p. 63
Intervention III: Parametric Update via SFT/DPO (Level 3)
- For dimensions resistant to inference-time steering, value-specific preference data pairings support Direct Preference Optimization to update model parameters.Preferred responses are those with smaller Wasserstein distance to the target human-values distribution than rejected responses.Source: Intervention III: Parametric Update… · p. 63
- This Level 3 strategy permanently changes model parameters and is reserved for dimensions whose value shifts cannot be achieved through prompt or reasoning interventions.The escalation therefore moves from inference-time methods to parametric updating only when lower-level interventions are insufficient.Source: Intervention III: Parametric Update… · p. 63, Intervention I: Prompt Engineering… · p. 63, Intervention II: Chain-of-Thought… · p. 63
Intervention IV: Continued Pre-training (Level 4)
Continued pre-training is reserved for dimensions that remain pre-train locked after all lower-cost interventions fail. It uses a curated, target-aligned corpus to shift the value distribution while limiting unrelated distributional noise.
Intervention IV: Continued Pre-training (Level 4)
- Continued pre-training is triggered when all preceding intervention levels fail to exceed τ, classifying the dimension as pre-train locked.Source: Intervention IV: Continued… · p. 63
- The intervention trains on a curated corpus enriched with texts aligned to the target value orientation.Source: Intervention IV: Continued… · pp. 63–64
- Documents are scored by geometric proximity to vtarget in Schwartz space, retaining those within ϵ while downsampling texts reinforcing the undesired orientation.This targeted construction shifts values in the intended direction without broad distributional noise and is used only after prescription validation rules out lower-cost interventions.Source: Intervention IV: Continued… · p. 64
D.2 Comparative Metric: Effect-per-Cost Efficiency
The study defines an effect-per-cost ratio to compare Environment, Cognition, and Prior interventions using a common FLOP-based cost unit. This normalized metric measures value displacement per computation and supports Pareto-efficient intervention selection, but does not determine intervention levels.
- The effect-per-cost ratio compares Environment, Cognition, and Prior interventions, with Prior including SFT/DPO and continued pre-training.The three classes correspond to Prompt, CoT, and parametric interventions in the hierarchy.Source: §D.2 · p. 64
- Intervention costs are converted into FLOPs: prompting is negligible, CoT cost depends on additional generated tokens, and Prior cost depends on training tokens processed.The cost approximations follow standard scaling-law estimation practices [78].Source: §D.2 · p. 64
- The metric normalizes value displacement by computational cost, enabling cross-class comparison and establishing a Pareto frontier for cost-effective intervention selection.Per-model costs and efficiency scores are provided in the released code and data.Source: §D.2 · p. 64
- Effect-per-cost efficiency does not determine intervention levels; prescriptions instead follow the effectiveness analysis described in the main text.Source: §D.2 · p. 64
Related papers
This paper cites
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2018
- Language Models are Few-Shot Learners2020
- Training language models to follow instructions with human feedback2022
- GPT-4 Technical Report2023
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning2025
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer2019