PaperlayerView exact revision on arXiv

Source-linked AI summary

Epistemic Familiarity is Associated With Belief Stability in Large Language Models

Samantha Dies, Courtney Maynard, Germans Savcisens, Tina Eliassi-Rad

arXiv:2511.19166v4cs.CL

TL;DR

LLMs may destabilize their truth judgments when semantic assumptions change, but the role of epistemic familiarity remains unclear. P-StaT evaluates this question with matched perturbations in representational and behavioral settings, finding greater instability for unfamiliar Synthetic than familiar Fictional statements. The findings associate epistemic familiarity with stability and motivate stability-based complements to accuracy benchmarks.

  • Problem

    It remains unclear whether LLM truth judgments are organized coherently and whether their stability under semantic reframing depends on epistemic familiarity.

  • Method

    P-StaT evaluates belief stability by varying semantically plausible Neither statements under matched perturbations across representational and behavioral settings.

  • Results

    Unfamiliar Synthetic perturbations generally induce greater epistemic instability than familiar Fictional perturbations, particularly in behavioral evaluations.

  • Takeaways & Limitations

    Epistemic familiarity is associated with stability under semantic reframing, and belief-set analyses can complement factuality and robustness evaluations.

  • Takeaways & Limitations

    Familiarity is operationalized through statement construction and cultural embeddedness as proxies for pretraining exposure, which cannot fully isolate it from correlated factors such as domain coverage or memorization.

Abstract

from arXiv · show

Large language models (LLMs) are widely used as information sources, yet small changes in semantic assumptions can destabilize their beliefs. We introduce P-StaT (Perturbation Stability of Truth), a framework for evaluating belief stability under matched semantic perturbations in both representational and behavioral settings. Across 21 LLMs and three domains, we compare perturbations involving familiar Fictional statements against synthetically generated and unfamiliar Synthetic statements. Unfamiliar Synthetic perturbations generally induce greater epistemic instability than familiar Fictional perturbations, with behavioral belief retraction rates frequently exceeding 0.5 (50%). Finally, exploratory clustering analyses reveal recurring themes among retracted statements, including ambiguity, technical terminology, and obscure concepts. These results show that epistemic familiarity is systematically associated with stability under semantic reframing, suggesting that stability-based analyses can complement accuracy-based benchmarks when evaluating LLM robustness and reliability. Code and data are available at https://github.com/samanthadies/P-StaT.

1 Introduction

P-StaT evaluates whether semantic assumptions destabilize LLM truth judgments by comparing familiar Fictional and unfamiliar Synthetic perturbations across representational and behavioral settings. The framework addresses a gap between veracity probing and behavioral robustness, finding greater instability for unfamiliar Synthetic perturbations.

  • P-StaT evaluates belief stability by varying which semantically plausible Neither statements are treated as True under perturbed assumptions.
  • Fictional statements are culturally embedded and plausibly familiar from pretraining, whereas Synthetic statements are generated to minimize prior exposure.
  • Unfamiliar Synthetic perturbations generally induce greater epistemic instability than familiar Fictional perturbations, particularly in behavioral evaluations.
  • The study introduces Fictional and lexically matched Synthetic datasets for controlled evaluation of LLM belief stability.
  • The framework unifies representational probing and behavioral prompting under matched perturbations, enabling cross-setting stability evaluation.
  • Across the framework, retractions and expansions quantify changes between baseline and perturbed belief sets.

2 Related Work

Related work spans epistemic distinctions, veracity probing, behavioral stability, and perturbation-based robustness. P-StaT connects these directions by examining whether semantic perturbations produce corresponding changes in representations and behavior.

  • Prior work studies LLM epistemic distinctions, veracity representations, behavioral sensitivity, and robustness largely as separate research directions.
  • P-StaT connects belief and epistemic stability with veracity probing, behavioral stability, and perturbation-based robustness.
  • Veracity Probing: Veracity probing examines whether True and False statements occupy separable latent regions and whether hidden representations encode veracity signals beyond behavior.
  • LLM Behavioral Stability: Behavioral-stability research documents sensitivity to paraphrases, sycophancy, jailbreaks, and multi-turn interactions, while leaving representational correspondence unresolved.
  • Perturbation-Based Robustness: Perturbation-based robustness studies preserve meaning while changing surface realization or context, and P-StaT modifies semantic assumptions without changing model parameters.

3 Methodology

P-StaT evaluates how matched semantic assumptions alter LLM truth judgments across representational and behavioral settings. It distinguishes familiar Fictional from unfamiliar Synthetic Neither statements and quantifies belief changes through retractions and expansions.

  • P-StaT: Perturbation Stability of Truth: P-StaT applies matched semantic perturbations across probing and prompting evaluations of LLM truth judgments.The same assumption Θ is implemented by relabeling statements for probes and by constructing belief contexts for prompting.
  • Operationalizing Familiarity of Neither Statements: Fictional statements represent familiar Neither claims, whereas Synthetic statements are constructed to minimize prior exposure while preserving specified lexical distributions.Fictional entities are plausibly represented in training data; Synthetic variants are unfamiliar constructed entities.
  • Experimental Scope: The evaluation spans City Locations, Medical Indications, and Word Definitions, with a fixed veracity layer used for representational comparisons.The fixed layer is selected for True-versus-Not-True separability and held constant across perturbation conditions.
  • P-StaT: Perturbation Stability of Truth: Each perturbation Θ relabels selected Neither statements as True, allowing the same semantic assumption to be evaluated in both representational and behavioral settings.The induced labels define retrained probes or belief contexts while keeping the perturbation itself shared across settings.
  • P-StaT: Perturbation Stability of Truth: Epistemic retractions are previously believed True statements that lose belief status under perturbation, while expansions capture newly believed statements.Normalized rates account for differences in baseline performance, and retractions are treated as the stronger instability signal.

4 Experiments

Experiments evaluate P-StaT across 21 open-source LLMs and three domains using lexical, representational, probabilistic, and clustering analyses. The design tests whether stability differences reflect familiarity rather than low-level artifacts and examines recurring themes in retractions.

  • Experimental Setup: The experiments evaluate 21 open-source LLMs spanning Gemma, Llama, Mistral, and Qwen families, including base and chat-tuned variants.Multiple architectures and tuning variants provide coverage across diverse LLM families.
  • Artifact Analyses: Bigram, activation-space, and next-token analyses test whether stability differences could arise from lexical or representational artifacts rather than familiarity.These analyses compare lexical structure, veracity-representation distances, and estimated relative familiarity.
  • Behavioral Evaluation: K = 100 statements are sampled without replacement to construct behavioral belief contexts, while a Noise condition supplies non-semantic controls.Prompting uses cross-domain True statements for Noise; probing uses Gaussian activation vectors matched to representation statistics.
  • Behavioral Evaluation: The behavioral prompt asks whether each statement is True, False, or Neither, with predictions restricted to three options for deterministic comparison.Chat templates are used for chat-tuned LLMs, and option a maps to True while b and c map to Not True.
  • Retraction Themes: Retracted statement activations are clustered with PCA, UMAP, and HDBSCAN, followed by contrastive prompting to generate candidate semantic themes.Clustering is performed separately for each dataset–LLM pair with hyperparameters selected using cluster-quality metrics.
  • Robustness Checks: Smaller belief contexts and an alternative baseline yield a largely unchanged qualitative ordering of perturbation conditions.Additional K ∈ {20,50} experiments and baseline variations produce qualitatively similar results.

5 Results

Across three domains, unfamiliar Synthetic perturbations generally produced greater instability than familiar Fictional perturbations, especially behaviorally, despite matched lexical and representational structure. Retractions also concentrated around ambiguity, weak grounding, and specialized or context-dependent concepts.

  • Familiarity Operationalization: Synthetic (TF) and Synthetic (Fi) statements approximately preserved the lexical distributions of True/False and Fictional statements, respectively, reducing lexical explanations for stability differences.
  • Familiarity Operationalization: Fictional statements had similar or lower within-group Euclidean distances than Synthetic conditions, indicating they were not more heterogeneous across datasets and LLMs.
  • Familiarity Operationalization: Synthetic statements remained least familiar in token-probability rankings despite partially matching other statement types’ lexical and representational structure.The average familiarity ordering was True, False, Fictional(T), Fictional, Synthetic(TF), and Synthetic(Fi).
  • Epistemic Stability under Familiar and Unfamiliar Perturbations: 0.51/0.53 and 0.49/0.67 median behavioral retraction rates for Synthetic perturbations exceeded Fictional rates of 0.27/0.17 and 0.34/0.29 in City Locations and Word Definitions.Median behavioral retraction rates often exceeded 0.5.
  • Epistemic Stability under Familiar and Unfamiliar Perturbations: Representational retraction rates stayed below 0.02, 0.04, and 0.08 for City Locations, Medical Indications, and Word Definitions, respectively.Synthetic conditions nevertheless frequently showed comparable or greater instability, particularly for Medical Indications and Word Definitions.
  • Recurring Themes in Retracted Statements: Retractions frequently involved ambiguous referents, rare or technical concepts, qualifier-sensitive biomedical relationships, and semantically graded categorizations.Exploratory analyses suggest instability concentrates in identifiable semantic regions rather than appearing uniformly across statements.

6 Discussion

The discussion interprets the results as evidence that unfamiliar semantic assumptions are associated with greater instability, while matched lexical and representational analyses argue against purely low-level explanations. P-StaT therefore offers a controlled complement to accuracy-based evaluation for studying fragile truth judgments.

  • Interpretation: Across representational and behavioral settings, unfamiliar perturbations generally produced equal or greater instability than familiar perturbations, especially behaviorally.
  • Interpretation: Matched lexical and representational structure, combined with familiarity differences, suggests the stability gap is not driven solely by low-level artifacts.
  • Recurring Retraction Patterns: Clustering suggests retracted statements are not random but instead often ambiguous, technical, or context-dependent.
  • Implications: P-StaT provides a controlled setting for studying how familiarity interacts with representational organization and behavioral robustness.
  • Implications: Stability-based analysis may complement accuracy benchmarks by identifying semantically fragile claims or domains that conventional evaluations can leave vulnerable.

7 Conclusion

P-StaT evaluates LLM truth-judgment stability under matched semantic perturbations across representational and behavioral settings. Across 21 LLMs and three factual domains, unfamiliar Synthetic perturbations generally produced greater instability than familiar Fictional perturbations, while recurring retraction themes included ambiguity and specialized terminology.

  • Conclusion: P-StaT evaluates LLM truth-judgment stability under matched semantic perturbations in representational and behavioral settings.
  • Conclusion: Across 21 LLMs and three factual domains, unfamiliar Synthetic perturbations generally induced greater instability than familiar Fictional perturbations.
  • Conclusion: Recurring retraction themes involved ambiguity, weak grounding, specialized terminology, and semantically borderline relationships.
  • Conclusion: Lexical, representational, and probabilistic analyses suggest the effects were not driven solely by low-level artifacts.

8 Limitations

The study’s familiarity operationalization and controlled perturbations constrain what its stability findings establish. It does not cover all semantic variation or settings where model representations evolve.

  • Familiarity is approximated through statement construction and cultural embeddedness, so it cannot be fully isolated from domain coverage or memorization effects.Lexical, representational, and probabilistic analyses reduce several confounds but do not eliminate them.
  • The perturbation design covers a limited set of factual domains and does not exhaustively represent semantic variation.The authors suggest extending P-StaT to disputed claims and evolving facts.
  • P-StaT evaluates controlled semantic perturbations rather than naturally occurring conversational dynamics or deployment-time adversarial attacks.Its stability signal emphasizes epistemic retractions, while other applications may require alternative notions of stability.
  • Representational analyses use fixed LLM parameters and therefore do not test stability when representations evolve through continual learning or retrieval augmentation.

9 Ethical Considerations

The paper frames P-StaT as a diagnostic robustness tool rather than an exploitation method. Its findings identify semantic conditions associated with instability while acknowledging potential misuse.

  • P-StaT may help robustness evaluation by identifying semantic conditions under which LLM truth judgments become unstable.
  • The perturbation analysis could potentially be misused to design prompts or contexts intended to induce instability in deployed systems.The authors state that their experiments are diagnostic, offline, and conducted on fixed open-source LLMs.
  • P-StaT complements accuracy benchmarks by analyzing how truth judgments reorganize under controlled semantic variation, rather than supporting exploitation or belief steering.

Funding

The supplied passages document study materials, controls, data partitions, model coverage, and methodological details rather than funding information. They describe datasets, synthetic and fictional statements, controls, and experiments across 21 LLMs.

  • The experiments use City Locations, Medical Indications, and Word Definitions datasets with affirmative and negated statement forms.
  • Synthetic statements use generated entities and filtering to produce grammatically coherent but intentionally unfamiliar, semantically meaningless claims.
  • Fictional statements draw on culturally embedded sources across locations, medical indications, and word definitions, while remaining non-factual by construction.
  • Noise provides non-semantic controls through matched random activation sequences for probing and cross-domain True statements for prompting.
  • Data are split into approximately 55% training, 20% calibration, and 25% testing, with identical partitions across conditions.
  • The study evaluates 21 open-source LLMs spanning Gemma, Llama, Mistral, and Qwen families, including base and chat-tuned variants.

E LLM-level Results

Across individual models, unfamiliar Synthetic perturbations generally produce greater instability than familiar Fictional perturbations, especially behaviorally. These qualitative patterns recur across model families and remain broadly stable across methodological variations.

  • Across LLM families, absolute instability varies substantially between models and settings, but several qualitative perturbation patterns repeat consistently.
  • Unfamiliar Synthetic perturbations frequently yield larger retraction rates than familiar Fictional perturbations, especially in behavioral evaluations.Synthetic (TF) and Synthetic (Fi) often rank among the largest behavioral instabilities across individual LLMs.
  • Behavioral expansions are generally smaller and less separated across perturbation types, whereas Synthetic (TF) often produces the highest representational expansion rates.
  • Retraction counts generally mirror the rate-based results, with unfamiliar Synthetic conditions frequently exceeding familiar Fictional conditions.
  • Representational perturbation ordering remains largely unchanged across neighboring veracity layers, with Synthetic conditions comparable to or above Fictional conditions.
  • Behavioral ordering remains largely unchanged across smaller belief contexts and alternative baselines, suggesting semantic composition matters more than context size or baseline choice.

H.2 LLM-level Familiarity Results

Familiarity analyses find that Fictional statements receive higher estimated familiarity than Synthetic statements, while unfamiliar Synthetic perturbations generally produce greater instability. Additional analyses identify recurring semantic themes among retracted statements and show that probe choice can strongly affect apparent instability.

  • Familiarity results: Fictional statements occupy an intermediate familiarity regime, while Synthetic conditions receive the lowest object-token probabilities across nearly all evaluated LLMs.True and False statements generally receive the highest probabilities, and the familiarity pattern is consistent across datasets and models.
  • Familiarity results: The familiarity pattern supports treating Synthetic statements as comparatively unfamiliar and Fictional statements as more familiar through likely pretraining representation.This interpretation follows the object-token probability results and the construction of the two statement families.
  • LLM-level retraction rates: Unfamiliar Synthetic perturbations frequently produce larger behavioral retraction counts than familiar Fictional perturbations across 21 LLMs.Overall retraction rates remain comparatively small in the sAwMIL analysis, but Synthetic conditions often match or exceed Fictional instability.
  • Retraction themes: Clustered retractions repeatedly involve ambiguity, sparse grounding, unusual semantic relationships, and competing categorizations, with domain-specific variants across locations, medicine, and word definitions.Examples include ambiguous place names, qualifier-sensitive medical treatments, and polysemy or near-synonymy in word definitions.
  • Probe robustness: Mass-Mean shows retraction rates from about 0.3 to nearly 1.0 under Noise, but its instability is attributed to centroid sensitivity rather than genuine representational instability.The analysis therefore treats sAwMIL as better suited for quantifying P-StaT stability.
Loading 2511.19166v4…