PaperlayerView exact revision on arXiv

Source-linked AI summary

ValueGraph: Value-Signal Guided Graph Pre-training for Contextualized User Representation

Yitong Han, Wei Gao, Yi Zhao, Prasanta Bhattacharya, Fengzhu Zeng, Mohammad Amanlou

arXiv:2609.00057v1cs.CLcs.AIcs.LG

TL;DR

Existing user representations often miss value-relevant differences hidden by similar content or interaction patterns. ValueGraph uses inferred moral-value signals as noisy auxiliary guidance within graph pre-training, combining semantic, structural, contrastive, and clustering objectives. Across stance detection and Twitter bot detection, it reports consistent gains over strong graph, text, PLM, GNN, and LLM baselines.

  • Problem

    Existing methods can treat users with similar content exposure or graph neighborhoods as similar even when their underlying values and behavioral tendencies differ.

  • Method

    ValueGraph combines masked graph autoencoding with contrastive learning over inferred value similarity and clustering to learn contextualized user representations.

  • Results

    ValueGraph shows consistent gains on stance detection and Twitter bot detection over strong graph-based, text-based, PLM, inductive GNN, and text-only LLM baselines.

  • Takeaways & Limitations

    Noisy but theory-informed value signals provide useful auxiliary guidance for socially informed user modeling without predicting users’ true values.

  • Takeaways & Limitations

    Results are dataset-specific, and the inferred signals should not be used for individual-level profiling or decision-making without safeguards.

Abstract

from arXiv · show

Value signals are aggregated user-level moral representations that capture users' inferred value-related tendencies from their online discourse. User behavior on social media is shaped not only by what users say or whom they interact with, but also by the value signal through which they express attitudes. Existing user representation methods largely miss this value-relevant dimension. We propose ValueGraph, a graph pre-training framework that uses automatically inferred moral-value signals as noisy auxiliary signals for contextualized user representation. From post-reply graphs, ValueGraph learns semantic and structural representations and further aligns users through relative value similarity with contrastive and clustering objectives. Rather than treating inferred values as gold psychological labels, ValueGraph uses them as soft constraints for representation learning. Experiments on stance detection and twitter bot detection show consistent gains over strong text-based, graph-based, and text-only LLM baselines, highlighting value-signal guidance as a useful inductive bias for socially informed user modeling.

Introduction

Existing user representations often infer similarity from content or interaction patterns, which can confuse shared exposure with shared values. ValueGraph addresses this gap by using inferred moral-value signals as auxiliary guidance for graph-based contextualized user representations.

  • Motivation: Existing methods learn user embeddings from text, visual content, or network structure, often assuming similar exposure or neighborhoods imply similar users.These assumptions can overlook the factors underlying users’ behavior.
  • Motivation: Shared engagement can reflect exposure, controversy, or platform dynamics even when users hold opposing attitudes.Interaction-based similarity may therefore produce brittle representations as social contexts shift.
  • Approach: ValueGraph derives 10-dimensional user-level value signals by aggregating MoralBERT’s post-level moral scores under Moral Foundations Theory.The signals are used as auxiliary supervisory cues rather than explicit psychological labels.
  • Approach: ValueGraph learns contextualized user representations through masked graph autoencoding, value-similarity alignment, contrastive learning, and clustering.The framework models textual content, interaction context, and inferred value information jointly.
  • Evaluation: Experiments on stance detection and Twitter bot detection show consistent gains over graph pre-training, PLM, inductive GNN, and text-only LLM baselines.Ablations attribute improvements to jointly modeling semantic, structural, and value-signal information.

Related Work

Prior work develops user representations from text, visual content, network structure, and graph pre-training, while moral-value detection has largely remained separate from graph-based user modeling. ValueGraph connects these lines by using inferred value signals as auxiliary guidance for user representation learning.

  • User Representation: Prior user-representation methods rely on text, visual content, network structure, or multi-view fusion, but rarely use moral signals as an explicit inductive bias.This leaves value-relevant information underused in user modeling.
  • Graph Pre-training: Graph pre-training improves generalization by exploiting graph structure and neighborhood similarity, with later work examining transfer across similar or different domains.These approaches primarily emphasize structural and neighborhood information.
  • AI and Human Values: Transformer-based models detect moral values in text, while value-alignment research focuses on instruction following and preference-based reward modeling.These research directions do not directly address graph-based user representation.
  • Positioning: ValueGraph instead uses automatically inferred value signals as auxiliary signals for graph-based user representation learning.The framework links text-level value inference with graph-based representation learning.

Problem Formulation

The problem formulation represents users through their authored posts and the reply structure connecting those posts. The target is a compact embedding that combines semantic information with structural context for downstream behavior and content understanding.

  • Graph and Authorship: Each conversation graph contains posts as nodes and directed reply edges, while an authorship mapping assigns every post to its user.The formulation uses these graph and authorship relations as the basis for user representation.
  • Graph and Authorship: The post aggregation function ϕ maps each user to the set of posts they authored.This set provides the user’s textual evidence for representation learning.
  • Representation Objective: The encoder fθ maps each user u to a compact embedding z_u ∈ R^d.The embedding is intended to support downstream behavior and content understanding.
  • Representation Objective: User embeddings aggregate semantic information from authored posts and structural context from post-post reply relations.The resulting representations are expected to capture semantic, relational, and value-relevant behavioral cues.

Methodology

ValueGraph uses two-stage graph pre-training to combine conversational semantics and structure with noisy, inferred moral-value signals. It builds value-guided user representations through similarity-based contrastive learning and clustering.

  • Foundation GNN Pre-training: ValueGraph first pre-trains a foundation GNN on a post-reply graph using masked autoencoding to obtain noise-resilient representations.Reply edges enable multi-hop message passing over conversational patterns, while masked reconstruction supports unsupervised graph pre-training.
  • Value-Guided Pre-training: Post embeddings are aggregated into user embeddings, which are then refined using value signals derived from MoralBERT.The framework focuses on post-reply relations because follow or friendship links are platform-dependent and unavailable across datasets.
  • Value-Signal Construction: MoralBERT produces ten foundation-specific scores per post, aggregated into 10-dimensional user vectors that represent noisy moral-framing signals rather than gold user values.The signal is intended to be useful through relative differences across users, not exact post-level predictions.
  • Value-Signal Construction: Value similarity is computed from Euclidean distances between inferred value vectors and converted into scores with a Gaussian radial basis function kernel.The kernel bandwidth σ is set to the median of sampled distances.
  • Value-Guided Pre-training: The user-level contrastive loss pulls value-similar users together and pushes value-dissimilar users apart, using sampled positive and negative sets.Positive and negative pairs are constructed through percentile-based similarity thresholds, with thresholds relaxed when either set is underpopulated.
  • Value-Guided Pre-training: Periodic K-means clustering adds compactness and margin terms, encouraging users toward assigned centroids while separating distinct cluster centroids.The combined objective uses contrastive learning for pairwise value consistency and clustering for global embedding-space structure.

Theoretical Analysis

The theoretical analysis characterizes how ValueGraph organizes user embeddings using inferred value similarity and clustering geometry. Graph pre-training supplies the semantic and relational context for this organization.

  • Value-Signal Similarity Preservation: Value-signal contrastive learning encourages learned embedding similarities to preserve the ordering induced by inferred value-signal similarity.The result is stated for sampled user pairs under fixed positive and negative sets and convergence of the user-level contrastive loss.
  • Cluster Compactness and Separation: The clustering objective minimizes distances from users to assigned centroids and penalizes centroid pairs that fall below a margin, promoting compactness and separation.The compactness term acts within clusters, while the margin term acts between distinct cluster centroids.
  • Overall Interpretation: Together, the objectives organize user embeddings with inferred value signals while graph pre-training supplies semantic and relational context.This combines value-based organization with information learned from the conversational graph.

Experiments and Results

ValueGraph is evaluated on stance detection and Twitter bot detection against graph, language-model, GNN, and text-only LLM baselines. It delivers consistent gains, while ablations and visualization support the contribution of jointly modeling semantic, structural, and value-signal information.

  • Stance Detection: ValueGraph is evaluated on MT_CSD and RumourEval19 using established stance models with integrated post-level embeddings.The experiments compare against graph pre-training, pretrained language models, inductive GNNs, and GPT-5.4.
  • Stance Detection: 63% acc and 58% macF1 on MT_CSD yield relative gains of 5% and 5.5% over GraphMAE2, respectively.GraphMAE2 records 60% acc and 55% macF1 as the strongest pretrained baseline.
  • Stance Detection: 77% acc and 71% macF1 on RumourEval19 correspond to a 20.3% macF1 gain over ModernBERT and 57.8% over GTN.These comparisons use the strongest PLM and graph-based baselines, respectively.
  • Twitter Bot Detection: RoBERTa’s text-only bot-detection configuration improves macF1 from 54.8% to 64.3% and recall from 62.5% to 81.0% with ValueGraph embeddings.ValueGraph embeddings consistently outperform non-LLM baseline representations across models.
  • Twitter Bot Detection: ValueGraph gains remain useful without bot labels or downstream fine-tuning, although significance decreases when full multimodal inputs already dominate the decision boundary.The reported decrease is especially associated with macF1 in FTUG configurations.
  • Ablation Study: Neither Lcls nor Luser alone improves consistently, whereas the full model jointly integrating structural pretraining, clustering supervision, and value guidance improves across tasks.For example, adding Lcls raises GLAN accuracy from 41.2% to 50.4% but lowers macF1 from 40.7% to 34.3% on MT_CSD.
  • User Clustering Analysis: t-SNE visualization shows clearer bot-human separation for ValueGraph than for the compared text and graph encoders.The visualization is presented as dataset-specific qualitative evidence of improved representation separability.

Conclusion

ValueGraph combines graph pre-training with inferred value signals to capture textual, relational, and value-relevant behavioral cues. Across stance detection and Twitter bot detection, it shows consistent gains over strong baselines.

  • ValueGraph combines semantic and structural graph pre-training with contrastive learning over inferred value similarity and clustering.
  • The framework captures textual, relational, and value-relevant behavioral cues without predicting users’ true values.
  • ValueGraph shows consistent gains over strong baselines on stance detection and Twitter bot detection.

Ethics Statement

The work uses publicly available social-media datasets under their original access conditions and licenses. It treats MoralBERT-derived vectors as noisy aggregate signals rather than users’ true psychological values, and limits interpretation to research use with safeguards.

  • The study uses publicly available social-media datasets under their original access conditions and licenses.
  • MoralBERT-derived vectors are used only as noisy aggregate signals for representation learning, not as users’ true psychological values.
  • Results are dataset-specific and should not support individual-level profiling or decisionmaking without appropriate safeguards.
  • The authors will release code and processing scripts but not raw social-media content or personally identifiable information.

Full Proof of Theorem 1

The proof explains how the InfoNCE objective aligns embedding similarity with inferred value-signal similarity. Normalized embeddings and softmax probabilities yield updates that pull value-similar users together and push value-dissimilar users apart.

  • The proof assumes normalized embeddings so cosine similarity can be treated as a dot product.
  • The InfoNCE gradient is derived by differentiating its terms and expressing the second term through softmax probabilities.
  • Positive pairs have high inferred value-signal similarity, whereas negative pairs have low similarity, so the objective pulls value-similar users together and pushes dissimilar users apart.
  • When the positive pair is close in representation, it dominates the softmax and the update primarily pulls the anchor toward that pair.
  • The learned embedding similarity is encouraged to preserve the ordering induced by inferred value-signal similarity.

Full Proof of Theorem 2

The proof shows that the clustering loss makes embeddings compact within clusters while maintaining separation between different centroids. Compactness pulls users toward assigned centroids, and the margin term pushes insufficiently separated centroids apart.

  • The compactness loss minimizes squared Euclidean distance between each user embedding and its assigned centroid.
  • If a user embedding deviates from its assigned centroid, optimization pulls it toward that centroid and encourages intra-cluster compactness.
  • When distinct centroids are at least margin m apart, their separation loss is zero and applies no further separation force.
  • When centroid distance d is below m, the gradient pushes the centroids apart until they reach the margin.
  • Because Lcls includes compactness and margin terms, minimizing it keeps same-cluster embeddings close while separating different cluster centroids.

Moral Foundations Background

The paper grounds value-signal modeling in Moral Foundations Theory and describes the data, training configuration, and staged optimization used by ValueGraph. Supplementary ablations report that combining clustering and user-level consistency objectives yields the strongest and most stable performance.

  • Moral Foundations Theory: Moral Foundations Theory provides the framework for assessing moral foundations, including Care/Harm, Fairness/Cheating, and Loyalty/Betrayal.
  • Hyperparameters: The reported configuration uses τ = 0.07, β = 0.5, k = 10, K = 5, λ = 0.05, and m = 1.
  • Datasets: The pre-training resources include Reddit and Twitter conversations, while downstream evaluation uses MT_CSD, RumourEval19, and TwiBot-22 benchmarks.The corpora preserve conversational structures such as reply interactions and cover diverse discussion settings.
  • Training procedure: ValueGraph samples similar and dissimilar users from inferred value profiles, encodes their posts with a graph encoder, and computes a value-guided user-level contrastive loss.The training procedure periodically adds clustering loss before updating the encoder with the combined objective.
  • Ablation results: The full model achieves the strongest and most stable performance across models and tasks, while clustering and user-level consistency improve over Stage 1 alone.The ablation attributes gains to progressively incorporating task-aware discrimination, interaction structure, and value-aware relational information.
Loading 2609.00057v1…