Measuring Psychological Constructs from Social Media Text Using the Word Embedding Projection Approach

1 Introduction

• The introduction frames dictionary methods as transparent but potentially sparse in short texts, while contextual models can be harder to interpret as fixed constructs.It identifies the validity and longitudinal comparability of embeddings for individual constructs as less clear than their use for macro-level phenomena.Source: §1 · p. 1
• WEPA adapts semantic projection for individual psychological measurement, using theory-driven anchor words to define axes onto which user text is projected.Scores indicate expressed psychological salience in language, not direct observations of latent states or external motivational conditions.Source: §1 · p. 2
• The study contributes context-specific anchor-word development, measurement validation, and cross-temporal semantic-axis stability analysis in a Chinese digital fitness community.WEPA is presented as a psychometric implementation of semantic projection, not a universal replacement for dictionary, supervised, or contextual-language-model approaches.Source: §1 · p. 2
• The observational panel comprises 177,829 Keep users and 2,668,298 weekly observations, evaluated through four studies.Study I examines anchor validation, human-coded benchmarks, and axis stability; Studies II–III test associations with subsequent exercise duration; Study IV explores relative-week aggregate trajectories.Source: §1 · p. 2

2.1 Text as a Basis for Psychological Measurement

• Language patterns are treated as reflective indicators of latent constructs, under assumptions that constructs generate recurring expressions, features share variance, and individual indicators can change without changing construct meaning.These are theoretical assumptions about how digital traces relate to latent constructs.Source: §2.1 · pp. 2–3

2.2 Conceptual Space and Construct Representation

• WEPA represents word meaning in embedding space, defines construct-specific axes with theory-driven anchors, and projects user text to produce scores that can be aggregated when appropriate.Projection scores are observable indicators of expressed construct salience, not replacements for latent constructs; bipolar and multidimensional constructs can be represented through their poles or separate dimensions.Source: §2.2 · p. 3

2.3 Measurement Stability and Longitudinal Comparability

• Longitudinal text changes are interpretable only if they reflect construct-related expression rather than changing word meanings or unstable measurement procedures.Static embeddings, fixed axes, and anchor-word prototypes are intended to support stability, but do not constitute a strict test of measurement invariance.Source: §2.3 · pp. 3–4

2.4 Alternative Text-Based Measurement Approaches

• WEPA links construct definitions to continuous embedding-space measurement and is designed for constructs with meaningful semantic poles, with dictionary-based scoring serving as its closest benchmark.The approach occupies an intermediate position among method families; Table 1 summarizes conceptual trade-offs rather than supervised benchmarks across all model families.Source: §2.4 · pp. 4–5, Table 1 · p. 5

2.5 Computational Implementation of WEPA

• The computational implementation turns anchor-word sets and user text into projection-based construct scores.The section introduces construction of semantic axes, text vectorization, and score computation as the workflow’s core steps.Source: §2.5 · p. 5, §2.5.1 · p. 5

2.5.1 Core Computational Procedure

• WEPA averages positive and negative anchor-word vectors into prototypes, normalizes their difference into an axis, averages valid text-word vectors, and projects the text vector onto that axis.Out-of-vocabulary tokens contribute zero directional information but remain in the denominator, pulling low-coverage texts toward neutral; scores indicate alignment with the positive pole, with larger absolute values indicating greater deviation from neutral.Source: §2.5.1 · pp. 5–6

2.5.2 Implementation Steps

• The study trains domain-specific static embeddings on a cleaned 816 MB fitness corpus, using 200 dimensions, a 15-word context window, and 20 iterations.For 50 high-frequency fitness terms, 90% of retrieved neighbors were judged semantically relevant in the fitness domain; the implementation is provided in an open-source Python package with documentation and basic tests.Source: §2.5.2 · p. 6

3.1 Validation Strategy

• The four-study strategy moves from measurement validation to criterion validity and then exploratory relative-week trajectories of aggregate constructs.The trajectories align users by weeks since registration.Source: §3.1 · p. 6, Figure 2 · p. 8

3.2 Data and Sample

• The Keep data combine user-generated posts with exercise records; the network-expanded sample is not a probability sample of the platform population.After cleaning and exclusions, the final panel contains 177,829 users and 2,668,298 user-week observations.Source: §3.2 · pp. 6–7

3.3 Variable Measurement

• Weekly exercise duration, transformed as ln(x+1), serves as the behavioral criterion, while weekly text yields WEPA scores for goal-setting constructs and four self-efficacy sources.The four self-efficacy dimensions are also summed into an overall score.Source: §3.3 · p. 7
• Time-varying controls cover demographics, expressive volume, behavioral inertia, and social incentives; continuous predictors are standardized in regressions.The correlation matrix uses winsorized and standardized data.Source: §3.3 · pp. 7–8
• Goal commitment correlated with goal difficulty at r=−0.544, self-efficacy dimensions at r=0.79–0.96, and social engagement with exercise duration at r=0.696.The authors note the shared variance among self-efficacy dimensions and use social controls in regression models.Source: Table 5 · p. 10, §3.3 · p. 11

4.1.1 Anchor-Word Development and Validation

• Goal difficulty captures online obstacle, fatigue, and challenge language rather than objective goal difficulty; social-persuasion expressions may reflect stable support or compensatory help-seeking.Source: §4.1.1 · p. 11
• Anchor development retained 232 theoretically relevant words after expert review, with Krippendorff’s α=0.8249.The process began with 260 candidates and retained 251 after vocabulary screening.Source: §4.1.1 · p. 11
• Every construct’s anchor words showed the expected within-pole versus between-pole cosine pattern, supporting semantic-axis construction despite moderate absolute similarities.The diagnostic compares average similarity within each pole and between poles.Source: §4.1.1 · p. 12, Table 7 · p. 12

4.1.2 Human Benchmark and Algorithm Comparison

• WEPA scored all 536 physiological-state texts (ρ=0.850), versus dictionary coverage of 52/536 (9.7%) and ρ=0.161; goal-specificity ρ was 0.895 versus 0.733.These are the reported Spearman correlations with human annotations; Table 8 presents the benchmark comparison.Source: Table 8 · p. 13, §4.1.2 · p. 13
• Human benchmarks test quantified goal specificity and implicit affective-somatic physiological states, not all seven constructs.The physiological-state expressions are described as a more difficult test because they can be implicit and fragmented in short posts.Source: §4.1.2 · pp. 12–13

4.1.3 Cross-Temporal Stability of Semantic Axes

• Year-specific embeddings from 2015–2021 were aligned to a full-period reference; cosine similarity assessed axis orientation and Spearman correlation assessed anchor-word ordering.The authors place less emphasis on 2015 because its corpus was sparse.Source: §4.1.3 · pp. 13–14
• After 2016, rank-order consistency remained high while axis orientation varied across constructs; this supports longitudinal interpretability but does not prove measurement invariance.Figure 3 distinguishes axis stability in panel (a) from rank-order consistency in panel (b).Source: §4.1.3 · p. 14, Figure 3 · p. 14

4.2 Study II: Criterion Validity of Goal-Setting Constructs

• Study II evaluates whether text-derived commitment, specificity, and difficulty scores are associated with subsequent exercise behavior.The hypotheses concern users’ expressed psychological states, not experimentally assigned goal conditions.Source: §4.2 · p. 15, §4.2.1 · p. 15

4.2.1 Hypotheses

• The hypotheses predicted positive associations for commitment and specificity, but a negative association for difficulty as expressed in online fitness text.The authors interpret difficulty language as potentially signaling obstacles, fatigue, or frustration in this context.Source: §4.2.1 · p. 15

4.2.2 Model Specification

• Separate individual fixed-effects models linked each weekly goal-setting score to next-week log exercise duration, adjusting for time-varying controls and clustering standard errors by user.Goal commitment and difficulty were correlated (r = −0.544), but separate models were chosen for interpretability, not because joint estimation was infeasible.Source: §4.2.2 · p. 15

4.2.3 Results

• Goal commitment and specificity were positively associated with later exercise duration, while goal difficulty was negative; all three associations were statistically significant.Coefficients were 0.069, 0.019, and −0.053, respectively (all p < 0.001); the authors characterize the raw-minute equivalents as modest associations, not large intervention effects.Source: §4.2.3 · p. 16, Table 9 · p. 16

4.3 Study III: Criterion Validity of Self-Efficacy Dimensions

• Study III tests whether WEPA distinguishes behavioral associations across self-efficacy’s four theoretically distinct source dimensions.The dimensions are mastery experience, vicarious experience, social persuasion, and physiological states.Source: §4.3 · pp. 16–17, §4.3.1 · p. 17

4.3.1 Hypotheses

• The hypotheses predicted positive associations for mastery, vicarious experience, and physiological states, but a negative association for social persuasion in this platform context.The authors caution that the social-persuasion hypothesis is about user-expressed support-related language, not a general claim about self-efficacy theory.Source: §4.3.1 · p. 17

4.3.2 Model Specification

• The primary joint model assesses each theoretically distinct self-efficacy source’s association with subsequent behavior and its relative contribution conditional on the other three sources.The dimensions were highly correlated (r = 0.79–0.96); VIFs for MastExp and SocPer were 13.09 and 10.31, and the maximum condition index was 8.56. These diagnostics call for cautious interpretation of coefficient magnitudes without invalidating the joint specification.Source: §4.3.2 · pp. 17–18

4.3.3 Results

• In the joint model, mastery, vicarious experience, and physiological states were positively associated with later exercise duration, while social persuasion was negative; overall self-efficacy was positive.The respective coefficients were 0.117, 0.122, 0.025, −0.231, and 0.008 (all p < 0.001).Source: §4.3.3 · p. 18

4.4 Study IV: Exploratory Relative-Week Dynamics

• Mastery experience, vicarious experience, and social persuasion declined early, flattened after about Week 20, and later fluctuated near zero.Mastery experience generally remained highest among the three, though their differences narrowed after the early stage.Source: §4.4 · p. 19
• Goal specificity rose gradually from the lowest starting level but remained below zero, while goal difficulty approached and then stayed near zero after about Week 20.Figure 4 plots mean standardized WEPA scores aggregated by relative week.Source: §4.4 · p. 19, Figure 4 · p. 20
• Physiological states and goal commitment showed smaller declines and partial later movement toward zero than the goal-related trajectories.The authors characterize these changes as mild aggregate adjustments.Source: §4.4 · pp. 19–20
• Week 20 is a visual reference, not a statistically estimated breakpoint; later-week patterns describe retained observable users, not the original registration cohort.Users were aligned by registration week and scores standardized using full-sample means and standard deviations before aggregation.Source: §4.4 · pp. 18–20

5 Robustness Checks

• The checks span textual sparsity, technical implementation, anchor-word perturbation, subgroup heterogeneity, and measurement-error bias.Technical, anchor-word, and subgroup checks use goal commitment as a representative construct; SIMEX covers all core constructs.Source: §5 · p. 20

5.1 Sensitivity to Textual Sparsity

• After excluding texts with fewer than five valid tokens, focal coefficients retained their directions and significance; shorter- and longer-text subgroups also had stable directions.This suggests the main patterns were not primarily driven by extremely sparse user–week texts.Source: §5.1 · p. 21, Table 11 · p. 21

5.2 Technical Robustness

• Across alternative embedding architectures and settings, WEPA–benchmark correlations stayed above 0.86 and near the main specification’s ρ = 0.890, outperforming the dictionary baseline.The authors conclude benchmark agreement was not driven by one embedding architecture or hyperparameter setting.Source: §5.2 · p. 21, Table 12 · p. 22

5.3 Dictionary Robustness

• Goal-commitment coefficients ranged from 0.067 to 0.072 after removing up to three positive–negative anchor pairs, with maximum baseline deviation of 4.35%.The authors interpret this stability as evidence that the score reflects the broader anchor-word set rather than a few high-frequency words.Source: §5.3 · p. 22, Table 13 · p. 22

5.4 Cross-Group Robustness

• Goal commitment was positively associated with future exercise duration for both female (β = 0.074) and male (β = 0.059) subsamples; the between-group difference was not significant (p = 0.087).The authors report this as stability across gender subsamples.Source: §5.4 · pp. 22–23, Table 14 · p. 23

5.5 Estimation Robustness

• SIMEX estimated attenuation of 11.2%–15.1% across constructs; the goal-commitment estimate changed from 0.0689 to 0.0792, a bias of −13.0%.The corresponding bias for self-efficacy was −13.2%; correction did not change coefficient signs or establish causality.Source: §5.5 · p. 23, Figure 5 · p. 24, Table 15 · p. 25

6.1 Main Findings

• WEPA showed strong agreement with human annotation, broader text coverage than the dictionary baseline, and construct scores meaningfully associated with subsequent exercise behavior.Relative-week analysis also recovered interpretable aggregate temporal patterns; these findings indicate measurement performance, predictive associations, and exploratory descriptive utility, not causal effects or strict measurement invariance.Source: §6.1 · p. 25
• Goal commitment and specificity were positively associated with later exercise duration, while goal difficulty was negatively associated; three self-efficacy sources were positive and social persuasion was negative.Difficulty-related language often captured perceived obstacles, fatigue, or reduced persistence; social-persuasion mentions may become salient when users struggle or seek reassurance, not because support weakens self-efficacy.Source: §6.1 · p. 25

6.2 Contributions to Theory and Measurement

• WEPA operationalizes constructs as interpretable projections along theory-grounded bipolar semantic axes, complementing dictionary-based methods and deep-learning models.Its application to more than 2.6 million user–week observations demonstrates practical feasibility, while broader portability remains for future evaluation.Source: §6.2 · pp. 25–26
• The study treats text-based measurement as layered validation, separating human-judgment agreement, behavioral association, temporal comparability, and descriptive dynamics.These components address distinct validity questions rather than one undifferentiated form of validity.Source: §6.2 · p. 26
• Text-based scores capture expressed psychological salience rather than external motivational conditions, particularly for goal difficulty and social persuasion.Difficulty language may reflect obstacles or fatigue, while persuasion language may reflect users’ own mentions of encouragement or support.Source: §6.2 · p. 26

6.3 Practical Implications

• WEPA construct scores may help platforms monitor aggregate psychological signals that behavioral logs alone can miss and support retrospective evaluation of community dynamics.Changes in goal commitment, perceived difficulty, or self-efficacy language may indicate motivational strain, declining persistence, or support needs.Source: §6.3 · p. 26

6.4 Limitations and Future Research

• Evidence comes from one Chinese fitness-oriented platform, with context-specific embeddings and anchor dictionaries that limit direct portability across platforms, languages, and cultures.Network-based sampling and excluding users with insufficient records may overrepresent socially active and persistent users.Source: §6.4 · pp. 26–27
• Validation did not establish strict measurement invariance; WEPA best fits bipolar constructs, and correlated self-efficacy dimensions and weekly aggregation constrain interpretation.Weekly aggregation trades measurement stability against temporal precision; the study is observational, so reported associations are predictive rather than causal.Source: §6.4 · p. 27

• The study received no external funding.Source: Funding · p. 27

• The study was exempt from institutional ethical review because it used publicly accessible, de-identified social media text without intervention, participant interaction, or identifiable information.Analyses were conducted at the aggregated group level.Source: Institutional Review · pp. 27–28

• The study received a waiver of informed consent because it used publicly accessible, de-identified secondary data without direct user contact or identifiable individual-level reporting.Source: Informed Consent · p. 28

Data Availability

• WEPA scoring code and the complete Chinese anchor-word dictionaries are available in the open-source cntext Python package, version 2.2.0.The repository includes installation instructions and a minimal WEPA example.Source: Data Availability · p. 28
• The cntext repository provides API descriptions and basic tests for semantic-axis construction and text scoring.Source: Data Availability · p. 28
• Raw platform data are not publicly redistributed to protect user privacy.Source: Data Availability · p. 28