PaperlayerView exact revision on arXiv

Source-linked AI summary

Beyond What Is Visible: Context-Aware Sentiment Across Online Communities Among Crisis Events

Zihui Ma, Yiheng Chen, Lingyao Li, Chen Chen, Fan Yang, Xiao Huang, Haozheng Liu, Yuhan Wang, Runlong Yu, Gregory B. Baecher

arXiv:2610.22551v1cs.CY

TL;DR

Uneven social-media participation and imbalanced language-model data complicate reliable, locally representative disaster sentiment estimates. The paper combines text and aggregate mobility with socioeconomic and hazard-informed cross-city representations, evaluating adaptive learning on two disasters. The adaptive model achieves the strongest overall sentiment-classification performance, while contextual grounding can reduce reliance on uneven digital participation and text-only representations.

  • Problem

    Uneven online participation and socially and geographically imbalanced language-model data challenge reliable, locally representative disaster sentiment estimates.

    Source: §1 · p. 2

  • Method

    The framework combines text and aggregate mobility with socioeconomic and hazard-informed city representations, transferring evidence from structurally related cities to supplement sparse observations.

    Source: §4.2 · p. 7, §4.3 · p. 7, §7 · p. 20

  • Results

    Across the Southern California wildfires and Hurricane Helene in Florida, the adaptive model achieves the strongest overall sentiment-classification performance.

    Source: §7 · p. 20

  • Takeaways & Limitations

    Contextual grounding can improve sentiment inference while reducing reliance on uneven digital participation and text-only representations that may carry geographic biases.

    Source: §7 · p. 21

    The analysis is limited to city-level representations and two disaster cases, and it evaluates the computational framework rather than user interaction with its outputs.

    Source: §6.4 · p. 20

Abstract

from arXiv · show

Social media offers timely insight into disaster experiences, yet online communities may not represent those affected on the ground. Uneven participation can bias crisis information systems toward data-rich locations and obscure communities with fewer digital traces. We address this with an adaptive cross-community learning framework that combines text and aggregate mobility signals with socioeconomic and hazard-informed community representations, supplementing sparse local observations with evidence from structurally related cities. We evaluate the framework on the January 2025 Southern California wildfires and 2024 Hurricane Helene in Florida. The adaptive model achieves the highest F1-score in both cases (0.5923 and 0.7615), while augmentation has its strongest effect where original observations are limited. The framework also reveals finer cross-city community-level sentiment variation and stronger sentiment-mobility correspondence than text-only models, with relationships differing across disaster contexts. These findings support crisis systems that better account for context-dependent evidence and improve interpretation of underrepresented community signals.

1 Introduction

Social media provides timely disaster information, but online participation is uneven and may not represent affected communities. Language-model capabilities do not resolve the challenge of producing reliable, locally representative sentiment estimates.

1 Introduction

  • Social media posts support situational awareness by reporting local conditions, requests for assistance, concerns, and evolving risks.The passage describes social media as an important source for disaster response.Source: §1 · p. 2
  • Self-selected users and unequal digital access leave some communities underrepresented, potentially biasing computational understanding toward digitally visible places.The passage identifies vulnerable and digitally disconnected populations as especially underrepresented.Source: §1 · p. 2
  • Contextual language models enable fine-grained classification, yet imbalanced training data can amplify inequities and leave sentiment estimates locally unrepresentative.The passage also notes LLMs’ computational cost, prompt sensitivity, and limited transparency in commercial models.Source: §1 · p. 2

estimates of community sentiment when online evidence is uneven across affected places?

The framework combines cross-city community representations with text and aggregate mobility to contextualize sentiment when online participation is uneven. Its contributions include cross-disaster evidence on how contextual grounding changes community-level sentiment interpretation.

estimates of community sentiment when online evidence is uneven across affected places?

  • The framework uses socioeconomic and demographic similarity to transfer evidence from data-rich to data-sparse areas while preserving locally observed signals.These cross-city relationships serve as a weak structural prior rather than assuming communities are equivalent.Source: estimates of community sentiment when… · p. 2
  • It integrates aggregate mobility with textual representations to provide behavioral context for interpreting online expression.The framework operates at two levels: inter-city similarity and text-plus-mobility contextualization.Source: estimates of community sentiment when… · p. 2
  • The work contributes a framework combining cross-city representation learning and multimodal contextualization for disaster sentiment analysis under uneven participation.It also presents cross-disaster evidence on contextual grounding’s effects on community-level sentiment representation and interpretation.Source: estimates of community sentiment when… · pp. 2–3
  • The framework has implications for making sparse, transferred, and context-dependent evidence more visible to crisis-information-system users.The passage frames this as an implication for system design.Source: estimates of community sentiment when… · p. 3

2 Related Work

Prior work identifies unequal digital participation and limits in sentiment-model generalization as challenges for disaster analysis. Context-aware approaches address a gap left by fairness methods focused on observed training data.

2.1 Digital Inequity in Crisis Communication

  • Social media is used across crisis phases for preparedness, response, and recovery, but self-selected participation leaves marginalized and digitally disconnected populations underrepresented.The cited examples include demographic disparities and barriers faced by elderly populations.Source: §2.1 · p. 3

2.2 Context-aware sentiment modeling

  • Contextual encoders capture nuanced language, yet sentiment models can generalize poorly across regions, domains, and disaster events.The passage cites CrisisBench and WILDS as benchmarks documenting degradation across geographies or event types.Source: §2.2 · p. 3
  • Fairness methods address observed-data bias, but do not directly address spatial and behavioral context underlying uneven disaster participation.The cited approaches include augmentation, representation-level debiasing, and robust optimization.Source: §2.2 · pp. 3–4
  • Mobility, socioeconomic characteristics, and hazard exposure offer complementary signals for contextualizing online sentiment beyond text-only modeling.The passage connects these signals to reducing reliance on the volume and composition of local posts.Source: §2.2 · p. 4

3 Study Context and Data

The study compares the Southern California wildfires with Hurricane Helene in Florida using social-media, mobility, and community-context data. The data are harmonized at the city level across both cases.

3.1 Disaster Cases

  • The January 2025 Southern California wildfires and 2024 Hurricane Helene provide contrasting disaster settings across urban, suburban, coastal, inland, and less urbanized communities.The wildfire case includes heterogeneous urban–rural communities; Helene affected coastal, metropolitan, and inland Florida communities.Source: §3.1 · p. 4

3.2 Online Community Data

  • X posts were collected with case-specific keyword queries, so the observations reflect visible platform users rather than a representative sample of affected residents.California queries included wildfire-related terms; Helene queries included event-specific and broader hazard terms.Source: §3.2 · pp. 4–5

3.3.1 Aggregate Mobility.

  • Device-level point-of-interest visits were converted into daily tract-level origin–destination flows, excluding pairs with fewer than five trips.The flows provide aggregate community-level behavioral context and are not linked to individual social-media users.Source: §3.3.1 · p. 5

3.3.2 Socioeconomic and Hazard Context.

  • Socioeconomic attributes describe demographic, economic, housing, infrastructure, and urbanization conditions, while FEMA variables capture case-specific wildfire or hurricane risk.The attributes are harmonized to a common city-level representation.Source: §3.3.2 · p. 5

3.4 Data Preparation and Alignment

  • After preprocessing and alignment, the dataset contains 29 California and 29 Florida communities, each represented by posts, mobility, and socioeconomic and hazard attributes.The California set includes 13 urban and 16 non-urban communities; Florida includes 14 urban and 15 non-urban communities.Source: §3.4 · pp. 5–6

4 Methods

The framework combines post-level text and aggregate mobility with risk-informed relationships among communities, then adapts a shared model using weighted evidence from related cities. It uses staged training and evaluates predictive performance alongside representation, mobility correspondence, and component contributions.

4.1 Framework Overview

  • The framework combines post-level multimodal learning, cross-city community representations, and similarity-guided adaptation to supplement sparse local evidence.A shared model is trained across communities before cross-city relationships guide target-community adaptation.Source: §4.1 · p. 6

4.2 Post-level Contextual Learning

  • Text and aggregate mobility are fused for sentiment classification; mobility supplies community-level context, not individual-user behavior.The text encoder maps the [CLS] representation to a fixed-length embedding, while a mobility encoder produces a parallel representation for fusion.Source: §4.2 · p. 6
  • Training first uses Mistral-generated noisy labels, then human annotations refine the decision boundary while both encoders remain frozen.In the refinement stage, only the fusion network and classifier are updated; the resulting model is the global model.Source: §4.2 · p. 7

4.3 Cross-city Community Representation Learning

  • Socioeconomic and hazard features yield risk-ordered city embeddings through a weighted ordinal objective, not proof of city equivalence or hard ranking constraints.The objective learns graded relationships using hazard-risk proximity as its ordering signal.Source: §4.3 · p. 7
  • Risk-difference pools define positive, semi-positive, and negative training roles, while only cities with labeled observations can donate transfer examples.Nearest-neighbor similarity identifies candidate donors but does not imply equivalent experiences, needs, or sentiment distributions.Source: §4.3 · pp. 7–8

4.4 Similarity-guided City Adaptation

  • Local observations retain full weight, while examples from top-K related cities receive similarity-based weights; donor contribution also depends on transferred sample count.This preserves the distinction between directly observed and transferred supervision while letting communities with limited local data draw selectively on related locations.Source: §4.4 · pp. 8–9
  • During adaptation, the text and mobility encoders stay frozen while the fusion network and classifier update on local and transferred observations.Transferred samples are weighted by learned similarity, and each donor's total contribution also depends on its sample count.Source: §4.4 · p. 8

4.5 Evaluation Setup

  • Evaluation compares seven model configurations using accuracy, macro-F1, and class-wise F1, alongside analyses of representation, mobility correspondence, and component contributions.The analyses examine augmentation, mobility integration, cross-city learning, encoder freezing, and alternative ways of incorporating mobility.Source: §4.5 · p. 9

5 Results and Evaluations

Across the California wildfire and Florida hurricane, the adaptive model leads overall F1 performance, while the analyses examine representational coverage, local sentiment variation, sentiment–mobility patterns, and design choices. The results also show that augmentation, city-wide representation learning, frozen encoders, and structured mobility fusion have distinct effects across these settings.

5.1.1 Classification Performance.

  • F1 = 0.5923 in California and 0.7615 in Florida make the adaptive model the top overall performer in both cases.Global-Fusion reaches 0.5549 and 0.7483, respectively; local adaptation does not consistently improve the pure-text model on its own, particularly in Florida.Source: §5.1.1 · pp. 9–10, Table 1 · p. 10
  • The adaptive model leads neutral and positive sentiment in both cases, but Mistral has the highest California negative-class F1 at 0.5383.In Florida, the adaptive model leads all classes: negative 0.7885, neutral 0.7927, and positive 0.7034.Source: §5.1.1 · p. 10, Table 2 · p. 10

5.1.2 Urban–Non-Urban Representation.

  • Augmentation yields the largest representational changes in communities with limited original observations, while its effects are less pronounced in Florida.The passage identifies Merced’s positive sentiment shift from 0% to 34% and negative shift from 50% to 39%; most Florida city negative-sentiment distributions remain closely aligned before and after augmentation.Source: §5.1.2 · pp. 11–12, Figure 2 · p. 11

5.2.1 Spatial Sentiment Variation.

  • The adaptive model reveals more localized sentiment differences than pure-text and LLM models, including differing sentiment directions in nearby Pasadena and Altadena.In Florida it retains the broadly negative regional tendency while increasing the sample-size-weighted standard deviation to 7.22 percentage points, versus 6.66 for Pure-text and 6.11 for LLM.Source: §5.2.1 · p. 12, Figure 3 · p. 13

5.2.2 Sentiment-Mobility Relationship.

  • Mobility embeddings reorganize sentiment representations differently by disaster: into more distinct local regions in California and more continuous differentiation in Florida.Average sentiment and mobility correlate positively in California and predominantly negatively in Florida; in Altadena, adaptive and pure-text R^2 values are 0.606 and 0.551, respectively.Source: §5.2.2 · pp. 13–14, Figure 4 · p. 14, Figure 5 · p. 15

5.3.1 Generalization of cross-tity representations.

  • Learning city representations across all available cities produces smoother, more coherent similarity structures than learning only from cities covered by social media.The comparison uses 938 California cities and 956 Florida cities in the socioeconomic datasets; the Florida analysis also reports a coherent structure from 29 broader-set locations.Source: §5.3.1 · pp. 15–16, Figure 6 · p. 15

5.3.2 Training strategy for model stability.

  • Freezing both text and mobility encoders while training only the perception layer consistently performs better across all metrics.The comparison is between adaptation with frozen and unfrozen encoders.Source: §5.3.2 · p. 16, Figure 7 · p. 16

5.3.3 Effectiveness of multimodal fusion.

  • City-specific fusion outperforms prompt-based mobility input, with F1-scores rising from 0.35 to 0.56 in the wildfire and 0.69 to 0.76 in the hurricane.The comparison treats mobility either as verbalized textual input or as a separate numerical modality.Source: §5.3.3 · p. 16, Table 3 · p. 16

6 Discussion

The discussion argues that crisis sensing must account for where online expressions originate and how communities respond behaviorally. Online content should not be treated as uniformly representative of ground truth.

6 Discussion

  • Disaster sensing should account for where online expressions originate and how communities respond, rather than treating online content as uniformly representative of ground truth.The authors frame this as relevant to supporting community expression, representation-aware interfaces, and emergency decision-making.Source: §6 · pp. 16–17

6.1 Supporting Context-Rich Crisis Communication

  • Contextual grounding can expose local distinctions without asking users for extra input, and its value depends on ambiguity in the original text.The framework draws on socioeconomic, hazard, and mobility information; broader contextual evidence can also compensate for uneven participation.Source: §6.1 · p. 17

6.2 Representation-Aware Interface Design

  • Interfaces should show post volume, coverage, and class balance alongside estimates, and distinguish observed information from similarity-transferred estimates.The discussion recommends provenance labels and uncertainty cues so users can judge how much local evidence supports an estimate.Source: §6.2 · p. 18
  • Sentiment and mobility should remain coordinated but distinct because their relationship varies across cities and disaster types.Combining them into one community-condition score could obscure differences; a coordinated view can show where signals converge or diverge.Source: §6.2 · p. 18

6.3 Equitable Crisis Decision-Making

  • Framework outputs should direct attention and verification, not automate resource allocation, and should be interpreted alongside other evidence.The discussion names emergency calls, shelter demand, infrastructure outages, traffic conditions, damage assessments, and local reports as potential corroborating evidence.Source: §6.3 · p. 19

6.4 Limitations and Future Work

  • Keyword retrieval and inferred locations leave uneven coverage; transfer can supplement sparse posts but cannot recover absent residents or ensure representativeness.Transfer also depends on whether socioeconomic and hazard attributes capture meaningful relationships among cities.Source: §6.4 · p. 20
  • Mobility and sentiment are incomplete signals: aggregate movement may reflect routine or economic activity, and mobility input makes their correspondence non-independent validation.Sentiment labels can also collapse posts combining emotional expression, factual reporting, assistance requests, and institutional communication.Source: §6.4 · p. 20
  • The study covers city-level data from two disasters, and its interface and decision-support recommendations have not been evaluated with users.The authors propose finer-scale evaluation across more hazards, regions, and times, plus co-design and user evaluation.Source: §6.4 · p. 20

7 Conclusion

The framework combines text and aggregate mobility context with similarity-guided transfer based on socioeconomic and hazard characteristics to address uneven online evidence. Across two disasters, it achieves the strongest overall sentiment-classification performance, with augmentation especially helpful where observations are limited.

7 Conclusion

  • The adaptive model achieves the strongest overall sentiment-classification performance across the Southern California wildfires and Hurricane Helene in Florida.Augmentation yields the largest gains in representational coverage in communities with limited original observations.Source: §7 · p. 20
  • The framework preserves finer-grained cross-city community-level sentiment variation than text-only and LLM-based approaches.It also strengthens correspondence between online sentiment and observed mobility patterns, with the relationship's direction varying across disaster contexts.Source: §7 · p. 20
  • Crisis systems are more informative when they show how much evidence is locally observed versus transferred and where linguistic and behavioral signals converge or diverge.Such transparency can help users identify informational blind spots, interpret outputs cautiously, and determine where additional local verification may be needed.Source: §7 · p. 21

Ethics

The study uses licensed social-media data under platform and ethical research requirements. It protects privacy through aggregate processing and anonymization, and uses consented human annotation.

Ethics

  • Social-media data were accessed under an academic research license for noncommercial scholarly purposes and are not publicly available.Source: Ethics · p. 25
  • Records were anonymized and processed in aggregate, with no personally identifiable information collected, stored, or disclosed.Source: Ethics · p. 25
  • Trained researchers performed sentiment annotations with informed consent, following digital-research and data-governance requirements.Source: Ethics · p. 25

Sentiment prompt

The sentiment instructions assign each tweet one of three labels and specify how to handle emotional, informational, and ambiguous content. The supplied section also describes screening census-tract predictors for the city-wide learning layer.

Sentiment prompt

  • Each tweet receives one sentiment label: negative, neutral, or positive, with the output restricted to the requested JSON fields.Source: Sentiment prompt · pp. 25–26
  • The policy maps distress, danger, loss, or anger to negative; relief, gratitude, help, or hope to positive; and factual or ambiguous content to neutral.When uncertain, annotators choose neutral; emojis, intensifiers, negations, sarcasm, hashtags, and mentions count only when sentiment is unambiguous.Source: Sentiment prompt · p. 25
  • VIF screening removed predictors until surviving variables were below 5, yielding 37 additional predictors alongside 22 core ACS indicators.The resulting 59 tract-level variables provide structured inputs for city embeddings; Table 4 reports final VIF values.Source: Descriptive statistics of the variables · p. 26, Table 4 · p. 26

Annotation and model selection

The study compares sentiment classifiers using manually annotated California and Florida disaster tweets and evaluations of eight LLMs. Mistral-8B was selected for downstream analysis, while the evaluation also reports systematic interpretation biases in tested LLMs.

Annotation and model selection

  • Tweets for the California and Florida cases were collected from Brandwatch using case-specific search terms and locations.The California terms include wildfire-related variants, while Florida terms include Hurricane Helene and related storm terms.Source: Annotation and model selection · pp. 26–27
  • Two trained annotators labeled 2,000 tweets per case as positive, neutral, or negative, using emotional distress, support, and factual content to distinguish the classes.Source: Annotation and model selection · p. 27
  • Krippendorff’s Alpha was used to assess annotation consistency, and the reported final reliability indicates substantial consistency and interpretive alignment.The measure adjusts agreement for chance; higher values indicate stronger adjusted agreement.Source: Annotation and model selection · pp. 27–28
  • Eight LLMs were evaluated with a standardized workflow for location detection, event relevance, and sentiment classification under zero-shot and few-shot settings.Only minimal preprocessing was applied to retain the social-media data's inherent noise.Source: Annotation and model selection · p. 28
  • Mistral-8B achieved the highest and most stable accuracy among evaluated systems, closely followed by DeepSeek, and was selected as the baseline sentiment classifier.The selection also considered efficient A100-GPU inference and suitability for fine-tuning under constrained computational settings.Source: Table 7 · p. 29, Annotation and model selection · p. 29
  • The evaluation reports that tested LLMs can achieve competitive accuracy while showing systematic sentiment-interpretation biases for nuanced, community-specific, or geographically contextualized language.Source: Annotation and model selection · p. 29

Related papers

Loading 2610.22551v1…