PaperlayerView exact revision on arXiv

Source-linked AI summary

Poetic Heritage for Culturally Grounded Emotional Support: An Interaction Design Framework and Its Multimodal Agentic Instantiation

Yangming Zhang, Zhiqian Li, Bin Wu, Qi Li, Jie Xu, Yunpeng Song, Liang Zhao

arXiv:2608.22639v1cs.HC

TL;DR

Digital emotional-support systems often remain culturally generic, motivating the study of poetic tradition as a culturally grounded interactive medium. The paper develops a design framework, instantiates it as Poemithy, and evaluates multimodal and text-only versions with 50 participants. Both conditions improved affective outcomes within sessions, while multimodality showed clearer advantages on selected experiential measures.

  • Problem

    Digital emotional-support systems may overlook culturally specific needs, while how poetic traditions should structure AI-mediated support remains empirically underexplored.

  • Method

    The paper develops an interaction design framework and instantiates it as Poemithy, a multimodal LLM-enabled multi-agent system for guided reflection through classical Chinese poetry.

  • Results

    The study distinguishes outcomes observed across both conditions from additional experiential differences associated with multimodal presentation.

  • Takeaways & Limitations

    The findings support culturally grounded content and structured guidance as anchors for design, with multimodal presentation potentially strengthening resonance and engagement.

  • Takeaways & Limitations

    LLM-based emotional-support systems carry risks around safety, transparency, accountability, dependency, and privacy.

Abstract

from arXiv · show

Digital systems increasingly mediate emotional support, yet their interactions often remain culturally generic. Accordingly, we examine how a poetic tradition can be operationalized as a culturally grounded interactive medium and how generative AI can support such engagement. The resulting interaction design framework translates staged literature-based support and tradition-specific poetic aesthetics into guidance for digital system design. Poemithy instantiates the framework as a multimodal, LLM-enabled multi-agent system for guided reflection through classical Chinese poetry. A controlled between-subjects study with 50 participants compared text-only and multimodal versions. Both conditions showed medium-to-large within-session improvements in affect, anxiety, and emotion regulation, while between-condition tests detected no differences in these changes. Among secondary post-session user-experience measures, the clearest observed differences favored multimodality in perceived attunement, perceived task success, and engagement; usability and hedonic quality were descriptively higher, while workload did not differ detectably. Post-only cultural ratings were descriptively favorable in both conditions for cultural identification, poetry-engagement and dissemination intentions, and perceived cultural enrichment. Together, the findings suggest that culturally grounded content and structured guidance should anchor system design, while multimodal presentation may strengthen resonance and engagement. More broadly, the work shows how generative AI can mediate engagement with poetic heritage in culturally grounded emotional-support interactions.

1 Introduction

The paper addresses how poetic traditions can become culturally grounded media for interactive emotional support. It develops a framework and instantiates it in Poemithy, then evaluates multimodal presentation against text-only interaction.

  • Culturally generic emotional-support systems may overlook users’ backgrounds, communicative norms, lived experiences, and culturally specific support resources.
  • Poetry therapy offers a staged design precedent involving receptive engagement, expressive creation, and symbolic or ceremonial closure.
  • Poetic traditions can function as culturally situated language for sensing, expressing, and transforming emotion, but their digital affordances remain difficult to access.
  • Existing AI poetry work demonstrates technical feasibility but does not explain how a particular poetic tradition should inform emotional-support system design.
  • The framework connects staged literature-based support with tradition-specific aesthetics, and Poemithy realizes it as a multimodal, LLM-enabled multi-agent system using classical Chinese poetry.
  • A controlled between-subjects experiment with N=50 compares multimodal and text-only interfaces sharing the same workflow, cultural content, and system logic.
  • The findings distinguish outcomes observed across both conditions from additional experiential differences associated with multimodal presentation.

2 Background and Related Work

Related work frames poetry therapy as structured emotional engagement and highlights cultural sensitivity in digital support. Prior AI-poetry systems show varied possibilities, while staged culturally grounded workflows and multimodality remain underexplored.

  • Poetry Therapy and Culturally Situated Emotional Support: Poetry therapy uses structured poetic engagement, commonly involving curated poems, creative writing, and symbolic or ceremonial acts.
  • Poetry Therapy and Culturally Situated Emotional Support: Related literary-support models describe movement from encounter through expression toward reflective integration, while allowing non-strict sequencing.
  • Poetry Therapy and Culturally Situated Emotional Support: Classical poetry traditions have been associated with emotional restoration, but poetic heritage remains better established culturally than as systematized digital intervention.
  • AI-Mediated Emotional Support and Poetic Reflection: Users often seek support grounded in their backgrounds and lived experiences, whereas generic mental-health chatbots may overlook local stigma, privacy expectations, language practices, and resources.
  • AI-Mediated Emotional Support and Poetic Reflection: Prior AI-mediated poetry systems support recommendation, listening, immersive presentation, somatic introspection, and creative expression, but their organization as staged workflows remains unclear.
  • Multimodality: Multimodal emotional-support systems may provide additional cues and help users make sense of diffuse feelings, yet richer media do not necessarily strengthen core well-being outcomes.
  • Multimodality: The literature therefore motivates treating poetic tradition as a culturally grounded medium while decomposing content, delivery, engagement strategy, and support structure into distinct design components.

3 Theoretical Framework

The paper proposes a three-layer framework that treats poetic tradition as an interactional medium for culturally grounded emotional support. It connects staged support processes, tradition-sensitive poetic aesthetics, and interactive system realization.

  • Three-Layer Framework: The framework organizes culturally grounded emotional support into literature-based support processes, poetic aesthetics, and interactive system realization.The three layers are Encounter–Articulation–Ritualization, aesthetic dimensions of poetic expression, and system capabilities.
  • Literature-Based Emotional Support Process: Encounter, Articulation, and Ritualization guide users from culturally meaningful material to shaped expression and culturally meaningful closure.The stages respectively support guided literary engagement, expressive formation through poetic conventions, and marked completion through performance or closure.
  • Interactive System Realization: The framework separates Support Interaction Functions from Multimodal Interface Affordances so cultural materials, scaffolding, and presentation channels can be traced separately.Support functions operationalize the staged process, while interface affordances render poetic aesthetics through tradition-appropriate modalities.
  • Aesthetic Dimensions: Atmospheric Anchoring, Structural Order, and Symbolic Performance define sensory-agnostic experiential dimensions for tradition-specific poetic materials.These dimensions address affective orientation, formal organization of experience, and culturally marked enactments of completion.
  • Tradition-Sensitive Adaptation: Applying the framework requires tradition-sensitive adaptation linking support stages, aesthetic affordances, and realization capabilities to a tradition’s literary and ritual practices.Concrete interactional realization remains an empirical design question rather than a fixed universal mapping.
  • Classical Chinese Poetry Instantiation: For classical Chinese poetry, Pictorial, Architectural, and Musical Beauty illustrate how poetic imagery, composition, and sound populate the framework.The design probe shows how a tradition’s practices can instantiate the three-layer structure.

4 Formative Research

Formative research used an early prototype, pilot study, and expert consultation to resolve open instantiation choices. The resulting findings favored sequential guidance, accessible empathic dialogue, multiple expressive entry points, culturally grounded content, and concrete closure rituals.

  • 4.1 From the Framework to Instantiation Requirements: The formative process combined an early prototype, pilot user study, and expert consultation to derive traceable instantiation requirements.These activities addressed stage progression, dialogue style, scaffolding, and Ritualization choices.
  • 4.2 Early Formative Prototype: The prototype covered all three stages, with Encounter combining consultation and poem matching, Articulation supporting sketching or poem generation, and Ritualization supporting recitation.The flow enforced Encounter first but allowed free navigation between Articulation and Ritualization.
  • 4.3 Pilot User Study: 4.06 to 4.36 were the mean stage-level ratings for emotional and cultural/aesthetic experience across Encounter, Articulation, and Ritualization.Overall ratings were usability M=4.20, willingness to use M=4.24, and perceived emotional-support helpfulness M=4.25.
  • 4.3 Pilot User Study: Sequential progression should be enforced because optional navigation confused users and made the overall journey feel fragmented.The revised design therefore guided users through the stages, including the Articulation–Ritualization transition.
  • 4.3 Pilot User Study: Conversational style should be modern, accessible, empathic, and interwoven with literary engagement rather than overly literary or merely diagnostic.Pilot feedback favored cultural authenticity in the poem itself, deeper interpretation, and dialogue as a central part of Encounter.
  • 4.3 Pilot User Study: Articulation should offer non-verbal entry points because participants particularly valued drawing-based expression and the system’s interpretation of their sketches.This finding motivated multiple expressive channels rather than requiring users to articulate feelings verbally.
  • 4.3 Pilot User Study: Users wanted cross-temporal cultural presence beyond disembodied poem text, while experts recommended direct-address closure rituals instead of poet images.The expert recommendation treated direct address as a stronger affective route to connection with the poet.
  • 4.4 Expert Consultation: Expert consultation added deeper emotional and imagery annotation, mixed guided choices with open dialogue, psychoeducational or referral support for significant distress, and unresolved audio-design disagreement.Experts warned that relying only on LLM parametric knowledge could yield culturally generic or inaccurate interpretations, and they differed over chanted versus ambient audio.

5 System Design

Poemithy instantiates a three-stage, culturally grounded workflow through coordinated dialogue agents, functional modules, shared session state, and poetry retrieval. Each stage couples a support function with a tradition-specific experiential affordance, while multimodal components render poetic imagery, composition, and performance.

  • Architecture Overview: Poemithy coordinates six dialogue agents and four functional modules through shared session state containing the emotion profile, selected poem, co-created poem, and dialogue history.Components read from and write to this state as the workflow progresses.
  • Architecture Overview: The guided session proceeds through Encounter, Articulation, and Ritualization, each foregrounding one tradition-specific experiential affordance grounded in the Three Beauties.The interface labels these stages Receiving & Exploration, Writing & Catharsis, and Recitation & Ceremony.
  • Stage 1: Encounter: Encounter matches an emotion-situation profile with a culturally appropriate poem, then combines poem-specific media, interpretation, audio, and reflective dialogue.Consultation elicits the user's situation, feelings, intensity, and poetry preferences before poem presentation and reflection.
  • Stage 1: Encounter: Multimodal generation derives poem-specific visual storyboards and ambient soundscapes from imagery, mood, and scene progression to support Atmospheric Anchoring during Encounter.The assets are rendered as ink-style picture cards, animated backgrounds, and ambient soundscapes while dialogue continues.
  • Stage 2: Articulation: Articulation translates diffuse feelings into imagery and an editable calligraphic poem through guided dialogue, optional sketching, culturally grounded allusions, and iterative revision.AI scaffolding lowers the classical poetry literacy barrier while supporting alternative expressive entry points.

6 User Study

The user study compared multimodal and text-only versions of the same Poemithy workflow in a controlled experiment with 50 Chinese university students. The conditions shared orchestration and stage workflow but differed in presentation modality.

  • Participants: 50 Chinese university students aged 18–28 participated in the controlled comparative experiment.Participants were randomly assigned to multimodal (n = 25) or text-only (n = 25) conditions.
  • Participants: Poetry literacy scores ranged from 6 to 17, with 14% beginners, 72% intermediate, and 14% advanced participants.The literacy measure summed four 1–5 background items and showed α = .80.
  • Participants: Participants meeting PHQ-9 ≥15 or GAD-7 ≥15 screening cutoffs for moderate-to-severe depressive or anxiety symptoms were excluded.The screening procedure was used to reduce potential risk.
  • Experimental Conditions Control: The multimodal and text-only conditions shared the same agent orchestration, stage workflow, and underlying poetry knowledge base.The conditions differed only in the modality of presentation.
  • Experimental Conditions Control: The multimodal interface added ambient sound, illustrations, animation, audio recitation, optional sketch-based expression, calligraphic rendering, and a visual closing letter.The text-only condition presented the same textual content in a standard chatbot interface without these elements.

Appendix A.

A controlled comparison of text-only and multimodal Poemithy sessions found sizable within-session improvements across affective outcomes in both conditions, but no detected between-condition differences in change. Multimodality instead showed clearer advantages in perceived attunement, task success, engagement, and related experiential measures.

  • Affective outcomes: Both conditions showed robust pre–post improvements across positive affect, negative affect, state anxiety, and emotion-regulation difficulty.Every within-condition change remained significant after Holm correction, with all adjusted p< .001.
  • Affective outcomes: Between-condition tests detected no differences in change scores for the four affective outcomes.The comparison therefore distinguished shared short-term affective change from modality-associated experiential differences.
  • Qualitative experience: Participants described poem matching as understanding, co-creation as externalizing feelings, recitation as embodied release, and the virtual poet’s letter as cross-temporal dialogue.Participants also linked cultural familiarity and recognizing shared struggle across centuries with emotional resonance.
  • User experience: The multimodal condition scored higher on perceived poem–feeling fit and feeling understood, while differences were smaller or undetected for accompaniment and new perspective.Perceived poem–feeling fit was 4.40 versus 3.80 (p= .0078, d= 0.79), and feeling understood was 4.24 versus 3.56 (p= .0150, d= 0.72).

7 Discussion

The discussion frames poetic heritage as an interactive medium and separates culturally grounded support functions from multimodal interface affordances. The study’s shared workflow improved affect in both conditions, while multimodality primarily strengthened experiential quality, motivating a content-first, modality-second design principle.

  • Framework: Poetic heritage is conceptualized as an interactive medium for everyday emotional support rather than content alone.The framework connects staged literature-based support, tradition-specific aesthetics, and interactive system realization.
  • Framework: The framework recasts receptive, expressive, and ceremonial activities as interaction roles and separates support functions from multimodal interface affordances.This separation allows interface roles to be traced without treating any single modality as essential.
  • Empirical interpretation: Both conditions shared the poetry-centered workflow and showed within-session affective improvements, whereas the clearest between-condition differences concerned experiential quality.The discussion identifies perceived attunement, task success, engagement, and related experience measures as the main modality-associated contrasts.
  • Design implications: The proposed content-first, modality-second principle establishes culturally grounded materials and progressive scaffolding before adding multimodal affordances.Multimodal features should make stage-specific aesthetic qualities more perceptible and engaging.
  • Design implications: Multimodal augmentation was most coherent when it amplified culturally inherent qualities, including atmosphere, rapid attunement, and non-verbal articulation, rather than generic sensory embellishment.Participants associated soundscapes with atmosphere, dynamic visuals with attunement, and the drawing canvas with a non-verbal channel.
  • Interaction structure: Dedicated interface components redistributed procedural load, improved perceived integration and flow, and left the dialogue channel for fewer, richer substantive inputs.Participants also described the three-stage structure as helping maintain focus and reduce dialogue load.
  • User agency: The staged workflow preserved creative agency by making users decision-makers in selecting imagery and shaping poems, while generation quality and editing affordances remained prerequisites for co-creation.Participants valued jointly authored outputs, but mechanically assembled results weakened the experience.
  • Accessibility and culture: Favorable changes across beginner-to-advanced poetry literacy suggest that interpretive dialogue, imagery prompts, and graduated composition can lower entry barriers without removing culturally specific material.The observed models did not indicate that greater prior literacy was required for larger improvements.

8 Conclusion

The paper presents a culturally grounded interaction design framework and instantiates it in Poemithy, a multimodal, LLM-enabled multi-agent system using classical Chinese poetry. In a 50-participant controlled study, both conditions improved affective outcomes, while multimodality showed clearer differences in experiential quality.

  • The framework connects staged literature-based emotional support with tradition-specific poetic aesthetics for interactive-system design.
  • Poemithy instantiates the framework through classical Chinese poetry as a multimodal, LLM-enabled multi-agent system for guided emotional reflection.
  • Medium-to-large within-session improvements in affect, anxiety, and emotion regulation occurred in both multimodal and text-only conditions, with no detected between-condition differences.
  • The clearest secondary user-experience differences concerned perceived attunement, perceived task success, and engagement.
  • Post-only ratings were descriptively favorable in both conditions for cultural identification, poetry-engagement and dissemination intentions, and perceived cultural enrichment.
  • The framework offers a reusable structure for other poetic traditions when materials, aesthetics, and interaction design are grounded locally.

9 Ethical Considerations

Poemithy is framed as non-clinical reflective support rather than therapy, with safeguards for crisis disclosures and privacy requirements for deployment. The paper also identifies potential mismatches in retrieval-based poetry curation as an ethical limitation.

  • Poemithy is designed as non-clinical reflective emotional support, not therapy or a substitute for professional mental-health services.
  • The interface displays a persistent non-clinical-scope notice and crisis hotline number.
  • During the study, experimenter oversight safeguarded crisis-level disclosures; deployment would require automated monitoring and referral protocols.
  • Retrieval-based poetry matching may produce scenario mismatches that feel dismissive rather than supportive, motivating future feedback loops.
  • Sensitive emotional narratives, voice recordings, and user-created poems were stored in an encrypted cloud database.
  • Production deployment would require explicit consent protocols and compliance with local privacy regulations.

A Study Workflow and Experimental Conditions

The controlled study used the same sequence and underlying interaction logic in both conditions, varying presentation modality and available interface features. The multimodal version added generated visual and audio elements while preserving the textual content.

  • Figure 13 summarizes the study sequence shared by both experimental conditions.
  • Both interfaces preserved the same prompts, poem set, stage sequence, and agent logic.
  • The experimental contrast was limited to presentation modality and available interface features.
  • In the Stage 1 poem-reception step, the multimodal interface displayed generated poetry cards and animated videos alongside text, reading audio, chat, and ambient-sound controls.
  • The text-only condition preserved the same textual content through a plain chatbot interface.

B Participant Profile and Screening

The main study included 50 screened participants, with poetry literacy measured using a four-item self-assessment and categorized into four levels. The study also examined affective improvement across literacy strata using descriptive and robust model-based analyses.

  • The controlled study sample comprised N=50 participants with a complete anonymized profile reported in Table 2.
  • Poetry literacy was derived from four background-questionnaire items scored 1 to 5 and summed to a 4–20 score.
  • Scores of 4–8, 9–14, 15–18, and 19–20 were categorized as Beginner, Intermediate, Advanced, and Professional, respectively.
  • Table 3 reports affective improvement across three observed self-reported poetry-literacy strata and continuous HC3-robust models.

C Exploratory Poetry-Literacy Accessibility Analysis

The section examines exploratory relationships between poetry literacy and post-session responses, alongside the system’s staged reflective interaction and multimodal poetic activities.

  • Poetry-literacy analysis: Positive values represent favorable affective change, calculated as post-minus-pre for PANAS-PA and pre-minus-post for PANAS-NA, STAI-SF, and S-DERS.Poetry literacy was standardized before regression; coefficients represent estimated improvement differences associated with a one-standard-deviation increase in literacy.
  • Post-session cultural measures: Participants rated post-session cultural identification, learning orientation, poetry-engagement intentions, dissemination intentions, and perceived cultural enrichment.Items used a 5-point Likert scale and asked participants to report genuine feelings after the session.
  • Post-session cultural measures: Poetry-engagement items assessed saving favorite verses, continuing classical-style composition or adaptation, and reading or reciting poetry to express or regulate emotions.These measures addressed both continued creative participation and personal poetic practice.
  • Post-session cultural measures: Dissemination items assessed using classical Chinese poetry in everyday communication, recommending emotion regulation through poetry, and supporting accessible promotion by platforms.The items covered personal expression, interpersonal recommendation, and broader platform support.
  • System interaction and analysis: The qualitative protocol examined helpfulness, poem-emotion connection, the guided sequence, co-creation participation, interface features, and changing relationships to classical Chinese poetry.Two researchers independently coded 30% of transcripts, reconciled disagreements, and applied the codebook to the remaining transcripts.
  • System interaction and analysis: The system’s interaction sequence included consultation, preference extraction, poem recommendation and interpretation, imagery guidance, classical transformation, recitation guidance, feedback, and a poet letter.Prompts used session context, user preferences, emotional summaries, retrieved poetry candidates, and multimodal rendering when active.

H Relationship to Prior Work

The present article extends two earlier non-archival abstracts by providing a fuller account of the framework, system, evaluation, and implications.

  • Earlier related outputs: An earlier UbiComp/ISWC demonstration described an early Poemithy prototype and its three-stage interaction concept.The demonstration is identified as a preliminary component related to the present article.
  • Earlier related outputs: A CHI poster summarized the multimodal-versus-text-only controlled study.The poster is distinguished from the present article’s fuller treatment.
  • Present article’s extension: The present article adds the full framework development and design rationale, a complete system account, a substantially fuller mixed-method evaluation, and broader implications.These elements were outside the scope of the earlier extended abstracts.
Loading 2608.22639v1…