PaperlayerView exact revision on arXiv

Source-linked AI summary

From Qualitative Text to Integrated Evidence: The Parallel-Then-Integrated (PTI) Procedure

Elyas Irankhah, Ashley Hagaman, Samah Fodeh

arXiv:2610.19164v1cs.CL

TL;DR

Researchers have less guidance for integrating multiple computational representations of the same qualitative corpus, whose outputs may differ in meaning and form. PTI keeps ontology-guided coding and data-driven themes distinct before aligning and interpreting them; in the Nepal demonstration, they partly converged but also represented some domains differently.

  • Problem

    Guidance is limited for aligning multiple computational representations of the same qualitative corpus, whose outputs may differ in conceptual meaning, structure, and scale.

    Source: Introduction · p. 2, Mixed Methods Integration in… · p. 3

  • Method

    PTI independently generates ontology-guided and data-driven representations, aligns them in a crosswalk, assesses overlap and uniqueness, and integrates them in a joint display.

    Source: Introduction · p. 2, Unsupervised NMF theme representation · p. 10, Conclusion · p. 19

  • Results

    Ontology-guided and data-driven approaches represented the same participant-response text differently, with some domains partly converging and others represented mainly by one approach.

    Source: Discussion · p. 18, Conclusion · p. 19

  • Takeaways & Limitations

    The crosswalk and joint display make inclusion, alignment, and divergence decisions traceable while preserving the representations as distinct.

    Source: Conclusion · p. 19

    The small cohort limits the strength of supervised findings, which are exploratory and do not support substantive claims about perinatal mental health.

    Source: Limitations · p. 18

Abstract

from arXiv · show

Mixed methods research offers established strategies for integrating qualitative and quantitative evidence, but less guidance for integrating two computational representations of the same qualitative corpus. This methodological study introduces Parallel-Then-Integrated (PTI), a four-step procedure for explicit and traceable integration. We present PTI using 1,203 interview paragraphs from 29 mothers in Nepal. The two representations were ontology-guided Patient Voice (PV) coding with PVminer and data-driven themes generated through non-negative matrix factorization (NMF). A theme-complete crosswalk documented convergence, divergence, and preprocessing-sensitive differences. Depression status, based on Patient Health Questionnaire-9 (PHQ-9) scores, served as a secondary external anchor for exploratory association checks. PTI provides a structured approach for integrating computational representations of qualitative text within mixed methods research.

Introduction

Computational text analysis creates new ways to represent qualitative data, but two computational representations of the same corpus are not automatically comparable. PTI addresses this gap by keeping representations distinct before aligning, assessing, and interpreting them together.

Introduction

  • PTI addresses limited guidance for aligning and interpreting multiple computational representations of the same qualitative corpus, which may differ in conceptual meaning, structure, and scale.The procedure makes the integration process explicit and traceable rather than proposing a theory-building framework.Source: Introduction · p. 2, Mixed Methods Integration in… · p. 4

Objectives

  • The study develops and illustrates PTI using ontology-guided Patient Voice coding and data-driven NMF to examine overlapping, distinct, and potentially unstable information in the same corpus.It also uses PHQ-9-based depression status only as a mother-level external anchor for exploratory association checks.Source: Objectives · pp. 2–3

Mixed Methods Integration in Computational Text Analysis

  • Computational outputs may share a source without representing the same concept, so alignment decisions and links back to participant text require interpretation.Existing integration approaches were not developed specifically to operationalize pre-integration alignment of multiple computational representations from one corpus.Source: Mixed Methods Integration in… · pp. 3–4

Parallel-Then-Integrated (PTI) Procedure

  • PTI independently quantitizes the same corpus, aligns outputs to shared domains, assesses convergence and uniqueness, then integrates evidence through a joint display.Its four heuristic checks—domain uniqueness, feature overlap, signal increment, and interpretive richness—are considered separately, without a single score or decision threshold.Source: Parallel-Then-Integrated (PTI) Procedure · p. 4
  • PTI’s heuristic checks examine different aspects of relationships between representations, including conceptual domains, numeric features, external-anchor signals, and joint insights.Specific coding and topic-modeling methods can vary across applications.Source: Parallel-Then-Integrated (PTI) Procedure · pp. 4–5

Data Source

The demonstration draws on translated interview transcripts from a community-based perinatal mental health study of adolescent and young mothers in Chitwan, Nepal.

Data Source

  • The data came from a community-based perinatal mental health study of adolescent and young mothers in Chitwan, Nepal.Participants were recruited through postnatal clinics and immunization camps.Source: Data Source · p. 5
  • Interviews were conducted and transcribed verbatim in Nepali, then translated into English while preserving culturally meaningful terms.Source: Data Source · p. 5
  • The analytic corpus included participant responses only; interviewer questions, template text, and non-speech metadata were excluded.Interviews elicited experiences of young motherhood and passive-sensing technology.Source: Data Source · p. 5

Study Design

The study compares two text representations using participant paragraphs and mother-level depression status as an external anchor. It preprocesses the corpus, aggregates paragraph-level features to mothers, and follows PTI’s alignment and integration steps.

Study Design

  • The illustrative sample comprised 29 mothers, including nine depressed and 20 non-depressed, with depression defined by a PHQ-9 cutoff of ≥9.Paragraph-level codes and theme weights were aggregated to mother-level features before modeling.Source: Study Design · pp. 5, 7
  • PTI applies ontology-guided and data-driven methods independently to text units, aligns their outputs in a theme-complete crosswalk, assesses overlap and uniqueness, and builds a joint display for integrated interpretation.The procedure records supporting evidence and assesses signal increment against the external anchor when one is available.Source: Study Design · p. 6
  • Preprocessing reduced 4,253 transcript rows to 1,242 participant-response paragraphs, then removed 39 short turns to yield 1,203 analytic paragraphs.The short-turn filter excluded turns with fewer than five alphabetic words after normalization; all 29 mothers retained at least 12 paragraphs.Source: Study Design · pp. 6–7, Table 2 · p. 7
  • The demonstration workflow transforms participant responses into parallel PVminer and NMF representations, then aligns and integrates them through PTI.Cross-validated logistic regression supplies a secondary external-anchor association check, while the joint display is the primary analytic output.Source: Figure 1 · p. 7

PVminer codebook and feature construction

PVminer applies the Patient Voice ontology to participant-response paragraphs, producing codebook labels grounded in text spans. Paragraph labels are aggregated into mother-level density features for analysis.

PV ontology coding

  • PVminer assigns ontology-defined communication and psychosocial labels, including Social Determinants of Health sub-codes, to participant-response text.The ontology structures patient and provider language into predefined domains; its SDOH sub-codes include economic stability, neighborhood and built environment, and social and community context.Source: PVminer codebook and feature… · pp. 7–8
  • The selected 70B checkpoint had approximately 85% code F1, 83% sub-code F1, and 88% span F1 on the PVminer expert gold standard.Each PV record includes a code, a permitted sub-code, and a supporting text span.Source: PVminer codebook and feature… · p. 8

Feature construction

  • Across 1,203 paragraphs, 41 observed Code–Sub-code pairs were aggregated into a 29 × 41 mother-level density-feature matrix.Each density represents the proportion of a mother’s paragraphs assigned to a given pair; grounded spans were not predictors in the external-anchor analysis.Source: PVminer codebook and feature… · p. 9

PV external-anchor association model

The PV and NMF representations were converted to mother-level features for exploratory association checks with depression status. The procedures used fold-specific PV feature selection, while the NMF representation was fit on the full corpus and is transductive.

PV external-anchor association model

  • Within each LOOCV fold, five PV sub-codes were selected using only the 28 training mothers before fitting ℓ2-regularized logistic regression.The held-out mother’s outcome did not influence feature selection or model fitting.Source: PV external-anchor association model · p. 9
  • The PV analysis treated AUC as an external-anchor association check, not as diagnostic performance or sufficient construct validation.PVminer quantifies codebook-based psychosocial domains and does not diagnose depression.Source: PV external-anchor association model · p. 10

Unsupervised NMF theme representation

  • NMF modeled the same 1,203 paragraphs without imposing the PV ontology, using unigram and bigram TF–IDF features.Bootstrap stability selected k = 6, with mean Jaccard similarity of 0.410; paragraph theme weights were averaged to create a 29 × 6 mother-level matrix.Source: Unsupervised NMF theme representation · pp. 10–11, Table 4 · p. 11
  • Because NMF was fit on all 1,203 paragraphs before cross-validation, held-out mothers’ text contributed to the representation and the AUCs are exploratory transductive estimates.Depression status was not used to fit the unsupervised representation.Source: Unsupervised NMF theme representation · p. 10

Cross-stream alignment and joint-display decisions

PTI aligns the ontology-guided and data-driven representations through a theme-complete crosswalk, then determines which themes contribute a useful shared perspective. A combined model tests the concatenated features against the external anchor.

Cross-stream alignment and joint-display decisions

  • The crosswalk classifies alignment as convergence, partial convergence, or divergence using ontology definitions, NMF terms, representative paragraphs, and grounded PV spans.Each theme’s five highest-weight paragraphs were reviewed.Source: Cross-stream alignment and… · p. 11
  • All themes remain in the crosswalk, but only themes adding a clear and useful perspective enter the joint display.Weak, indirect, or preprocessing-sensitive links are retained in the crosswalk but excluded from the display.Source: Cross-stream alignment and… · p. 11

Combined PV–NMF external-anchor model

  • The combined logistic-regression model concatenates fold-selected PV and NMF features to test for incremental external-anchor signal beyond either representation alone.Within each training fold, up to five PV sub-codes and four NMF themes were selected; the effective NMF contribution was two themes, T0 and T2.Source: Combined PV–NMF external-anchor model · p. 12

Statistical modeling

The analysis compares PV–NMF, PVminer, and NMF representations using mother-level, cross-validated logistic regression. Regularization and permutation testing support exploratory assessment of association with depression status.

Statistical modeling

  • Text-derived features were aggregated to one row per mother before regularized logistic regression compared the three representations.The models used the same mother-level evaluation, with held-out predictions generated through cross-validation.Source: Statistical modeling · p. 12
  • Because the cohort was small and depressed cases limited, models used ℓ2 regularization and were not treated as clinical screening tools.They assessed preliminary external-anchor association at the mother level.Source: Statistical modeling · p. 12
  • LOOCV held out one mother per fold and computed AUC from predictions across all 29 held-out mothers.AUC compares predicted probabilities for randomly selected depressed and non-depressed mothers.Source: Statistical modeling · p. 13
  • Permutation testing used 500 shuffles of mother-level depression labels, rerunning the full cross-validation procedure for each shuffle.The one-sided permutation p-value assessed whether observed AUC exceeded chance.Source: Statistical modeling · p. 13

Results

PTI’s results combine a theme-complete crosswalk and joint interpretation to compare ontology-guided PV coding with data-driven NMF themes. The external-anchor checks were exploratory, while conceptual and interpretive evidence remained primary.

Results

  • Conceptual and interpretive evidence was primary; NMF themes were discovered on all 1,203 paragraphs before LOOCV, making NMF and combined AUCs transductive estimates.The combined model selected five PV and four NMF features at most per fold, with an effective contribution of five PV and two NMF features.Source: Results · p. 13

What ontology-guided PV coding captured

  • PV coding covered ontology-defined psychosocial domains, with partnership codes capturing relational and communication experiences and SDOH codes covering contextual domains.PV densities had a LOOCV AUC of 0.706 (permutation p = 0.106), indicating a directional association with depression status.Source: What ontology-guided PV coding captured · p. 14

What data-driven NMF captured

  • NMF identified participant-language patterns including family and in-law talk, emotional difficulty and help-seeking, activities and preferences, and denial or negation.Six themes reached a LOOCV AUC of 0.680 (permutation p = 0.090); contraction normalization disrupted T4, so it was excluded from integrated interpretation, while T2 remained only in the crosswalk and T5 in the joint display.Source: What data-driven NMF captured · p. 14

Cross-stream alignment, assessment, and integrated interpretation

  • In the economic–relational example, PV identifies economic instability while NMF places the account in family and husband talk, linking hardship with violence in the participant’s relational context.The alignment illustrates how one account’s experience is divided across the two computational representations.Source: Cross-stream alignment, assessment,… · p. 15
  • For affective language about irritation and infant crying, PV captures socioemotional expression while NMF places the paragraph in an emotional-difficulty context.This is partial convergence: both represent emotional experience, but at different levels.Source: Cross-stream alignment, assessment,… · p. 16
  • 0.756 combined-model LOOCV AUC exceeded 0.706 for PV and 0.680 for NMF, an exploratory signal from NMF features T0 and T2 excluded from the joint display.The crosswalk and joint display remained the primary interpretive integration.Source: Table 5 · p. 13, Cross-stream alignment, assessment,… · p. 16
  • The crosswalk found no direct NMF counterpart for Shared decision-making, no matching PV construct for T5 routine disruption, and only partial PV correspondence for T2.T4 was excluded after preprocessing sensitivity checks; mean absolute correlation was |r| = 0.152 across 246 feature pairs.Source: Cross-stream alignment, assessment,… · p. 16

Discussion

PTI makes the alignment, comparison, and interpretation of two computational representations traceable. In the Nepal demonstration, the representations partly converged, while preprocessing sensitivity and the distinction between interpretive value and external-anchor association qualified their integration.

Discussion

  • Tables 6 and 7 present the theme-complete cross-stream crosswalk and the joint display of alignment and meta-inferences.Table 6 maps all six NMF themes to relevant PV domains and records unmatched, excluded, and preprocessing-sensitive themes.Source: Table 6 · p. 17, Table 7 · p. 17
  • The Nepal analysis found partial convergence between ontology-guided and data-driven representations, while some domains appeared mainly in one representation.The denial and negation theme became unstable after contraction normalization, indicating that preprocessing choices may influence observed divergence.Source: Discussion · p. 18
  • PTI distinguishes themes contributing to external-anchor association from those selected for integrated qualitative interpretation.Baby care and health (T0) and activities and preferences (T2) contributed to the combined external-anchor model but were excluded from the joint display.Source: Discussion · p. 18
  • Human interpretation remains central for deciding whether computational outputs are conceptually related and what their relationship means in context.PTI makes alignment decisions and the reasoning behind integrated conclusions traceable.Source: Discussion · pp. 16, 18

Limitations

The study’s supervised findings are exploratory, and the demonstration does not support substantive claims about perinatal mental health. The cohort, transductive NMF fitting, modest solution stability, and unvalidated ontology constrain interpretation.

Limitations

  • The small cohort of 29 mothers, including nine depressed cases, limits supervised findings to exploratory interpretation and does not support substantive perinatal mental-health claims.NMF was fit on the full corpus before cross-validation, making NMF and combined-model estimates transductive.Source: Limitations · p. 18
  • NMF solution stability was modest, and the T4 denial and negation theme was unstable after contraction normalization.These properties constrain interpretation of the theme and preprocessing-sensitive divergence.Source: Limitations · p. 18
  • The PV ontology was developed for clinical dialogue and has not been validated for translated Nepali perinatal narratives.Unmatched themes may reflect codebook coverage gaps, translation effects, labeling error, limited ontology fit, or model instability.Source: Limitations · p. 18

Conclusion

PTI offers a transparent, auditable procedure for integrating two computational representations of one qualitative corpus. The study used de-identified transcripts from a parent study, with no new participant contact.

Conclusion

  • PTI makes alignment, inclusion, and divergence decisions traceable while integrating ontology-guided coding and data-driven topic modeling.The perinatal demonstration suggests the representations may contain partially non-overlapping information useful for integration, with the balance potentially varying across corpora, codebooks, and computational methods.Source: Conclusion · p. 19

Ethical considerations

  • The analysis used de-identified participant interview transcripts, included quotations without direct identifiers, and involved no new contact with participants.Ethical approval and consent for the parent study were obtained by its original investigators.Source: Ethical considerations · p. 19
Loading 2610.19164v1…