PaperlayerView exact revision on arXiv

Source-linked AI summary

Computational KJ-Ho: An Analyst-Bias-Free Insight Extraction Framework from Large-Scale Qualitative Data Using Domain-Specialized LLMs

Kasumi Ban

arXiv:2608.16467v1cs.HCcs.CLcs.CY

TL;DR

Qualitative insight work is constrained by analysts’ cognitive capacity, and analysts can reach different conclusions from identical data. This concept paper proposes Computational KJ-Hō, a CPT- and SFT-trained, three-layer framework that preserves data-emergent KJ analysis. Preliminary studies support CPT-based domain specialization, while the framework awaits empirical validation.

  • Problem

    The paper asks whether KJ's data-emergent analysis can be computationally extended beyond an individual analyst's cognitive capacity while drawing on marketing-research knowledge.

    Source: §2.1 · p. 8

  • Method

    It proposes a three-layer framework that uses CPT to specialize an LLM for marketing research before SFT trains its analytical operations.

    Source: §3.1 · p. 14, §7 · p. 33

  • Results

    Preliminary studies support domain specialization: agreement rose from 0.091 to 0.317 with a model-generation update, but semantic precision remained below expert needs.

    Source: §4.3 · p. 21

  • Takeaways & Limitations

    The preliminary evidence indicates that general-purpose models alone do not yet reach the semantic precision required by domain experts in the studied marketing-analysis tasks.

    Source: §4.3 · p. 21

    The framework is presented as a concept paper before empirical validation, which the authors say will require considerable effort and time.

    Source: §1.5 · p. 7

Abstract

from arXiv · show

The qualitative research methodologies that underpin consumer-insight generation - the KJ method, Grounded Theory, and Thematic Analysis - share a structural constraint: the cognitive processing capacity of the human analyst. Replication research further shows that conclusions vary substantially across analysts analyzing identical data (analyst bias). This paper proposes Computational KJ-Ho (the Kawakita Jiro method), a theoretical framework that computationally realizes the KJ method's epistemology - letting structure emerge from the data itself without imposing the analyst's preconceptions - an orientation we term "analyst-bias-free." The framework employs a domain-specialized LLM built through continued pre-training (CPT) on a marketing-research corpus and supervised fine-tuning (SFT) on expert-curated insight pairs, organized as a three-layer architecture: data structuring, insight extraction, and strategy generation. Two preliminary studies in the Japanese marketing context support the necessity of CPT-based domain specialization. The paper makes five contributions: (1) a theoretical integration of the KJ method, Grounded Theory, and Peircean abduction into a single epistemological commitment of data-driven explanation generation; (2) a three-layer architecture leveraging domain-specialized embeddings for cross-interview analysis; (3) two novel evaluation metrics, InsightExtraction-F1 and MarketingQA; (4) explicit engagement with the WEIRD problem, centering a non-Western methodology; and (5) five practice-derived problem formulations from nearly three decades of marketing-research practice, translated into design requirements. The human analyst retains a supervisory role. This is a concept paper presented ahead of empirical validation.

1 Introduction

The introduction presents qualitative research as facing intertwined scalability and analyst-bias problems, proposes Computational KJ-Hō to generate explanatory structures from data, and positions the framework as awaiting empirical validation.

1.1 The Scalability Crisis in Qualitative Research

  • Traditional qualitative research faces a coupled scale-and-bias crisis: one study can require six weeks for 40 interviews, while analysts fully agreed with original papers in only 34% of 100 reanalyses.Digital sources and pressure to deliver richer insights faster have further raised the scale challenge.Source: §1.1 · p. 3

1.2 The Advent of LLMs: Possibilities and Remaining Challenges

  • Prior LLM work reports 80–88% agreement with expert coding and a 168-fold throughput gain, but existing approaches leave five domain, cross-dataset, evaluation, strategy, and cultural-validity gaps.The cited gains come from Grounded Theory coding tasks and Neo-Grounded Theory, respectively.Source: §1.2 · pp. 3–4

1.3 Proposed Approach: Computational KJ-Hō (the Kawakita Jiro Method)

  • Computational KJ-Hō makes data, rather than analysts’ conscious preconceptions, the basis for generated explanatory structure, while acknowledging that trained models retain premises.The framework aims to reduce structural social bias, including WEIRD-centric epistemology imposed on non-WEIRD markets.Source: §1.3 · pp. 4, 6
  • The three-layer architecture prepares qualitative data, uses a domain-specialized LLM for structuring, grouping, and insight generation, then translates insights into marketing tactics.The third layer can define target segments, develop message concepts, and propose quantitative-survey designs.Source: §1.3 · pp. 4, 6

1.4 Contributions

  • The framework integrates the KJ method, Grounded Theory, and abductive reasoning, and supports cross-dataset analysis through marketing-specialized embeddings and CPT followed by SFT.The paper places a non-Western qualitative methodology at its core.Source: §1.4 · p. 6
  • The paper proposes InsightExtraction-F1 and MarketingQA as complementary measures [51], treats cultural validity as a design requirement, and derives design requirements from nearly three decades of practice.The experience spans Japan, North America, Europe, China, Southeast Asia, India, and Australia.Source: §1.4 · pp. 6–7

1.5 Positioning of This Paper

  • The authors present the work as a concept paper and declaration of ongoing research, published before empirical validation while seeking broad feedback.They state that industry collaboration is indispensable to validation and will require considerable effort and time.Source: §1.5 · p. 7

2 Intellectual Foundations

The KJ method, Grounded Theory, and Peircean abduction converge on generating explanations from observed data rather than imposing prior hypotheses. Computational KJ-Hō aims to extend that commitment to large-scale qualitative analysis using domain-specialized models.

2.1 The KJ Method [37]: Bottom-Up Integration as an Epistemological Commitment

  • The KJ method turns fieldwork data into labels, groups, spatial diagrams, prose, and team consensus, with structure emerging through iterative integration.Its sequence moves from collecting and fragmenting information to grouping similar items, mapping relationships, and reviewing the diagram collectively.Source: §2.1 · p. 7

2.2 Grounded Theory [28]: Hypothesis-Generating Analysis

  • Grounded Theory develops concepts and theoretical explanations from observed patterns rather than testing prior propositions [28]; Glaser rejected imposed categories, while Strauss and Corbin used analytical frameworks.The study aligns with data-grounded theory construction and treats qualitative inquiry as hypothesis-generating rather than hypothesis-testing.Source: §2.2 · p. 9

2.3 Abduction [50]: The Logical Structure of Insight

  • Abduction generates the most plausible hypothesis to explain observed facts, and the framework’s “functional abduction” is intended to produce consumer insights absent from training data and unspecified as hypotheses.Unlike arbitrary invention or mechanical extrapolation, abduction is constrained by the current state of knowledge; KJ and Grounded Theory both use it to generate provisional explanations from observed regularities.Source: §2.3 · pp. 9–10

2.4 Integration: Data-Driven Explanation Generation as a Unifying Principle

  • The KJ method, Grounded Theory, and abduction share a commitment to generating explanatory elements from data rather than imposing them from outside.Kawakita describes “letting the data speak”, Glaser and Strauss seek “discovering theory”, and Peirce emphasizes the “most plausible explanation”.Source: §2.4 · p. 10
  • Computational KJ-Hō operationalizes data-driven hypothesis generation to extend analysis beyond human cognitive capacity using a domain-specialized LLM.The framework treats computational mechanisms as serving, not replacing, the methodological inquiry.Source: §2.4 · pp. 11–12
  • Its design principles include marketing-domain embeddings, bottom-up groupings, visualized emergent structure, and human-readable insight descriptions grounded in the data.These principles prioritize consumer-behavior semantics over general linguistic patterns and avoid predefined categories.Source: §2.4 · pp. 11–12

3 Related Work: LLMs in Qualitative Research

Prior work applies LLMs to qualitative analysis, marketing research, and human–AI collaboration, but the surveyed studies leave gaps in domain adaptation, cross-dataset analysis, evaluation, strategic output, and non-Western traditions. Computational KJ-Hō is positioned as addressing these gaps through CPT-based semantic-space reshaping and human-expert evaluation.

3.1 Full Automation and Semantic Resolution

  • NGT is the closest technical predecessor, but Computational KJ-Hō reshapes embeddings through CPT rather than using general-purpose embeddings, targeting finer marketing distinctions.NGT reported a 168-fold speedup and a 0.904 quality score versus 0.883 for manual coding; Computational KJ-Hō sets that score as a target. SFT-TA is the closest training-strategy reference: the proposed approach places CPT before SFT so domain knowledge precedes task adaptation.Source: §3.1 · pp. 13–14

3.2 Human–AI Collaborative Approaches

  • Human–AI collaboration tools support coding and interpretation, but studies retain human judgment and report limits in context, scale, or generalizability.CollabCoder augments rather than replaces human decisions; ThemeViz found greater utility than ChatGPT with no observed hallucinations, yet most participants regarded AI as a tool. MindCoder keeps interpretation human, while Prescott et al. found weaker reliability for individual AI coding decisions and recommended hybrid work.Source: §3.2 · pp. 14–15

3.3 LLMs in Marketing Research

  • Marketing-research LLM work includes synthetic respondents and survey substitution, whereas Computational KJ-Hō analyzes what real consumers said rather than generating fictitious respondents.The two approaches address different problems—data-collection cost and speed versus analysis depth and scale—and are described as complementary.Source: §3.3.1 · p. 15

3.3.1 LLMs as Synthetic (Surrogate) Respondents

  • Brand et al. frame LLMs as synthetic respondents, reporting human-comparable conjoint responses and improved alignment after fine-tuning.This research mode uses LLMs as computational substitutes for human consumers in survey-based research, unlike Computational KJ-Hō's analysis of actual consumer statements.Source: §3.3.1 · p. 15

3.3.2 LLMs as a Substitute for Quantitative Surveys

  • Findings on LLM substitution in surveys are mixed, while customer-needs extraction shows SFT can match expert accuracy with 600 examples.Reported concerns include non-reproducibility, sign reversal, limited behavior-generation accuracy, and emotion or context limits; the paper places these respondent-generation problems outside its scope and instead proposes domain-specialized CPT followed by SFT for analyzing real-human qualitative data.Source: §3.3.2 · pp. 15–16

3.5 LLM-Based Analytical Support Toolkits: Prior Work Sharing This Study's Problem Awareness

  • DeTAILS speeds qualitative analysis and guards against code hallucination, but its evaluation highlights risks to reflexivity, interpretive validity, novice scrutiny, and backtracking.The paper argues prompt-based correction alone cannot achieve analyst-bias-free analysis, contrasting DeTAILS's RAG with CPT-based semantic-space reshaping plus human-expert evaluation.Source: §3.5 · p. 16

3.6 Methodological Reflection

  • The review identifies five gaps: no CPT domain adaptation, single-dataset analysis, missing domain-specific metrics, no path to marketing decisions, and reliance on Western traditions.Computational KJ-Hō is designed to address these problems through its computational and evaluation structures; the methodological reflection also connects qualitative LLM use to technological reflexivity and human oversight.Source: §3.6 · pp. 16–17

4 The KJ-Method Computational Structure · 4.1 Overview of the Computational Structure · Layer 2: Extraction and structuring of consumer insights from that data

Computational KJ-Hō uses a three-layer analytical architecture whose outputs feed forward, while each layer has independent evaluation criteria for improving output quality.

4.1 Overview of the Computational Structure

Layer 2: Extraction and structuring of consumer insights from that data

  • The layers feed into one another sequentially, but each retains distinct design decisions and independent evaluation criteria; a domain-specialized LLM drives the architecture.Source: Layer 2: Extraction and structuring of… · p. 17
  • CPT draws on marketing-domain resources spanning consumer research, business, economics and law, academic literature, research methods, and practitioner knowledge; the six-resource composition remains subject to curation.Source: Layer 2: Extraction and structuring of… · pp. 17–18
  • The CPT corpus also includes Japanese-style company histories as benchmark data, covering both enduring and bankrupt firms to avoid survivorship bias and capture best and worst practices.These histories are intended to inform the third layer's learning.Source: Layer 2: Extraction and structuring of… · p. 18
  • SFT uses expert-curated pairs linking consumer-generated text to derived insights and the resulting marketing strategy and measures, with each sample reviewed by at least two independent curators.Source: Layer 2: Extraction and structuring of… · p. 18
  • Alt et al. [3] found that expert re-annotation relabeled over 50% of difficult TACRED examples and raised baseline average F1 from 62.1 to 70.1, supporting expert review over crowdsourcing.They also found that models may use surface-level heuristics when relations are not substituted.Source: Layer 2: Extraction and structuring of… · p. 18
  • The envisioned empirical-study model has 17 to 27 billion parameters and is designed for local deployment without external APIs, balancing complex reasoning with confidentiality and operational autonomy.The target is deployment within a single research institution's or firm's computing environment, from fine-tuning through tactical prototyping.Source: Layer 2: Extraction and structuring of… · p. 18

4.2 The Three-Layer Architecture … Step 1—Semantic Clustering (KJ method: group formation)

The three-layer architecture structures qualitative data into domain-specialized semantic representations, extracts consumer insights without preset hypotheses, and uses bottom-up clustering to reveal cross-segment patterns. Its semantic grouping is designed to reflect consumer-psychology distinctions relevant to marketing research.

Layer 1: Ingestion and Structuring of Qualitative Data

  • Layer 1 segments qualitative data into atomic utterances or coherent fragments, enriches them with respondent attributes and intent, emotion, and motivation, then embeds them in the model’s learned semantic space.These multidimensional representations link utterances to respondent attributes.Source: Layer 1: Ingestion and Structuring of… · p. 19

Layer 2: Consumer-Insight Extraction

  • Layer 2 analyzes structured data without requiring hypotheses, detecting latent themes and segment differences, structuring consumer psychology, and identifying purchase motivations and barriers for marketing-relevant insights.Source: Layer 2: Consumer-Insight Extraction · p. 19
  • The core process follows the KJ-method sequence of grouping labels, assigning headings, and spatially arranging them.Source: Layer 2: Consumer-Insight Extraction · p. 19

Step 1—Semantic Clustering (KJ method: group formation)

  • Domain-specialized embeddings cluster utterances by consumer-behavior semantics rather than general-language similarity, forming groups bottom-up and enabling cross-tabulation to reveal segment-specific patterns.The method does not set categories or code systems in advance.Source: Step 1—Semantic Clustering (KJ method:… · p. 19
  • CPT-trained embeddings distinguish conceptually close terms such as “high price” and “premium” by their opposing consumer motivations, producing groupings more directly tied to action design than general embeddings used with BERTopic.Source: Step 1—Semantic Clustering (KJ method:… · p. 19
  • Clustering methods such as hierarchical agglomerative clustering determine cluster counts from embedding structure using an SFT-tuned stopping criterion, and analyze the corpus simultaneously across interviews, segments, or time points.This allows a cluster to combine material across sources and reveal patterns that single-corpus analysis may miss.Source: Step 1—Semantic Clustering (KJ method:… · p. 19
  • CPT embeddings encode marketing-research distinctions such as price sensitivity versus perceived value, approximating expert judgments of which data belong together and grounding the second layer’s clustering.This semantic resolution is intended to maintain phenomenological replication of expert grouping judgments.Source: Step 1—Semantic Clustering (KJ method:… · pp. 19–20

Step 2—Assigning Meaning to Groups (creating headings) · Step 3—Structuring the Relationships Among Groups (spatial arrangement and structuring)

Step 2 turns each cluster into an evidence-linked marketing insight, while Step 3 maps relationships among the resulting groups into an explicit structure for strategy planning.

Step 2—Assigning Meaning to Groups (creating headings)

  • For each cluster, the SFT-trained model creates a marketing-significant heading integrating its data and linking to supporting evidence for human review.Training on expert-curated pairs helps the model internalize expressions professional marketing researchers recognize as analytically valuable.Source: Step 2—Assigning Meaning to Groups… · p. 20

Step 3—Structuring the Relationships Among Groups (spatial arrangement and structuring)

  • Step 3 makes causal, contradictory, hierarchical, and complementary relationships among groups explicit in a structured map for strategy construction and planning.The map can show, for example, a purchase motivation opposed to a barrier or a latent need above a behavioral pattern.Source: Step 3—Structuring the Relationships… · p. 20

Layer 3: Strategy Generation

Layer 3 turns structured consumer insights into marketing strategy and implementable outputs, extending the framework from academic methodology to practical business application.

Layer 3: Strategy Generation

  • Outputs include an insight-structure statement, target segment, business opportunities, brand messages and creative ideas, an evidence-cited analytical report, and proposals for further research.The report includes an executive summary and detailed analysis; further research addresses issues unresolved by the analysis.Source: Layer 3: Strategy Generation · p. 20
  • This layer does not directly correspond to the KJ method’s core process; it extends the framework to practical business application.The described core process is collecting, organizing or structuring, and consensus building or collective decision-making.Source: Layer 3: Strategy Generation · p. 20

4.3 The Role of Domain Specialization: Why General-Purpose LLMs Are Insufficient

Preliminary studies show that general-purpose LLMs can structure consumer data but miss distinctions important to marketing researchers. The section argues that CPT-based domain specialization is needed to internalize marketing knowledge, improve analytical quality, and keep confidential data under organizational control.

4.3 The Role of Domain Specialization: Why General-Purpose LLMs Are Insufficient

  • Agreement on 779 sake reviews was 0.091 for GPT-3.5-turbo and 0.317 for GPT-4o-mini, despite processing in about 10 minutes versus 19 hours manually [33].GPT-4o-mini reproduced the human analysis’s three clusters and top-level classification; explicit sentiment reached 0.783, while fine-grained distinctions such as drinking temperature and price range fell short of human performance.Source: §4.3 · p. 21
  • Analysis of 958 AI-chat interview records extracted consumption contexts as quantitative linkages of situation, motivation, and behavior using 7W2H labels.A top combination was mood change/reward × emotional satisfaction × relaxation consumption; the section treats 7W2H as a domain-universal descriptive lens, not a project-specific prior hypothesis.Source: §4.3 · p. 21
  • CPT reshapes semantic representations to preserve distinctions marketing analysis needs, while SFT teaches the model to produce evidence-based, actionable, and ideally non-obvious insights [36].Examples include preference intent versus behavior, price evaluation versus value perception, and satisfaction versus brand loyalty; researchers supply objectives and constraints, while the model supplies the analytical framework.Source: §4.3 · p. 21
  • A self-hosted 17-to-27-billion-parameter model can cost roughly one-tenth as much as a comparable frontier API while keeping confidential research data in a controlled environment.The cost rationale rests on usage-based API billing versus low marginal inference costs after GPU investment, with the advantage growing at scale; quantitative comparison is deferred to a later empirical paper.Source: §4.3 · p. 22
  • Few-shot prompting, RAG, and prompt engineering cannot fully substitute for CPT: they lack adapted representations or risk analyst hypotheses, whereas CPT+SFT internalizes domain knowledge [64] [11].The section positions those alternatives as external augmentation or prompt crafting, and CPT+SFT as white-box model fine-tuning; domain-knowledge internalization is presented as a precondition for analytical quality.Source: §4.3 · p. 22

4.4 The Operational Definition of “Analyst-Bias-Free”

“Analyst-bias-free” means structurally minimizing the analyst’s prior hypotheses in early analysis, not eliminating all bias or claiming statistical objectivity. The framework uses embeddings to let patterns emerge bottom-up while retaining human expertise for final interpretation.

4.4 The Operational Definition of “Analyst-Bias-Free”

  • The operational definition is a design that minimizes analysts’ prior-hypothesis intervention early, using embedding-space proximity to let qualitative patterns emerge bottom-up.It does not claim complete freedom from bias; it computationally enacts the KJ method’s instruction to “let the data speak.”Source: §4.4 · p. 22
  • The CPT-refined model acts as a “Computational Mirror” of collective domain knowledge, embodying “Technological Reflexivity” and helping mitigate premature thematic foreclosure.The approach does not claim quantitative or statistical objectivity.Source: §4.4 · pp. 22–23
  • The LLM proposes qualitative analytical results as a rigorous starting point, not a replacement for human expertise, supporting researchers’ accountability in final interpretation [21].The framework connects specialized-LLM rigor with holistic KJ integration to provide a transparent, reproducible path from raw data to strategic consumer insights.Source: §4.4 · p. 23
  • The framework targets skilled analysts’ fixation on industry common sense, precedent, and project-specific “shoulds,” which can cause contradictory consumer data to be excluded.It does not reject hypotheses altogether, but calls for a hypothesis-free posture when first facing raw consumer voices.Source: §4.4 · p. 23
  • The framework distinguishes accumulated domain knowledge from project-specific analyst bias: CPT and SFT internalize the former, while reviewed training data exclude the latter.Multiple experts mutually review SFT data to dilute individual idiosyncrasies; the distinction is posed for measurement through three falsification paths in Section 6.5.Source: §4.4 · p. 23

4.5 The Interdisciplinary Positioning of “Analyst-Bias-Free”

Computational KJ-Hō is framed as hypothesis-generating abduction from observed data, not analysis constrained by prior hypotheses. Human analysts set the objective and evaluate and refine outputs, while the framework brings together knowledge from multiple disciplines.

4.5 The Interdisciplinary Positioning of “Analyst-Bias-Free”

  • “Analyst-bias-free” means hypothesis-generating: insights, relationships, and strategic implications emerge from observed patterns, while humans set the objective and evaluate and refine outputs for implementation.The passage describes these human decisions as necessary for the system’s data structuring to have value.Source: §4.5 · p. 23
  • Following Inoue, abduction explains why “hypothesis-free” misnames the KJ process: its design structurally excludes the analyst’s established theories, industry standards, and validation criteria.The KJ method’s abductive question is what label best explains a group of cards.Source: §4.5 · pp. 23–24
  • Computational KJ-Hō fuses marketing-research practice, the KJ method, abduction, Grounded Theory, and large language models as an interdisciplinary effort to establish standards for judging knowledge validity.The study crosses disciplines and journal communities while conducting analysis unbound by any particular validation criteria.Source: §4.5 · p. 24

5 The Evaluation Structure

The section proposes evaluating both intermediate analytical structure and final insight quality, using new metrics alongside expert judgment. It defines how those measures are calibrated and interpreted, and outlines the MarketingQA benchmark and model comparisons.

5.1 Why Existing Metrics Are Insufficient

  • BLEU/ROUGE miss semantic, contextual, and downstream insight value, while Pi et al. [51]’s LOGOS evaluates codebook structure; the paper therefore recommends evaluating both intermediate and final outputs.LOGOS assesses codebook reusability, descriptive fitness and coverage, parsimoniousness, and consistency, but not the business value or psychological distinctions of consumer insights.Source: §5.1 · p. 25

5.2 InsightExtraction-F1

  • InsightExtraction-F1 measures precision, recall, and their harmonic mean against consensus insights produced by skilled researchers analyzing the same interview corpus.Researchers individually analyze the corpus using conventional qualitative methods, then integrate their insights through structured discussion to establish the gold standard.Source: §5.2 · pp. 25–26
  • Practitioners adjudicate unmatched insights: valid human-missed discoveries are excluded from the precision denominator and assessed for novelty, separating human-level agreement from beyond-human discovery.Unsupported outputs are not treated the same as valid discoveries missed by human analysts.Source: §5.2 · p. 26
  • InsightExtraction-F1 matching uses independent frozen embeddings and a practitioner-calibrated cosine-similarity threshold, avoiding circular self-grading; sensitivity analysis found stability across 0.6–0.8.The starting threshold is 0.70, while 0.65 is the minimum feasibility target and 0.70 the primary research target; these are normative goals, not established benchmarks.Source: §5.2 · p. 26

5.3 The MarketingQA Benchmark

  • MarketingQA is a Japanese multiple-choice benchmark spanning four marketing-research knowledge domains, with a normative CPT-model accuracy target of at least 0.75 and the pre-CPT model as baseline.The proposed benchmark should be released openly to support reproducibility and measurement of domain-specialization gains.Source: §5.3 · pp. 26–27
  • The evaluation compares general-purpose, CPT-only, and CPT+SFT models on MarketingQA and InsightExtraction-F1 to separate domain-knowledge gains from task-specific analytical-operation gains.Concrete baseline and post-CPT performance values are deferred to the empirical-phase paper.Source: §5.3 · p. 27

5.4 The Human-Expert Evaluation Protocol and Metrics

  • Blind expert comparison rates usefulness and actionability on five-point scales and assesses whether insights are novel discoveries absent from human analysts’ output.The evaluators are skilled marketing researchers unfamiliar with the input data and unaware whether each insight came from the system or human analysis.Source: §5.4 · p. 27
  • Novelty follows Chen et al.’s [14] metric without a fixed target and serves as a diagnostic; high novelty may indicate either valuable discovery or unsupported business potential, requiring human judgment.The extent to which domain-specialized AI finds insights missed by humans remains an empirical-phase question; novelty should be interpreted with other metrics and qualitative inquiry.Source: §5.4 · pp. 27–28

6 Discussion

The discussion frames Computational KJ-Hō as a data-grounded but human-supervised approach whose cultural scope, collaborative design, and central analyst-bias-free claim require further scrutiny and validation.

6.1 Epistemological Implications: Is the LLM's Processing “Functional Abduction”?

  • The authors describe “functional abduction” as structural equivalence to abductive reasoning, not proof of Peircean inference, and require human oversight of consumer-psychology claims.They argue that novelty alone is insufficient: outputs must also be data-grounded and explanatory, and cannot independently guarantee their claims.Source: §6.1 · p. 29

6.2 De-Westernizing AI Epistemology (the WEIRD Problem)

  • Computational KJ-Hō centers a non-Western method and Japanese-language domain specialization to ground AI-generated meaning in the data’s cultural, linguistic, and professional context.The framework’s effectiveness in other countries remains to be validated; the authors plan validation in another non-Western culture.Source: §6.2 · p. 29

6.3.1 Redistributing Human–AI Collaboration: The “Human-on-the-Loop” Paradigm

  • The analyst-bias-free design removes humans from coding, not judgment: people define objectives and constraints, evaluate outputs, and make strategic decisions.The system autonomously generates an insight map; analysts can accept, reject, refine, or request re-analysis, while business action still requires human contextual judgment.Source: §6.3.1 · pp. 29–30

6.3.2 Reliability Requirements

  • An evidence-link architecture connects each insight element to its source utterances, enabling audit and evidence-based strategic judgment rather than blind approval of AI output.The paper treats process transparency as necessary for meaningful supervisory control and for downstream stakeholders to assess the insights.Source: §6.3.2 · p. 30

6.3.3 Implications for CSCW Research

  • The framework redesigns collaborative marketing research, but CSCW questions remain around automation bias, intermediate human input, and trust calibration across expertise levels.The proposed division assigns pattern recognition to the model and evaluation and decisions to humans; whether that split is optimal, and how domain and computing expertise should coordinate, merit investigation.Source: §6.3.3 · pp. 30–31

6.4 A Rapidly Maturing Research Field: Timing and Strategic Implications

  • As qualitative-analysis systems proliferate, the paper argues that its integration of CPT-based specialization with a pipeline from analysis to actionable strategy makes publication timely.The authors also position the contribution as bringing long-practice researcher experience and a non-Western methodology to a scalability problem they consider worth solving at scale.Source: §6.4 · p. 31

6.5 Limitations and Future Work

  • The current text-focused framework needs multimodal CPT, better weak-signal detection, and validated measures of insight quality, strategic usefulness, and interpretive depth.InsightExtraction-F1 and MarketingQA thresholds are theoretically derived starting points; empirical work must connect their values to practical usefulness.Source: §6.5 · pp. 31–32
  • CPT+SFT may not transfer beyond marketing or be reproducible without substantial domain data, while a Japan-centered approach is only a first step toward cultural pluralism.The central analyst-bias-free claim remains falsifiable through embedding comparisons, blind comparisons with human and general-purpose LLM teams, and agreement across independently trained CPT models.Source: §6.5 · p. 32

7 Conclusion

Computational KJ-Hō proposes using a domain-specialized LLM to realize the KJ method’s data-emergent epistemology at scale. The framework aims to overcome qualitative analysis’s cognitive constraint and, if validated, widen access to consumer understanding.

7 Conclusion

  • Analysts can hold only about 200 data points in productive analytical tension, after which cognitive overload threatens context sensitivity, contradiction, and openness to the unexpected.The paper frames this as a structural constraint on qualitative research’s capacity to process large volumes of data.Source: §7 · p. 33
  • Computational KJ-Hō proposes realizing the KJ method’s data-emergent, analyst-bias-free commitment at scale through a domain-specialized LLM.The framework treats this as a methodological extension that preserves qualitative research’s epistemological commitment, not merely an efficiency gain.Source: §7 · p. 33
  • Its five contributions integrate KJ, Grounded Theory, and abduction; propose a CPT-based architecture and insight metrics; center non-Western methodology; and translate practice problems into design requirements.The five practice-grounded problem formulations draw on nearly three decades of global practical experience.Source: §7 · p. 33
  • If validated, the framework could extend beyond marketing research and democratize the analytical infrastructure needed to derive consumer understanding, including for organizations without large research teams.The proposed benefit is broader access to analysis, not to data, which the passage describes as already abundant.Source: §7 · p. 33

Related papers

Loading 2608.16467v1…