PaperlayerView exact revision on arXiv

Source-linked AI summary

Thinking Outside the Bots: Human Ideas in AI's Semantic Blind Spots Are Not Inferior

Kibum Moon, Dan R. Johnson, Kostadin Kushlev, Benjamin Lira, Indre Viskontas, James C. Kaufman, Roger E. Beaty, Angela L. Duckworth, Adam E. Green

arXiv:2610.17324v1cs.CLcs.AIcs.CY

TL;DR

Prior research documented reduced diversity in LLM outputs, but whether ideas beyond that narrower range represent a meaningful loss remained unclear. The paper uses EDGE to identify human ideas weakly represented by task-matched AI outputs across three open-ended tasks. Across all three, human responses reached AI-sparse semantic regions, and ideas in those regions were as good as or better than ideas more similar to AI on task-relevant outcomes.

  • Problem

    Whether ideas beyond the narrower range of LLM outputs represent valuable thinking or merely less effective ideas remains unresolved.

    Source: Human Ideas in AI’s Semantic Blind… · pp. 3–4

  • Method

    EDGE maps AI semantic blind spots by assessing human ideas against task-matched AI comparison sets.

    Source: Analytic Strategy · p. 14

  • Results

    Across three tasks, human responses reached semantic regions sparsely covered by matched AI outputs, and ideas in those blind spots were as good as or better on task-relevant outcomes.

    Source: Discussion · p. 24

  • Takeaways & Limitations

    The findings suggest that human idea diversity may be advantageous across diverse forms of idea generation.

    Source: Discussion · p. 24

    The interpretation assumes that weak AI representation does not merely reflect idiosyncratic variation or poor fit to task demands.

    Source: Human Ideas in AI’s Semantic Blind… · p. 5

Abstract

from arXiv · show

AI-generated outputs are frequently perceived as highly creative, but also tend to be more similar to each other (homogenized) than human ideas. Does the human heterogeneity that extends beyond AI reflect valuable thinking that AI misses? Alternatively, is greater human heterogeneity driven by lower-quality ideas and less effective thinking that AI avoids? We developed Expansion of Divergent Generative Expression (EDGE) to map AI's semantic "blind spots" by assessing how strongly/weakly individual human ideas were represented in a task-matched AI semantic comparison set. Specifically, we compared human-generated vs. AI-generated ideas across three open-ended tasks: college admissions essays, alternative uses for everyday objects, and interpersonal advice. Across all three tasks, human responses extended into semantic regions sparsely covered by AI outputs. These distinctively human responses showed no meaningful deficits in creativity, quality, or effectiveness, were associated with equal-or-better academic performance, and with credibly superior ratings and achievement for some outcomes. These ideas were thus not merely poor-performing variation; they performed as well as or better than ideas that overlapped more strongly with AI ideas. More distinctive human ideas tended to receive higher creativity ratings, but this relationship was weak or reversed for AI ideas. Taken together, these results suggest that differentiation among human-generated ideas does not reflect poor thinking that AI can safely omit, and instead may afford both individual and population-level advantages.

Human Ideas in AI’s Semantic Blind Spots Are Not Inferior · Method · Study Overview

The paper asks whether human ideas weakly represented by matched LLM outputs are valuable or merely lower-quality variation, and introduces EDGE to identify such ideas. It examines these questions across three open-ended tasks using coverage and outcome noninferiority analyses.

Human Ideas in AI’s Semantic Blind Spots Are Not Inferior

  • Prior diversity comparisons did not establish whether human responses weakly represented in LLM outputs retain value, motivating independent evaluation of their task-relevant outcomes.The paper calls the semantic regions occupied by these responses AI semantic blind spots.Source: Human Ideas in AI’s Semantic Blind… · p. 4
  • EDGE identifies responses in less well-represented parts of semantic space using isolation from dense response groups and weak alignment with any single group.Higher EDGE scores indicate that similar responses are relatively uncommon in the comparison set.Source: Human Ideas in AI’s Semantic Blind… · p. 5
  • The study tests whether human ideas weakly represented by matched LLM outputs perform worse, or as well as or better than ideas more similar to AI.The authors predict that ideas in AI semantic blind spots will not be inferior on task-relevant outcomes.Source: Human Ideas in AI’s Semantic Blind… · p. 5

Study Overview

  • Matched human and LLM responses were analyzed across college admissions essays, alternative uses for everyday objects, and interpersonal advice.The tasks span consequential writing, brief idea generation, and everyday problem solving.Source: Study Overview · p. 5
  • Coverage Analysis compared mean EDGE scores for human and LLM responses in repeated, task-matched samples.This analysis tested whether matched LLM responses left task-specific semantic blind spots occupied by heterogeneous human responses.Source: Study Overview · pp. 5–6
  • Outcome Noninferiority Analysis examined whether task-relevant outcomes declined as human responses became less well represented by matched LLM outputs, and whether any decline was substantively meaningful.The analysis assessed the cost of occupying task-specific semantic blind spots.Source: Study Overview · pp. 5–6

Datasets and Response Sources … Outcome Measures

The study compared human responses with task-matched AI response pools across admissions essays, alternative uses, and creative problem solving. It assessed essays against academic outcomes and evaluated response creativity, originality, and solution quality using human ratings or task-specific automated measures.

Datasets and Response Sources

  • The essay sample comprised 1,600 essays from matriculated students at three U.S. institutions, restricted to application cohorts from 2022 or earlier.Essays were randomly sampled from eight institution–prompt strata, with 200 essays per stratum.Source: Datasets and Response Sources · pp. 6–7
  • Five essay prompts yielded a matched AI pool of 1,000 GPT-5-mini responses, with each essay treated as one response-level unit.The eight institution–prompt strata represented five distinct prompts.Source: Datasets and Response Sources · p. 7

Alternative Uses Task

  • The Alternative Uses Task sample included 481 human participant–object bundles and 481 matched AI bundles for brick and knife prompts.Each AI bundle contained the same number of individual uses as its corresponding human bundle; bundles, not individual uses, were the analysis units.Source: Alternative Uses Task · p. 7

Creative Problem-Solving Task

  • The Creative Problem-Solving Task compared 1,018 undergraduate responses to Becky’s Problem with 1,018 GPT-5-mini responses to the same prompt.The AI generation instructions targeted approximately 54 words in about three sentences, approximating the human mean of 53.77 words and 2.58 sentences.Source: Creative Problem-Solving Task · p. 8

Outcome Measures

  • Essay authors’ external outcomes were SAT-equivalent scores and cumulative college GPA, obtained from university records.When both SAT and ACT scores were available, the higher SAT-scale score was retained.Source: Outcome Measures · p. 8
  • Human-rated outcomes covered AUT originality and CPS originality and quality, using 5-point rating scales.AUT originality was averaged across uses within each participant–object bundle; CPS ratings came from separate panels of two to four trained raters.Source: Outcome Measures · p. 9
  • Task-specific automated assessments scored essays for creativity, AUT uses for originality, and CPS solutions for originality and effectiveness.The admissions evaluator correlated .82 with mean expert ratings in a held-out set of 111 essays; CAP provided the AUT CLAUS and CPS-AI models.Source: Outcome Measures · p. 9

Ethical Compliance · EDGE Measure · Analytic Strategy

EDGE measures how weakly responses are represented in a task-matched comparison set, and the analyses used distinct comparison sets to study human–AI semantic coverage and outcomes.

Ethical Compliance

  • Admissions essays and associated academic records had IRB approval; AUT and CPS analyses used existing datasets reported by Patterson et al. (2023) and Luchini, Maliakkal, et al. (2025).Source: Ethical Compliance · p. 10

EDGE Measure

  • EDGE scores rise with greater distance from dense semantic concentrations, ambiguity about cluster membership, or both; scores depend on the comparison set and do not directly measure creativity, quality, or effectiveness.Source: EDGE Measure · p. 10
  • Responses were represented using three open-source sentence-embedding models to reduce dependence on any single semantic-similarity representation.Source: EDGE Measure · p. 10
  • Essays and CPS responses were segmented into sentences, AUT uses were embedded separately, and component scores were averaged across sentences or uses within each analytic unit.Source: EDGE Measure · pp. 10–11
  • EDGE combines GLOSH outlyingness with fuzzy c-means cluster ambiguity, calculated as one minus maximum membership; six component scores are standardized and averaged, with higher final scores indicating weaker representation.Source: EDGE Measure · p. 11

Analytic Strategy

  • The two primary analyses used distinct comparison sets: coverage compared human and AI responses in shared semantic space, while noninferiority scored each human response against itself and task-matched AI outputs; a third analysis clarified what EDGE captures.Source: Analytic Strategy · p. 14

Coverage Analysis · Bayesian Outcome Noninferiority Analysis

The analyses tested whether human ideas occupied semantic regions sparsely represented by AI and whether ideas less represented by AI showed meaningfully poorer outcomes.

Coverage Analysis

  • Each coverage draw sampled 120 human and 120 matched AI response units, then recomputed clustering and EDGE scores for that sampled set.Source: Coverage Analysis · p. 14
  • Coverage contrasts subtracted mean AI EDGE from mean human EDGE; 1,000 draws were generated per essay stratum, AUT object, and CPS scenario, with essay and AUT contrasts equally averaged.Source: Coverage Analysis · pp. 14–15
  • For each task, the median and shortest interval containing 95% of draw-level contrasts were reported; an interval wholly above zero supported higher human mean EDGE.Source: Coverage Analysis · p. 15

Bayesian Outcome Noninferiority Analysis

EDGE and Assessments of Originality and Creativity · Results · Outcome Noninferiority Analysis

Across the three tasks, human responses occupied semantically sparser regions than matched AI responses, yet higher AI-reference EDGE was not linked to meaningfully poorer outcomes. EDGE captures distributional semantic position rather than originality itself, and its relationship with creativity or originality differed between human and AI responses.

EDGE and Assessments of Originality and Creativity

  • EDGE measures how weakly a response is represented in a specified comparison distribution, not how original or creative it is judged to be; their relationship is empirical, not definitional.A response can be judged original despite being common among AI outputs, or occupy a sparsely covered AI-semantic region without being judged especially original.Source: EDGE and Assessments of Originality… · p. 17

Results

  • Human mean EDGE exceeded matched AI mean EDGE across admissions essays (0.991), AUT (1.295), and CPS (1.141), with positive median contrasts across all eight essay strata and both AUT objects.The 95% resampling intervals were [0.931, 1.048] for essays, [1.126, 1.426] for AUT, and [.993, 1.278] for CPS.Source: Results · pp. 18–19, Figure 3 · p. 19

Outcome Noninferiority Analysis

  • Across all nine outcomes, every 95% HDI lower bound was above the −.10 noninferiority margin; four associations were credibly positive beyond noninferiority, while five were compatible with no association but not meaningful harm.Source: Outcome Noninferiority Analysis · p. 20
  • For admissions essays, AI-reference EDGE was positively associated with essay creativity (median β = .35, 95% HDI [.31, .40]) and SAT (median β = .11, 95% HDI [.04, .18]); GPA was near zero but noninferior.The GPA association was median β = .03, 95% HDI [−.02, .08], with P(β > −.10) = 1.000.Source: Outcome Noninferiority Analysis · p. 20
  • For AUT, EDGE associations with human-rated originality (median β = .03, 95% HDI [−.06, .12]) and CLAUS originality (median β = .01, 95% HDI [−.09, .10]) included zero but remained noninferior.Their respective P(β > −.10) values were .997 and .987.Source: Outcome Noninferiority Analysis · p. 20
  • For CPS, EDGE was positively associated with human-rated originality (median β = .20, 95% HDI [.13, .26]) and CPS-AI originality (median β = .17, 95% HDI [.11, .23]); quality and effectiveness were near zero but noninferior.Quality was median β = .02, 95% HDI [−.05, .08], and effectiveness was median β = .04, 95% HDI [−.02, .11].Source: Outcome Noninferiority Analysis · pp. 20–21
  • Higher EDGE was associated with higher human originality or creativity across all three tasks, and human slopes and human–AI differences exceeded the .10 small-effect benchmark.The human slopes were .67 for essay creativity, .45 for AUT CLAUS originality, and .54 for CPS-AI originality; the corresponding human–AI differences were Δβ = .62, .56, and .43.Source: Outcome Noninferiority Analysis · p. 22, Figure 5 · p. 23
  • The corresponding AI associations were small and positive for essay creativity, negative for AUT CLAUS originality, and positive but substantially smaller than the human slope for CPS-AI originality.AI median β values were .05 for essays, −.11 for AUT, and .11 for CPS.Source: Outcome Noninferiority Analysis · p. 22

Discussion

Across three idea-generation tasks, human ideas reached semantic regions sparsely covered by matched AI outputs and were not inferior on task-relevant outcomes. The findings suggest that this human heterogeneity may benefit both individuals and the collective pool of ideas.

Discussion

  • Across three tasks, human ideas occupied AI-sparse semantic regions and were as good as or better than more AI-like human ideas on task-relevant outcomes.This pattern held across essay sampling strata, both Alternative Uses Task objects, and the Creative Problem-Solving scenario.Source: Discussion · p. 24
  • In essays, more AI-distinct ideas received higher creativity ratings; in CPS, they received higher human- and AI-rated originality, while AUT originality and CPS quality and effectiveness met noninferiority criteria.Those latter associations were not credibly positive.Source: Discussion · p. 24
  • Applicants whose essays extended further into AI semantic blind spots had higher SAT scores; their college GPA association was positive and noninferior, but not credibly positive.The authors interpret these associations as evidence that less AI-represented ideas may reflect academic strengths of their producers.Source: Discussion · pp. 24–25
  • Because the underrepresented human ideas met task-relevant quality conditions, reduced collective idea diversity could prune candidate solutions without a compensating quality gain.The authors argue that such diversity may benefit innovation by expanding the pool from which ideas are selected.Source: Discussion · pp. 25–26
  • Human originality and creativity assessments aligned with greater semantic sparsity across tasks, whereas AI associations were small for essays and CPS and reversed for AUT.The authors caution that high creativity ratings alone do not show that AI outputs broaden the collective idea space; EDGE also captures idea density and collective structure.Source: Discussion · pp. 26–27
  • The study compared human-only ideas with AI-generated ideas, while proposing EDGE for future assessments of human–AI hybrid creativity and interventions to expand AI semantic reach.The authors note that parameter-based increases in AI heterogeneity can reduce coherence, and suggest EDGE can test whether interventions expand semantic reach without sacrificing value.Source: Discussion · p. 27

Supplemental Materials for · Human Ideas in AI’s Semantic Blind Spots Are Not Inferior

The supplied material identifies “Thinking Outside the Bots:” and a supplemental-references figure for “Human Ideas in AI’s Semantic Blind Spots Are Not Inferior.”

Supplemental Materials for

Human Ideas in AI’s Semantic Blind Spots Are Not Inferior

  • Figure S1 is titled “Supplemental References.”Source: Figure S1

Supplemental Method

The supplemental method describes how human responses and GPT-5-mini outputs were assembled and matched across admissions essays, alternative uses, and interpersonal problem solving.

Admissions essays

  • The admissions-essay sample contained 1,600 essays balanced across eight institution–prompt strata representing five prompts, with shared-prompt AI pools reused across institution comparisons.The AI pool comprised 1,000 essays, 200 per prompt, with target lengths based on each prompt’s median human essay length.Source: Supplemental Method · p. 38

Alternative Uses Task

  • For the Alternative Uses Task, 481 human participant–object bundles containing 3,425 uses were matched with 481 GPT-5-mini bundles of the same sizes (Patterson et al., 2023).Uses remained grouped within bundles; AI generation requested the exact number of distinct uses in each matched bundle.Source: Supplemental Method · p. 40

Creative Problem-Solving Task

  • For Becky’s Problem, 1,018 human responses were matched with 1,018 GPT-5-mini responses, each treated as one complete response-level unit (Luchini et al., 2025).AI instructions targeted about 54 words in three sentences, approximating the human mean length of 53.77 words and 2.58 sentences.Source: Supplemental Method · p. 41

External Person-Level Outcomes

External outcomes for admissions essay authors were assessed using SAT-scale scores and cumulative college GPA from university records. SAT analyses covered a subset of authors, while GPA was available for all authors.

External Person-Level Outcomes

  • SAT-scale scores were derived using the official 2018 ACT/SAT concordance, retaining the higher SAT-scale score when both SAT and ACT scores were available.Scores came from university records.Source: External Person-Level Outcomes · p. 45
  • Cumulative GPA was available for all 1,600 essay authors.GPA was obtained from university records.Source: External Person-Level Outcomes · p. 45
  • The SAT analysis included 794 authors from the elite private university and selective liberal-arts college, excluding the regional private university because SAT availability was sparse.Source: External Person-Level Outcomes · p. 45

Human-Rated Response Outcomes … Aggregation and Standardization

The methods define how human-rated and automatic response outcomes were scored, how semantic representations and clusters were built, and how EDGE components were aggregated and standardized. These procedures varied by task and analysis set.

Human-Rated Response Outcomes

  • Human-rated outcomes covered all 481 AUT bundles for originality and 1,017 of 1,018 CPS responses for originality and quality; full response pools remained available for Coverage Analysis.AUT originality averaged use-level scores at the participant–object level, while CPS ratings assessed complete solutions (Patterson et al., 2023; Luchini et al., 2025).Source: Human-Rated Response Outcomes · p. 45

Automatic Assessments of Creativity, Originality, and Effectiveness

  • The same task-specific automatic assessments were applied to human and matched AI responses: a fine-tuned GPT-4.1-mini evaluator for essays and Penn State’s CAP tools for AUT and CPS.CLAUS scored AUT-use originality, averaged within participant–object bundles; CPS-AI assessed complete-solution originality and effectiveness (Patterson et al., 2025; Luchini et al., 2025). Essay creativity judgments were supplied by 22 researchers (Moon et al., 2026).Source: Automatic Assessments of Creativity,… · pp. 45–46
  • CPS-AI used raw predictions: 48 distinct human responses had out-of-range originality or effectiveness predictions, while no AI predictions fell outside the nominal −2 to 2 range.The evaluator scored 2,036 complete CPS responses; 22 human originality and 28 human effectiveness predictions exceeded the upper bound, with two responses exceeding it on both outcomes.Source: Automatic Assessments of Creativity,… · p. 46

Text Representation

  • Admissions essays and CPS responses were sentence-segmented before embedding; each model used its corresponding tokenizer, batch padding, and truncation at 512 tokens, and final-hidden-layer first-token vectors entered clustering directly without additional length normalization or dimensionality reduction.The models were avsolatorio/NoInstruct-small-Embedding-v0 (Solatorio, 2024b), sentence-transformers/all-mpnet-base-v2 (Reimers & Gurevych, 2019; Song et al., 2020), and avsolatorio/GIST-Embedding-v0 (Solatorio, 2024a).Source: Text Representation · pp. 46–47

Clustering Parameter Selection

  • Clustering parameters were selected separately by embedding model and scoring corpus using outcome-blind criteria: HDBSCAN minimized mean GLOSH, while fuzzy c-means maximized membership-ambiguity range.HDBSCAN candidates used minPts values {3, 5, 8, 11, 14, 17}; fuzzy c-means tested 12 combinations of k = {3, 4, 5, 6, 7, 8} and m = {1.05, 1.10}, with ties resolved in candidate order.Source: Clustering Parameter Selection · p. 47

Aggregation and Standardization

  • EDGE averaged six standardized components—two computed in each of three embedding spaces—across response sentences or AUT uses, then averaged those components into a unit-level score.Each sentence or use contributed equally; AUT bundles contained 1–23 uses (M = 7.1).Source: Aggregation and Standardization · p. 48
  • EDGE components were standardized within each scoring set, then scores were standardized again within task-specific sampling strata, separately for each panel.Coverage Analysis standardization used all 240 human and AI units within a matched response group and draw; AI-reference scoring used human target units sharing a prompt, object, or problem and reference panel.Source: Aggregation and Standardization · p. 48

AI-Reference EDGE … Supplemental Results

Across supplemental analyses, human responses extended beyond AI reference panels across all matched groups, without falling below outcome noninferiority thresholds; human EDGE was also more strongly associated with originality or creativity assessments than AI EDGE. The section details how AI-reference EDGE and its outcome models were constructed and visualized.

AI-Reference EDGE

  • Each human response was scored in a separate clustering calculation alongside a matched AI reference panel, excluding other human responses from that calculation.Reference panels could include some of the same AI responses.Source: AI-Reference EDGE · pp. 48–49

Model Specification

  • Outcome models treated observed mean EDGE as a noisy measure of latent EDGE, standardized outcomes within matched response groups, and did not impute missing outcomes.Measurement-error standard deviations came from each response’s mean EDGE across reference panels; residual correlations between latent EDGE and outcome submodels were fixed at zero.Source: Model Specification · p. 49
  • The interaction coefficient represented the human-minus-AI slope difference; task-level human slopes, AI slopes, and differences were calculated within each draw, then averaged across eight admissions essay strata or two AUT objects.Source: Model Specification · p. 50

Visual Explanation of the Two EDGE Components

  • PaCMAP displays illustrate cluster-membership ambiguity and GLOSH outlyingness, but the reported analyses calculated EDGE components in original high-dimensional embedding spaces.Display coordinates were rescaled or transformed to clarify spatial patterns; outlyingness was shown as percentile ranks across pooled human and AI points within each task.Source: Visual Explanation of the Two EDGE… · p. 50

Supplemental Results

  • All 11 matched response groups had positive median human-minus-AI EDGE contrasts with 95% intervals above zero; ten were positive in 1,000 of 1,000 draws and one in 998.The exception was the regional private university’s challenge-or-failure prompt.Source: Table S2 · p. 51, Supplemental Results · pp. 51–52
  • The institution-equal admissions contrast was 1.165 (95% resampling interval [1.106, 1.214]), positive in 1,000 of 1,000 draws.This sensitivity analysis equally weighted the three institutions after averaging prompts within each institution.Source: Supplemental Results · p. 52
  • Every 95% HDI had a lower bound above the −0.10 noninferiority margin.Noninferiority was defined as a lower bound exceeding −0.10.Source: Table S4 · p. 54, Supplemental Results · p. 54
  • Each task-level human–AI difference in EDGE associations with originality or creativity assessments was positive in 1,000 of 1,000 draws.The paired differences compare human and AI EDGE coefficients for the assessments.Source: Table S5 · p. 55, Supplemental Results · p. 55

Stability Across AI-Reference Panels

Averaging across 10 AI-reference panels yielded stable relative EDGE scores under the current reference-sampling procedure.

Stability Across AI-Reference Panels

  • Across admissions strata, single-panel ICCs were .945–.987 and 10-panel ICCs .994–.999; AUT single-/10-panel ICCs were .797/.975 (brick), .793/.975 (knife), and CPS ICCs were .909/.990.ICC(C,1) represents a single panel; ICC(C,10) represents the mean of 10 panels.Source: Stability Across AI-Reference Panels · pp. 56–57, Table S6

Descriptive Associations · Supplemental References

The section presents descriptive associations between EDGE and four automatic assessments, with separate human and AI response patterns shown in Figure S1.

Descriptive Associations

  • Figure S1 displays EDGE associations with essay creativity, CLAUS originality, CPS-AI originality, and CPS-AI effectiveness.The figure title describes assessments among human and AI responses.Source: Descriptive Associations · p. 58
  • Each plotted point represents one human or AI response unit.Regression lines were fitted separately for human and AI responses; labels report slopes (β) and sample sizes.Source: Descriptive Associations · p. 58
  • EDGE scores were standardized across human and AI responses within each matched group and draw, then averaged across draws for each response.Source: Descriptive Associations · p. 58
  • Assessment scores were standardized across human and AI responses within each panel.Source: Descriptive Associations · p. 58
  • Regression lines were fitted separately for human and AI responses using response-level mean EDGE scores.Source: Descriptive Associations · p. 58
  • AUT denotes Alternative Uses Task, CPS denotes Creative Problem-Solving Task, CLAUS denotes Cross-Lingual Alternate Uses Scoring, and CPS-AI denotes its automatic creative problem-solving scorer.Source: Descriptive Associations · p. 58

Related papers

Loading 2610.17324v1…