PaperlayerView exact revision on arXiv

Source-linked AI summary

Deconstructing Student Perceptions of Generative AI (GenAI) through an Expectancy Value Theory (EVT)-based Instrument

Cecilia Ka Yuk Chan, Wenxin Zhou

arXiv:2305.01186v1cs.CYcs.AI

TL;DR

This study examines how students’ knowledge, familiarity, perceived value, and perceived cost relate to their intention to use GenAI. Using an EVT-based factor structure and CFA, it finds perceived value most strongly positively correlated with intention, while perceived cost is weakly negatively correlated.

  • Problem

    The study examines how students’ knowledge, familiarity, perceived value, and perceived cost relate to their intention to use GenAI.

    Source: Literature on student’s perception of… · p. 4

  • Method

    The study applies EVT and confirmatory factor analysis to test a proposed factor structure for relationships among survey items and constructs.

    Source: Literature on student’s perception of… · p. 4, Methodology · p. 7

  • Results

    Perceived value had the strongest positive correlation with intention to use GenAI, perceived cost had a weak negative correlation, and knowledge had a significant but weak relationship.

    Source: Discussion · p. 10

  • Takeaways & Limitations

    Institutions should foster AI literacy and guide students in effective use and practical applications of GenAI, beyond providing basic knowledge.

    Source: Discussion · p. 10, Implications · p. 11

    The 405-participant, single-time-point higher-education sample may not represent the broader student population or capture changing perceptions over time.

    Source: Limitation · p. 12

Abstract

from arXiv · show

This study examines the relationship between student perceptions and their intention to use generative AI in higher education. Drawing on Expectancy-Value Theory (EVT), a questionnaire was developed to measure students' knowledge of generative AI, perceived value, and perceived cost. A sample of 405 students participated in the study, and confirmatory factor analysis was used to validate the constructs. The results indicate a strong positive correlation between perceived value and intention to use generative AI, and a weak negative correlation between perceived cost and intention to use. As we continue to explore the implications of generative AI in education and other domains, it is crucial to carefully consider the potential long-term consequences and the ethical dilemmas that may arise from widespread adoption.

Literature on student’s perception of AI and GenAI

Research on student perceptions of AI and GenAI is described as scarce, with prior studies examining varied educational settings and reporting mixed perceptions. This study applies EVT to investigate associations between students’ knowledge, familiarity, perceived value, perceived cost, and intention to use GenAI.

Literature on student’s perception of AI and GenAI

  • Research on student perceptions of AI, particularly GenAI, is scarce, motivating an EVT-based study of factors correlated with intention to use GenAI.The study focuses on knowledge, familiarity, perceived value, and cost.Source: Literature on student’s perception of… · pp. 2, 4
  • In US and German studies, students valued AI teaching assistants’ usefulness and communication, but were more critical of ChatGPT’s scientific accuracy than its linguistic quality.The US study also noted concerns about reduced human interaction and errors; prior AI experience in the German study related to more positive attitudes.Source: Literature on student’s perception of… · pp. 2, 4
  • Chinese EAP students viewed AI language-learning apps positively despite limited personalization and feedback, while TikTok content highlighted ChatGPT interest alongside academic-integrity concerns.Source: Literature on student’s perception of… · p. 3
  • Studies in India found positive views of AI in higher education but hesitation about replacing faculty, and reported that five perceived attributes significantly influenced ChatGPT adoption.Prior AI exposure influenced perceptions in the broader higher-education study.Source: Literature on student’s perception of… · p. 3
  • A Ghanaian study found no significant relationship between students’ perceptions and intention to use ChatGPT, despite stated adoption intentions and perceived advantages.Source: Literature on student’s perception of… · p. 3
  • Dutch business students expressed concern that their institutions were not adequately preparing them for AI work environments.The study identified needs for updated curricula, facilities, and more comprehensive AI training.Source: Literature on student’s perception of… · pp. 3–4

Expectancy-Value Theory and Other Frameworks

The section frames students’ intention to use GenAI through EVT, relating expectancy and value to research questions about knowledge, perceived value, and perceived cost. Value includes attainment, intrinsic, and utility components, while cost captures potential barriers.

Expectancy-Value Theory and Other Frameworks

  • EVT describes engagement decisions in terms of expectations of success and the value attached to an activity; expectancy is belief in one’s ability to succeed.Value includes attainment, intrinsic, utility, and cost components.Source: Expectancy-Value Theory and Other… · p. 4
  • The first research question asks whether students’ knowledge of and familiarity with GenAI correlate with intention to use it.The section proposes that familiarity and knowledge may shape expectancy beliefs, but frames this as a possibility rather than a demonstrated result.Source: Expectancy-Value Theory and Other… · p. 4
  • The second research question asks whether perceived value correlates with intention, with attainment, intrinsic, and utility value identified as relevant components.Examples include anticipated academic or digital gains, enjoyment, and practical benefits such as time savings or feedback.Source: Expectancy-Value Theory and Other… · p. 5
  • The third research question asks whether perceived cost correlates with intention; costs include effort, time, reduced educational value, social interaction, or holistic competencies.The section proposes that students may be less likely to adopt GenAI if perceived costs outweigh benefits.Source: Expectancy-Value Theory and Other… · p. 5

Why Expectancy-Value Theory?

The study selects EVT because it foregrounds motivation and decision-making around choices, goals, and performance. It also directly addresses perceived value, cost, knowledge, and familiarity as factors relevant to intention to use GenAI.

Why Expectancy-Value Theory?

  • EVT was chosen over UTAUT, TAM, and TPB because its focus on motivation and decision-making aligns with the study’s emphasis on choices, goals, and performance.The authors state that the other models offer insights into technology adoption but do not fully capture EVT’s motivational factors.Source: Why Expectancy-Value Theory? · p. 5
  • EVT suits the study because it accounts for perceived value and cost in GenAI use intentions, as well as students’ knowledge and familiarity with the technology.Source: Why Expectancy-Value Theory? · pp. 5–6

Methodology

The study used an online, cross-sectional survey of Hong Kong university students, with EVT-grounded questionnaire factors and confirmatory factor analysis. The analysis combined descriptive statistics with tests of the proposed factor structure.

Methodology

  • A cross-sectional online questionnaire gathered Hong Kong university students’ familiarity, knowledge, perceived value, perceived cost, and intention regarding GenAI in teaching and learning.Source: Methodology · p. 6
  • The survey contained 23 five-point Likert questions and used convenience sampling, distributing the questionnaire by bulk email to students.The authors note that convenience sampling may not produce a representative sample, though it allows efficient data collection.Source: Methodology · p. 6
  • Analysis proceeded from descriptive means and standard deviations to CFA, which tested the EVT-derived factor structure and relationships between items and factors.CFA was used without EFA because the authors considered the factors theoretically established and aimed to test the hypothesized structure.Source: Methodology · pp. 6–7
  • 405 students provided valid data; the sample’s average age was 23.87 years, and 51.4% were male.Source: Methodology · p. 7
  • For Q1, 33.6% reported never using GenAI and the mean was 2.32; Q11 and Q10 were neutral, while the other items showed an overall agree perception.The Q1 frequency responses were never 33.6%, rarely 22.0%, sometimes 28.9%, often 9.6%, and always 5.9%.Source: Methodology · p. 7

Validity and reliability of the scales

CFA indicated good fit for the knowledge, perceived value, and perceived cost constructs, and all three scales had acceptable internal consistency. Perceived value was positively correlated with intention, while knowledge and perceived cost also showed significant correlations.

Validity and reliability of the scales

  • All three constructs showed good CFA fit across χ²/df, RMSEA, SRMR, CFI, and TLI indices.χ²/df was below 3, RMSEA below 0.07, and SRMR below 0.080 for all constructs; CFI exceeded 0.95 for all, while perceived-value TLI was above 0.90 but below 0.95.Source: Validity and reliability of the scales · p. 8, Table 3 · p. 9
  • Cronbach’s alpha was 0.824 for knowledge, 0.874 for perceived value, and 0.759 for perceived cost, each above 0.7.The authors describe the three scales’ internal consistency as acceptable.Source: Table 4 · p. 9, Validity and reliability of the scales · p. 9
  • r=0.603 (p<0.001) for perceived value and intention; attainment, intrinsic, and utility values also correlated positively with intention.The subscale correlations were attainment value r=0.587 (p<0.001), intrinsic value r=0.456 (p<0.001), and utility value r=0.5502 (p<0.001).Source: Table 5 · p. 9, Validity and reliability of the scales · p. 9
  • r=0.179 for knowledge and r=-0.301 for perceived cost were both significantly correlated with intention.The association between perceived cost and intention was stronger than the association between knowledge and intention, but negative.Source: Table 5 · p. 9, Validity and reliability of the scales · p. 9

Discussion

Perceived value was the strongest positive correlate of intention to use GenAI, while perceived cost had a weak negative correlation. The discussion connects value to learning and workplace benefits, and addresses knowledge and perceived costs as separate considerations.

Discussion

  • All EVT factors—knowledge, perceived value, and perceived cost—were significantly correlated with intention; perceived value was the strongest positive correlate, while cost was weakly negative.Source: Discussion · p. 10
  • Students associated GenAI’s value with workplace advantages and learning gains, including academic performance and digital competence; utility included efficiency, personalized and immediate feedback, and idea generation.Source: Discussion · p. 10
  • Knowledge had a statistically significant but weak relationship with intention; the discussion says basic knowledge alone is insufficient and recommends practical-use guidance and training.It also recommends building AI literacy so students can turn GenAI’s advantages into valuable outcomes in learning and work.Source: Discussion · p. 10
  • Perceived costs include academic integrity, privacy, and security concerns; the discussion suggests social and experiential learning and interpersonal interaction to address apprehensions.The authors also note concerns about GenAI’s influence on higher education’s value and the development of holistic competencies.Source: Discussion · p. 10

Implications

The discussion presents a validated EVT-based instrument for understanding student perceptions of GenAI and connects its findings to curriculum design, targeted interventions, and further research.

Implications

  • Curricula and professional development should foster AI literacy, guide effective use, and highlight practical applications in learning and work.The authors connect these elements to students’ academic pursuits and future careers.Source: Implications · p. 11
  • Educators can tailor interventions to different student groups by identifying concerns and motivations that vary across disciplines or cultural contexts.The discussion proposes using such differences to better address student needs.Source: Implications · p. 11
  • The validated EVT-based instrument addresses a gap in robust, theoretically grounded measures of student attitudes toward GenAI adoption.The authors describe it as a foundation for future research and practice.Source: Implications · p. 11
  • The EVT framework captures knowledge alongside perceived value and cost as dimensions shaping students’ intention to use GenAI.The discussion describes these as cognitive and affective dimensions of student perceptions.Source: Implications · pp. 10–11
  • The instrument can be adapted to examine educators’ knowledge of GenAI, its classroom value, and perceived implementation costs.The discussion notes that this could inform professional development opportunities for teaching staff.Source: Implications · p. 11
  • Comparisons across populations and educational contexts could help identify contextual factors and inform tailored adoption policies and interventions.The authors include students and educators across educational levels, countries, and disciplines among possible comparison groups.Source: Implications · p. 11

Limitation

The study’s conclusions are bounded by its sample, timing, setting, self-reported measures, and validation choices.

Limitation

  • The 405-participant sample, single-time-point design, higher-education focus, and reliance on self-reports constrain representativeness and generalizability.Self-reports may be subject to social desirability or recall bias.Source: Limitation · p. 12
  • The study did not examine how cultural background, personal experiences, or learning styles may shape students’ perceptions of GenAI.The authors identify these individual differences as possible influences.Source: Limitation · p. 12
  • Skipping exploratory factor analysis may leave alternative factor structures or item-loading issues undetected.The authors consider CFA-only validation justifiable given the theoretical basis and hypothesis-testing focus, while recommending further validation studies.Source: Limitation · p. 12

Conclusion

The conclusion leaves the value-cost balance and ethical implications of GenAI adoption as open questions. Correlations suggest an association with intention to use, but do not establish a stopping threshold.

Conclusion

  • The conclusion raises whether ethical, integrity, and human-value concerns could make perceived costs outweigh GenAI’s value.It frames this as an open question about whether society might stop using GenAI or find reversal too late.Source: Conclusion · p. 12
  • The strong positive association of perceived value and weak negative association of perceived cost with intention to use leave the balance point for continued use unresolved.The authors state that their correlation analysis does not identify a threshold at which students would stop using AI.Source: Conclusion · p. 12
  • Estimating a stopping point would require more detailed analysis, possibly regression, while student experiences and priorities may prevent a universal threshold.The conclusion notes these individual differences as a challenge to identifying one cutoff.Source: Conclusion · p. 12
  • The authors call for careful consideration of potential long-term consequences and ethical dilemmas as GenAI adoption spreads.They connect weighing these factors with informed decisions about GenAI’s future and alignment with societal values.Source: Conclusion · pp. 12–13
Loading 2305.01186v1…