Source-linked AI summary
Educational impacts of generative artificial intelligence on learning and performance of engineering students in China
Lei Fan, Kunyang Deng, Fangxue Liu
TL;DR
The study examines how Chinese engineering students use generative AI and addresses limited systematic evidence about its effects in engineering education. Using an anonymized questionnaire survey of 148 students, it finds perceived gains in learning efficiency, initiative, and creativity alongside concerns about inaccurate outputs, over-reliance, and specialized engineering reliability.
Problem
Systematic evidence is limited on how generative AI affects Chinese engineering students’ learning initiative, critical thinking, and self-cognition, despite concerns about inaccurate outputs and over-reliance.
Method
The study analyzes anonymized questionnaire data from 148 engineering students across diverse Chinese regions, institutions, disciplines, and educational levels.
Results
The survey found perceived improvements in learning efficiency, initiative, independent thinking, and creativity, while students also reported inaccurate outputs, over-reliance, and doubts about specialized engineering capability.
Takeaways & Limitations
Effective integration should emphasize clear usage guidelines, curriculum-tailored policies, training, improved accuracy, literature search, and professional-software integration.
Takeaways & Limitations
The study relies on self-reported data and does not examine the long-term effects of generative AI on learning, skills, academic outcomes, or career readiness.
Abstract
from arXiv · showhide
With the rapid advancement of generative artificial intelligence(AI), its potential applications in higher education have attracted significant attention. This study investigated how 148 students from diverse engineering disciplines and regions across China used generative AI, focusing on its impact on their learning experience and the opportunities and challenges it poses in engineering education. Based on the surveyed data, we explored four key areas: the frequency and application scenarios of AI use among engineering students, its impact on students' learning and performance, commonly encountered challenges in using generative AI, and future prospects for its adoption in engineering education. The results showed that more than half of the participants reported a positive impact of generative AI on their learning efficiency, initiative, and creativity, with nearly half believing it also enhanced their independent thinking. However, despite acknowledging improved study efficiency, many felt their actual academic performance remained largely unchanged and expressed concerns about the accuracy and domain-specific reliability of generative AI. Our findings provide a first-hand insight into the current benefits and challenges generative AI brings to students, particularly Chinese engineering students, while offering several recommendations, especially from the students' perspective, for effectively integrating generative AI into engineering education.
1. Introduction
The study addresses the limited systematic evidence on how Chinese engineering students use generative AI and how it affects learning, while examining integration challenges and prospects.
- Motivation and rationale: Traditional engineering education can neglect student initiative and practical problem-solving, while generative AI may support autonomy, competence, and relatedness.These needs are identified by self-determination theory as improving engagement and learning effectiveness.
- Challenges: Generative AI raises concerns about plagiarism, academic integrity, assessment fairness, biased or inaccurate outputs, and over-reliance that may weaken independent thinking.These concerns are especially consequential in technical engineering disciplines requiring high accuracy.
- Research gap: The study responds to a lack of systematic analysis of generative AI’s effects on engineering students’ initiative, critical thinking, and self-cognition in the Chinese educational context.It frames technology adoption as interacting with learning motivation, educational culture, and social norms.
- Study scope: Using questionnaire data from diverse academic backgrounds, the study examines AI use frequency, purposes, effects on learning behavior and outcomes, challenges, and future integration strategies.The research questions cover use patterns, learning experience, challenges, and expectations for integration into engineering education.
2. Literature review
The literature presents generative AI as a tool for personalized, interactive, and problem-based learning, while linking its educational role to feedback, constructivist learning, self-determination, and technology-acceptance frameworks.
- Educational applications: Generative AI can provide personalized learning, real-time feedback, automated resources, adaptive content, and interactive educational environments.These capabilities are described as supporting more engaging and customized learning experiences.
- Educational applications: Generative AI supports problem-based and collaborative learning by helping students apply theoretical knowledge in practical contexts.The literature connects these activities with critical thinking, reflective communication, problem-solving, and engagement in engineering concepts.
- Educational theories: Immediate, targeted feedback is presented as a mechanism for improving learning efficiency throughout the learning process.The review traces this emphasis to reinforcement learning theory and structured instructional design.
- Educational theories: Constructivist learning theory frames learning as active knowledge construction through exploration, reflection, and critical inquiry.Related work applies constructivist principles to independent problem solving, learning analytics, engagement, and self-regulated learning.
- Educational theories: Self-determination theory links effective learning with autonomy, competence, and relatedness, which generative AI may support through adaptive and interactive environments.Technology acceptance and multiple intelligences theories are also identified as frameworks for evaluating student attitudes and learning.
3. Methodology
The study uses an anonymized questionnaire survey of 148 Chinese engineering students from varied regions, institutions, disciplines, and educational levels to quantify AI use and perceptions.
- Survey design: The study employs an anonymized questionnaire survey with n = 148 to examine generative AI use, perceptions, challenges, and potential curriculum integration.The quantitative design provides a broad view of AI’s role in engineering education in China.
- Sampling: Participants represent multiple Chinese regions, institution types, engineering disciplines, and educational levels.The sample includes undergraduates and postgraduates from fields including computer science, civil, mechanical, and electrical engineering.
- Instrument: The questionnaire contains 21 scale-based questions, 7 multiple-choice questions, and one open-ended question.The scale items quantify usage frequency, complex problem-solving preferences, learning impacts, challenges, and future applications.
- Ethics: The research protocol received ethics approval, and participants provided informed consent.The University Research Ethics Review Panel of Xi’an Jiaotong-Liverpool University approved the protocol on 21 September 2024.
- Sampling: The study combines purposeful sampling for representation with convenience sampling for broad access to respondents.Data were collected from October 9 to November 1, 2024, through online and offline participation.
4. Reliability and validity analysis of surveyed data
The questionnaire demonstrated strong internal consistency and good factor-analysis suitability, supporting the reliability and validity of the surveyed data.
- Reliability: Cronbach’s α exceeded 0.8 for all five variables, with an overall value of 0.879, indicating strong internal consistency.The reliability analysis covered the 21 scale-based questions.
- Validity: KMO = 0.867 indicates good sampling adequacy for factor analysis.The KMO test evaluates whether the sample is suitable for factor analysis.
- Validity: Bartlett’s test returned p = 0.000, rejecting the identity-matrix null hypothesis and confirming that factor analysis is appropriate.Together with the KMO result, this supports the questionnaire’s reported validity.
5.1 Frequency and purpose of using generative AI by Chinese engineering students
Generative AI was already a frequent part of Chinese engineering students’ academic routines, supporting resource discovery, report preparation, data analysis, literature review, and idea generation. ChatGPT was the most popular tool, although tool preferences may shift as new systems emerge.
- Many respondents first encountered generative AI in 2023–2024, while newly emerging tools may influence later tool preferences.
- ChatGPT was used by 77.03% of respondents, ahead of Wenxin Yiyan at 41.89%.
- 20.95% of students used generative AI daily, while 41.89% used it multiple times per week.
- 55.41% used generative AI to find learning resources or concept explanations, followed by report compilation at 54.73% and data analysis at 52.03%.
- 32.43% frequently used AI for assignments and 41.22% used it occasionally, making it a regular part of many academic workflows.
- 50.68% often used AI for literature searches or reviews, while 47.97% used it to generate initial design or research ideas.
5.2 The impact of generative AI on Chinese engineering students' learning
Students generally perceived generative AI as improving learning efficiency, initiative, and creativity, while its effects on independent thinking were mixed. Improved efficiency did not consistently translate into perceived academic performance gains, and heavy reliance could undermine motivation.
- 88.52% of respondents reported improved learning efficiency, including 36.49% reporting significant improvement.
- 64.19% reported improved learning initiative, although 6.76% reported a decline linked to possible over-reliance on AI.
- Frequent reliance on AI to complete learning tasks may weaken autonomy, competence, intrinsic motivation, and active learning.
- 47.97% perceived improved independent thinking, while 34.46% reported almost no change and 17.56% reported weakening.
- 58.78% reported improved creativity, but 29.73% saw almost no change and 11.48% reported a negative effect.
- Many students did not perceive improved academic performance despite reporting greater learning efficiency and initiative.
5.3 Challenges faced by Chinese engineering students in using generative AI
Chinese engineering students identified inaccurate outputs, over-reliance, usability difficulties, limited specialization, ethical concerns, and weak privacy confidence as major challenges. These concerns point to needs for more reliable, usable, discipline-specific, and responsibly governed tools.
- 62.16% identified inaccurate generated content as the most prominent challenge.
- 39.86% were concerned about over-reliance on AI reducing independent problem-solving ability.
- 20.27% reported usability difficulties, including challenging interfaces and insufficient technical support.
- Students and institutions were urged to improve technical reliability, interfaces, support systems, and affordability.
- Nearly 40% expressed concerns about AI’s effectiveness for highly specialized engineering problems.
- 58.11% rated ethical issues as important or very important, while 54.05% reported average satisfaction with data privacy.
5.4 Expectation on generative AI use from Chinese engineering students
Students generally supported integrating generative AI into engineering education, but favored partial integration over wholesale replacement of traditional teaching. They also expected foundational and practical training, clear institutional rules, and discipline-specific improvements.
- 43.92% favored partial integration of AI into selected courses or scenarios, while 20.95% supported full integration.
- 41.89% were uncertain that AI would replace traditional teaching, and complete replacement lacked strong overall support.
- 49.32% wanted in-depth practical AI training and 43.24% wanted basic introductory courses.
- 55.41% supported clear university usage guidelines, while 47.3% favored curriculum-specific policies.
- 68.91% viewed AI’s future educational prospects as broad or very broad.
- Over 60% requested greater accuracy for discipline-specific problems, better literature search, and professional-software integration.
- Schools and policymakers were encouraged to create discipline-tailored integration plans and training covering technical, practical, and ethical issues.
6. Discussion
Generative AI is widely used by engineering students for academic tasks and is perceived to improve learning efficiency, initiative, creativity, and, for some students, independent thinking. However, concerns about over-reliance, accuracy, ethics, privacy, self-report bias, limited longitudinal evidence, and sample representativeness constrain conclusions and inform calls for responsible integration.
- Use and learning impacts: Postgraduate students used generative AI more frequently, while students across fields and levels used it for academic writing, data analysis, and concept clarification.
- Use and learning impacts: Generative AI enhanced learning efficiency and active learning through instant feedback and rapid content generation, while also improving creativity and independent thinking for some students.
- Use and learning impacts: 17.56% of respondents feared that generative AI weakened independent thinking through over-reliance on immediate AI feedback.
- Challenges and integration: 58.11% considered generative AI ethics important or very important, while only 37.84% expressed satisfaction with AI tools’ privacy protection.
- Challenges and integration: Students wanted better discipline-specific accuracy, literature search, professional-software integration, data processing, and deeper answers to professional questions.
- Limitations and future research: Self-reported data may be biased because students can exaggerate AI use or perceive learning gains without corresponding actual improvement.
- Limitations and future research: The study does not assess generative AI’s long-term effects on learning, competency development, academic outcomes, or career readiness.
7. Conclusion
The study surveys generative AI use among engineering students in China and evaluates its effects, challenges, and future integration. Students report substantial learning benefits and optimism, but accuracy, over-reliance, ethics, privacy, and specialized engineering performance remain central concerns requiring structured guidance and training.
- The questionnaire study examines generative AI’s potential and challenges among engineering students in China.
- Learning effects: 88.52% reported improved productivity, 64.19% increased learning initiative, 58.78% greater creativity, and nearly 48% enhanced independent thinking.
- Learning effects: Nearly half of respondents felt generative AI did not improve academic performance despite reporting greater efficiency and active-learning engagement.
- Challenges: 62.16% identified inaccurate AI content as a challenge, 39.86% cited over-reliance, and nearly 40% doubted AI could handle specialized engineering problems.
- Future integration: 20.95% supported full integration and 43.92% favored partial integration of generative AI into engineering education.
- Future integration: 55.41% favored clear usage guidelines, 47.3% supported curriculum-tailored policies, and 46.62% advocated training for students and faculty.
Appendix A
Appendix A presents the questionnaire domains used to assess generative AI’s learning effects, future applications, usage frequency, challenges, and intended use among engineering students.
- Learning capabilities: The questionnaire measures generative AI’s effects on learning efficiency, independent thinking, creativity, and self-directed-learning motivation.
- Future applications: Future-application items assess optimism, engineering-education potential, career growth, innovative potential, and possible replacement of traditional teaching.
- Usage scenarios: Usage-frequency items examine AI use for generating initial ideas, reading literature, professional coursework, and writing reports.
- Challenges and concerns: Challenge items assess professional adaptability, academic-performance impact, ethical importance, data-privacy satisfaction, and inaccurate information exposure.
- Intended use: Intended-use items examine reliance on generative AI for initial innovation searches and as a primary solution to interdisciplinary problems.