Source-linked AI summary
The Ethics of AI in Education
Kaska Porayska-Pomsta, Wayne Holmes, Selena Nemorin
TL;DR
The chapter examines ethical concerns surrounding AI applications and their implications for people’s quality of life and opportunities. It reviews key Ethical AI concepts, highlights unique AIED aspects, and identifies areas for improvement toward actionable Ethical AIED practices.
Problem
AI applications raise ethical concerns with implications for people’s quality of life and opportunities.
Method
The chapter reviews key Ethical AI concepts and highlights unique aspects of the AIED domain.
Results
The chapter identifies areas for improvement in the field with respect to ethics.
Takeaways & Limitations
The chapter proposes an actionable approach to Ethical AIED practices that bridges ethical considerations and practice.
Takeaways & Limitations
AIED data collection can be limited, and social, environmental, and ethical concerns remain relevant in education.
Abstract
from arXiv · showhide
The transition of Artificial Intelligence (AI) from a lab-based science to live human contexts brings into sharp focus many historic, socio-cultural biases, inequalities, and moral dilemmas. Many questions that have been raised regarding the broader ethics of AI are also relevant for AI in Education (AIED). AIED raises further specific challenges related to the impact of its technologies on users, how such technologies might be used to reinforce or alter the way that we learn and teach, and what we, as a society and individuals, value as outcomes of education. This chapter discusses key ethical dimensions of AI and contextualises them within AIED design and engineering practices to draw connections between the AIED systems we build, the questions about human learning and development we ask, the ethics of the pedagogies we use, and the considerations of values that we promote in and through AIED within a wider socio-technical system.
1. Introduction
AI’s expansion into education raises ethical questions about bias, harm, fairness, equity, autonomy, and the values embedded in learning technologies. The chapter examines these issues within AIED’s wider socio-technical context and proposes an Ethics of AIED framework.
- AI applications can reproduce discriminatory practices and assumptions embedded in historical, social, and institutional systems.
- AIED ethics remains fledgling, with limited critical examination of systems’ ethical value, safety, and trustworthiness in socio-technical contexts.
- AIED’s positive self-image rests partly on assumptions that educational systems guard against unintended consequences and that AI promotes inclusive, high-quality education.
- Because fitting an existing educational system does not guarantee ethical practice, fairness, equity, and autonomy must be examined in education-specific contexts.
- The chapter reviews Ethical AI concepts, contextualises them within AIED, maps forms of bias across AI and socio-technical contexts, and identifies future directions.
- AIED offers a distinctive setting for studying ethics because its systems interact with cognition and may influence long-term learning behaviour.
2. What counts as an ethical approach?
An ethical approach to AI requires principles such as beneficence, non-maleficence, autonomy, justice, explicability, and accountability, but applying them remains context-dependent and insufficiently concrete.
- The meaning of being good in AI remains an open question shaped by ethical, individual, socio-political, and contextual perspectives.
- Floridi and Cowls’ framework identifies beneficence, non-maleficence, autonomy, justice, and explicability as principles for trustworthy and responsible AI.
- Explicability supports the other principles by exposing how AI works and enabling autonomy and accountability.
- Universal, domain-independent principles facilitate dialogue across engineering and social-scientific perspectives but lack concreteness.
- Guidance for operationalising ethical principles in specific AI designs and contexts remains scarce.
- Ethical concepts vary with changing socio-cultural norms, individual circumstances, and conflicts among stakeholder interests and needs.
3. Ethics of AI: key dimensions and concerns
AI ethics involves overlapping disciplinary understandings of bias, fairness, and harm, requiring attention to how bias enters systems and how ethical consequences are assessed across development and deployment.
- Ethical AI analysis examines the sources and consequences of violated principles to articulate related harms and complex socio-cultural relationships.
- Bias has overlapping definitions across disciplines, so practitioners must examine when and how it enters AI systems and how it can be mitigated.
- General forms of bias: Moral, legal, and statistical biases can diverge, meaning that correcting one form of bias may not make a system ethically better.
- General forms of bias: AI systems can reproduce structural inequalities or stereotypes even when data or algorithms are not statistically or legally biased.
- Algorithmic bias and sources of bias: Bias can enter or be reinforced at any stage from task definition through deployment and feedback, with earlier choices affecting downstream ethical quality.
- Algorithmic bias and sources of bias: Data bias may arise from sampling, preprocessing, labelling, collection software, sampling strategies, or human interpretation.
- Algorithmic bias and sources of bias: Ethical assessment should consider data provenance, acquisition, preprocessing, and labelling, alongside recommendations to mitigate algorithmic bias.
- Equity and representation: Fairness concerns in AIED include limited generalisability and the unequal representation of populations in data.
4. Ethical considerations for AI in Education
AIED ethics extends broader AI concerns into questions about access, representation, pedagogy, fairness, transparency, and human agency. The chapter highlights both existing ethical practices and persistent blind spots affecting diverse learners and educational contexts.
- Broader ethical dimensions: AIED inherits broader concerns about algorithmic bias, harms, fairness, transparency, and explainability while adding education-specific questions about learning, teaching, and values.The field’s educational context makes ethical considerations relevant to both technical systems and pedagogical practices.
- Inclusion and representation: Culturally and linguistically aligned technologies may support learners’ scientific reasoning and help remove barriers to learning and academic achievement.The cited example contrasts interaction in African American English and mainstream American English.
- Inclusion and representation: Limited attention to non-mainstream education represents a missed opportunity for more inclusive practices and for using digital technologies to support special-needs learners.The chapter connects representational gaps with the underdevelopment of learning-support innovations.
- Flexible support: Flexible, negotiable systems can support shared mechanisms across learners’ environments, helping educators and caregivers understand needs and provide holistic support.Open Learner Models and negotiated support are presented as ways to build on learners’ strengths rather than define them by deficits.
- Fairness and data: Fairness evaluation may require more than one approach for each AIED scenario, while representationally balanced data cannot be guaranteed through proportional sampling alone.These constraints reflect the context-dependent nature of fairness and data representation.
- Transparency and agency: Open Learner Models give users ownership over data, acknowledge model inaccuracy, and support transparency, explainability, learning, and criticality.The chapter presents OLMs as a promising direction for increasing ethical value in AIED practices and solutions.
5. Towards a framework of Ethics of AIED
The chapter develops a discussion-oriented Ethics of AIED framework that separates pedagogy from the socio-technical context surrounding education. It maps ethical questions, biases, and risks across four design and deployment foci.
- Framework foundations: The proposed framework begins with data, models, and education, then examines their intersections and the interaction between AI systems and individual human cognition.It is presented as a first step intended to stimulate discussion and remain open to refinement.
- Framework foundations: The amended framework distinguishes pedagogy from the socio-technical context and comprises four foci: Data, Models/Algorithms, Pedagogy, and Socio-technical context.This restructuring clarifies two fundamental dimensions of education: educational practices and the context in which education occurs.
- Practical use: The framework links each identified bias to example problems, questions for practitioners, and indicative consequences or risks.Its purpose is to support forward-thinking, ethics-conscious AIED research and development by design rather than only list prohibitions.
- Design and deployment questions: Design questions address whether data represent target populations, models align with diverse behaviours and cultures, pedagogies accommodate varied needs, and systems support human adjustment.The framework also asks whether systems reinforce or challenge dominant educational cultures and beliefs.
- Design and deployment questions: The framework identifies risks including representational harm, inequitable accessibility, unfair outcomes or processes, inappropriate predictors, and prescriptive or harmful technologies.Examples include conflating login duration with engagement and misinterpreting learner behaviour in assessment.
6. Conclusions
AI can benefit education, but its implementation raises social, environmental, and ethical concerns involving access, privacy, market concentration, behavioural influence, and broader socio-technical effects. The chapter argues for an interdisciplinary, context-aware, actionable approach to Ethical AIED that connects algorithmic design with wider educational systems and values.
- Ethical risks: AI implementation in education can create unfair practices around access and perpetuate bias against historically oppressed groups.The chapter links these concerns to unequal access to technologies and education itself.
- Ethical risks: Concentrated control of educational technologies can create privacy risks and allow dominant platforms to influence educational policies and practices.The chapter identifies concentration of personal information and potential monopolisation of algorithmic research and development as primary concerns.
- Ethical risks: Optimisation systems can capture detailed information and manipulate behaviour, creating risks beyond algorithmic bias and discrimination.The chapter emphasises that examining only algorithms misses effects arising from optimisation across the broader socio-technical system.
- Socio-technical context: Understanding AIED ethics requires attention to development, deployment, exploitation, service-provider incentives, and the power of actors shaping decisions and behaviours.The chapter presents context-awareness as necessary for appraising AIED work beyond declared good intentions.
- Future directions: A comprehensive Ethical AIED approach calls for horizon scanning and interdisciplinary dialogue across learning sciences, neuroscience, sociology, and philosophy.It also requires questioning what education is for and what educational systems AIED supports.
- Future directions: Ethics by design should connect micro-level algorithmic decisions with macro-level socio-technological considerations and support actionable Ethical AIED practices.The chapter frames this connection as a way to explore unknown ethical issues and promote the beneficence of education for all.