Generative AI May Reinforce Social Biases in Software Engineering Education

• Potential bias matters because educational AI can influence learning experiences and shape educational and career perceptions, while GenAI’s impact on software engineering education remains largely unexplored.Source: §1 · p. 1
• The study examines team formation and visual content generation as underexplored GenAI applications in software engineering education.Source: §1 · p. 1
• The evaluation varies student qualifications and personal attributes in team assignments and analyzes images generated from real course-material prompts.The team-formation evaluation covers three LLMs and 11,200 decisions; the visual task analyzes 500 images.Source: §1 · p. 1
• GenAI models exhibit biases toward specific genders, nationalities, and skin tones in educational tasks.Source: §1 · p. 1

• Prior research documents gender, racial, and nationality biases in LLMs, along with biases involving other demographic factors and domains [19] [22] [53].Source: §2 · p. 2
• Educational studies examine demographic disparities in college-major recommendations and generated or selected educational content, while team formation and visual content generation remain underexplored [53].Source: §2 · p. 2
• Software engineering bias research covers hiring, regional role assignment, and gendered task descriptions, while prior work has largely focused on narrow tasks and limited educational contexts [4] [34] [37] [47].Source: §2 · p. 2
• The paper extends team-assignment research by considering educational settings, class standing, implicit demographic signals, and AI-generated visual content for software engineering materials [37].Source: §2 · p. 2

3.1 Task 1: Team Formatting

• The team-formation task assigns 28 students to four SWEBOK-based teams: Interface Design, Core Development, Database, and Quality Assurance.Source: §3.1 · p. 2
• Student names indirectly signal gender and nationality, while profile qualifications represent technical skills and experience relevant to team placement.Source: §3.1 · p. 2
• Three qualification strategies provide no skills, year of study, or technical skill profiles, with skill profiles designed to support two suitable team options.Source: §3.1 · pp. 2–3
• Name-to-qualification mappings are varied, and each class version is tested across 10 runs with GPT-4.1, GPT-5.2, and DeepSeek V3.2.Table 1 summarizes the experimental structure and total runs for each LLM.Source: §3.1 · p. 3, Table 1 · p. 3

3.2 Task 2: Visual Content Generation

• Visual-generation prompts are drawn from human-centered topics in four software engineering courses at three U.S. universities.Source: §3.2 · p. 3
• The visual task uses 25 prompts and two image-generation models, producing 500 images in total.Source: §3.2 · p. 3

• Plain and Level-based team assignments are analyzed with multinomial logistic regression, while Skill-based assignments use six pairwise binary logistic regressions on over 99% of on-pair predictions.The pairwise approach addresses sparse outcomes outside the two valid team labels for each qualification.Source: §4 · p. 3
• Regression odds ratios use Core Development, female, and China as reference categories, and the analyses do not apply multiple-comparison corrections.Source: §4 · p. 3
• Image labels record gender and skin tone, with skin tone used as a proxy for racial bias and labels based on agreement between two annotators.The coding scheme includes light, medium, dark, unsure, diverse, and unknown skin-tone categories.Source: §4 · p. 3
• Prompt categories follow SWEBOK roles, and Representation Bias Percentage reports demographic-label prevalence by category alongside chi-square tests and Cramér’s V.Source: §4 · p. 4
• Single-person images are tested for men and light-skinned individuals versus other categories, while multi-person images are assessed for gender and whether skin tones are diverse or uniform.Source: §4 · p. 4

5.1 Task1: Team Formatting

• Across all three team-formation persona strategies, the models showed significant biases associated with gender and nationality.The specific assignment patterns differed across plain, level-based, and skill-based personas.Source: §5.1 · p. 4

5.1.1 Plain Personas.

• In plain personas, men were more likely than women to be assigned to Core Development, while nationality was associated with different team assignments.Across models, men were at least 80% less likely to be assigned to Interface Design than women; for example, Nigerian names were more likely to be assigned to Quality Assurance and South Korean names to Interface Design than Chinese names.Source: §5.1.1 · p. 4, Table 2 · p. 4

5.1.2 Level-based Personas.

• Seniors were much less likely than freshmen to be assigned to Interface Design (GPT 4.1: OR=0.0004, GPT-5.2: OR=0.004, DeepSeek: OR=0.001), while seniors and juniors tended toward Core Development.The authors note that linking class standing to team assignment may imply that Interface Design is less technically demanding, an assumption that may not hold across curricula or development contexts.Source: §5.1.2 · pp. 4–5

5.1.3 Skill-based Personas.

• 99.2% of skill-based persona assignments matched one of the two intended teams, but gender still influenced which valid team was selected.For Core Development versus Interface Design, male personas were favored for Core Development relative to female personas (GPT 4.1: OR=2.53, GPT-5.2: OR=2.81, DeepSeek: OR=1.43).Source: §5.1.3 · p. 5, Table 4 · p. 6

5.2.1 Single-person.

• For single-person images, both models significantly favored men (GPT Image 2: p<0.001, V=0.64; Imagen 4: p<0.001, V=0.65), and light-skinned individuals (GPT Image 2: p<0.001, V=0.57; Imagen 4: p<0.001, V=0.61).More than 95% of single-person images in Database, Debugging, and Individual Programming depicted light-skinned individuals; AI and Testing prompts yielded more diverse representations with GPT Image 2 than Imagen 4.Source: §5.2.1 · pp. 5–6

5.2.2 Multi-person.

• Multi-person images were more balanced by gender than single-person images: GPT showed no significant preference for men, while Imagen 4 did (p<0.001, V=0.25).Both models tended to generate groups with diverse skin tones, and most prompt categories predominantly produced such groups.Source: §5.2.2 · p. 6, Figure 1 · p. 6

6 Discussion

• Models linked male students with Core Development and women with Interface Design, while favoring senior students for development roles.Single-person images also overrepresented light-skinned men, while multi-person images were more balanced.Source: §6 · pp. 6–7
• Single-person images overrepresented light-skinned men, whereas multi-person generation showed more balanced representations.The discussion attributes greater reliance on learned demographic patterns to single-person generation.Source: §6 · pp. 6–7
• Adding task-relevant information reduced but did not eliminate bias; demographic attributes still influenced choices among multiple valid assignments.Skill-based personas led to more appropriate assignments.Source: §6 · p. 6
• The authors highlight model development and prompt design as important for mitigating bias.Repeated exposure to biased decisions or representations may reinforce stereotypes and contribute to broader inequalities over time.Source: §6 · p. 7
• Names signaling gender or nationality can affect outputs, so educational decision prompts should specify criteria and minimize unnecessary demographic cues.The discussion also calls for scalable evaluation frameworks to identify domain-specific biases before educational adoption.Source: §6 · p. 7

7 Conclusion & Future Work

• Team assignment and image generation can exhibit patterns favoring particular genders, nationalities, or skin tones.The authors note that undesirable outputs may affect learners and, over time, contribute to broader societal impacts.Source: §7 · p. 7
• Future work can investigate ways to provide information that reduces GenAI models’ reliance on assumptions learned from potentially imbalanced training data.Source: §7 · p. 7
• The authors identify domain-specific fairness evaluation before deployment in sensitive areas such as education and healthcare as an important research direction.Source: §7 · p. 7

A Persona Examples

• The listed student profiles include names and computer-science academic years, such as freshman, senior, and junior.Source: A Persona Examples · p. 10
• Skill-based personas associate each student with experience related to two of four potential teams.The examples show interface design with quality assurance, core development with quality assurance, and interface design with database experience.Source: A Persona Examples · p. 10
• Persona strategies provide different levels of information and all include a student name that may signal gender or nationality.The examples are designed to reflect information course instructors may have when forming teams.Source: Figure 2 · p. 10

B Prompt Templates

• The prompt-template materials include a template for the visual content generation task.Source: B Prompt Templates · p. 11
• The same prompt templates were used across all evaluated models.Source: Figure 3 · p. 11