The Role of Implicit and Explicit Demographic Signals in Large Language Model-based Student Assessment

• Demographic context may appropriately shape instructional feedback, but essay scores should depend on essay quality rather than student demographics.Feedback may account for a learner’s first language, while demographic adaptation risks reinforcing stereotypes or unequal expectations.Source: §1 · p. 1
• Implicit demographic cues can arise from linguistic style or prior conversation and may shape scoring and feedback even when attributes are unstated.The study asks whether explicit and implicit cues systematically influence behavior and how effects differ across educational tasks.Source: §1 · p. 2
• Prior LLM fairness work has focused mainly on gender, with fewer studies examining race or socioeconomic background.This study addresses that gap using realistic profiles and real prompt histories to distinguish baseline behavior from explicit and implicit effects.Source: §1 · p. 2
• Six LLMs are compared across three educational tasks under default, explicit, and implicit demographic conditions using real-user profiles and conversation histories.The study assembles 192,480 LLM-generated responses across these conditions.Source: §1 · p. 2

2 Defining and Measuring Demographic Sensitivity in Education

• Demographic sensitivity is systematic model-behavior variation attributable to demographic context with task-relevant inputs held constant, measured across scoring, feedback, and metalinguistic QA.The tasks represent scoring, instructional feedback, and reasoning-oriented behavior.Source: §2 · p. 2

2.1 Datasets

• AES and formative feedback use Persuade 2.0 essays, metalinguistic QA uses ELQA questions, and the study samples 40 tasks from each dataset.AI Gap supplies real user prompt histories and demographic profiles for contextual information.Source: §2.1 · pp. 2–3

2.2 Prompting Strategy

• The three conditions are an unconditioned model default, explicit demographic attributes in a persona prompt, and implicit cues from ten prior user prompts and simulated responses.The same educational task input is held fixed while demographic context varies.Source: §2.2 · p. 3, §2.3 · p. 3

2.3 Evaluation

• AES is evaluated against human scores, while formative feedback and QA use linguistic metrics; demographic effects are analyzed with Lasso and KS tests.AES additionally uses paired t-tests with Bonferroni correction; the study applies statistical tests to each task and demographic profile.Source: §2.3 · p. 4

2.4 Models

• Six instruction-tuned models from proprietary and open-source families are evaluated under all three conditions, producing 192,480 inference calls.The models vary in family and size, and each processes every task under default, implicit, and explicit prompting.Source: §2.4 · p. 5

3.1 Automated Essay Scoring

• Llama-70B’s implicit scores rose +1.57 points relative to its own default (p < 0.001), the largest deviation across models and conditions.Table 1 reports mean AES score deviations from human ratings and Model by condition.Source: §3.1 · p. 5, Table 1 · p. 5
• Explicit AES demographic attributes generally produced small score shifts; Qwen-30B penalized higher-education users (µb = −0.11), while implicit effects included Llama-70B overscoring.Significant education-level distribution differences appeared only under Exp, including Llama-3B: D(edu2, edu0) = 0.65 and Qwen-30B: D(edu2, edu0) = 0.60.Source: §3.1 · pp. 5–6

3.2 Metalinguistic QA

• In QA, explicitly stated higher education generally corresponded to less-readable responses, while implicit readability adaptation was weaker and less consistent.For ARI, explicit effects appeared across four models; FRE results also showed lower readability for higher education under explicit conditioning across five models.Source: §3.2 · pp. 6–7
• QA response-length effects were more significant under Imp than Exp; laptop users and users with higher education received longer responses.The Imp-versus-Exp regression-line coefficient was m = 1.20.Source: §3.2 · p. 7
• Higher education corresponded to more positive QA sentiment under Exp, with an average 0.3 difference between the lowest and higher education levels; Qwen-30B showed a smaller Imp effect.Other implicit sentiment effects included technology, religion, and hobby attributes, but were not consistent across models.Source: §3.2 · pp. 7–8
• For QA BERTScore F1, US nationality had positive explicit coefficients in four models, while age and SES effects sometimes reversed between Exp and Imp.Higher education, lower age, and higher SES were also associated with responses more similar to the default under explicit conditioning.Source: §3.2 · p. 8

3.3 Formative Feedback

• In formative feedback, implicit conditioning produced more readable responses for several persona attributes, with readability effects concentrated mainly in Llama-70B.Writing-related personas had a FRE coefficient of µb = +3.67; no explicit effects were reported for these attributes.Source: §3.3 · p. 8
• Implicit feedback length shifted for several Qwen-4B contexts, while sentiment effects appeared only under Imp and were attributed exclusively to Llama-70B.Black or African American ethnicity was associated with longer Qwen-4B feedback (+1.35, Cnt=5).Source: §3.3 · pp. 8–9

4 Related Work

• LLMs are increasingly used for educational scoring, feedback, tutoring, question generation, and learning analytics.These uses offer opportunities for scalable and personalized instructional support.Source: §4 · p. 9
• Educational deployment raises validity, reliability, transparency, and fairness concerns, including the risk of reproducing or amplifying demographic, linguistic, cultural, and socioeconomic biases.Source: §4 · p. 9
• Most existing studies focus on a limited set of explicit attributes, while this study examines broad explicit demographics and implicit conversational characteristics across three educational tasks.Source: §4 · p. 9

5 Conclusion

• The controlled study examines explicit and implicit demographic signals across three educational assessment tasks using six LLMs.Source: §5 · p. 9
• Demographic effects are strongly task-dependent: explicit persona conditioning leaves most models relatively stable in essay scoring, while implicit cues produce larger, less predictable effects, especially for Llama-70B.In open-ended tasks, models show systematic differences in readability, sentiment, and response length.Source: §5 · p. 9
• The authors argue that high-stakes scoring requires demographic invariance, while instructional contexts may justify adaptations such as readability adjustments.They warn that current LLMs cannot reliably separate task-relevant information from demographic cues.Source: §5 · p. 9

Limitations

• The evaluation uses only 120 task items and 200 user profiles, and the authors say larger datasets and additional educational tasks would strengthen the study.Source: Limitations · p. 9
• Because several demographic attributes are heavily skewed toward majority groups, their regression coefficients are uninterpretable as genuine group contrasts and are excluded from the results.For example, 96% of profiles report English as their first language.Source: Limitations · pp. 9–10
• Prompt histories may not represent educational use, task writers’ demographics may conflict with conditioning signals, and English-only experiments cannot properly assess grammatical gender effects.The authors state that the task signal is held constant across demographic conditions, preserving the validity of their findings despite potential conflicting signals.Source: Limitations · p. 10
• Bonferroni correction may be overly conservative and miss significant results, although the authors say the significant effects they report likely reflect real condition effects.Source: Limitations · p. 10

Ethics Statements

• The study uses public AI Gap demographic profiles collected with ethical oversight and informed consent, and its experiments aim to identify potential harms before deployment.Source: Ethics Statements · p. 10

A List of demographic attributes

• Demographic variables include ordinal, single-value categorical, and multi-value categorical attributes, with ordinal values converted to integers and infrequent categories grouped or excluded from tests.The AI Gap tables list the ordinal and single-value categorical attributes; multi-value attributes appearing in fewer than 10 profiles are discarded.Source: A List of demographic attributes · pp. 12–13, Table 7 · p. 13, Table 8 · p. 13

B Prompts

• Prompts first provide explicit or implicit context, or no added context for the model default, then present a task prompt that is agnostic to the conditioning.Source: B Prompts · p. 13

B.1 Providing contextual information

• The three conditions are Model, with no context; Imp, with prior user questions and simulated responses; and Exp, with an explicit demographic description.Only the response to the final user message is analyzed.Source: §B.1 · pp. 13–14

B.2 Performing the educational tasks

• Essay scoring rates overall writing quality, formative feedback summarizes essay strengths and weaknesses, and metalinguistic QA uses a Stack Exchange question without additional information.Source: §B.2 · p. 14

C.1 Linguistic Metrics

• ARI and FRE are computed with textstat, sentiment with an education-fine-tuned BERT-based classifier, and BERTScore analysis uses the F1 score.Source: §C.1 · p. 14

C.2 Statistical Tests

• The OLS model and Bonferroni correction use statsmodels, while the KS test uses scipy.stats.Source: §C.2 · p. 14

D Models

• Table 12 lists the experimental models, and GPT models were queried through the OpenAI API.The passage does not name the models listed in the table.Source: D Models · p. 14
• Open-weight models were prompted locally from HuggingFace after deployment with vLLM.Source: D Models · p. 14
• All models used temperature 0.0; Llama 3.3 70B and Qwen3 30B used bitsandbytes quantization and a 32,768-token context limit.The context limit did not apply to the smaller Llama 3.2 7B and Qwen3 4B or to GPT models.Source: D Models · p. 14

E.1 Topic Modeling on User Prompts

• Across 15 topics, the most populated were General Q&A (T0, n = 240), Career & Resume (T10, n = 185), and AI & Concept Explanation (T11, n = 179).Source: §E.1 · p. 14
• Topic shares varied by subgroup: women skewed toward career/resume, men toward general Q&A, and lower- versus top-SES groups toward factual lookup versus finance.Source: §E.1 · p. 14

E.2 Automated Essay Scoring

• AES score-deviation shifts in the KS test occurred only under explicit personas, not implicit conditioning.Source: §E.2 · p. 14

E.3 Formative Feedback

• AoA regression flagged Qwen-4B tech:Laptop under Exp; KS gaps were larger across more distant education levels, and UK–US nationality differences appeared in three models.Source: §E.3 · pp. 14, 16
• Under Imp, 11 FRE predictors were significant and all positive; Llama-70B writing-email profiles had +3.67, while Exp had none.Source: §E.3 · p. 16
• Only Exp conditioning showed significant KS-test results; Llama-3B SES reached its largest D, 0.66, for ses4 vs. ses7, while US nationality in Exp yielded higher readability than UK.Source: §E.3 · p. 16
• Although ARI effects were sparse, GPT-nano English was +0.46 under Exp and Llama-70B social-media use was +0.73 under Imp; UK users received more readable responses.Source: §E.3 · p. 16
• Qwen-4B produced longer Imp feedback for several profiles, including Black or African American ethnicity (+1.35), the only demographic-group attribute beyond language and technology-use patterns.Source: §E.3 · p. 16
• Sentiment effects occurred only under Imp and only for Llama-70B; all listed attribute effects were positive, led by brainstorming (+0.32).Source: §E.3 · p. 16
• BERTScore F1 effects were the smallest in absolute terms, suggesting personas primarily reshaped readability, tone, and length rather than semantic content.Source: §E.3 · p. 16

E.4 Metalinguistic QA

• Sentiment effects were exclusive to Imp and Llama-70B; all 13 significant predictors were positive and spanned demographic and behavioral attributes.Source: §E.4 · p. 16
• For attributes significant under both settings, ARI effects were positively correlated between Exp and Imp (0.56), with smaller effects under Imp.Source: Figure 3 · p. 17, §E.4 · p. 17
• SES and education most strongly affected readability in Llama-70B; UK users received more readable responses, with UK–US differences significant in 10 of 40 items.Source: §E.4 · p. 17
• Under Exp, higher education and SES had negative FRE coefficients, while UK nationality had positive coefficients; Imp effects appeared for different models and attributes.Source: §E.4 · p. 17
• FRE KS gaps grew with education distance: Llama-70B D(edu0, edu2)=0.65(±0.11), while no LLM showed significant SES differences.Source: §E.4 · pp. 17–18
• For shared significant attributes, response-length effects were similar across conditions but larger under Imp than Exp.Source: Figure 5 · p. 18, §E.4 · p. 18
• AoA differences were marginal; explicit conditioning yielded slightly more difficult texts for higher education and English, while implicit coefficients were smaller.Source: §E.4 · p. 18

F Use of AI Assistants

• Claude AI was used to refine the manuscript's writing and grammar.Source: F Use of AI Assistants · p. 18