PaperlayerView exact revision on arXiv

Source-linked AI summary

From Student Risk Prediction to SC2R: Semantics-Constrained Counterfactual Recourse for Educational Decision Support

Ngoc Luyen Le, Marie-Hélène Abel, Bertrand Laforge

arXiv:2608.17618v1cs.IRcs.AI

TL;DR

Educational risk models identify students at risk but do not specify interventions that are actionable and feasible under educational constraints. SC2R generates discrete counterfactual intervention plans and validates them semantically, producing compact plans at scale while revealing infeasibility that optimization alone can miss.

  • Problem

    Educational risk prediction does not by itself provide interventions that are actionable, contextually appropriate, and operationally feasible.

  • Method

    SC2R combines predictive modeling, integer-programming recourse over discrete actions, an intervention vocabulary, and SHACL validation of explicit constraints.

  • Results

    127,972 plans achieved 1.0 model validity and SHACL conformance, averaged 1.075 actions, and semantic validation exposed infeasibility under availability constraints.

  • Takeaways & Limitations

    SC2R distinguishes plans that are merely model-valid from those feasible under explicit educational constraints, supporting machine-checkable recourse before practical use.

  • Takeaways & Limitations

    The offline, model-based, observational evaluation does not establish that following generated plans would improve student outcomes.

Abstract

from arXiv · show

Learning analytics models can identify students at risk of poor performance, but they do not directly indicate which interventions are feasible, actionable, and compatible with educational constraints. This paper introduces SC2R, a semantics-constrained counterfactual recourse framework for educational decision support. SC2R combines a calibrated predictive model, integer-programming-based recourse generation over discrete action variables, a lightweight RDF vocabulary for intervention-plan representation, and SHACL validation for enforcing timing, budget, immutability, and availability constraints. The framework is evaluated offline on the OULAD dataset using snapshots constructed relative to each assessment at two decision horizons. Results show that the predictive component provides strong performance, that compact intervention plans can be generated at scale, and that semantic validation reveals infeasible plans that lighter optimization-only settings would otherwise accept. Rather than claiming causal improvement in student outcomes, this work shows that counterfactual recourse becomes more operationally meaningful in education when recommendations are not only model-valid, but also semantically feasible and machine-checkable.

I. INTRODUCTION · II. RELATED WORK

The paper argues that educational decision support needs actionable, contextually appropriate, and operationally feasible recommendations beyond risk prediction. SC2R unifies predictive modeling, constrained counterfactual recourse, intervention vocabularies, and SHACL validation to produce machine-checkable, semantically feasible plans.

  • I. INTRODUCTION: Learning analytics predicts failure, disengagement, and dropout risk, but risk scores alone do not provide actionable or operationally feasible support recommendations.Instructors, advisors, and student support services need recommendations suited to educational contexts.
  • I. INTRODUCTION: Counterfactual explanations and algorithmic recourse identify changes intended to produce more desirable model outcomes while emphasizing actionability.Educational application remains challenging because feature-space methods may generate unsuitable recommendations.
  • I. INTRODUCTION: Educational recommendations must respect assessment timing, bounded intervention effort, immutable learner attributes, and pedagogical-resource availability.Usefulness also depends on stability, practical validity, and alignment with stakeholder needs.
  • I. INTRODUCTION: SC2R combines predictive modeling, integer-programming-based recourse over discrete action variables, an OWL/SKOS intervention vocabulary, and SHACL validation.The framework targets timing, budget, immutability, and optional availability constraints.
  • II. RELATED WORK: SC2R addresses the integration gap by combining prediction, constrained counterfactual recourse, and semantic validation into machine-checkable, interpretable, and semantically feasible intervention plans.The framework is positioned against prior work that generally treats prediction, recourse, and semantic technologies separately.
  • II. RELATED WORK: Prior learning analytics research demonstrates predictive models’ value for identifying students who may require support, while emphasizing that predictive performance alone is insufficient for educational usefulness.The field covers prediction of failure, disengagement, and dropout from educational traces, assessment histories, and learner profiles.
  • II. RELATED WORK: Prior recourse research studies sparsity, diversity, plausibility, robustness, and feasibility, increasingly framing recourse as intervention under actionability constraints.Most existing approaches remain mainly feature-space based, creating challenges for educational recommendations.
  • II. RELATED WORK: SHACL offers a W3C-standard mechanism for validating RDF graphs against declarative constraints, but semantic technologies have rarely been integrated into counterfactual recourse pipelines.This makes SHACL suitable for timing, budget, immutability, and resource-availability conditions.

III. PROBLEM FORMULATION

The problem is to generate low-burden, human-readable intervention plans that achieve a target predictive outcome from student snapshots while satisfying explicit semantic feasibility constraints. Educational recourse is therefore formulated as constrained decision support rather than geometric feature perturbation.

  • Student snapshots: Student states x_t ∈ R^p are observed at predefined times relative to an upcoming assessment or learning milestone.Snapshots use information typically available in learning management systems, with examples including d − 14 and d − 7.
  • Intervention plans: A counterfactual plan π comprises discrete, human-readable actions that modify learner engagement or study behavior before the next assessment.Applying π transforms the student state within the remaining time window.
  • Optimization formulation: The optimization objective minimizes intervention burden subject to f_h(x_t ⊕ π) ≥ τ, π ∈ A, and S(G_π) = 1.The threshold τ = 0.60 shown in Fig. 1 is illustrative only; c(π) denotes weighted intervention cost.
  • Operational constraints: Feasible plans must meet timing, budget, and immutability constraints governing schedulability, intervention burden, and non-actionable student attributes.Timing is bounded by the snapshot-to-deadline window, while budget remains below a configurable upper bound.
  • Decision-support framing: A recommendation is useful only when it is model-valid, feasible to enact, and understandable to human stakeholders.Candidate plans are retained only when they satisfy both the prediction target and semantic constraints.

A. Predictive Component

SC2R uses a calibrated logistic-regression pipeline to predict next-assessment success probabilities from student snapshots. The transparent, efficient linear baseline supports actionable recourse, while calibration is assessed with the Brier score because recourse operates on predicted probabilities.

  • Predictive model: The predictive component maps each student snapshot to a probability of success for the next assessment at a specified decision horizon.The predictor is denoted f_h(x_t) for horizon h.
  • Predictive model: The primary predictor is a calibrated logistic-regression pipeline with numeric-feature imputation and standardization before classification.The linear model is selected for transparency, computational efficiency, and compatibility with actionable recourse formulations for linear decision boundaries.
  • Predictive model: Calibration is assessed with the Brier score because downstream recourse operates on predicted probabilities rather than hard class labels.The broader implementation includes gradient-boosted trees and neural architectures, but these models are supplementary rather than the main paper’s foundation.

B. Recourse Generation

SC2R generates recourse by searching over allowable, interpretable intervention actions for a minimum-cost plan that moves a student’s predicted state across the target threshold. It uses integer programming for discrete, combinatorial interventions, while recognizing that model validity alone does not establish feasibility.

  • Recourse objective: The recourse component searches allowable action space A for an intervention plan π minimizing weighted cost c(π) while satisfying fh(xt ⊕π) ≥τ.Actions are defined as interpretable intervention increments derived from learner engagement and study-related variables.
  • Optimization formulation: The main solver uses integer programming because intervention variables are discrete, feasibility conditions are combinatorial, and the objective minimizes intervention burden.This formulation selects plans over discrete action variables rather than arbitrary perturbations in feature space.
  • Validity condition: A plan is model-valid when its post-intervention state crosses the predictive decision threshold, meaning fh(xt⊕π) ≥τ.The passage explicitly distinguishes this model-validity condition from broader feasibility requirements.

C. Semantic Validation Layer

SC2R’s semantic validation layer represents candidate intervention plans as structured RDF graphs and checks them with SHACL constraints. It enforces operational feasibility—including timing, budget, availability, and immutability—so recourse artifacts can be machine-checked before downstream use.

  • Representation and validation: SC2R combines a lightweight intervention-plan ontology with SHACL-based validation to make candidate plans operationally meaningful.The layer provides both representation and checking mechanisms for intervention plans.
  • Representation and validation: Plans are modeled as cbe:ActionPlan instances linked to cbe:Action objects, students, assessments, snapshot days, due days, and budgets.The vocabulary uses cbe:hasAction, cbe:forStudent, and cbe:forAssessment to structure intervention plans.
  • Representation and validation: Each candidate plan is encoded as an RDF graph Gπ, preserving a machine-readable distinction between the overall plan and its individual actions.The graph uses the intervention vocabulary before semantic feasibility is checked.
  • Feasibility constraints: SHACL constraints enforce timing, budget, and optional activity availability conditions, including snapshotDay <= from <= to <= dueDay and sum(actionCost) <= budget.Availability is checked when valid activity time windows are known.
  • Feasibility constraints: The semantic layer restricts plans to allowable intervention variables rather than fixed learner characteristics or non-actionable assessment metadata.This immutability principle distinguishes semantic feasibility from geometric proximity in feature space and supports checking structured intervention artifacts before downstream use.

V. EXPERIMENTAL SETUP

The experimental setup defines the dataset, snapshot protocol, comparison methods, and evaluation metrics used in the study.

  • Experimental Setup: The study evaluates its approach using a specified dataset, snapshot protocol, comparison methods, and evaluation metrics.These components establish the basis for the experimental evaluation.

A. Dataset and Snapshot Protocol

SC2R is evaluated on OULAD using module-presentation-level splits and assessment-relative snapshots at two pre-assessment horizons. The snapshots use only information available by each decision time, with features spanning engagement, alignment, assessment history, deadlines, and immutable learner descriptors.

  • Dataset: SC2R is evaluated on OULAD, combining demographics, assessment schedules and scores, VLE metadata, and daily interaction summaries across module presentations.The implementation uses the studentInfo, assessments, studentAssessment, vle, and studentVle tables.
  • Snapshot protocol: Two snapshots are constructed for each eligible student and assessment at t = d−14 and t = d−7, using only information available up to each snapshot.Post-snapshot interactions and assessments are excluded, while early assessments with negative snapshot times are handled during preprocessing.
  • Features and target: Features include rolling 7-, 14-, and 28-day activity clicks, availability alignment, prior assessment history, time-to-deadline variables, and immutable learner descriptors.The prediction target is next-assessment pass/fail, with pass defined as score ≥40.

B. Comparison Methods · C. Evaluation Metrics

The study compares SC2R’s calibrated logistic-regression predictor and integer-programming recourse formulation with supplementary predictive models, a Wachter-style counterfactual baseline, and a stricter availability-constrained variant. Evaluation covers predictive quality and recourse quality through model, semantic, cost, action, plausibility, and stability measures.

  • B. Comparison Methods: SC2R uses a calibrated logistic-regression model as its primary reference predictor.Stronger nonlinear models are included only as supplementary analyses.
  • B. Comparison Methods: Supplementary predictive analyses include XGBoost, a TabTransformer-style architecture, and a BiLSTM-based model.These models are not identified as the primary predictor.
  • B. Comparison Methods: The main recourse method is an integer-programming formulation over discrete action variables.This formulation generates intervention plans using discrete actions.
  • B. Comparison Methods: A Wachter-style counterfactual search serves as the recourse baseline on a controlled 200-case subset.The baseline comparison is restricted to the stated subset.
  • B. Comparison Methods: A stricter integer-programming variant adds availability constraints to assess richer semantic validation.The comparison tests the added value of incorporating availability into validation.
  • C. Evaluation Metrics: Predictive quality is measured with AUC, F1, accuracy, average precision, and Brier score.These metrics define the predictive-quality evaluation dimension.
  • C. Evaluation Metrics: Recourse quality is measured using model validity, SHACL conformance, intervention cost, number of actions, predicted probability gain, k-nearestneighbor plausibility, and stability under noise and retraining.These measures assess both semantic feasibility and practical recourse characteristics.

VI. EXPERIMENTAL RESULTS · A. Predictive Quality

The calibrated logistic-regression predictor performs strongly at both decision horizons, with usable calibration for downstream recourse generation. Stronger nonlinear models provide only modest additional AUC gains, so logistic regression remains SC2R’s reference predictor.

  • A. Predictive Quality: AUC is 0.884 at d−14 and 0.889 at d−7 for the calibrated logistic-regression baseline.The later snapshot benefits from more recent learner evidence.
  • A. Predictive Quality: Brier scores are 0.128 and 0.122 at the two decision horizons, indicating usable calibration for downstream recourse generation.F1 and average precision follow the same pattern as AUC.
  • A. Predictive Quality: On the d−14 split, XGBoost and the TabTransformer-style model reach an AUC of 0.898.These stronger nonlinear models improve modestly relative to the calibrated logistic baseline.
  • A. Predictive Quality: On the d−14 split, the MLP reaches an AUC of 0.895.The improvement remains modest relative to the calibrated logistic baseline.
  • A. Predictive Quality: On the d−14 split, the BiLSTM reaches an AUC of 0.888.The improvement remains modest relative to the calibrated logistic baseline.
  • A. Predictive Quality: Logistic regression remains the reference predictor for SC2R because nonlinear-model improvements are modest.The supplementary checks do not materially change the paper’s central picture.

B. Recourse Quality at Scale · C. Effect of Semantic Validation

SC2R generates compact, model-valid and semantically conformant intervention plans at scale, while semantic validation exposes infeasibility under availability constraints. Robustness is higher to input noise than to retraining.

  • B. Recourse Quality at Scale: 127,972 scored plans achieved 1.0 model validity and 1.0 SHACL conformance at scale.The plans averaged 1.075 actions, mean total cost 10.357, and mean predicted probability gain 0.383.
  • B. Recourse Quality at Scale: 1.075 actions was the average plan size, indicating that generated interventions remained low-cardinality while meeting predictive and semantic objectives.The same evaluation reported a mean total cost of 10.357 and mean predicted probability gain of 0.383.
  • B. Recourse Quality at Scale: 6.950 was the mean 5nearest-neighbor distance, suggesting recourse-adjusted states remained close to observed learner profiles without becoming trivial copies.Stability under noise and retraining was evaluated alongside semantic ablation in Fig. 5.
  • C. Effect of Semantic Validation: 24.127 vs. 24.175 were the average costs for IP and Wachter-style recourse, with both achieving perfect model validity and SHACL conformance under baseline constraints.The comparison used a controlled 200-case ablation of IP recourse, Wachter-style recourse, and IP recourse with availability constraints.
  • C. Effect of Semantic Validation: 0.875 was the SHACL conformance rate after activating availability constraints in the IP pipeline, without reducing optimization cost.This demonstrates that semantic validation can reveal infeasible plans accepted by lighter optimization-only settings.
  • C. Effect of Semantic Validation: 0.656 noise stability exceeded 0.228 retrain stability, indicating greater robustness to small input perturbations than to changes in the learned decision boundary.These robustness indicators were reported for the generated plans in Fig. 5.

VII. DISCUSSION · VIII. CONCLUSION AND PERSPECTIVE

The discussion positions SC2R as an efficient framework for generating compact educational intervention plans and semantically validating their feasibility. The conclusion emphasizes machine-checkable recourse while limiting claims about causal effectiveness because evaluation was offline, model-based, and observational.

  • VII. DISCUSSION: SC2R’s predictive component provides a strong and well-calibrated basis for downstream recourse generation.
  • VII. DISCUSSION: Integer-programming recourse produces compact intervention plans at scale, supporting efficient operationalization of discrete recourse.
  • VII. DISCUSSION: The semantic layer changes how recourse outputs are interpreted by enforcing a feasibility-oriented, machine-checkable representation of candidate plans.
  • VII. DISCUSSION: The offline, model-based, observational evaluation does not establish that following generated plans would improve student outcomes.
  • VII. DISCUSSION: The simplified action vocabulary derives from structured learner activity signals rather than richer pedagogical interaction data.
  • VII. DISCUSSION: Lower retrain stability suggests that some recommendations may remain sensitive to changes in the modeling context.
  • VIII. CONCLUSION AND PERSPECTIVE: SC2R is best understood as a framework for machine-checkable and semantically feasible educational recourse, not as evidence of causal intervention effectiveness.
  • VIII. CONCLUSION AND PERSPECTIVE: Using OULAD, the conclusion reiterates that calibrated logistic-regression prediction, scalable compact recourse, and semantic validation jointly support educational decision support.
Loading 2608.17618v1…