PaperlayerView exact revision on arXiv

Source-linked AI summary

RegulaTutor: A Theory-Informed Framework for Long-Term Personalized Learning

Yonggang Li, Boqian Han, Rui Sheng, Yushi Sun, Zixin Chen

arXiv:2610.13399v1cs.AI

TL;DR

Long-term tutoring must translate evolving learner histories into appropriate teaching, yet retrieving relevant records does not itself explain how they should guide instruction. RegulaTutor uses CAMM-guided interpretations and evidence-linked policies, while PairRoute refines the system from execution traces. Evaluations report average gains in evidence acquisition and diagnosis, with teaching gains on several backbones.

  • Problem

    Long-term tutoring must connect current tasks to accumulated learner history, while retrieval alone does not specify how records should guide teaching.

    Source: §1 · p. 1

  • Method

    RegulaTutor uses CAMM-guided interpretations to derive evidence-linked learner-specific teaching policies, and PairRoute routes execution failures to harness revision or reviewed model supervision.

    Source: §1 · p. 2

  • Results

    RegulaTutor improves evidence acquisition and diagnosis on average across evaluated configurations, with Teaching gains on several backbones.

    Source: §1 · p. 3, §6 · p. 9

  • Takeaways & Limitations

    The work offers a foundation for exploring lifelong tutoring in which instructional support can evolve as learners’ knowledge, goals, and contexts change.

    Source: §6 · p. 9

    LONGTUTOR evaluates independent history snapshots and does not measure real learners’ retention, transfer, or progress over time.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Long-term personalized tutoring requires language models to translate a learner's evolving history into appropriate instructional support. Existing LLM-based tutoring systems use memory and retrieval to personalize responses, but the pedagogical rationale linking historical evidence to teaching strategies is not always explicit. We introduce RegulaTutor, a theory-informed framework for long-term personalized tutoring that connects historical evidence, learner interpretation, and learner-facing instruction. Guided by self-regulated learning (SRL) research, its tutoring harness operationalizes a cognitive, affective, metacognitive, and motivational (CAMM) framework for deriving evidence-supported learner interpretations and teaching policies. The resulting policies retain links to their supporting evidence and are updated as new observations arrive. This staged workflow also supports systematic refinement through PairRoute, an offline procedure that uses intermediate execution records to distinguish harness, model, and mixed failures. PairRoute proposes and evaluates targeted harness revisions, then constructs reviewed stage-level supervision for confirmed model failures remaining under the selected harness. We focus this refinement on historical evidence acquisition, which underpins personalized instruction and allows outputs to be checked against learning records and reference answers. RegulaTutor improves historical-evidence accuracy, learner diagnosis, and judged teaching quality on average across evaluated model configurations. PairRoute results show an endpoint improvement in standalone-model historical-evidence accuracy over the refinement trajectory. Together, these results highlight RegulaTutor's potential to support lifelong personalized education through instructional policies that evolve with learner histories and targeted improvements to the tutoring system.

1 Introduction

Long-term tutoring must connect accumulated learner history to appropriate instructional decisions, but retrieving relevant records alone does not determine how they should guide teaching. RegulaTutor makes that connection explicit through CAMM-guided interpretations and evidence-linked policies, while PairRoute uses execution traces to refine the system offline.

1 Introduction

  • Long-term tutoring must relate current tasks to accumulated learner history as difficulties persist, recur, or change.Memory retrieval alone does not specify whether evidence should prompt reteaching, procedural scaffolding, or reflection.Source: §1 · pp. 1–2
  • RegulaTutor uses self-regulated learning research and CAMM processes to derive evidence-supported interpretations that inform learner-specific teaching policies.Policies specify support mode, explanation depth, references to prior episodes, and metacognitive or motivational support while retaining links to supporting records.Source: §1 · p. 2
  • Learner history and tutoring policies update as new observations arrive, supporting an inspectable path from accumulated history to instructional choices.The shared prompt also encourages goal setting, self-checking, and reflection when appropriate.Source: §1 · p. 2
  • PairRoute uses intermediate execution records to distinguish harness, model, and mixed failures, evaluate targeted harness revisions, and prepare reviewed supervision for residual model failures.The refinement focuses on historical evidence acquisition, which can be checked against learning records and reference answers.Source: §1 · p. 2
  • Across evaluated model configurations, RegulaTutor improves historical evidence acquisition and learner diagnosis on average.The focused Qwen3.5-4B study examines successive checkpoints under different harness versions and model–harness interactions.Source: §1 · p. 3
  • The paper evaluates RegulaTutor on long-history tutoring and separately examines harness revisions and model updates in a focused Qwen3.5-4B study.The introduction identifies evidence acquisition, diagnosis, and teaching as evaluation targets.Source: §1 · p. 3

2 Method

RegulaTutor turns learner history into instructional decisions through evidence acquisition, CAMM-guided interpretation, and evidence-linked policies that inform learner-facing directives. PairRoute then uses execution traces to diagnose failures, revise the harness, and prepare reviewed stage-level model supervision under the selected harness.

2 Method

  • RegulaTutor connects learner history, evidence-supported interpretations, and adaptive teaching policies, while PairRoute uses execution traces for offline refinement.The method presents tutoring and system refinement as related workflows.Source: §2 · p. 3

2.1 From Learner History to Instructional Decisions

  • At each interaction, the current task and learner history inform an interpretation, policy update, and response; model weights and shared harness rules stay fixed during tutoring.Learner history and policy evolve across interactions.Source: §2.1 · p. 3
  • Learner history accumulates observations, dialogue, and policy state, while a bounded retrieval view supplies context to individual model calls.The history may also be supplied as a self-contained record for retrospective tasks.Source: §2.1 · p. 3
  • The Evidence Resolution Loop retrieves focused records or consolidates observations across history chunks, then resolves answers with source identifiers and checks citations.Bounded retrieval expansion addresses unresolved information needs; its outputs preserve source evidence and per-need answers with support status.Source: §2.1 · p. 3
  • CAMM-guided interpretation connects admitted evidence and source-event statistics to tentative learner claims that retain supporting evidence, uncertainty, and instructional implications.Unsupported inferences are also recorded to avoid them.Source: §2.1 · p. 3
  • Teaching policies map supported interpretations to support mode, explanation depth, history references, and metacognitive or motivational support, then screen updates for evidence and confidence.The harness penalizes unnecessary switching, resolves competing assignments, retains policy provenance, and re-examines interpretations as evidence arrives.Source: §2.1 · p. 4
  • RegulaTutor combines interpretations and policies into instructional directives, constructs a final response-generation input, and records intermediate execution for offline refinement.Recent dialogue preserves speaker roles, and the shared prompt encourages goal setting, self-checking, and reflection.Source: §2.1 · p. 4

2.2 PairRoute: Trajectory-Guided Offline System Refinement

  • PairRoute diagnoses failures from stage inputs and outputs, revises the harness using training trajectories, then prepares reviewed supervision for confirmed model failures under the selected harness.Router1 evaluates bounded revisions with the model fixed and preserves successful behavior; Router2 re-diagnoses after harness changes, reviews teacher targets, and trains on reviewed corrections and correct outputs.Source: §2.2 · p. 5

3 Experiments

The experiments evaluate long-history tutoring and component contributions on LongTutor, then examine PairRoute’s model refinement and crossed model–harness configurations.

3.1 Experimental Setup

  • LongTutor’s 1,000-case human-annotated split tests evidence acquisition, diagnosis, and teaching using 199-record histories across six model configurations.Fixed-backbone comparisons use final harness H1, selected through PairRoute-guided refinement.Source: §3.1 · p. 5

3.2 Main Results

  • 13.2 percentage points in Evidence accuracy and 11.6 points in Diagnosis accuracy are the average RegulaTutor gains across nine fixed-backbone comparisons.The comparisons use final harness H1; Table 1 reports results for four local Qwen checkpoints and five API backbones.Source: Table 1 · p. 6, §3.2 · p. 6
  • Across Qwen checkpoints, RegulaTutor gains 15.5–36.2 percentage points in Evidence accuracy and 18.3–24.1 points in Diagnosis accuracy; two API backbones also gain in Evidence and Teaching.DeepSeek-v4-pro gains 10.9 Evidence points and 0.220 Teaching points; Gemini gains 18.2 Evidence points and 0.506 Teaching points.Source: §3.2 · p. 6

3.3 Ablation Studies

  • Removing CAMM interpretation lowers Evidence by 2.0 points and Teaching by 0.164, while fixing the policy lowers them by 1.5 points and 0.144.These are within-batch paired contrasts; the batch’s absolute reference score is not merged with the main H1 run.Source: §3.3 · p. 7, Table 2 · p. 7

3.4 Trace-Guided Refinement and Compatibility

  • PairRoute uses execution records to separate harness failures from model errors, tests targeted harness revisions, and builds reviewed stage-level training examples for residual model failures.The study uses 369 discovery cases, a separate 32-case probe set, and a disjoint 100-case development set; only reviewed discovery examples enter training.Source: §3.4 · p. 7
  • On the 100-case development set, Evidence accuracy under H1 rises from 212/300 (70.7%) with M0 to 250/300 (83.3%) with M3.The first model update gains 6.0 percentage points under H1, with a paired case-bootstrap 95% interval of [1.7, 10.3].Source: Table 3 · p. 7, §3.4 · p. 7
  • Without RegulaTutor, Evidence accuracy rises from 20.0% at M0 to 28.2% at M3, while judged Teaching rises from 2.631 to 2.789.The paired gains are 8.2 Evidence points (95% CI [6.3, 10.0]) and 0.158 Teaching points (95% CI [.088, .226]).Source: §3.4 · pp. 7–8, Table 4 · p. 8

4 Related Work

Prior tutoring research connects learner activity and learning theory to instructional support, while RegulaTutor emphasizes an explicit path from longitudinal evidence to teaching decisions and refinement.

4 Related Work

  • Earlier systems such as MetaTutor, Wayang Outpost/MathSpring, and CLASS connect learner observations or learning-science principles with instructional support.LLM tutoring frameworks add generation, planning, and memory, while LongTutor evaluates history use across evidence acquisition, diagnosis, and teaching.Source: §4 · p. 8
  • RegulaTutor combines longitudinal evidence management, CAMM-guided interpretation, persistent learner-specific policies, and learner-facing SRL support.Incremental session memory maintains learner history and policies across interactions.Source: §4 · p. 8
  • Prior co-evolution work updates harnesses and models, motivating evaluation of the executing model together with its interface.PairRoute applies failure-guided refinement to evidence acquisition, revises harnesses, and uses confirmed residual model failures as reviewed stage-level correction targets.Source: §4 · pp. 8–9

5 Discussion

The discussion highlights how learner interpretations, teaching quality, and personalization continuity pose distinct considerations for long-term tutoring.

4 Discussion

  • A past error remains a valid record even after a learner overcomes the difficulty, so interpretation lifetimes should be managed separately from supporting records.The authors suggest linking interpretations to supporting evidence and revision conditions.Source: §5 · p. 9
  • Removing compiled directives increases Evidence while reducing judged Teaching, exposing a distinction between recovering facts and deciding how they should shape instruction.The authors suggest optimizing evidence processing subject to explicit Diagnosis and Teaching criteria.Source: §5 · p. 9
  • Crossed checkpoint results show that model-update effects depend on inference configuration, raising a continuity requirement when learner histories and policies persist across system updates.The authors propose replaying held-out histories and stored policies before and after updates to test whether supported decisions are preserved.Source: §5 · p. 9

6 Conclusion

The paper concludes that RegulaTutor and PairRoute offer a foundation for exploring lifelong personalized tutoring with evolving learner histories and targeted system refinement.

6 Conclusion

  • LongTutor evaluations show average gains in evidence acquisition and diagnosis, with Teaching gains on several backbones; PairRoute shows development-set gains in evidence processing and characterizes model–harness compatibility.The paper presents RegulaTutor as a foundation for exploring instructional support that evolves alongside learners.Source: §6 · p. 9

Limitations

The paper bounds its claims through evaluation and implementation limitations, while identifying safeguards and requirements for responsible use with real learners. It also documents AI-assisted editing and release-readiness requirements.

Limitations

  • LongTutor tests independent history snapshots, not real learners’ retention, transfer, or progress over time.Teaching scores rely on model judges, with some configurations sharing the evaluator’s model family.Source: Limitations · p. 9
  • The refinement study is limited to one local backbone, three supervised rounds, and an adaptively reused development set.GRPO and stage-level SFT differ in supervision, objectives, budgets, and checkpoint selection, so score differences cannot be attributed to optimization method alone.Source: Limitations · p. 9
  • The harness uses engineering heuristics for learning-related evidence, without calibrated psychological-trait estimates or established causal effects of interventions.Source: Limitations · p. 9

AI use statement

  • AI tools were used to polish writing and readability, and all assisted edits were reviewed by the authors.Source: AI use statement · p. 10

Ethics statement

  • The design separates observations from episode-specific interpretations and discourages unsupported motivation or affect claims, but does not establish fairness or educational safety.Real-learner use requires data permissions, privacy protections, and review of instructional-decision consequences; the draft reports no new human-participant study or institutional approval.Source: Ethics statement · p. 10

Reproducibility statement

  • The release requires immutable model and harness revisions, learner-level split manifests, compiled snapshots, evaluator settings, and outputs for each pair comparison.Formal result admission requires matched generation and scoring artifacts plus an overlap audit; obsolete or incompatible runs are excluded, and no anonymous public artifact link is claimed.Source: Reproducibility statement · p. 10

A RegulaTutor Implementation Details

RegulaTutor maintains learner histories, resolves cited evidence, and turns supported interpretations into linked, updateable teaching policies. PairRoute uses execution records to select harness revisions and identify residual model calls for supervision.

A.1 History and Evidence Management

  • Snapshot mode reconstructs learner state from a complete history, whereas incremental mode combines a new observation with stored history and policy for an ongoing session.Events are deduplicated and chronologically ordered; tutor dialogue remains separate from scored learner observations.Source: §A.1 · p. 13
  • The Evidence Resolution Loop retrieves focused records for single-attempt questions or maps history chunks for multi-session questions, then resolves answers with source identifiers.The harness checks references and output structure, and may expand retrieval or repeat resolution when support is insufficient.Source: §A.1 · p. 13
  • Table 5 distinguishes event-count history and retrieval budgets from per-call token output limits.Source: Table 5 · p. 13

A.2 From Learner Evidence to Teaching Decisions

  • CAMM keeps cognitive and metacognitive interpretations separate while combining motivational and affective signals in one implementation field; interpretations record supporting observations and inference limits.Table 6 links admitted learner evidence to instructional support, and absence of a signal does not establish a learner trait.Source: §A.2 · p. 13, Table 6 · p. 14
  • CAMM recomputes statistics over ERL-cited events; repeated difficulty requires at least two incorrect outcomes on a concept, while prior-success support requires at least two scored events and accuracy of at least 0.7.Confidence values and acceptance thresholds are fixed engineering heuristics, not calibrated psychological probabilities.Source: §A.2 · p. 13
  • Policy proposals are ranked by confidence, guideline relevance, and evidence support, with switching cost favoring continuity and supporting observations retained in the resulting policy.Teaching directives specify support mode, explanation depth, history references, and inferences to avoid; the final input also includes the current task, learner statistics, resolved evidence, and recent dialogue.Source: §A.2 · pp. 13–14

B.1 Failure Diagnosis and Harness Refinement

  • At each PairRoute round, Router1 proposes evidence-access or presentation repairs, candidates are evaluated with the model fixed, and Router2 reviews residual failures under the selected harness for supervision eligibility.The current experiments evaluate the resulting trajectory and crossed model–harness pairs, not an advantage over alternative routing or update algorithms.Source: §B.1 · p. 14
  • The routing table organizes decisions by evidence visible to the implicated model call; separate Codex reviewers inspect calls, while the pipeline checks identities, references, and review structure.Router1 repairs receive manual refinement before evaluation, and Router2 reviews calls under the selected harness so training targets match the student interface.Source: Table 7 · p. 14, §B.1 · p. 14

B.2 Stage-Level Supervision and Training

PairRoute builds reviewed supervision from execution under a selected harness, while later matched evaluations characterize model–harness compatibility. The protocols cover long-history tutoring and an exploratory interactive-transfer study.

B.2 Stage-Level Supervision and Training

  • Training uses 369 discovery cases with 199-record histories and three evidence queries; silver references support review rather than serving as copied targets.The three queries are information extraction, multi-session reasoning, and hallucination control.Source: §B.2 · pp. 14–15
  • Each stage-level example supervises one recorded call, and reviewed corrections are combined with correct student stages, including stages from failed tasks.Teacher targets must be valid in full, and hidden references and routing judgments are excluded from student inputs.Source: §B.2 · p. 15

B.3 Selected Harness and Refinement Rounds

  • H1 is the final retained PairRoute harness, adding current-context binding and an indexed factual-support audit; later candidates were not retained.Both query classification and support checking remain fallible model/harness operations.Source: §B.3 · p. 16

B.4 Matched Comparisons

  • Matched crossed evaluations characterize updates along one refinement trajectory, but do not establish that routing outperforms an alternative update method.When the harness is retained and observations are reused, the interaction contrast is zero.Source: §B.4 · p. 16

C Datasets and Evaluation Protocols

  • TeachArena is an exploratory transfer study of simulated interaction and tool execution under a runtime different from the primary LongTutor benchmark.The paper describes its results as protocol-specific evidence about interface compatibility.Source: C Datasets and Evaluation Protocols · p. 16

C.1 Benchmarks and Metrics

  • LongTutor reports Evidence as the mean of IE, MR, and HC, Diagnosis as four-class accuracy, and Teaching as the mean of four rubric scores on a 1–5 scale.The overall judgment is reported separately as an auxiliary measure.Source: §C.1 · p. 16

C.2 Data Splits

PairRoute separates discovery, probe, and development data for harness refinement and model training. Evaluation settings distinguish LongTutor benchmark comparisons from protocol-specific TeachArena tests and other runtime configurations.

C.2 Data Splits

  • PairRoute uses separate discovery, probe, and development sets, but reuses probe and development adaptively during the executed refinement procedure.The three LongTutor adaptation sets are disjoint from Human Gold1000 by case, learner, and complete-history identity.Source: §C.2 · p. 17, Table 9 · p. 17

C.3 Evaluation Settings

  • LongTutor comparisons use paired case-level bootstrap intervals over 1,000 cases; TeachArena intervals condition on tasks completed by both configurations.LongTutor intervals are descriptive and unadjusted for multiplicity; TeachArena intervals describe recorded-run sample uncertainty, not variation across seeds or judges.Source: §C.3 · p. 17

D.1 Baseline Configurations

  • The baselines include model-only inference, DeepTutor, native Codex, and the initial and final harnesses H0 and H1.Native Codex uses a distinct runtime, so its comparison does not isolate a harness effect; calls and retries are accounted for rather than assumed computationally equal.Source: §D.1 · p. 18

D.2 Ablation Definitions

The component analyses support contributions from CAMM interpretation and adaptive policy selection. Separate probes examine a cumulative GRPO training path and response-mode effects on TeachArena task completion.

D.2 Ablation Definitions

  • Removing CAMM reduces Evidence by 2.03 points and Teaching by 0.164, while using a static policy reduces them by 1.50 points and 0.144, respectively.Diagnosis changes were not detected for either contrast.Source: §D.2 · pp. 18–19

D.3 GRPO Training-Path Probe

  • Evidence rises from 47.23% to 64.57% across two cumulative GRPO checkpoints, a paired gain of 17.33 points (95% CI [15.9, 18.7]).Diagnosis and Teaching remain statistically unchanged; the separate-batch absolute scores are not merged with the main SFT checkpoint table.Source: §D.3 · p. 19

D.4 Interface Controls

  • Relaxing strict schema enforcement sharply improves completion for DeepSeek and Gemini, showing that interface reliability must be distinguished from instructional quality.The response-mode effect differs across backbones; Qwen's JSON-object run has 93 unterminated-string errors and only 2/136 scored tasks.Source: §D.4 · p. 19

E Additional Results and Analyses

The additional analyses qualify LongTutor gains by query type, report conditional and full-task outcomes on TeachArena, and characterize compatibility across crossed model–harness evaluations. The comparisons use distinct splits and protocols.

E Additional Results and Analyses

  • The additional analyses use the same 1,000 LongTutor cases and completed Teaching judgments, while development scores use a different split and evaluator.Paired intervals are reported in Tables 14 and 17, and query-level Evidence results in Table 15.Source: E Additional Results and Analyses · p. 19, §E.1 · p. 20

E.1 LongTutor: Paired and Query-Level Effects

  • Excluding the self-judged DeepSeek-v4.1-flash condition, the eight Direct/H1 pairs average gains of 15.7 Evidence and 11.4 Diagnosis percentage points.M0, M1, and Gemini improve on all three Evidence query types; hallucination control improves reliably on seven of eight configurations judged independently of the tested backbone.Source: §E.1 · p. 20

E.2 Interactive Transfer and Interface Stress Test

  • On TeachArena, all three dialogue paired intervals include zero, while Qwen and DeepSeek workflow intervals exclude zero in the negative direction.Full-task rewards assign zero to failed executions, whereas paired intervals condition on tasks completed by both arms; dialogue and workflow failures vary by backbone.Source: §E.2 · pp. 20–21

E.3 Model–Harness Compatibility

  • H1 scores below H0 on Evidence at every Qwen checkpoint, with all four paired intervals excluding zero; the largest decrease is 16.20 percentage points at M3.These crossed evaluations characterize compatibility along one refinement trajectory, not performance against an independently trained alternative update method.Source: §E.3 · p. 21
Loading 2610.13399v1…