PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepEdu-v1: Efficient and Scalable Agentic LLMs for Vietnamese Education

Quang Nguyen, Hieu Nguyen, Hien Hoang, Toan Pham, Cong Tran, Nam Vu

arXiv:2609.31568v1cs.AI

TL;DR

Vietnamese AI tutoring must reconcile data-sovereignty and curriculum-grounding needs with the hardware costs of self-hosting long-context models. DeepEdu combines cluster-level selective attention with a self-improving, playbook-curating agentic layer. The paper reports lower latency and higher agentic accuracy in its evaluated settings.

  • Problem

    Vietnamese educational deployment needs on-premise handling of student data and curriculum-grounded local knowledge, while long-context inference poses hardware costs.

    Source: §I · pp. 1–2

  • Method

    DeepEdu pairs cluster-level long-context inference with an agentic layer that curates an evolving playbook rather than updating model weights.

    Source: §VII · p. 20

  • Results

    Agentic accuracy rises from 70.0% to 79.5% on complex tasks, while the deployed agentic layer on SCR reduces mean time-to-first-token by roughly 2×.

    Source: §VII · p. 21

  • Takeaways & Limitations

    The paper presents SCALE as combining long-context efficiency with curriculum-grounded self-improvement for on-premise educational deployment.

    Source: §VII · p. 20

    The authors flag an experimental limitation as future work.

    Source: §VII · p. 21

Abstract

from arXiv · show

AI tutoring could markedly improve learning outcomes for students in developing regions such as Vietnam, yet the two obvious paths both fall short. Cloud assistants such as ChatGPT route sensitive student data to foreign servers---violating data-sovereignty laws such as Vietnam's Decree 53---and, pre-trained on Western-centric corpora, are not organized around the national textbook curriculum, so their knowledge of local content is unsystematic and frequently hallucinated. Self-hosting an open model keeps data on-premise but hits a two-fold wall: post-training quantization (AWQ, GPTQ) tames the static weight footprint, yet the dynamic KV cache and prefill latency of long tutoring contexts still cause out-of-memory failures and slow responses on consumer GPUs, while the model keeps hallucinating on region-specific material. We present DeepEdu-v1, an AI-tutoring system for Vietnamese education built on SCALE (Self-improving Context-Aware Learning Engine), a framework with two innovations. First, a long-context inference engine amortizes token selection from per-sub-chunk to per-cluster granularity; on long-context retrieval it issues x7.7 fewer retrieval calls than a state-of-the-art selective-attention baseline, cutting prefill latency (TTFT) by roughly 35% while matching or improving task accuracy. Second, a self-improving agentic layer continuously curates a verified playbook from past interactions instead of fine-tuning, a design intended to progressively reduce reliance on dominant-language priors as trustworthy local knowledge accumulates. In its deployed configuration, DeepEdu achieves a nearly x2 TTFT speedup over standard vLLM serving and lifts agentic accuracy from 70.0% to 79.5% on complex tasks, with the strongest per-track gains across financial-reasoning and interactive-agent benchmarks.

I. INTRODUCTION

DeepEdu addresses data-sovereignty, curriculum-grounding, and hardware constraints in Vietnamese AI tutoring with SCALE, which combines long-context inference and a self-improving agentic layer. The introduction reports fewer retrieval calls and lower prefill latency, alongside gains on agentic benchmarks.

I. INTRODUCTION

  • Vietnamese tutoring must keep sensitive student data on-premise and ground answers in the textbook curriculum, where generic models’ local knowledge is unsystematic and frequently hallucinates.Long tutoring contexts add KV-cache and prefill costs; continuously fine-tuning on evolving curricula is described as computationally prohibitive and prone to catastrophic forgetting.Source: §I · pp. 1–2
  • SCALE pairs Similarity Chunk Rolling, which amortizes selective sparse attention from sub-chunk to cluster granularity, with a playbook-curating agentic layer that avoids weight updates.The playbook is verified and curriculum-grounded, and the agentic layer is intended to reduce reliance on dominant-language priors.Source: §I · p. 2
  • 7.7× fewer retrieval invocations and roughly 35% lower prefill latency than TokenSelect, while preserving or improving task accuracy on structured-retrieval tasks.The deployed configuration also reports a nearly 2× TTFT speedup and strongest per-track gains across grounded financial-reasoning and interactive-agent benchmarks.Source: §I · p. 2

A. The Evolution of Large Language Models

  • LLM development progressed from scaled Transformer architectures and open-weight models to tool-augmented agents capable of complex interactive tasks.The introduction frames personalized education as one application of those agents.Source: A. The Evolution of Large Language… · p. 2

B. Foundation Models for Education

  • LLM-based tutoring extends beyond static intelligent tutoring systems toward personalized feedback and adaptive scaffolding, but accessible-hardware costs and localized-curriculum misalignment remain deployment challenges.The related-work discussion also surveys tool-augmented agents as a foundation for complex interactive applications, including personalized education.Source: A. The Evolution of Large Language… · p. 2, B. Foundation Models for Education · p. 3

C. Post-Training Quantization for Efficient Deployment

  • Quantization methods such as GPTQ and AWQ reduce static model-weight footprints, but do not resolve dynamic KV-cache growth during long-context inference.GPTQ uses an approximate second-order procedure; AWQ selectively preserves salient weights through per-channel scaling.Source: C. Post-Training Quantization for… · p. 3

D. Knowledge Distillation for Compact Educational Models

The related-work discussion covers model compression, long-context inference, low-resource-language adaptation, localized hallucination, and self-evolving agents. It positions DeepEdu as combining long-context efficiency with curriculum-grounded self-improvement without fine-tuning.

D. Knowledge Distillation for Compact Educational Models

  • Knowledge distillation can transfer reasoning trajectories to compact students, but its training cost and static knowledge limit adaptation to evolving curricula without repeated re-distillation.The reviewed methods include reverse-KLD optimization and step-by-step reasoning-trajectory distillation.Source: D. Knowledge Distillation for Compact… · p. 3

E. Long-Context Inference Optimization

  • Long-context methods extend context windows and exploit attention sparsity, while prefill latency remains costly and the interaction between selective sparse attention and chunked prefill remains under investigation.System optimizations mitigate GPU-memory bottlenecks but do not reduce attention’s fundamental computational complexity.Source: E. Long-Context Inference Optimization · pp. 3–4

F. LLMs on Low-Resource Languages

  • Vietnamese and other low-resource languages are underrepresented in pre-training corpora, and reviewed adaptation approaches range from cross-lingual prompting and in-context learning to continual pre-training.These approaches seek language adaptation without necessarily requiring full pre-training from scratch.Source: F. LLMs on Low-Resource Languages · p. 4

G. Hallucination in Localized and Cultural Contexts

  • Western-centric training data and distribution mismatch are linked in the reviewed work to hallucinations and degraded reasoning in low-resource languages and non-Western cultural contexts.The discussion identifies localized runtime grounding as a route to supplying regional knowledge without continual retraining.Source: G. Hallucination in Localized and… · p. 4

H. Agentic Systems and Self-Evolving Context

  • Self-evolving agent approaches use dynamic context memory to preserve domain insights without standard fine-tuning, but prompt brevity bias and monolithic rewriting can lose technical nuance or accumulated knowledge.The discussion describes context collapse as a risk of iterative rewriting in long-context windows.Source: H. Agentic Systems and Self-Evolving… · pp. 4–5

I. Discussion and Positioning

  • DeepEdu is positioned at the intersection of on-premise deployment, long-context efficiency, self-improvement without fine-tuning, and localization, which prior directions address only in part.The authors characterize it as, to their knowledge, the first framework to combine these properties in one system.Source: §I · p. 5, Table I · p. 5

III. PRELIMINARIES

The preliminaries establish the inference bottlenecks and selective sparse attention that SCALE builds on. They identify redundant per-sub-chunk retrieval as an opportunity for cluster-level amortization.

A. LLM Inference and Multi-Head Attention

  • Prefill is compute-bound, while the KV Cache grows with context; chunked prefill caps peak activation memory but does not remove the cost of attending to the growing cache.The attention softmax has quadratic complexity in sequence length.Source: A. LLM Inference and Multi-Head… · p. 5

B. Selective Sparse Attention

  • Selective sparse attention dynamically chooses a small subset of cached tokens for query-aware attention, reducing the need to attend to the full cache.TokenSelect conditions its selection on the current query, with a retrieval call per step.Source: B. Selective Sparse Attention · p. 6
  • Because similar consecutive sub-chunks can retrieve overlapping tokens, independent selection repeats retrieval work M times per outer chunk.SCALE groups similar consecutive sub-chunks and invokes selection once per cluster.Source: B. Selective Sparse Attention · p. 6

C. Agentic Context Engineering

Agentic Context Engineering adapts an agent through inspectable playbook edits rather than weight updates. The observations report high similarity between consecutive sub-chunks and support query-relevant playbook retrieval and cluster-level selection.

C. Agentic Context Engineering

  • Agentic Context Engineering maintains a structured playbook of natural-language entries and applies validated edits distilled from interactions, leaving model weights unchanged.The framework leaves open how edits are produced and which entries condition each interaction.Source: C. Agentic Context Engineering · p. 7

IV. MOTIVATIONS AND OBSERVATIONS

  • High consecutive-sub-chunk similarity supports clustering; a tunable threshold controls the trade-off between amortization and selection granularity, while cluster-level retrieval closely matches per-sub-chunk top-k indices.These observations motivate cluster-level sparse selection.Source: §IV · p. 7
  • Query-relevant playbook rules are faster and more accurate than the full playbook, and a length-matched random control attributes the gain to query–rule matching.Audited errors also recur across superficially different tasks until the underlying weaknesses are addressed.Source: §IV · p. 7

A. Consecutive Sub-Chunk Similarity within Outer Chunks

B. Cluster Formation Under Tunable Similarity Threshold

Cluster formation uses a similarity threshold to trade retrieval amortization against selection granularity, while cluster-level retrieval retains high overlap with per-sub-chunk selection. Query-aware Playbook retrieval is faster and more accurate than supplying the full Playbook in the Formula evaluation.

B. Cluster Formation Under Tunable Similarity Threshold

  • Consecutive sub-chunks join a cluster when their similarity exceeds θ, with a maximum cluster size of 4096 tokens.The evaluation sweeps θ over 0.95, 0.97, 0.99, and 0.999; higher thresholds produce smaller clusters, especially at middle layers.Source: B. Cluster Formation Under Tunable… · p. 8, Table III · p. 8
  • At θ = 0.99, mean cluster size is 1628 tokens at Layer 20 and 4013 tokens at Layer 0, showing depth-dependent cluster formation.The paper relates the stronger fragmentation at middle layers to their more discriminative semantic representations.Source: Table III · p. 8, B. Cluster Formation Under Tunable… · p. 8
  • At θ = 0.95, 94% of Layer 20 clusters reach the eight-block maximum; at θ = 0.999, 83% revert to single-block selection.The threshold provides a tuning trade-off: lower values maximize amortization, while higher values increase selection granularity.Source: Table IV · p. 8, B. Cluster Formation Under Tunable… · p. 8

C. Top-k Overlap Under Cluster-Level Retrieval

  • 97.9% median top-k overlap at θ = 0.95, rising above 99% at θ = 0.99, indicates that anchor retrieval largely preserves per-sub-chunk selections.The overlap stays above 88% across measured clusters; Table V reports overlap ratio at k = 8192.Source: Table V · p. 9, C. Top-k Overlap Under Cluster-Level… · p. 9

D. Query-Aware Retrieval for Efficient Playbook Execution

  • Semantic retrieval supplies the ten Playbook entries most relevant to the query and context, while the random control supplies ten randomly selected entries.The comparison is evaluation-only, using a fixed, fully curated Formula Playbook rather than the un-adapted Origin Playbook.Source: D. Query-Aware Retrieval for Efficient… · p. 9, Figure 1 · p. 9
  • 79.5% accuracy with semantic top-10 retrieval versus 74.5% with the full Playbook, while mean TTFT falls from 0.392 to 0.068 seconds.The evaluation used a fixed Formula Playbook, the same backbone and decoding configuration, and 200 examples in every condition.Source: Figure 1 · p. 9, D. Query-Aware Retrieval for Efficient… · p. 9
  • Random top-10 retrieval reaches only 71.5% accuracy, so the semantic-retrieval gain is not explained by shorter prompts alone.Random retrieval lowers latency nearly as much as semantic retrieval but removes useful guidance; the full Playbook also carries distractors and higher prefill cost.Source: D. Query-Aware Retrieval for Efficient… · pp. 9–10

E. Systematic and Recurring Agent Failures

Formula-run failures often recur across distinct tasks, motivating proactive failure-memory grounding. The methodology pairs this self-improving agentic layer with Similarity Chunk Rolling, which uses anchor-based clusters to reduce repeated retrieval while limiting drift.

E. Systematic and Recurring Agent Failures

  • 72.2% of verified failure observations are related to at least one later observation, and 55 are related to at least six later observations.In the 500-task offline Formula run, 126 tasks (25.2%) produced a Ground-Truth-verified failure; some underlying weaknesses recur across different surface questions.Source: E. Systematic and Recurring Agent… · p. 10, Figure 2 · p. 10

V. METHODOLOGY

  • SCALE combines an efficient long-context inference engine with an agentic layer that curates a localized educational Playbook without expensive weight updates.The engine keeps long Playbook-conditioned contexts tractable under hardware budgets, while the agentic layer grounds inference in locally accurate knowledge.Source: §V · p. 10

A. Efficient Long-Context Inference Engine

  • Similarity Chunk Rolling groups consecutive sub-chunks with high mutual similarity, selects once per cluster, and computes sparse attention over the cluster in one parallel pass.It reorganizes selection and attention to run once per cluster rather than once per sub-chunk.Source: A. Efficient Long-Context Inference… · p. 10

1) Cluster representation and similarity measure:

  • SCR represents clusters with sub-chunk fingerprints and uses normalized fingerprints to reduce cosine-similarity checks to a dot product.The implementation avoids repeated L2-norm computation through a fused dot-product kernel.Source: 1) Cluster representation and… · p. 10

2) Anchor-based cluster formation:

  • Anchor-based clustering admits each subsequent sub-chunk only if it remains similar to the cluster’s first sub-chunk, preventing cumulative drift.At θ = 0.95 with Lmax = 8L, most clusters reach maximum size and retrieval scoring is reduced by about 8× relative to per-sub-chunk selection.Source: 2) Anchor-based cluster formation · pp. 10–11

3) Cluster-level retrieval and parallel attention:

Cluster-level retrieval uses the first sub-chunk’s fingerprint to select a shared set of critical KV-cache entries, then computes sparse attention for the cluster in parallel. The shared retrieval approximates distant context, while intra-cluster attention remains exact and causally masked.

3) Cluster-level retrieval and parallel attention:

  • The first sub-chunk’s fingerprint serves as the cluster query, producing one top-k index set shared by all cluster members instead of independent retrievals.The admission threshold makes member fingerprints nearly collinear with the anchor, so its token ranking closely matches the rankings members would produce independently.Source: 3) Cluster-level retrieval and… · p. 12
  • The procedure concatenates cluster queries and computes sparse attention in one forward pass with a causal mask.The mask ensures each query attends only to preceding keys; the resulting output is written to the cluster’s output positions.Source: 3) Cluster-level retrieval and… · p. 13
  • The shared retrieval approximates distant global context, while local intra-cluster attention is computed exactly.The gathered keys combine the shared top-k set, attention sinks, and a local window spanning the cluster.Source: 3) Cluster-level retrieval and… · p. 13

B. Self-Improving Agentic Context Engineering

SCALE adapts an agentic Generator–Reflector–Curator loop for tutoring, using a growing curriculum playbook as runtime knowledge while keeping the backbone fixed. Its roles make playbook use and feedback-driven updates traceable.

B. Self-Improving Agentic Context Engineering

1) Agentic Architecture Overview:

  • The loop persists its playbook and optional failure memory across episodes, with RAE, FMB, and adversarial stress testing added as distinct mechanisms.The playbook and memory are described as auditable, editable, and deployable under data-sovereignty constraints.Source: 1) Agentic Architecture Overview · pp. 13–14

2) Generator - Producing Tutoring Trajectories:

  • The Generator returns a tutoring trajectory, answer, and identifiers for used playbook entries, making advice attributable during later reflection.Trajectories can include worked explanations, Socratic hints, or tool-use traces; they are transient evidence, not retained hidden state.Source: 2) Generator - Producing Tutoring… · p. 14

3) Reflector - Extracting Insights from Execution Feedback:

  • The Reflector diagnoses responses against available feedback and tags cited entries, turning instance-level feedback into reusable explanations without exposing the original student record.FMB stores distilled diagnoses of prior failures and supplies analogous cases to the Reflector; it does not replace the execution-guiding playbook.Source: 3) Reflector - Extracting Insights… · pp. 14–15

4) Curator - Incremental Playbook Updates:

The Curator turns reflections into localized playbook edits, while retrieval and periodic refinement bound context growth. An adversarial curriculum adds synthetic stress tests to expose playbook weaknesses.

4) Curator - Incremental Playbook Updates:

  • The Curator converts reflections into typed, localized playbook edits, adding absent rules or updating counters and merging overlaps instead of rewriting the full prompt.Validated operations leave the existing playbook unchanged when malformed output produces an empty delta.Source: 4) Curator - Incremental Playbook… · p. 15

5) Incremental Updates and Context Collapse Prevention:

  • Periodic merging of semantically redundant entries and RAE retrieval limit context growth; localized updates preserve unrelated rules and reduce prompt noise and inference cost.RAE selects relevant playbook bullets for generation, while entry identifiers let the Reflector assign credit or blame to rules actually used.Source: 5) Incremental Updates and Context… · p. 15

6) Adversarial Curriculum:

  • The adversarial curriculum generates one playbook-conditioned challenge and routes mismatches through the usual loop, but does not edit the playbook directly.Because the generated target has no independent verifier, the authors characterize this as active stress testing, not formal evidence of error.Source: 6) Adversarial Curriculum · pp. 15–16

VI. EXPERIMENTS AND RESULTS

The experiments compare SCR with TokenSelect on long-context benchmarks, ablate agentic mechanisms, and evaluate their combined deployed configuration. The supplied long-context results show latency savings while preserving or improving accuracy.

1) Experimental Setup:

  • SCR is compared with TokenSelect on long-context benchmarks, while the agentic layer is ablated separately and then evaluated atop SCR in the deployed configuration.The inference-engine evaluation includes InfiniteBench and RULER; the RULER setup covers eight retrieval tasks across context lengths from 4,096 to 131,072 tokens.Source: §VI · p. 16, A. Long-Context Inference Engine · p. 16, 1) Experimental Setup · p. 16

2) Accuracy and Latency on InfiniteBench:

3) Scaling with Context Length on RULER:

4) Ablation: Effect of Lmax and Threshold θ:

The ablations expose trade-offs in SCR’s cluster settings and test added agentic mechanisms against an ACE baseline using verified supervision. The supplied financial-reasoning results show gains from adversarial curriculum, with FMB favoring FiNER.

4) Ablation: Effect of Lmax and Threshold θ:

  • On R.KV, increasing Lmax from 1024 to 4096 lowers TTFT from 9.9 s to 8.3 s and raises accuracy from 94.0% to 98.0%.Code.D and En.Dia slightly favor Lmax = 1024, so practitioners may reduce cluster size when fine-grained reasoning is paramount.Source: 4) Ablation: Effect of Lmax and… · p. 18
  • Across θ values from 0.95 to 0.99, accuracy varies by only ±2 points with near-identical latency; θ = 0.95 is adopted for more aggressive amortization.Higher thresholds remain available when finer cluster granularity is desired.Source: 4) Ablation: Effect of Lmax and… · p. 18

1) Experimental Setup:

  • Agentic ablations use Ground Truth labels or deterministic outcomes, hold backbone and evaluation settings fixed, and test AppWorld normal and challenge trajectories.The Origin baseline is the ACE Generator–Reflector–Curator loop with the added mechanisms disabled.Source: 1) Experimental Setup · pp. 18–19

3) Interactive Agents: AppWorld:

DeepEdu’s deployed configuration combines the agentic layer with SCR, reporting higher complex-task accuracy and lower AppWorld TTFT. AppWorld ablations show that retaining diagnosed failures matters for durable gains, while the conclusion notes that native-curriculum grounding remains to be evaluated.

3) Interactive Agents: AppWorld:

  • Adding the Failure Memory Bank recovers the best AppWorld average (10.4) and best normal TGC (22.0) after the adversarial curriculum alone leaves the aggregate at 8.9.RAE lifts the Origin average from 9.3 to 10.4; the adversarial curriculum layered on RAE improves only challenge SGC (0.7 →2.2).Source: 3) Interactive Agents: AppWorld · p. 19

C. End-to-End Efficiency: SCR on the Agentic Layer

  • The deployed SCR-plus-agentic configuration is evaluated on AppWorld normal and challenge splits with the same Playbook and Qwen3-4B-Instruct-2507 backbone.The comparison uses ACE served through vLLM as Origin and reports TTFT, TPOT, and aggregate input/output tokens.Source: C. End-to-End Efficiency: SCR on the… · pp. 19–20
  • SCR lowers AppWorld mean TTFT from 11.96 to 5.51 seconds, with 5.75 seconds on Challenge (52.2% reduction; 2.09× faster).Figure 6 reports aggregate inference statistics for the normal and challenge splits; SCR targets prefill, while the underlying TokenSelect selection cache governs decode latency.Source: Figure 6 · p. 20, C. End-to-End Efficiency: SCR on the… · p. 20

VII. CONCLUSION

  • Accuracy on complex tasks rises from 70.0% to 79.5%, while mean TTFT falls by roughly 2× in the deployed configuration.The conclusion reports the strongest per-track gains across finer-grained subtasks and adversarial splits; AppWorld normal is 2.17× faster.Source: §VII · p. 21
  • Experiments validate the enabling mechanisms on established benchmarks, but direct evaluation of localized grounding on native, region-specific curricula remains future work.The conclusion identifies that evaluation as important to the goal of trustworthy AI tutoring in resource-constrained regions.Source: §VII · p. 21

APPENDIX

The appendix reports nine SCR configurations across six InfiniteBench tasks, with representative subsets shown in the main-text tables.

APPENDIX

  • The appendix reports all nine SCR configurations across all six InfiniteBench tasks.The configurations span θ ∈ {0.95, 0.97, 0.99} and Lmax ∈ {1024, 2048, 4096}.Source: APPENDIX · p. 21
  • R.Num and R.PK are saturated at 100% accuracy for every method.Source: APPENDIX · p. 21
  • The representative subsets in main-text Tables VI and VII are drawn from the complete configuration grids.Source: APPENDIX · p. 21
Loading 2609.31568v1…