PaperlayerView exact revision on arXiv

Source-linked AI summary

TRACE-BN: Transferring Bangla-English Tutoring Behavior to a Sub-1B Offline Language Model

Khan Raiyan Ibne Reza, Sanjana Aktar Maria, Mohammad Tushar Abdullah, Asfee Bhuiyan Leen, Sumaiya Tabassum Nimi

arXiv:2608.15223v1cs.CL

TL;DR

Bangla-English learners need tutoring that goes beyond translation, but existing Bangla-focused models are not designed for structured instructional interaction. TRACE-BN pairs curriculum-guided tutoring traces with Qwen3-0.6B fine-tuning, improving schema validity, reference-based translation scores, and all evaluated tutoring dimensions.

  • Problem

    Existing Bangla-focused models are not designed for structured tutoring interactions combining translation, grammar explanation, learner-error identification, and targeted practice.

    Source: §I · p. 1

  • Method

    TRACE-BN constructs 4,099 curriculum-guided Bangla-to-English tutoring traces across seven fields and fine-tunes Qwen3-0.6B to generate their structure.

    Source: §I · p. 1, §III · p. 2

  • Results

    85.4% to 95.8% schema validity and 15.28 to 34.77 chrF++ accompanied gains across every evaluated tutoring dimension over untuned Qwen3-0.6B.

    Source: §I · p. 1, B. Pedagogical Quality and Failure… · p. 5

  • Takeaways & Limitations

    Curriculum-guided structured supervision can transfer tutoring structure and content to a sub-1B model under these resource constraints.

    Source: §VI · p. 6

Abstract

from arXiv · show

Bangla-English tutoring requires more than producing a correct translation: learners also need explanations of grammar differences, awareness of their likely errors, and targeted practice. We present TRACE-BN, a curriculum-guided dataset of structured tutoring traces for Bangla-speaking learners of English at the CEFR A1-A2 level. Each trace combines word-level glosses, literal and natural translations, Bangla grammar explanations, a plausible learner error, and a targeted practice question with its answer. The traces are generated by Gemini 3.5 Flash Lite as the teacher model from NCTB Classes 9-10 English curriculum units, then filtered for structural validity, script integrity, and semantic duplication. We transfer the resulting structured tutoring behavior to Qwen3-0.6B using LoRA with 4-bit quantization for resource-constrained offline deployment. On held-out inputs, schema validity increases from 85.4% to 95.8%, while, against teacher-model references, chrF++ improves from 15.28 to 34.77 and BLEU from 4.52 to 21.03. Field-level evaluation by two independent judges shows improvements across translation, grammar explanation, learner-error diagnosis, and practice alignment, while a human audit supports the quality of the supervision data. The results show that curriculum-guided structured supervision can transfer multi-component tutoring behavior to a sub-1B model under these resource constraints. The dataset, model checkpoints, and code are publicly available at https://huggingface.co/datasets/RaiyanKhaan/Trace-BN

I. INTRODUCTION

TRACE-BN frames Bangla-English tutoring as a structured instructional sequence and tests whether curriculum-guided traces can transfer that behavior to Qwen3-0.6B.

I. INTRODUCTION

  • TRACE-BN provides 4,099 curriculum-guided traces, each encoding a seven-field instructional sequence rather than a single input-output pair.The fields cover word-level glosses, literal and natural translations, Bangla grammar notes, a learner mistake, a practice question, and its answer; scaffolding is in Bangla and target-language content in English.Source: §I · p. 1
  • On 432 held-out examples, tuning raised schema validity from 85.4% to 95.8%, chrF++ from 15.28 to 34.77, and BLEU from 4.52 to 21.03.Two independent judges also found consistent improvements in translation, grammar explanation, learner-error diagnosis, and practice alignment.Source: §I · p. 1
  • The task combines translation with grammatical contrast, likely-error prediction, and targeted practice in one response, extending beyond the Bangla-English pair studied here.The introduction notes that existing Bangla-focused models are not designed around this structured tutoring interaction.Source: §I · p. 1
  • The adaptation study fine-tunes Qwen3-0.6B with LoRA and includes field-level evaluation and human validation of the supervision data.Evaluation compares the tuned model with the untuned base model and three zero-shot baselines.Source: §I · p. 1

II. RELATED WORK · A. Bangla NLP and Low-Resource Language Tutoring

Prior Bangla language models and low-resource tutoring systems address general language tasks or narrower learning needs, while TRACE-BN combines translation, grammar explanation, error prediction, and practice for Bangla-speaking English learners. It uses filtered teacher-generated traces to adapt a sub-1B model.

A. Bangla NLP and Low-Resource Language Tutoring

  • Existing Bangla models target general language understanding or generation, and KrishokChat focuses on agricultural advisory; none combines translation, grammar explanation, error prediction, and practice in one tutoring interaction –, [10].Source: A. Bangla NLP and Low-Resource… · p. 1
  • Fig. 1 shows curriculum-guided Bangla inputs becoming teacher-generated traces, which are filtered into TRACE-BN and used to adapt Qwen3-0.6B with LoRA.Source: Figure 1 · p. 2
  • AfriLangTutor, LEARN, CaptainA, and LangLearn address other language-learning needs, including multiturn dialogue, oral proficiency, pronunciation practice, and vocabulary drills [5], –.AfriLangTutor uses 8B and 12B models across ten African languages.Source: A. Bangla NLP and Low-Resource… · pp. 1–2
  • TRACE-BN instead targets a sub-1B model and one schema for translation, grammar explanation, and error-aware practice for a single language pair.Source: A. Bangla NLP and Low-Resource… · p. 2
  • A feasibility study identifies a need for Bangla LLMs but a shortage of high-quality pretraining and instruction-tuning data.Source: A. Bangla NLP and Low-Resource… · p. 2
  • TRACE-BN addresses that data gap for one downstream use with teacher-generated traces filtered for structural validity, script integrity, and semantic duplication.Source: A. Bangla NLP and Low-Resource… · p. 2

B. Small Models and Structured Generation · III. TRACE-BN DATASET CONSTRUCTION

TRACE-BN defines curriculum-scoped Bangla–English tutoring traces and applies small-model efficiency strategies to structured tutoring with a 0.6B target model. Its construction uses manually reviewed NCTB Classes 9–10 units and prompts Gemini 3.5 Flash Lite to generate strictly valid JSON traces.

B. Small Models and Structured Generation

  • TRACE-BN applies small-model efficiency strategies to structured tutoring with a 0.6B target model.Prior approaches KD-LoRA, DistilQwen2.5, LoRA-Gen, and PhoneLM each specialize or deploy smaller language models through a different efficiency strategy [14]–[17].Source: B. Small Models and Structured… · p. 2
  • Schema compliance alone does not establish correct or useful content, so the evaluation separates structural validity from translation and pedagogical quality.Prior work examined schema adherence and how format constraints affect language-model quality –[22].Source: B. Small Models and Structured… · p. 2

III. TRACE-BN DATASET CONSTRUCTION

  • TRACE-BN is based on manually selected and reviewed NCTB English curriculum units, topics, and grammar coverage for Classes 9–10.Fig. 2 shows a complete trace from the resulting dataset.Source: §III · p. 2
  • For each selected unit, specifications set the target learner level, grammar or language pattern, and an everyday context.They address beginner-relevant Bangla–English differences, including question formation, tense and auxiliary use, prepositions, introductory it and there, passive constructions, and idiomatic expressions.Source: §III · p. 2
  • Gemini 3.5 Flash Lite was prompted as the teacher model with the curriculum unit, target CEFR level, and Bangla input sentence to generate a strictly valid JSON trace.The prompt required seven fields and no free-form text outside the JSON object, as shown in Fig. 3.Source: §III · p. 2

A. Curriculum Coverage and Language Design · B. Trace Filtering and Final Dataset

TRACE-BN is a curriculum-spanning dataset of Bangla-English tutoring traces, filtered for quality and divided into training and evaluation sets. Educator ratings support the quality of its supervision data.

A. Curriculum Coverage and Language Design

  • The dataset covers 15 CEFR A1–A2 topic clusters and 13 NCTB grammar units.Topics include family, school, food, health, weather, directions, shopping, and daily routines; grammar includes passive voice, tenses, pronouns, prepositions, modals, tag questions, conditionals, sentence transformation, indirect narration, and introductory it and there.Source: A. Curriculum Coverage and Language… · p. 2
  • Bangla provides learner-facing grammar notes and practice questions, while target-language fields remain in English.Each trace includes word-level glosses, literal and natural English, a contrastive explanation, a plausible learner error, and a practice exercise with its answer.Source: A. Curriculum Coverage and Language… · p. 2, Figure 2 · p. 3

B. Trace Filtering and Final Dataset

  • Three automatic filters check JSON structure and field lengths, Bangla script integrity, and semantic duplicates within topic clusters.Deduplication uses paraphrase-multilingual-MiniLM-L12-v2 sentence embeddings and removes pairs with cosine similarity > 0.92.Source: B. Trace Filtering and Final Dataset · p. 2
  • 4,099 traces remained from 4,450 candidates after 351 removals: 180 structural errors, 45 script-integrity violations, and 126 semantic duplicates.Mean Bangla source and English target lengths were 5.5 and 6.6 words, respectively.Source: B. Trace Filtering and Final Dataset · p. 2
  • The partition contains 3,667 training examples and a 432-example evaluation split organized by topic.Held-out domains include weather conditions and idiomatic expressions; the passage reports no source-sentence overlap or detected lexical leakage between evaluation and training sets.Source: B. Trace Filtering and Final Dataset · p. 2, Figure 3 · p. 3
  • 4.71 ± 0.59 overall, with 95.7% rated ≥4, summarizes educators’ audit of TRACE-BN supervision; no factual or grammatical error reached consensus.Across five dimensions, mean ratings were 4.81 ± 0.58 for translation quality, 4.81 ± 0.62 for grammar explanations, 4.79 ± 0.68 for learner-mistake plausibility, and 4.44 ± 1.17 for practice-question alignment.Source: B. Trace Filtering and Final Dataset · p. 3

IV. MODEL ADAPTATION AND EVALUATION · A. LoRA Fine-Tuning

TRACE-BN tutoring traces are transferred to Qwen3-0.6B through quantized LoRA fine-tuning, with the final checkpoint evaluated on a held-out split.

A. LoRA Fine-Tuning

  • Qwen3-0.6B is fine-tuned with LoRA and 4-bit quantization on a single 16 GB Colab T4 to generate complete seven-field tutoring traces.The input combines a Bangla sentence with the target schema specification; training uses Unsloth.Source: A. LoRA Fine-Tuning · p. 3
  • LoRA uses rank r = 16, α = 32, zero dropout, and targets the q, k, v, o, gate, up, and down projection layers.Training runs for three epochs (621 optimizer steps) with a cosine learning-rate schedule peaking at 2 × 10−4.Source: A. LoRA Fine-Tuning · p. 3
  • The final-step checkpoint is evaluated on the 432-example split without hyperparameter tuning or checkpoint cherry-picking on evaluation data.Loss is logged every 51 steps (Fig. 5).Source: A. LoRA Fine-Tuning · p. 3

B. Baseline Models · C. Evaluation

The paper compares tuned Qwen3-0.6B with its untuned base and zero-shot baselines on 432 held-out Bangla inputs. Evaluation combines teacher-reference text metrics with judge-rated tutoring fields and paired bootstrap tests.

B. Baseline Models

  • The comparison includes tuned and untuned Qwen3-0.6B plus three zero-shot baselines: Gemma-4 E2B, TigerLLM-1B, and Llama-3.2-1B-Instruct.All receive the same task instruction, formatted with each model family’s native chat template, and are evaluated on the same 432 held-out Bangla inputs.Source: B. Baseline Models · p. 3
  • Qualitative examples contrast the base model’s literal substitutions or degenerative loops with the tuned model’s fluent translations and contrastive Bangla grammar explanations.Source: Figure 4 · p. 4

C. Evaluation

  • Schema validity requires valid JSON with all seven fields, including a parseable English natural_translation field.BLEU and chrF++ use extracted natural_translation strings when parseable; outputs without a valid translation field are scored as empty strings against the reference.Source: C. Evaluation · p. 4
  • The text metrics measure agreement with teacher-model references, not independently adjudicated translation quality.The teacher model generated both the training traces and the silver references.Source: C. Evaluation · p. 4
  • Two independent automated judges assess tutoring behavior on the held-out split against the corresponding TRACE-BN reference trace.The judges are GPT-4o-mini and DeepSeek-V4-Flash-0731; each receives the Bangla source sentence, reference trace, and generated response.Source: C. Evaluation · p. 4
  • The judges rate seven dimensions on a 0–4 anchored ordinal scale, from unusable output to fully correct.Dimensions include gloss, literal and natural translation, grammar explanation, learner-error diagnosis, practice alignment, and overall tutoring quality.Source: C. Evaluation · p. 4
  • The two judges score the same 432 held-out outputs, with mean field scores, score-at-least-3 proportions, and quadratic weighted Cohen’s κ reported.The field-level analysis distinguishes local errors from failures affecting the tutoring response more broadly.Source: C. Evaluation · p. 4
  • Paired bootstrap resampling with 1,000 iterations is used for metric comparisons, with p-values reported for the main comparisons.Source: C. Evaluation · p. 4

V. RESULTS

The results compare the tuned model with untuned and zero-shot baselines, then assess pedagogical quality, failure modes, trace-component contributions, qualitative behavior, and structural reliability.

V. RESULTS

  • The results cover baseline comparisons, pedagogical quality and failure modes, trace-component contributions, qualitative behavior, and structural reliability.The section first compares the tuned model with untuned and zero-shot baselines.Source: §V · p. 4

A. Overall Performance

TRACE-BN tuning improves Qwen3-0.6B’s schema validity and teacher-reference translation scores over the untuned model. The tuned model also delivers inference throughput above Gemma-4 E2B while targeting offline, on-device deployment.

A. Overall Performance

  • 95.8% schema validity (vs 85.4%), 34.77 chrF++ (vs 15.28), and 21.03 BLEU (vs 4.52) were achieved by tuned Qwen3-0.6B over untuned Qwen3-0.6B.The chrF++ gain was +19.49 over the base model (p < 0.05); Gemma-4 E2B remained the strongest translation baseline, with 50.91 chrF++ and 33.54 BLEU.Source: A. Overall Performance · pp. 4–5, Figure 6 · p. 5, Table II · p. 5
  • The model reaches 75.47 tokens/s, comparable to the untuned base model at 90.89 tokens/s and above Gemma-4 E2B at 45.97 tokens/s despite Gemma-4 E2B’s 2.3B parameters.This is consistent with the offline, on-device deployment the model targets.Source: A. Overall Performance · p. 5

B. Pedagogical Quality and Failure Analysis · C. Trace Component Ablation

TRACE-BN fine-tuning improves evaluated tutoring dimensions over untuned Qwen3-0.6B, and ablation results show higher chrF++ with richer trace supervision. The remaining errors are concentrated in idioms and fine-grained gloss decisions.

B. Pedagogical Quality and Failure Analysis

  • The largest gains were word-level gloss accuracy (+1.38), learner-error diagnosis (+1.24), grammar explanation accuracy (+1.21), and literal translation accuracy (+1.12).Source: B. Pedagogical Quality and Failure… · p. 5
  • TRACE-BN fine-tuning improved every evaluated tutoring dimension; natural translation rose from 0.26 to 1.31 and overall tutoring quality from 0.13 to 1.14 on the 0–4 scale.The consistent improvement extends beyond output formatting.Source: B. Pedagogical Quality and Failure… · p. 5
  • Agreement was κ = 0.788 for glosses, 0.754 for literal translation, 0.803 for natural translation, 0.700 for overall quality, 0.489 for grammar, 0.585 for diagnosis, and 0.529 for practice alignment.The authors frame field-level automated evaluation as a scalable complement to human validation, not a substitute for learner studies.Source: B. Pedagogical Quality and Failure… · p. 5
  • Remaining errors were concentrated in held-out idiomatic expressions and fine-grained word-level gloss decisions, distinct from schema-validity failures.Source: B. Pedagogical Quality and Failure… · p. 5

C. Trace Component Ablation

  • Translation-only training reached 31.84 chrF++, adding Bangla explanations reached 33.12, and the full seven-field TRACE-BN configuration reached 34.77.Source: C. Trace Component Ablation · p. 5
  • Richer structured supervision benefited the translation component itself, rather than only adding auxiliary output fields.Source: C. Trace Component Ablation · p. 5

D. Qualitative Behavior · E. Structured Reliability · VI. CONCLUSION

TRACE-BN transfers structured Bangla-to-English tutoring behavior to Qwen3-0.6B, with improved schema validity and tutoring quality. Qualitative examples, deployment checks, judge evaluations, and a human audit support the reported behavior, while learner impact remains to be tested.

D. Qualitative Behavior

  • The tuned model produces fluent translations with targeted grammar notes, correctly explaining auxiliary do insertion in English question formation.It avoids translating the Bangla question marker literally; the passage points to Fig. 4.Source: D. Qualitative Behavior · p. 5

E. Structured Reliability

  • 95.8% schema validity corresponds to 18 invalid outputs among 432, compared with 63 for the base model; GGUF export matched llama.cpp local-inference outputs.Source: E. Structured Reliability · p. 6

VI. CONCLUSION

  • 95.8% schema validity and a +19.49 chrF++ improvement over the base model summarize TRACE-BN’s reported gains on Qwen3-0.6B.The conclusion describes the learned traces as seven-field tutoring outputs and reports consistent improvement across evaluated tutoring dimensions.Source: §VI · p. 6
  • Two independent judges found consistent improvements across evaluated tutoring dimensions with moderate-to-substantial inter-judge agreement, and a 100-trace human audit supported supervision quality.Source: §VI · p. 6
  • TRACE-BN achieved higher schema validity and throughput than the larger Gemma-4 E2B zero-shot baseline, but remained behind it on translation quality.Source: §VI · p. 6
  • The supervision strategy may extend to other language pairs and curricula, but whether such tutors improve learners’ English proficiency requires controlled learner studies.Source: §VI · p. 6
Loading 2608.15223v1…