PaperlayerView exact revision on arXiv

Source-linked AI summary

TRACE-BN: Transferring Bangla-English Tutoring Behavior to a Sub-1B Offline Language Model

Khan Raiyan Ibne Reza, Sanjana Aktar Maria, Mohammad Tushar Abdullah, Asfee Bhuiyan Leen, Sumaiya Tabassum Nimi

arXiv:2608.15223v1cs.CL

TL;DR

Bangla-English tutoring needs structured explanations, learner-error diagnosis, and targeted practice beyond translation alone. TRACE-BN uses curriculum-guided tutoring traces to transfer this behavior to Qwen3-0.6B, raising schema validity from 85.4% to 95.8% and improving all evaluated tutoring dimensions.

  • Problem

    Existing Bangla-focused models are not designed for structured tutoring that combines translation, grammar explanation, learner-error identification, and targeted practice.

  • Method

    TRACE-BN constructs curriculum-guided bilingual tutoring traces and uses them to fine-tune Qwen3-0.6B with LoRA for structured trace generation.

  • Results

    Schema validity increases from 85.4% to 95.8%, while field-level evaluation shows consistent improvements across translation, grammar explanation, learner-error diagnosis, and practice alignment.

  • Takeaways & Limitations

    Curriculum-guided structured supervision can transfer multi-component tutoring behavior to a sub-1B model under resource-constrained offline deployment.

Abstract

from arXiv · show

Bangla-English tutoring requires more than producing a correct translation: learners also need explanations of grammar differences, awareness of their likely errors, and targeted practice. We present TRACE-BN, a curriculum-guided dataset of structured tutoring traces for Bangla-speaking learners of English at the CEFR A1-A2 level. Each trace combines word-level glosses, literal and natural translations, Bangla grammar explanations, a plausible learner error, and a targeted practice question with its answer. The traces are generated by Gemini 3.5 Flash Lite as the teacher model from NCTB Classes 9-10 English curriculum units, then filtered for structural validity, script integrity, and semantic duplication. We transfer the resulting structured tutoring behavior to Qwen3-0.6B using LoRA with 4-bit quantization for resource-constrained offline deployment. On held-out inputs, schema validity increases from 85.4% to 95.8%, while, against teacher-model references, chrF++ improves from 15.28 to 34.77 and BLEU from 4.52 to 21.03. Field-level evaluation by two independent judges shows improvements across translation, grammar explanation, learner-error diagnosis, and practice alignment, while a human audit supports the quality of the supervision data. The results show that curriculum-guided structured supervision can transfer multi-component tutoring behavior to a sub-1B model under these resource constraints. The dataset, model checkpoints, and code are publicly available at https://huggingface.co/datasets/RaiyanKhaan/Trace-BN

I. INTRODUCTION

TRACE-BN frames Bangla-English tutoring as a structured interaction combining translation, Bangla grammar explanation, learner-error diagnosis, and targeted practice. The study introduces a 4,099-trace curriculum-guided dataset and tests whether LoRA fine-tuning transfers this behavior to Qwen3-0.6B.

  • Dataset and task: The structured tutoring formulation combines translation with contrastive grammar explanation, learner-error prediction, and targeted practice in one trace.The formulation is presented as extending beyond the Bangla-English pair studied here.
  • Dataset and task: TRACE-BN contains 4,099 curriculum-guided Bangla-to-English tutoring traces spanning seven structured fields.The fields cover glosses, literal and natural translations, Bangla grammar notes, learner mistakes, practice questions, and answers.
  • Dataset and task: Each trace forms a complete instructional sequence rather than a conventional single input-output translation pair.Learner-facing scaffolding is in Bangla, while target-language content remains in English.
  • Model adaptation and evaluation: 95.8% schema validity was achieved by the tuned model, compared with 85.4% for the untuned model on 432 held-out examples.The model was Qwen3-0.6B fine-tuned with LoRA and evaluated against the untuned base model and three zero-shot baselines.
  • Model adaptation and evaluation: chrF++ increased from 15.28 to 34.77 and BLEU increased from 4.52 to 21.03 after tuning.Field-level evaluation by two independent judges showed consistent improvements across translation and grammar explanation, with human validation of the supervision data.

II. RELATED WORK · A. Bangla NLP and Low-Resource Language Tutoring

TRACE-BN addresses a gap in Bangla and low-resource language tutoring by combining translation, grammar explanation, learner-error prediction, and practice generation in one schema for a sub-1B model. It also responds to limited Bangla training data through teacher-generated traces filtered for structural and semantic quality.

  • A. Bangla NLP and Low-Resource Language Tutoring: Existing Bangla models target general language understanding or generation, while KrishokChat focuses on agricultural advisory rather than integrated language tutoring.The cited systems include TigerLLM, BanglaLlama, TituLLM, and KrishokChat.
  • A. Bangla NLP and Low-Resource Language Tutoring: None of these Bangla systems combine translation, grammar explanation, error prediction, and practice generation in a single tutoring interaction.
  • A. Bangla NLP and Low-Resource Language Tutoring: Other low-resource tutoring systems differ in scale and scope, covering multiturn dialogue, visual question answering, pronunciation practice, or flashcard vocabulary drills.AfriLangTutor uses 8B and 12B models across ten African languages; LEARN, CaptainA, and LangLearn target narrower instructional formats.
  • A. Bangla NLP and Low-Resource Language Tutoring: The TRACE-BN pipeline converts curriculum-guided Bangla inputs into teacher-generated tutoring traces, filters them, and adapts Qwen3-0.6B with LoRA.This summarizes the construction and adaptation pipeline shown in Figure 1.
  • A. Bangla NLP and Low-Resource Language Tutoring: TRACE-BN instead targets a sub-1B model with one schema spanning translation, grammar explanation, and error-aware practice for a single language.
  • A. Bangla NLP and Low-Resource Language Tutoring: A Bangla LLM feasibility study identifies insufficient high-quality pretraining and instruction-tuning data, a gap TRACE-BN targets for downstream tutoring.
  • A. Bangla NLP and Low-Resource Language Tutoring: Every TRACE-BN trace is teacher-generated and filtered for structural validity, script integrity, and semantic duplication before tuning.

B. Small Models and Structured Generation · III. TRACE-BN DATASET CONSTRUCTION

TRACE-BN extends small-model efficiency strategies to structured Bangla-English tutoring with a 0.6B target model, while separating schema reliability from translation and pedagogical quality. Its dataset is curriculum-guided and generated as strictly valid JSON tutoring traces for beginner learners.

  • B. Small Models and Structured Generation: TRACE-BN applies small-model efficiency strategies to structured tutoring with a 0.6B target model.
  • B. Small Models and Structured Generation: Structured generation is evaluated separately for schema validity, translation quality, and pedagogical quality because schema compliance alone may not ensure useful content.
  • III. TRACE-BN DATASET CONSTRUCTION: The dataset is based on the NCTB English curriculum for Classes 9–10, with manually selected and reviewed units, topics, and grammar coverage.
  • III. TRACE-BN DATASET CONSTRUCTION: Each selected unit specifies the target learner level, grammar or language pattern, and an everyday context.
  • III. TRACE-BN DATASET CONSTRUCTION: The specifications cover beginner-relevant Bangla–English differences including question formation, tense and auxiliary use, prepositions, introductory it and there, passive constructions, and idiomatic expressions.
  • III. TRACE-BN DATASET CONSTRUCTION: Gemini 3.5 Flash Lite generates each trace from the curriculum unit, target CEFR level, and Bangla input sentence as strictly valid JSON with seven required fields.The prompt also prohibits free-form text outside the JSON object.

A. Curriculum Coverage and Language Design · B. Trace Filtering and Final Dataset

TRACE-BN covers Bangla-English tutoring across CEFR A1–A2 topics and NCTB grammar units, with Bangla scaffolding and English target-language fields. Three automatic filters reduce 4,450 teacher-generated candidates to 4,099 final traces, which are split for evaluation and receive strong human validation ratings.

  • A. Curriculum Coverage and Language Design: TRACE-BN contains 4,099 examples spanning 15 CEFR A1–A2 topic clusters and 13 NCTB grammar units.Coverage includes everyday topics such as family, school, food, health, weather, directions, shopping, and daily routines, alongside 13 grammar areas.
  • A. Curriculum Coverage and Language Design: Learner-facing grammar notes and practice questions are provided in Bangla, while target-language fields remain in English.
  • B. Trace Filtering and Final Dataset: Three automatic filters enforce seven-field JSON structure, non-empty grammar notes, bounded gloss length, Bangla script integrity, and semantic deduplication.Semantic deduplication uses sentence embeddings from paraphrase-multilingual-MiniLM-L12-v2.
  • B. Trace Filtering and Final Dataset: 351 (7.9%) of 4,450 teacher-generated candidates were removed, leaving 4,099 final traces.Removals comprised 180 structural errors, 45 script-integrity violations, and 126 semantic duplicates; mean source and target lengths were 5.5 and 6.6 words.
  • B. Trace Filtering and Final Dataset: The dataset is partitioned into 3,667 training examples and a 432-example evaluation split organized by topic.
  • B. Trace Filtering and Final Dataset: The teacher-generation prompt frames the task as expert English tutoring for NCTB Class 9–10 exams and requires strictly valid JSON output.
  • B. Trace Filtering and Final Dataset: The evaluation split includes held-out domains such as weather conditions and idiomatic expressions, with no source-sentence overlap or detected lexical leakage.
  • B. Trace Filtering and Final Dataset: Human validation of 100 randomly sampled traces produced mean ratings of 4.81 for translation quality, 4.81 for grammar explanations, 4.79 for learner-mistake plausibility, 4.44 for practice alignment, and 4.71 overall.Three bilingual educators rated the traces on five pedagogical dimensions using a 1–5 Likert scale; 95.7% received ratings of at least 4.

IV. MODEL ADAPTATION AND EVALUATION · A. LoRA Fine-Tuning

TRACE-BN adapts Qwen3-0.6B with LoRA and 4-bit quantization to generate complete seven-field tutoring traces from Bangla sentences and schema specifications. Training uses a fixed three-epoch schedule on a single 16 GB Google Colab T4 GPU.

  • A. LoRA Fine-Tuning: Qwen3-0.6B is fine-tuned with LoRA and 4-bit quantization through Unsloth on a single 16 GB Google Colab T4 GPU.
  • A. LoRA Fine-Tuning: The training input combines a Bangla sentence with the target schema specification.
  • A. LoRA Fine-Tuning: The model is trained to generate the complete seven-field tutoring trace as a single response.
  • A. LoRA Fine-Tuning: r = 16, α = 32, and zero dropout define the LoRA configuration.
  • A. LoRA Fine-Tuning: The adaptation targets the q, k, v, o, gate, up, and down projection layers.
  • A. LoRA Fine-Tuning: Three epochs comprise 621 optimizer steps, using a cosine learning-rate schedule peaking at 2 × 10−4.
  • A. LoRA Fine-Tuning: Loss is logged at 51-step intervals, and the final-step checkpoint is evaluated on the 432-example split.

B. Baseline Models · C. Evaluation

The study compares TRACE-BN-tuned Qwen3-0.6B with its untuned base and three zero-shot baselines on 432 held-out Bangla inputs, using schema, translation, and field-level tutoring evaluations. The evaluation measures agreement with teacher references and granular instructional quality across seven tutoring dimensions.

  • B. Baseline Models: The comparison includes tuned Qwen3-0.6B, its untuned base, Gemma-4 E2B (2.3B), TigerLLM-1B, and Llama-3.2-1B-Instruct on 432 held-out Bangla inputs.All models receive the same task instruction formatted through their native chat templates.
  • B. Baseline Models: Qualitatively, the base model produces literal substitutions or degenerative loops, while TRACE-BN-tuned Qwen3-0.6B produces fluent translations with contrastive Bangla grammar explanations.The comparison is illustrated on representative held-out Bangla inputs.
  • B. Baseline Models: Training loss decreases steadily across three epochs and 621 steps, stabilizing near completion.The comparison models are not fine-tuned.
  • B. Baseline Models: BanglaLlama-3.2-3B was excluded from the full comparison after producing 0% schema-valid outputs on a stratified sample of 18 inputs.Its outputs could not be scored in the task’s evaluation format.
  • C. Evaluation: Schema validity requires valid JSON containing all seven fields, including a parseable English natural_translation field.BLEU and chrF++ use extracted natural_translation strings when parseable and score missing valid translation fields as empty strings.
  • C. Evaluation: BLEU and chrF++ measure agreement with teacher-model references because the teacher generates both training traces and silver references.The metrics are computed against reference natural_translation strings under the stated parsing rules.
  • C. Evaluation: Two independent automated judges rate seven tutoring dimensions on a 0–4 anchored ordinal scale, with 4 fully correct and 0 unusable.The dimensions cover glosses, literal and natural translation, grammar explanation, learner-error diagnosis, practice alignment, and overall tutoring quality.
  • C. Evaluation: The two judges score the same 432 held-out outputs using mean scores, the proportion scoring at least 3, and quadratic weighted Cohen’s κ.Paired bootstrap resampling with 1,000 iterations supplies p-values for the main metric comparisons.

V. RESULTS

The results section compares the tuned model with untuned and zero-shot baselines, then evaluates pedagogical quality, component contributions, qualitative behavior, failure modes, and structural reliability.

  • The section first compares the tuned model with untuned and zero-shot baselines.
  • It then examines pedagogical quality and failure modes.
  • Additional analyses assess individual trace components, qualitative behavior, and structural reliability.

A. Overall Performance

TRACE-BN tuning improves Qwen3-0.6B’s schema validity and reference-based translation scores on held-out inputs. The model also maintains high inference throughput for offline, on-device deployment.

  • Benchmark performance: 95.8% schema validity rises from 85.4%, while chrF++ increases from 15.28 to 34.77 and BLEU from 4.52 to 21.03.These results are measured on a shared held-out set of n = 432, using teacher-model outputs as silver translation references.
  • Benchmark performance: 50.91 chrF++ and 33.54 BLEU make Gemma-4 E2B the strongest translation baseline.The tuned model’s reported translation scores remain below this baseline on the same comparison.
  • Benchmark performance: +19.49 chrF++ over the untuned Qwen3-0.6B is statistically significant (paired bootstrap, 1,000 resamples, p < 0.05).The reported gain is also described as significant using paired bootstrap testing.
  • Deployment efficiency: 75.47 tokens/s exceeds Gemma-4 E2B’s 45.97 tokens/s while remaining below the untuned base model’s 90.89 tokens/s.This throughput is reported as consistent with the model’s targeted offline, on-device deployment, despite Gemma-4 E2B’s 2.3B parameters.

B. Pedagogical Quality and Failure Analysis · C. Trace Component Ablation

TRACE-BN fine-tuning improves all evaluated tutoring dimensions beyond schema formatting, while ablations show that richer structured supervision benefits translation quality. Remaining errors are localized mainly to idioms and fine-grained gloss decisions.

  • B. Pedagogical Quality and Failure Analysis: TRACE-BN fine-tuning improves every evaluated tutoring dimension relative to untuned Qwen3-0.6B.The largest gains are word-level gloss accuracy (+1.38), learner-error diagnosis (+1.24), grammar explanation accuracy (+1.21), and literal translation accuracy (+1.12).
  • C. Trace Component Ablation: The ablation evaluates three supervision configurations on Qwen3-0.6B with n = 432.The configurations are reported in Fig. 6(c) and Table IV.
  • B. Pedagogical Quality and Failure Analysis: Natural translation quality rises from 0.26 to 1.31, while overall tutoring quality rises from 0.13 to 1.14 on the 0–4 scale.The consistent improvement across fields suggests that fine-tuning extends beyond output formatting, although absolute scores remain below the rubric’s upper end.
  • B. Pedagogical Quality and Failure Analysis: Inter-judge agreement is substantial for glosses, literal and natural translation, and overall tutoring quality, but moderate for grammar explanation, error diagnosis, and practice alignment.Agreement values range from κ = 0.700 to κ = 0.803 for substantial dimensions and from κ = 0.489 to κ = 0.585 for moderate dimensions.
  • B. Pedagogical Quality and Failure Analysis: Remaining errors concentrate mainly in held-out idiomatic expressions and fine-grained word-level gloss decisions, rather than schema structure.These localized failures are distinct from structural failures captured by schema validity.
  • C. Trace Component Ablation: 31.84 chrF++ is reached by translation-only training, increasing to 33.12 with Bangla explanations and 34.77 with the full seven-field TRACE-BN configuration.The results suggest richer structured supervision benefits the translation component itself, not only auxiliary output fields.

D. Qualitative Behavior · E. Structured Reliability · VI. CONCLUSION

TRACE-BN transfers structured Bangla-to-English tutoring behavior to Qwen3-0.6B, combining fluent, targeted explanations with high schema reliability. The tuned model achieves 95.8% schema validity, improves chrF++ by 19.49 points over the base model, and shows consistent field-level gains supported by human auditing.

  • D. Qualitative Behavior: The tuned model produces fluent translations with targeted grammar notes, including correctly identifying auxiliary do insertion in English question formation.It avoids translating the Bangla question marker literally.
  • E. Structured Reliability: 95.8% schema validity corresponds to 18 invalid outputs out of 432, compared with 63 invalid outputs for the base model.The result concerns seven-field tutoring traces.
  • E. Structured Reliability: Matching outputs under llama.cpp local inference were confirmed after exporting the merged adapter to GGUF format.This verifies transfer to the target offline deployment setting.
  • VI. CONCLUSION: TRACE-BN is a curriculum-guided dataset for structured Bangla-to-English tutoring.The dataset supports generation of seven-field tutoring traces.
  • VI. CONCLUSION: +19.49 chrF++ improvement over the base model was achieved by Qwen3-0.6B while reaching 95.8% schema validity.The conclusion attributes these results to learning structured tutoring traces.
  • VI. CONCLUSION: Field-level evaluation by two independent judges showed consistent improvement across all evaluated tutoring dimensions.The judges demonstrated moderate-to-substantial inter-judge agreement.
  • VI. CONCLUSION: A 100-trace human audit supported the quality of the supervision data.The audit complements the field-level evaluation of tutoring behavior.
Loading 2608.15223v1…