PaperlayerView exact revision on arXiv

Source-linked AI summary

KIRA: A Full-Stack Reading Screening System with Dual-Pathway Analysis for Children

Maggie Dong, Rohit Mekkoth, Zixuan Wang, Cheol Jun Cho, Jiachen Lian, Gopala Anumanchipalli

arXiv:2610.28840v1cs.CLcs.LG

TL;DR

Early-grade reading screening needs recognition that works on children’s speech and assessment that captures more than word-level correctness. KIRA adapts word and phoneme recognizers and integrates them into WRE and ORF with a teacher-facing report. Fine-tuning lowers the reported phoneme error rate, while the deployed system remains validated at the component level.

  • Problem

    Adult-oriented recognition transfers poorly to children’s speech, and word-level scoring can miss intervention-relevant pronunciation errors.

    Source: §I · p. 1

  • Method

    KIRA fine-tunes word- and phoneme-level recognizers on children’s speech and integrates them in WRE and ORF within a teacher-facing screening pipeline.

    Source: §I · p. 1, §III · p. 2

  • Results

    23.00% PER after child-speech fine-tuning, compared with 41.66% PER for base HuPER; Parakeet-TDT-1.1B achieved 8.84% WER on MyST.

    Source: B. Phoneme Recognition · p. 3, A. Word ASR · p. 3

  • Takeaways & Limitations

    The web interface presents screening scores with per-word phoneme details, allowing proctors to inspect labeled errors beneath top-level results.

    Source: F. User Interface · p. 3

    Validation is at the component level; phoneme-tier benchmarking against expert labels, report comparisons, and classroom diagnostic-utility evaluation remain future work.

    Source: §V · p. 3

Abstract

from arXiv · show

Early-grade reading screening is critical for identifying children at risk of reading difficulty. Existing AI-based screeners are often limited in the scope of their analyses and evaluation metrics, or are proprietary systems that lack transparent model specifications and evaluation. We present KIRA, a dual-pathway screening system that pairs word-level fluency scoring with detailed phoneme-level error detection. We adapt pretrained word and phoneme recognizers to children's speech, reducing word error rate from 13.88% to 8.84% and phoneme error rate from 41.66% to 23.00%. Powered by these child-adapted models, we implement Word Reading (WRE) and Oral Reading Fluency (ORF), two foundational tasks in K-2 screening. The system integrates these recognizers with a web-based teacher-facing interface presenting fluency scores alongside phoneme feedback for potential targeted intervention. KIRA brings modern child-speech recognition into a full-stack, deployable, classroom-ready reading screener.

I. INTRODUCTION

Early-grade screening matters, but child-speech recognition and word-only scoring leave gaps in accuracy and intervention-relevant error detail. KIRA combines child-adapted word and phoneme recognition in WRE and ORF tasks, with a teacher-facing report.

I. INTRODUCTION

  • Foundational reading proficiency is a global concern, with an estimated two-thirds of ten-year-olds unable to read and understand a simple written story.Governments worldwide have promoted early-grade screening because early identification is essential for intervention.Source: §I · p. 1
  • Adult-trained recognition models transfer poorly to children’s speech, while prior work often isolates recognition or scoring instead of integrating capture and reporting.These limitations leave accuracy below screening needs and limit full-system coverage.Source: §I · p. 1
  • Word-level scoring can mark a word correct despite a substituted vowel, obscuring pronunciation patterns relevant to intervention.Meaningful screening therefore requires fluency scoring alongside sub-word error analysis [11] [13].Source: §I · p. 1
  • KIRA integrates word-level fluency scoring and parallel phoneme-level error attribution in a teacher-facing pipeline for WRE and ORF.WRE assesses isolated-word decoding, while ORF assesses connected-text reading fluency; both are widely used in K–2 screening batteries.Source: §I · p. 1

II. RELATED WORK

Prior research includes word-level assessment, phoneme-level analysis, and classroom-scale commercial systems, but KIRA situates itself at their intersection. Its work adapts modern word and phoneme recognizers to children’s speech.

II. RELATED WORK

  • Commercial systems have reached classroom scale, but undisclosed architectures, training, and benchmarks limit reproducibility and independent evaluation.The authors also note a risk of misidentifying at-risk readers.Source: §II · p. 1
  • Academic reading-assessment systems have separately emphasized word-level scoring or phoneme-level analysis.K-Function pairs phoneme transcription with LLM-driven scoring for children’s English, while Dudek et al. present a Polish-speaking children’s phoneme-level screening pipeline.Source: §II · p. 1
  • KIRA builds on child-speech adaptation research by fine-tuning modern word- and phoneme-level recognizers on children’s speech [9] [10] [19].Source: §II · p. 1

III. SYSTEM DESCRIPTION

KIRA organizes reading assessment as a modular pipeline that processes audio through complementary word- and phoneme-level recognition, scoring, and reporting. Its recognizers are adapted to children’s speech and deployed differently across ORF and WRE.

III. SYSTEM DESCRIPTION

  • KIRA’s application, recognition, scoring, and shared-report stages support ORF, which reads connected text, and WRE, which reads isolated words.Audio preprocessing includes a quality gate; word-level tasks additionally use Silero VAD to segment speech regions.Source: §III · p. 2

A. Recognition Modules

  • Both recognizers are fine-tuned on 20,632 MyST recordings; ORF uses both models on one recording, while WRE uses only phoneme recognition.The word model is Parakeet-TDT-1.1B; HuPER is trained on phonemes generated from MyST word transcripts and adapted in two stages, with the low-level feature extractor frozen in the second.Source: A. Recognition Modules · p. 2

B. Word Reading Pipeline

The WRE pipeline handles short recordings that may mix target and non-target speech, using voiced-region segmentation and alignment to assess pronunciation variants. It accommodates sound-outs and surfaces phoneme-level interface feedback.

B. Word Reading Pipeline

  • WRE uses voiced-region segmentation and target-word alignment to handle short recordings that may include proctor speech, sound-outs, or multiple attempts.The protocol limits proctor prompts to phrases without the target word, and non-target audio receives low-scoring alignments.Source: B. Word Reading Pipeline · p. 2
  • The WRE interface presents PER and PFER with optional LLM feedback, while expanded task views show heard phonemes and labeled errors.Source: Figure 2 · p. 2
  • Target-word pronunciation variants are assessed with PanPhon-based feature-weighted edit distance, so substitution costs reflect articulatory-feature distance.The curated list contains 170 K–2 sight words; 165 have one canonical pronunciation and five common function words have a reduced-form alternative.Source: B. Word Reading Pipeline · p. 2
  • Two alignments per voiced region—full-attempt and best-subsequence—accommodate mixed audio and disfluent sound-outs before candidates are classified and scored.The most robust candidate is compared against the closest pronunciation variant.Source: B. Word Reading Pipeline · p. 2

C. Oral Reading Fluency Pipeline

The ORF pipeline analyzes a child reading a passage in parallel at word and phoneme levels. It scores reading accuracy and provides per-word diagnostic feedback while gating phoneme results by coverage.

C. Oral Reading Fluency Pipeline

  • The two parallel analyses provide word-accuracy scoring and per-word phoneme diagnostic feedback for passage reading.Flagged words expand in the ORF report to show expected and heard phonemes with labeled errors.Source: C. Oral Reading Fluency Pipeline · pp. 2–3
  • Word-level analysis aligns the transcript to the reference, labels words correct, missing, or extra, and collapses adjacent missing/extra pairs into a wrong-word label.The count of correctly read reference words gives the word-accuracy score.Source: C. Oral Reading Fluency Pipeline · p. 2
  • Phoneme-level analysis decodes a passage-wide phoneme sequence, aligns it to reference phonemes, and attributes substitutions, deletions, and insertions to reference words.The model makes one forward pass, producing per-frame phoneme distributions at 20 ms resolution.Source: C. Oral Reading Fluency Pipeline · p. 2
  • Each word is gated by phoneme coverage; coverage below 0.60 marks it low-confidence and excludes it, while a word with no decoded phonemes is marked omitted.Coverage measures the fraction of reference phonemes decoded, regardless of correctness, so acoustic dropout is not counted as a reading error.Source: C. Oral Reading Fluency Pipeline · p. 3

D. Scoring

The scoring section distinguishes phoneme error types and reports task-specific reading measures for word and paragraph reading.

D. Scoring

  • Phoneme-level errors are labeled as substitutions, deletions, or insertions.A substitution replaces a reference phoneme, a deletion omits one, and an insertion adds an extra phoneme.Source: D. Scoring · p. 3
  • Word Reading reports per-word phoneme error rate and phonetic feature error rate (PFER).Source: D. Scoring · p. 3
  • Paragraph reading reports the number of words read correctly out of the passage total.Source: D. Scoring · p. 3

E. Report Generation

The report combines session-level task metrics with per-word phoneme details. An optional, privacy-permitting LLM layer can turn scored word results into plain-language narrative summaries for teachers.

E. Report Generation

  • Each session report includes a per-task metric summary and per-word reference and produced phoneme sequences with labeled substitutions, deletions, and insertions.Source: E. Report Generation · p. 3
  • A proctor may enable an optional LLM report layer for privacy-permitting deployments.When enabled, scored per-word results are sent to GPT-4o-mini to generate narrative summaries for both tasks.Source: E. Report Generation · p. 3
  • The clinical prompt directs the LLM to use plain language for classroom teachers and focus on phoneme-level error patterns rather than general encouragement.Source: E. Report Generation · p. 3

F. User Interface

KIRA presents a teacher-facing web interface for reading tasks and evaluates its word-ASR module on MyST child speech. The deployed Parakeet-TDT-1.1B reaches 8.84% WER on the MyST test set.

F. User Interface

  • The web interface lets proctors record or upload speech and receive reports, showing passage outcomes in a word-by-word grid with clickable phoneme details.Word Reading presents the per-word details as a scrollable list; the interface shows the top-level score first and sub-word evidence one tap beneath.Source: F. User Interface · p. 3

IV. EVALUATION

  • The word-ASR evaluation uses the MyST test set of 3,027 recordings at 16 kHz, averaging 10–15 seconds and containing children reading digits, sentences, and science content.Source: §IV · p. 3

A. Word ASR

  • 8.84% WER for Parakeet-TDT-1.1B compares with 8.95% WER for Parakeet-TDT-0.6B-v2; KIRA deploys the 1.1B model.Table I reports WER on 3,027 MyST test recordings and marks the deployed model in bold.Source: Table I · p. 3, A. Word ASR · p. 3

B. Phoneme Recognition

Fine-tuning HuPER on children’s speech lowered phoneme error on the test set, and the adapted model was selected as KIRA’s phoneme-recognition module.

B. Phoneme Recognition

V. CONCLUSION AND LIMITATIONS

KIRA pairs word-level fluency scoring with parallel phoneme-level error detection in a web-based proctor-facing interface. The authors report component-level validation and identify further benchmarking and classroom evaluation as future work.

V. CONCLUSION AND LIMITATIONS

  • KIRA is a full-stack K–2 screening pipeline combining word-level fluency scoring and parallel phoneme-level error detection in a web-based proctor-facing interface.Source: §V · p. 3
  • Validation is at the component level; planned work includes expert-label benchmarking, report comparisons, and classroom evaluation of diagnostic utility.Source: §V · p. 3

GENERATIVE AI DISCLOSURE

  • Claude and ChatGPT were used for manuscript grammar, language improvement, and condensing sections; the authors reviewed the content and take responsibility for the writeup.Source: GENERATIVE AI DISCLOSURE · p. 4
Loading 2610.28840v1…