Source-linked AI summary
Reading.help: Supporting EFL Readers with Proactive and On-Demand Explanation of English Grammar and Semantics
Sunghyo Chung, Hyeon Jeon, Sungbok Shin, Md Naimul Hoque
arXiv:2505.14031v2cs.HC
TL;DR
Online courses lack the situational support EFL readers need while reading, and dictionaries offer limited help identifying misunderstandings. The paper develops and evaluates Reading.help, combining difficulty detection with vocabulary, grammar, and context explanations; readers mainly used it to address text complexity and gain comprehension support.
Problem
EFL readers seeking self-education lack on-demand, situational support from online courses, while dictionaries provide limited help identifying potential misunderstandings.
Source: §1 · p. 2Method
Reading.help combines difficulty detection with vocabulary, grammar, and context explanations, refined with 15 South Korean EFL readers and evaluated with five readers and two education experts.
Source: §1 · p. 2Results
Readers mainly used Reading.help to address text complexity and gain comprehension support.
Source: §1 · p. 2Takeaways & Limitations
EFL reading tools should preserve productive effort and assess growing independence, rather than measuring success through faster reading alone.
Source: §8.2 · p. 14Detailed grammatical explanations overwhelmed participants, who subsequently stopped consulting that section.
Source: §8.2 · p. 14
Abstract
from arXiv · showhide
A large portion of texts is written in English, but readers who see English as a Foreign Language (EFL) often struggle to read texts accurately and swiftly. EFL readers seek help from professional teachers and mentors, which is limited and costly. In this paper, we explore how an intelligent reading tool can assist EFL readers. We conducted a case study with EFL readers in South Korea. We at first developed an LLM-based reading tool based on prior literature. We then revised the tool based on the feedback from a study with 15 South Korean EFL readers. The final tool, named Reading.help, helps EFL readers comprehend complex sentences and paragraphs with on-demand and proactive explanations. We finally evaluated the tool with 5 EFL readers and 2 EFL education professionals. Our findings suggest Reading.help could potentially help EFL readers self-learn English when they do not have access to external support.
1 Introduction
Reading.help supports university-educated EFL readers’ independent interpretation of English through difficulty detection and proactive or on-demand explanations. A South Korean case study informed its revision and evaluated its comprehension support and limitations.
1 Introduction
- English reading is important across academic, professional, and social contexts, yet formal education can be costly and inaccessible.Even formally educated learners may lack exposure to advanced vocabulary and cultural references outside classrooms.Source: §1 · p. 2
- Existing self-study resources leave gaps in situational support: online courses lack on-demand assistance, and dictionaries offer limited help identifying potential misunderstandings.Strategic translation can support comprehension and learning, but full-text translation may increase reliance and reduce engagement with vocabulary or syntax.Source: §1 · p. 2
- Reading.help combines difficulty identification and explanation modules for university-educated EFL readers willing to interpret English texts independently.Two specialized, interpretable neural models detect challenging words and sentences; explanations address vocabulary, grammar, and overall context.Source: §1 · p. 2
- An LLM explains automatically identified or manually highlighted text, while a secondary LLM evaluates and refines explanations.The explanation module also produces concise paragraph summaries to convey main ideas without overwhelming readers with details.Source: §1 · p. 2
- 87% average recall was reported when Reading.help recommended three elements per dimension, while readers mainly used it to address text complexity and support comprehension.LLM self-validation was comparable to human performance in detecting incorrect meanings, but remained limited in identifying other issues, including missing elements.Source: §1 · p. 2
- 15 South Korean EFL readers informed revisions through a pilot study, followed by an evaluation involving 5 EFL readers and 2 English-education experts.The evaluation assessed effectiveness and how readers benefited from Reading.help.Source: §1 · p. 2
2 Background and Related Work
Reading.help targets vocabulary, grammar, and comprehension difficulties through proactive and on-demand explanations, addressing a gap in EFL learning support among reading tools. Related research on intelligent reading and writing motivates its combination of NLP heuristics and LLM explanations.
2.1 Difficulties in EFL Reading
- Hirsch identifies fluency, vocabulary breadth, and domain knowledge as interdependent contributors to reading comprehension.Limited vocabulary delays connections between preceding and current text; insufficient domain knowledge can lead readers to construct an inappropriate mental model.Source: §2.1 · p. 2
- Text complexity and reading-strategy difficulties motivate unified support for vocabulary, grammar, and comprehension.Complex grammar, lengthy sentences, and difficulty inferring context can produce misinterpretation or slower reading. The proposed tool offers both on-demand help and proactive recommendations.Source: §2.1 · p. 2
2.2 Intelligent Reading Tools
- Prior tools offer comprehension questions through Marvista [10] and CReBot, chart–text links through Kori [44], and eye-based confusion detection.Other systems support skimming (SCIM [16]), keyword highlighting, and statistical-table interpretation (Statslator). Eye-movement research also predicts mind wandering.Source: §2.2 · p. 3
- Reading.help extends English-learning tools with proactive difficulty detection, on-demand grammar and context explanations, and verification intended to improve reader trust.The authors identify support for learning English syntax and semantics as a gap in prior reading tools.Source: §2.2 · p. 3
2.3 Intelligent Writing Tools
- Intelligent writing research motivates combining NLP heuristics with LLMs to address EFL readers’ problems through proactive and on-demand explanations.Learning to write English could potentially improve reading skills. Related work includes essay refinement through Langsmith, personalized related-work suggestions [9] [35] [37] [56], and human–LLM collaborative writing studies [28] [50].Source: §2.3 · p. 3
3 Design Requirements
The authors adopted an iterative design process, first creating Reading.help-Lite to assess their approach’s strengths and limitations. Their preliminary requirements call for proactive and on-demand explanations, layered detail, proficiency-sensitive support, and visual guidance.
DR1 Proactive and On-Demand Guidance
- Provide proactive guidance on potentially difficult text and on-demand help when readers request it.Proactive guidance could draw attention to overlooked complexity or text readers might interpret incorrectly.Source: §3 · p. 3
DR2 Explain vocabulary, grammar, and semantics
- Explain unfamiliar vocabulary, complex grammatical structures, and the core topic and main takeaways of a text segment.These explanations should support text comprehension.Source: §3 · p. 3
DR3 Drill-down to the Details
- Arrange feedback hierarchically, initially presenting a high-level summary and allowing readers to drill down into details.This structure should accommodate knowledge and information within limited screen space.Source: §3 · p. 3
DR4 User-centered adaptive support
- Tailor feedback to readers’ varying levels of English proficiency.Readers may differ in their formal education and social exposure to English.Source: §3 · p. 3
4 Reading.help-lite
Reading.help-Lite is an initial LLM-powered prototype that suggests vocabulary, grammar, and comprehension difficulties and lets readers request explanations for selected text. It was designed for preliminary user study to inform the final tool.
4.1.1 Workflow.
- Reading.help-Lite analyzes uploaded text for vocabulary, grammar, and comprehension difficulties, presenting suggested issues in color-coded boxes that users can open for explanations.Vocabulary analysis identifies unfamiliar words and phrases, comprehension analysis considers passage context, and grammar analysis addresses grammatical issues.Source: §4.1.1 · p. 4, Figure 2 · p. 4
4.1.2 Detailed explanations to users.
- Detailed explanations provide context-relevant definitions with Korean translations, passage-level meaning with alternative phrasings, and phrase-by-phrase grammatical analysis.Comprehension explanations present the main idea as a bullet point and explain the selected text’s intention within the passage. Grammar explanations describe individual keywords’ functions within phrases.Source: §4.1.2 · p. 4, §4.1 · p. 4
4.1.3 Additional Features.
- Readers can toggle subject-specific vocabulary highlighting, hover for definitions, and manually request vocabulary, grammar, or comprehension analysis for text the tool did not recommend.Explanations vary with the reader’s English proficiency.Source: §4.1.3 · pp. 4–5
4.2.1 Template for providing explanations.
- GPT-4o explanation prompts combine an English-instructor role, user proficiency, selected text, the entire passage, task instructions, and a prescribed response format.Iterative prompt engineering uses role assignment, concise templates, exact keywords for prompt chaining, and global versus task-specific variables to address unpredictable or verbose responses.Source: §4.2 · p. 5, §4.2.1 · p. 5
4.2.2 Template for recommendations (Help).
- A separate recommendation template generates comprehension, grammar, and lexical suggestions using role, proficiency, passage context, selected text, recommendation instructions, and an exact output format.This separates recommending potential difficulties from generating explanations.Source: §4.2.2 · p. 5, §4.2 · p. 5
4.3 Implementation details
- The initial LLM-based prototype supports preliminary user study and runs as a Next.js full-stack web application hosted on AWS during the study.The authors chose an LLM for accessibility and analytical capability; Next.js also handles backend data management and routing.Source: §4 · pp. 3–4, §4.3 · p. 5
5 Pilot Study
The pilot study assessed Reading.help-Lite’s effectiveness and identified improvements. Participants valued its support, but reported limitations in trust, comprehension coverage, explanation accessibility, and usability.
5.1 Study Details
- 15 South Korean degree-holding EFL readers, including 3 English educators, evaluated Reading.help-Lite through TOEFL reading tasks and interviews to identify improvements.Approximately 60-minute sessions included four five-minute tasks using approximately 200–250-word passages. A three-question proficiency test guided whether the LLM provided simplified explanations.Source: §5.1 · pp. 5–6, §5 · p. 5
5.2 Results
- 4.2 vocabulary, 2.1 comprehension, and 0.9 grammar explanations were sought on average per passage, making vocabulary the most frequently used support.Participants used the ‘help’ button 1.6 times versus 1.2 requests for specific explanations of selected text.Source: Figure 3 · p. 6, §5.2 · p. 6
- Participants valued Reading.help-Lite for personal improvement, and all reported that LLM recommendations were beneficial despite not expecting error-free responses.9/15 preferred the comprehension module and 8/15 preferred vocabulary. Comprehension explanations helped interpret complex sentences using their intention and passage context.Source: §5.2 · p. 6
5.3 Lessons Learned
- Participants questioned response reliability and difficult-word detection, needed whole-document guidance beyond self-selected text, and struggled with panels and unexplained interface colors.Selection-based assistance assumed users could recognize what they did not understand, although some were unsure where to seek guidance. These limitations informed the revised tool.Source: §5.3 · p. 6
- 12/15 participants reported minimal benefit from grammar explanations, whose verbosity overwhelmed users; simplified responses also remained difficult for some lower-proficiency readers.4/15 lower-TOEFL-proficiency users found responses overly verbose and difficult to comprehend. Participants suggested brief, concise grammar explanations.Source: §5.3 · p. 6, §5.2 · p. 6
6 Reading.help: Updated Version · 6.1 Interpretable CEFR Prediction Models
Reading.help addresses transparency and difficulty-calibration concerns with interpretable CEFR predictors for words and sentences. Both models outperform GPT-4o on average F1 and accuracy while providing faster inference.
6 Reading.help: Updated Version · 6.1 Interpretable CEFR Prediction Models
- Reading.help addresses pilot-study concerns about opaque recommendations and unclear difficulty by introducing interpretable word- and sentence-level predictors aligned with CEFR.CEFR supplies six proficiency levels, from A1 to C2, giving recommendations a standardized difficulty baseline.Source: §6.1 · pp. 6–7, §6 · p. 6
6.1.1 Lexical Complexity Estimator.
- The Lexical Complexity Estimator predicts word CEFR levels from frequency, character count, polysemy, and Qwen3-0.6B [72] embeddings.Embeddings average final hidden states from prompts containing the word and its part of speech. Dedicated feature experts feed an input-dependent weighting network, whose weighted representations are summed and classified.Source: §6.1.1 · p. 7
6.1.6 Model Evaluation.
- 65.2% versus 51.8% word-level average F1 and 75.0% versus 59.6% sentence-level average F1 favor the proposed estimators over GPT-4o.Model versus GPT-4o overall accuracy was 64.8% versus 52.9% for words and 87.4% versus 64.0% for sentences; grouped accuracy was 71.1% versus 62.3% and 89.7% versus 69.9%, respectively. GPT-4o led sentence-level F1 only at C2; testing accepted either annotated CEFR label as correct.Source: Table 1 · p. 8, §6.1.6 · p. 8
6.1.2 Sentence Readability Estimator.
- The Sentence Readability Estimator uses word count, average syllables per word, and Qwen3 sentence embeddings, with three experts and the same weighting, fusion, and classification approach.Its sentence-level feature selection draws on prior work [4] [53].Source: §6.1.2 · p. 8
6.1.3 Interpretability.
- Dynamic gating weights explain which linguistic features contributed to each individual CEFR prediction.Word and sentence predictions support recommendations filtered by the user's selected CEFR threshold. Users can inspect level probabilities and feature-contribution bars when requesting an explanation of a CEFR decision.Source: Figure 4 · p. 7, §6.1.3 · p. 8, Figure 5 · p. 9
6.1.4 Model Training.
- Both predictors train end-to-end in PyTorch, with LoRA adaptation of Qwen3 embeddings and independent binary CEFR classification for examples carrying multiple labels.Training uses BCEWithLogitsLoss, batch size 64, learning rate 2e-4, and 10 epochs with AdamW [51] on one NVIDIA A100 GPU.Source: §6.1.4 · p. 8
6.1.5 Datasets.
- Sentence data combine SCoRE and WikiAuto from CEFR-SP [4]; word data combine CEFR-J, Octanove Vocabulary Profile [17], and Oxford 5000.Sentence evaluation uses CEFR-SP's predefined split; word data use a 90%/10% training/test split. Word–part-of-speech entries are excluded when overlapping sources disagree by at least two CEFR levels.Source: §6.1.5 · p. 8
6.1.7 Inference efficiency.
- 0.05 s versus 0.611 s word-level inference and 0.052 s versus 0.592 s sentence-level inference make the proposed models approximately 12× and 11× faster than GPT-4o.These averages use batch size 1, comparing local inference on one NVIDIA A100 GPU against GPT-4o end-to-end API roundtrip latency from the same client machine.Source: §6.1.7 · p. 8
6.2 Automated Summary
Reading.help automatically generates paragraph-aligned summaries to help EFL readers grasp main ideas, with adjustable detail and direct anchoring to the source text.
6.2 Automated Summary
- Short summaries appear automatically when a document is uploaded, foregrounding topic sentences and key claims before readers examine paragraphs in detail.This feature addresses DR2: improving text comprehension.Source: §6.2 · p. 9
- Each summary stays anchored to its source paragraph, reducing context switching and preserving local coherence during skimming and review.Source: §6.2 · p. 9
- A summary toggle offers concise topic-sentence cues or detailed information, letting readers choose how much support they receive.Source: §6.2 · p. 9
6.3 Validation
Reading.help uses a second LLM to check generated explanations for grounding, relevance, and linguistic correctness. Validation provides explicit trust cues but does not eliminate all errors.
6.3 Validation
- The design retains LLMs for grammar generation and adds validation for reliability alongside a CEFR model intended to improve identification interpretability.LLMs remain necessary for their generative capabilities.Source: §6.3 · p. 9
- A second LLM checks generated lexical explanations, comprehension notes, and grammar analyses for grounding in selected text, requested assistance type, linguistic correctness, and non-contradiction.Grounding is checked against the selected span or paragraph.Source: §6.3 · p. 9
- The validator returns a valid/invalid decision and brief rationale, while validated items receive visible UI indicators.The dual-LLM scheme adds modest latency and makes trust cues explicit, but does not eliminate all errors.Source: §6.3 · p. 9
6.4 UI and Usability Improvements
Reading.help’s revised interface supports different proficiency levels through adjustable difficulty and summary detail, while providing proactive guidance and preserving on-demand explanations. It also shortens grammar explanations and makes second-LLM validation visible.
6.4 UI and Usability Improvements
- Readers can select a target difficulty from recommended CEFR levels and toggle between detailed and concise summaries.These controls make the interface adaptable to different proficiency levels.Source: §6.4 · pp. 9–10
- Readers retain on-demand explanations for selected spans, covering vocabulary, contextual comprehension, and phrase-level grammar.Vocabulary support includes definitions, idioms, usage, and Korean meanings. Comprehension support analyzes the selected text’s main idea within the broader context.Source: Figure 6 · p. 10, §6.4 · p. 10
- The interface recommends potentially difficult words and sentences, using colored highlights to distinguish grammar, vocabulary, and comprehension.Yellow denotes grammar, blue vocabulary, and purple comprehension. Interacting reveals a CEFR label; hovering displays a compact attribution chart showing which features contributed most to the predicted difficulty.Source: §6.4 · p. 10
- Uploaded documents automatically receive short, paragraph-aligned summaries displayed in a fixed left sidebar.This provides proactive guidance without requiring readers to request each summary.Source: §6.4 · p. 10
- Green icons identify responses confirmed by the second LLM, with validation rationales available on hover.The grammar module is also prompted to keep explanations brief, reducing verbosity.Source: §6.4 · p. 10
6.5 System Architecture and Implementation
Reading.help combines a web interface with a backend that estimates word and sentence CEFR levels and generates LLM explanations. Its pipeline filters reading assistance by proficiency level and uses a secondary LLM to validate explanations.
6.5 System Architecture and Implementation
- Reading.help pairs a web interface with a backend using the Lexical Complexity Estimator for word-level CEFR estimates and the Sentence Readability Estimator for sentence-level estimates.Both estimators run when a document is uploaded.Source: §6.5 · p. 10
- PyTorch estimators served through FastAPI batch token and sentence inference to reduce latency, surfacing only items above the user’s target CEFR level.The pipeline segments documents into paragraphs, sentences, and tokens, then reassembles predictions into the original text structure. Filtering uses the user’s chosen level or a history-inferred default.Source: §6.5 · p. 10
- A secondary validator LLM checks the primary LLM’s candidate explanation, returning a validity judgment and rationale that the interface displays through an indicator and available rationale.The validator receives context_paragraph, selected_span, component_type, and explanation, returning a JSON object with valid and rationale fields. The authors acknowledge that this validation is imperfect.Source: §6.5 · p. 10
7 Evaluation
Reading.help supported vocabulary lookup and sentence comprehension, while human experts judged most explanations valid. CEFR recommendations traded coverage for specificity, and education experts identified remaining challenges in personalization, learner motivation, and accuracy verification.
7.1 Study on EFL readers
- 5 Korean-educated EFL readers first marked difficulties without support, then revisited the same passages using Reading.help.Participants were at least undergraduate-level and did not major in English-related fields; materials comprised two novel excerpts, two Economist articles, and one deep-learning paper. Unsupported reading lasted 30 minutes, followed by a five-minute tutorial, 20 minutes using the tool, and a post-interview.Source: §7.1 · p. 11
7.2 Results
- 101 challenging text segments were highlighted by five participants, who used vocabulary modules most, followed by comprehension and grammar.Comprehension explanations supplied key terms and sentence context, while grammar explanations clarified ambiguous interpretations. Two participants reported that paragraph summaries helped them read and follow texts faster.Source: §7.2 · p. 11
- 40.82% average F1 at ≥C1 exceeded 30.38% at ≥B2, trading lower average recall (47.97% versus 87.64%) for higher average precision (37.77% versus 18.47%).Across five readings, recommendations matched user highlights when token-based Jaccard similarity exceeded 0.3. The ≥B2 threshold favored coverage, whereas ≥C1 favored specificity.Source: Table 3 · p. 12, §7.2 · p. 12
- CEFR gating reduced oversuggestion but missed rare forms, idioms, and context-dependent meanings when their estimated levels fell at or below the recommendation cutoff.The >B2 cutoff excluded invented or shortened terms assigned B2, a B1 idiom, and a B2 word used in a marked sense. Sense-aware cues were proposed to address confusing lower-CEFR words.Source: §7.2 · p. 12
- 88% vocabulary, 93% comprehension, and 90% grammar validity were reported by human experts evaluating Reading.help responses.Three English experts assessed 300 examples; of eight flagged vocabulary translations, six were semantically equivalent Korean paraphrases and two were completely incorrect. Comprehension errors involved reasoning beyond the text, while grammar errors frequently involved incorrect parts of speech.Source: §7.2 · pp. 12–13, Table 6 · p. 13
- 93% vocabulary, 97% comprehension, and 96% grammar validity were reported by LLM self-validation, with comprehension showing the highest agreement with human judgments.Precision, recall, and F1 treated valid responses as the positive class and human judgments as ground truth. Error diagnoses largely aligned with human assessments, except for omissions of Korean translations in vocabulary responses.Source: Table 6 · p. 13, §7.2 · p. 13, Table 5 · p. 13
7.3 Expert feedback
- Education experts valued improved comprehension, proactive recommendations, unlimited pressure-free questions, and explanations tailored to English proficiency.Feedback came from two English-teaching professionals. They emphasized support for both textual details and overall structure, and recommendations that expose difficulties learners may not recognize themselves.Source: §7.3 · p. 13
- Experts identified dependence on learner motivation, insufficient personalization, and difficult accuracy verification as remaining limitations.They suggested additional recommendations and visual highlights for less-motivated users, finer proficiency assessment, and better-customized explanations. Teachers wanted to use the tool without constantly validating generated content.Source: §7.3 · p. 14
8 Discussions
The discussion interprets Reading.help’s usage and recommendation results as support for proactive assistance, while noting content-validation concerns. It proposes adaptive, lightweight support that preserves productive effort, sustains attention, and augments rather than replaces teachers.
8.1 Implication of Results
- 5 to 6 vocabulary lookups and one or two comprehension requests per 400-word script indicate substantial reading effort and motivate proactive guidance.Readers also occasionally consulted grammar explanations and paragraph summaries.Source: §8.1 · p. 14
- 87.64% recall was reached in the B2 ≥ configuration, while recommending few items yielded approximately 37.77% precision.The authors favor broader recommendations to capture more information readers may not know. Agreement with human recommendations and response factuality remain validation concerns.Source: §8.1 · p. 14
8.2 Designing Adaptiveness for EFL Learning
- Adaptive assistance should personalize explanations and feedback while remaining brief by default, with additional detail revealed only on request.Personalization based on accumulated learner skills could target vocabulary, grammar, and comprehension, potentially accelerating reading and building confidence. Participants were overwhelmed by detailed grammatical explanations and subsequently stopped consulting them.Source: §8.2 · p. 14
- Reading assistants should preserve productive struggle and reduce support as proficiency grows, measuring success through increasing independence as well as faster reading.Consistent with the “no pain, no gain” principle, the authors suggest withholding hints for repeatedly encountered terms or gradually fading guidance.Source: §8.2 · p. 14
8.3 Engagement Strategies for EFL Readers
- Engagement design should compare task-aligned interest cues with distraction-free immersion, keeping attention on learning rather than competing for it.Interest cues should be sparse because attracting attention need not improve understanding; a distraction-free environment could offer a human-like tutor’s timely, tailored clarifications. Future assessment should compare both strategies while addressing guidance calibration, intervention pacing, and validation.Source: §8.3 · p. 14
8.4 Augmenting, Not Replacing: The Role of LLMs in English Reading Education
- LLM assistants should augment teachers through anytime adaptive support and pedagogical assistance, preserving human accountability, encouragement, social interaction, and the agency of teachers and learners.An envisioned assistant could detect learner difficulties and generate and grade comprehension questions. Teachers remain essential for motivation and adaptive care, especially for beginners or learners with low self-efficacy.Source: §8.4 · p. 15
9 Conclusion
Reading.help uses LLMs and interpretable neural networks to support EFL reading; experiments indicated effective CEFR-based recommendations and limited LLM verification of content correctness.
9 Conclusion
- Reading.help was refined from Reading.help-Lite through a pilot study to effectively support EFL readers.The tool uses LLMs and interpretable neural networks.Source: §9 · p. 15
- CEFR models effectively recommended parts of texts that EFL readers did not know.Source: §9 · p. 15
- LLMs showed limited ability to verify whether content was incorrect.Source: §9 · p. 15