Source-linked AI summary
Enhancing Procedural Writing Through Personalized Example Retrieval: A Case Study on Cooking Recipes
Paola Mejia-Domenzain, Jibril Frej, Seyed Parsa Neshaei, Luca Mouchel, Tanya Nazaretsky, Thiemo Wambsganß, Antoine Bosselut, Tanja Käser
TL;DR
Learners often struggle to write comprehensive, high-quality procedural texts, while feedback and examples are commonly static and insufficiently adapted to individual input or prior knowledge. RELEX predicts a learner recipe’s quality, retrieves higher-quality recipes, selects a semantically similar example in real time, and adds personalized explanations. In a study of 200 users, adaptive feedback positively affected user experience, while participants improved procedural-text quality by 15% on average.
Problem
Learners often struggle to write comprehensive, high-quality procedural texts, while feedback and examples are commonly static and insufficiently adapted to individual input or prior knowledge.
Method
RELEX predicts a learner recipe’s quality, retrieves higher-quality recipes, selects a semantically similar example in real time, and adds personalized explanations.
Results
In a study of 200 users, adaptive feedback positively affected user experience, while participants improved procedural-text quality by 15% on average.
Takeaways & Limitations
Personalized example-based learning can support procedural writing at scale while enabling examples and explanations to match learners’ topics and skill levels.
Takeaways & Limitations
The study leaves the long-term effects of repeated RELEX use for procedural writing skills to future longitudinal research.
Abstract
from arXiv · showhide
Writing high-quality procedural texts is a challenging task for many learners. While example-based learning has shown promise as a feedback approach, a limitation arises when all learners receive the same content without considering their individual input or prior knowledge. Consequently, some learners struggle to grasp or relate to the feedback, finding it redundant and unhelpful. To address this issue, we present RELEX, an adaptive learning system designed to enhance procedural writing through personalized example-based learning. The core of our system is a multi-step example retrieval pipeline that selects a higher quality and contextually relevant example for each learner based on their unique input. We instantiate our system in the domain of cooking recipes. Specifically, we leverage a fine-tuned Large Language Model to predict the quality score of the learner's cooking recipe. Using this score, we retrieve recipes with higher quality from a vast database of over 180,000 recipes. Next, we apply BM25 to select the semantically most similar recipe in real-time. Finally, we use domain knowledge and regular expressions to enrich the selected example recipe with personalized instructional explanations. We evaluate RELEX in a 2 x 2 controlled study (personalized vs. non-personalized examples, reflective prompts vs. none) with 200 participants. Our results show that providing tailored examples contributes to better writing performance and user experience.
1 Introduction
Procedural writing is difficult because it combines general writing knowledge with domain-specific knowledge, while existing example-based support is often static and insufficiently tailored. RELEX addresses this gap by retrieving higher-quality, topic-relevant examples and adaptive explanations for individual learners.
- 1 Introduction: Many learners struggle to produce complete, high-quality procedural texts, making effective feedback and examples important forms of support.The challenge applies across domains including manuals, code, lab protocols, and cooking recipes.
- 1 Introduction: Procedural writing requires both knowledge of text structure and knowledge of the application domain, preventing a single algorithmic solution.Examples can help learners borrow knowledge and abstract rules for similar problems, including in heuristic domains.
- 1 Introduction: Prior example-based learning has mainly addressed structured tasks, while tailored example retrieval in heuristic domains remains limited.Existing work has used feature-based similarity or unsupervised semantic similarity, and writing tools often provide explanations without corrective examples.
- 1 Introduction: RELEX retrieves higher-quality, topic-relevant recipes and enriches them with explanations tailored to each learner’s missing elements.Its pipeline predicts recipe quality with a fine-tuned LLM, retrieves higher-quality recipes, and selects the most semantically similar example using BM25.
- 1 Introduction: A 2 × 2 controlled study with 200 participants evaluates adaptive examples and feedback alongside reflective prompts, with static support as a control.The study examines effects on learner experience, writing, and related outcomes.
2 Related Work and Conceptual Background
Procedural writing is a domain-dependent heuristic skill whose quality involves structure, clarity, and specificity, and learners often omit steps, details, or domain-specific language. Example-based learning supports schema formation and transfer, but conventional examples, explanations, and writing feedback are commonly static rather than personalized.
- 2.1 Learning Procedural Writing Skills: High-quality procedural texts require structure, clarity, and specificity, yet learners commonly omit steps, details, ingredients, or domain-specific vocabulary.These difficulties have been documented in technical procedures and cooking recipes.
- 2.1 Learning Procedural Writing Skills: Because procedural writing depends on subject matter, teaching must address both how to structure a procedure and the topic being described.Cooking examples therefore involve procedural writing as the learning domain and cooking as the exemplifying domain.
- 2.2 Example-Based Learning in Heuristic Domains: Example-based learning helps learners observe or imitate solutions, build cognitive schemas, abstract general rules, and transfer them to related problems.In heuristic domains, examples contain two content levels and may require additional explanations or prompts.
- 2.2 Example-Based Learning in Heuristic Domains: Reflective prompts can deepen self-explanation and direct attention toward central aspects of examples, supporting conceptual knowledge and deficiency diagnosis.Their rationale follows the self-explanation effect, because passive or superficial explanations may limit learning.
- 2.2 Example-Based Learning in Heuristic Domains: Instructional explanations can improve early learning but later become redundant or cognitively burdensome when they are not adapted to prior knowledge or delivered timely.A prior study found self-explanation prompts yielded the most favorable learning outcomes, while instructional explanations produced the highest perceived learning.
- 2.3 Adaptive Learning: Personalized learning support can retrieve examples from existing collections and use language models to estimate the quality of peer-generated examples.Prior writing tools provide holistic or fine-grained feedback, but heuristic-domain suggestions and explanations remain less studied.
3 RELEX - Learning With Personalized Examples
RELEX supports procedural writing by combining tailored recipe examples, personalized annotations, and reflection guidance. Its pipeline selects a higher-quality, relevant recipe and enriches it with suggestions targeting learner-specific weaknesses.
- 3.1 User Interface: RELEX provides learners with tailored examples, highlighted missing elements, personalized explanations, and prompts to reflect on recipe strengths and weaknesses.
- 3.1 User Interface: The system targets procedural-writing organization, structural clarity, requisite detail, and appropriate specific vocabulary through annotated comparison recipes.
- 3.1 User Interface: RELEX was designed from ten user interviews that identified vagueness, missing steps, and difficulty reproducing instructions as common writing problems.
- 3.2.1 Offline Training and Annotation: Offline preparation uses more than 180,000 rated recipes, a cooking-domain-adapted language model, and rule-based annotation with expert suggestions.
- 3.2.1 Offline Training and Annotation: The annotation rules were manually evaluated across recipe segments, achieving a Cohen’s Kappa score of 0.85 between annotators.
- 3.2.2 Online Prediction and Selection: The retrieval pipeline predicts a learner recipe’s quality, retrieves recipes rated 0.4–0.8 stars higher, and selects a real-time comparison recipe within that range.
4 Experimental Design
The study used a randomized between-subjects experiment comparing adaptive versus non-adaptive feedback and reflective prompts versus none, with a static-support control group. Participants completed recipe-writing tasks, a furniture-assembly transfer test, surveys, and revision analyses.
- 4.1 Study Design: The experiment randomly assigned participants to four treatment groups crossing adaptive versus non-adaptive feedback with reflective prompts versus none, plus a static-support control group.The control group received general rules without a peer example, while treatment groups received different RELEX configurations.
- 4.2 Participants: The study recruited 200 paid Prolific participants and retained 187 for analysis after excluding incomplete or technically problematic cases.Participants were aged 18–30 and had at least an undergraduate degree.
- 4.3 Procedure: Participants completed three cooking-recipe writing tasks, with furniture-assembly pre- and post-tests used to assess transfer to another procedural domain.All groups completed the same pre-survey, writing tasks, and post-survey.
- 4.4 Measures and Analysis: Recipe-writing performance was measured using predicted stars and a quality score based on structure, clarity, and specificity criteria.Scores compared each participant’s first recipe with their last revised recipe.
- 4.4 Measures and Analysis: Transfer performance was calculated as the proportion of nine procedural-writing suggestions followed in the furniture-assembly task.Each suggestion received a binary score indicating whether it was followed.
- 4.4 Measures and Analysis: Open-ended perceptions were analyzed with BERTTopic, while keystroke-based revision behavior captured changes participants made after receiving feedback.Sentence embeddings from Sentence-BERT were clustered into participant-level topic sets.
5 Results
Adaptive feedback improved recipe-writing quality over time and enhanced several user-experience measures, while transfer effects were not significant. Participants also valued tailored suggestions and examples, although reflective prompts may have increased revision-related cognitive load in one comparison.
- 5.1 RQ1: Impact on Learners’ Experience: Adaptive-feedback groups scored significantly higher on perceived usefulness, attitudes toward use, behavioral intention, and perceived learning gain, but not perceived ease of use.The adaptive-feedback effect was significant for four of five constructs; ease of use did not differ significantly.
- 5.1 RQ1: Impact on Learners’ Experience: Participants valued comparing their recipes with examples, in-text highlighting, intuitive usability, and suggestions perceived as useful for beginners.In-text highlighting was valued by 11% of participants.
- 5.1 RQ1: Impact on Learners’ Experience: Participants in non-adaptive conditions requested more personalized or semantically similar recipes, while some adaptive users wanted examples sharing the same ingredients.Control-group feedback also described repeated, unchanged suggestions as becoming less useful.
- 5.2 RQ2: Effect on Learners’ Writing Performance: Adaptive-feedback users improved significantly more than non-adaptive users from their first to last recipe (p < .01), with adaptive conditions showing steeper quality-score gains.Overall quality scores increased across test time, with a significant time effect (F(1, 186) = 84.4, p < .0001) and time-by-condition interaction (F(4, 186) = 2.6, p < .05).
- 5.2 RQ2: Effect on Learners’ Writing Performance: Predicted stars improved significantly across recipes (F(1, 186) = 19.2, p < .0001), but no condition differences were found.There was no main effect of experimental condition, and planned comparisons found no differences between conditions.
- 5.2 RQ2: Effect on Learners’ Writing Performance: No significant effects were found for transfer-task improvement between conditions or for adaptive feedback and reflective prompts separately.The control group appeared to perform worse, but the reported transfer comparisons were not significant.
6 Discussion and Conclusion
RELEX’s personalized examples and adaptive feedback improved learners’ experience, writing performance, and revision engagement, while reflective prompts alone produced no significant quantitative gains. The study also identifies limits in the quality predictor and calls for real-world and longitudinal validation.
- Impact on learners’ experience (RQ1): Adaptive feedback significantly improved perceived learning gain, usefulness, behavioral intention, and attitudes toward use, while perceived ease of use did not differ between groups.Participants also described the tool as intuitive despite its additional personalized features.
- Impact on learners’ writing performance (RQ2): All groups improved transfer-task texts by 15% in structure and specificity, indicating gains in general procedural-writing principles across conditions.The transfer task involved furniture assembly rather than cooking recipes.
- Impact on learners’ revision behavior (RQ3): Adaptive feedback led users to revise more across later recipes, whereas overall revision time and revision counts declined from the first to the last recipe as recipe quality increased.The authors suggest learners made fewer changes because they had already incorporated feedback.
- Literature contributions: RELEX extends static example-based learning by retrieving better-quality, topic-relevant peer examples and adapting instructional explanations to each learner’s prior text.This design aims to reduce explanations becoming redundant or too complex.
- Reflective prompts: Reflective prompts did not significantly improve quantitative perception, performance, or transfer outcomes, possibly because the three-recipe study was too short for self-explanation effects to emerge.The authors propose that participants may have self-explained without writing.
- Limitations and future work: The quality predictor is limited because its subjective ratings combine writing quality with taste, producing only minor improvement over a static baseline.The authors made the code and models publicly available and suggest stronger models as one route to improved prediction.
Statements and Declarations
The project was substantially co-financed by SERI, and the authors report no relevant financial or non-financial interests.
- The project was substantially co-financed by the Swiss State Secretariat for Education, Research and Innovation (SERI).
- The authors have no relevant financial or non-financial interests to disclose.