Source-linked AI summary
Learning to Grade Efficiently: A Bandit-Driven Prompt-Selection Framework for Low-Cost LLM Essay Scoring
Olga Manakina, Igor Bogdanov
TL;DR
Fixed prompt selection does not address changing optimal configurations and the cost of evaluating every option. This paper uses a multi-armed bandit controller to adaptively select prompting strategies, achieving comparable accuracy to exhaustive search while reducing LLM calls by 78.4%.
Problem
Fixed prompt selection overlooks evolving optimal configurations and makes evaluating every prompt option financially unsustainable for large-scale assessment.
Method
The paper treats prompting strategies as arms in an adaptive multi-armed bandit controller, using four recipe variants and dynamically assembled prompt templates.
Results
78.4% fewer LLM calls and comparable accuracy to exhaustive grid search were achieved, while Multi-step assessment with calibration examples performed best at MAE 0.86 and QWK 0.57.
Takeaways & Limitations
Tracking token usage and latency alongside agreement metrics produces cost-reliability learning curves for balancing operational costs against psychometric validity.
Takeaways & Limitations
The initial study used only Gemini 2.5 on IELTS Task 2 essays, with a constant exploration rate of ϵ = 0.2.
Abstract
from arXiv · showhide
Large Language Models (LLMs) demonstrate strong capabilities in automated essay scoring (AES), but contemporary approaches typically employ fixed prompt selection, failing to address operational cost concerns and evolving optimal configurations. We propose a novel cost-aware approach that treats each prompt type as an arm in a multi-armed bandit (MAB) controller, enabling adaptive selection of optimal prompting strategies during inference. Our experiments on IELTS Writing Task 2 essays show that the MAB framework achieves comparable scoring accuracy to exhaustive grid search while reducing LLM calls by 78.4\% to find the best grading approach. We implemented four distinct grading recipes (multi-step vs. single-step assessment, with vs. without calibration examples) and found that the multi-step approach with examples achieves the highest accuracy. By tracking token usage and latency alongside agreement metrics, we produce the first cost-reliability learning curves for essay scoring, providing actionable insights for educational technology platforms that must balance operational costs against assessment validity. This work represents the first application of online control mechanisms to adaptively select prompting strategies in AES, transforming prompt selection from an offline hyperparameter optimization problem into an efficient online learning task.
1. INTRODUCTION
The paper addresses fixed prompt selection in automated essay scoring by introducing a cost-aware multi-armed bandit controller. Preliminary IELTS Task 2 experiments report comparable accuracy to exhaustive search with substantially fewer LLM calls, while tracking cost and reliability.
- Motivation: Fixed prompt selection leaves AES systems exposed to prompt-length costs and changing optimal configurations.Prompt costs vary with token usage and call frequency, while model updates, pricing changes, and essay characteristics can alter which configuration is best.
- Approach: The proposed controller treats each prompt-model combination as a bandit arm and uses agreement with examiner scores as its reward signal.The controller incrementally concentrates calls on prompt configurations that are reliable and cost-effective, reframing selection as online learning.
- Results: Approximately one-tenth as many LLM calls were required while achieving comparable QWK scores to exhaustive grid search.The result comes from preliminary experiments on the IELTS Writing Task 2 corpus.
- Contributions: The study presents adaptive online prompt selection as the first online control application for prompting strategies in AES.Prior published work is characterized as relying on fixed prompt templates.
- Contributions: Cost-reliability learning curves combine token usage and latency with agreement metrics for educational assessment platforms.The intended use is to help test providers and educational technology platforms balance cost considerations with psychometric validity.
- Scope: The initial study uses one model and focuses on IELTS Task 2, with broader models and datasets reserved for future work.The paper identifies Gemini Flash 2.5, GPT-4, Llama-3 70B, and ASAP as part of its current or planned scope.
2. RELATED WORK
Prior LLM essay-scoring research explores varied prompting methods but generally evaluates fixed prompt sets offline. Related NLP work shows that online controllers can reduce query or token costs, motivating an adaptive bandit approach for AES.
- LLM-Based Essay Scoring: LLM essay-scoring studies cover zero-shot, rubric-aligned, few-shot, chain-of-thought, comparative-judgment, and rationale-augmented prompting.Reported studies span multiple datasets and model configurations, with some methods improving QWK or transparency.
- Static and Cost-Blind Selection: Most LLM-AES studies select a single prompt template offline, while token budgets are rarely reported despite potentially long few-shot, rubric, and rationale prompts.A rubric-granularity study found that simplified rubrics maintained accuracy for three of four LLMs while cutting prompt length by more than half.
- Online Controllers: Outside AES, TRIPLE uses MAB for fixed-budget prompt identification with 50–80% of exhaustive-search LLM calls.Other adaptive systems include hierarchical MAB tuning for retrieval and contextual routing toward cheaper models meeting quality targets.
- Remaining Gap: This work embeds an MAB controller in an AES pipeline to select among four prompting techniques during inference.It connects adaptive efficiency research in NLP with writing assessment and reports reduced token use with human-level reliability.
- Datasets and Gap: The benchmark landscape includes ASAP, TOEFL11, and IELTS Writing Band-Score datasets, but published LLM studies use fixed prompt templates rather than online search.The IELTS Writing Band-Score set is described as containing approximately 1,200 essays.
3. METHODOLOGY
The methodology evaluates four modular IELTS grading recipes on Task 2 essays and dynamically assembles prompts from reusable templates. Multi-step recipes score criteria separately before calculating an overall score, whereas single-step recipes request direct assessment.
- Experimental Design: The study focuses on IELTS Writing Task 2 and compares four recipes combining single-step or multi-step assessment with or without essay examples.The multi-step design separately evaluates Task Response, Coherence and Cohesion, Lexical Resource, and Grammatical Range and Accuracy.
- Data: The corpus contains 787 Academic Task 2 compositions with official examiner-assigned band scores from 1 to 9.The essays address conventional Task 2 instructions involving arguments or perspectives on societal and academic subjects.
- Dynamic Prompt Assembly: A prompt library separates templates from runtime instantiation across multi-step versus single-step and example versus no-example conditions.Templates establish the LLM’s examiner role, specify assessment criteria, and may include annotated high- and low-scoring essays for calibration.
- Dynamic Prompt Assembly: At runtime, the system selects recipe-specific templates, injects the essay question and text, and constructs system/user message dictionaries.The resulting messages are assembled into the final prompt sequence transmitted to the LLM.
- Scoring Procedure: Multi-step recipes generate criterion-specific prompts sequentially and programmatically calculate the overall score, unlike single-step direct-assessment recipes.The design supports systematic comparison while maintaining consistent content across experimental conditions.
4. RESULTS
The MAB controller learned to favor Multi-step Ex, which delivered the strongest accuracy while adaptive selection substantially reduced token use and experimental cost versus exhaustive Grid Search.
- Learning Behavior and Approach Selection: Around 100 steps, assessment stabilized after exploration, with Multi-step Ex achieving the highest shaped reward at approximately -0.75.Single-step Ex followed at around -1.0; approaches without examples performed worse.
- Learning Behavior and Approach Selection: Approximately 350 pulls, or over 70% of selections, went to Multi-step Ex, while each other recipe received 40–65 pulls.The allocation reflects the controller’s learned preference for the most effective recipe.
- Scoring Accuracy: MAE was lowest for Multi-step Ex at approximately 0.85, followed by Single-step Ex at 1.0; Single-step NoEx and Multi-step NoEx reached approximately 1.45 and 1.55.Lower MAE indicates greater scoring accuracy.
- Scoring Accuracy: Multi-step Ex achieved the highest QWK, approximately 0.55, across both MAB and Grid Search experiments.Multi-step NoEx ranked second at around 0.35 despite its higher MAE, indicating that the metrics capture different agreement properties.
- Cost-Efficiency Tradeoffs: Multi-step Ex attained MAE 0.85 at about $0.0011 per essay, whereas Single-step Ex offered MAE 1.0 at $0.0003 and Single-step NoEx cost $0.0002 at MAE 1.45.Multi-step NoEx performed worst overall, with MAE 1.55 at $0.0004.
- Cost-Efficiency Tradeoffs: Approximately 9 million tokens for Grid Search versus 1.8 million for MAB, while total costs were about $1.4 versus $0.4 with comparable accuracy outcomes.Grid Search consumed tokens at approximately five times the rate of MAB.
- Ablation Study: Removing detailed rubric descriptions improved Multi-Step with Examples from MAE 0.965 and QWK 0.485 to MAE 0.862 and QWK 0.566 while reducing tokens from 7,862 to 7,402.The simplified approach also received 70.8% of pulls versus 66.4% for the detailed version.
5. DISCUSSION & FUTURE WORK
The study finds that adaptive prompt selection can reduce costs without compromising scoring quality, while highlighting limitations of the current evaluation and bandit design.
- Adaptive prompt selection via MAB achieved substantial cost savings without compromising scoring quality in large-scale assessment settings.
- Simplified prompts without detailed rubric explanations outperformed comprehensive prompts, suggesting that LLMs may have internalized academic writing assessment norms.
- Multi-step approaches performed strongly and may extend to other LLM assessment applications.
- The initial study used only Gemini 2.5 on IELTS Task 2 essays, limiting evidence about generalizability across models and essay types.
- The MAB implementation used a constant exploration rate of ϵ = 0.2, while adaptive ϵ strategies and contextual bandits remain future directions.
6. CONCLUSION
The paper introduces a MAB-based system that adaptively selects prompting strategies for automated essay scoring. Initial experiments matched exhaustive grid-search accuracy while substantially reducing LLM calls and token consumption.
- 78.4% fewer LLM calls and 72.8% lower token consumption were achieved with accuracy comparable to exhaustive grid search.
- Multi-step assessment with calibration examples performed best, achieving MAE 0.86 and QWK 0.57.
- Simplified prompts outperformed prompts containing detailed rubric criteria.
- Framing prompt selection as online learning enables assessment systems to adapt to changing model capabilities, pricing structures, and essay characteristics.