Source-linked AI summary
Rubric-to-Code Credit Assignment for Reinforcement Learning
Rui Jin, Jikai Chen, Yihan Chen, Hao Zhou, Demin Zhu, Kaichen Yang, Dong Wang, Chenyi Zhuang
TL;DR
Interactive web application generation requires credit assignment aligned with multiple user-facing requirements and their responsible code regions, which standard GRPO does not provide. RCCA combines rubric-driven tasks, hierarchical rewards, and evaluator-attributed token weighting; Ling-RCCA-Flash achieves strong results on MiniAppBench and ArtifactsBench, with reported cross-benchmark gains.
Problem
Standard GRPO collapses multiple functional outcomes into one sequence-level reward and applies the resulting advantage uniformly to all generated tokens.
Method
RCCA uses explicit functional rubrics, hierarchical rewards, and evaluator-generated attributions aligned with responsible code spans and generated tokens.
Results
41.25 on MiniAppBench improved Ling-3.0-Flash by 32.20 points, while 76.19 on ArtifactsBench improved the SFT model by 4.48 points.
Takeaways & Limitations
The results suggest transferable implementation-level gains across interactive web applications and visual artifact tasks.
Takeaways & Limitations
Evaluation focuses on interactive HTML/CSS/JavaScript applications and does not fully cover multipage applications, backend services, persistent storage, authentication, or deployment constraints.
Abstract
from arXiv · showhide
Interactive web application generation requires models to produce usable HTML, CSS, and JavaScript applications from natural language requests. Unlike conventional code generation, application quality depends on multiple user-facing functional requirements, each often tied to localized code regions such as event handlers, state updates, DOM fragments, or CSS selectors. Standard GRPO collapses these structured outcomes into a single sequence-level reward and applies the resulting advantage uniformly to all tokens, weakening credit assignment. We propose \textbf{Rubric-to-Code Credit Assignment} (RCCA), a reinforcement learning framework that converts rubric-level functional feedback into localized optimization signals over generated code. RCCA builds training tasks around explicit functional rubrics, uses a hierarchical reward to separate format, source-code, runtime, and functional failures, and aligns evaluator-generated textual attributions with responsible code spans and generated tokens. The resulting model, \textbf{Ling-RCCA-Flash}, scores 41.25 on MiniAppBench, improving Ling-3.0-Flash by 32.20 points and slightly surpassing Claude Opus 4.5. It also reaches 76.19 on ArtifactsBench, improving the SFT model by 4.48 points and establishing a new top score under the official ArtifactsBench leaderboard setting by surpassing the GPT-5 score by 3.64 points, suggesting transferable implementation-level gains.
1 Introduction
Interactive web applications require structured, user-facing functional evaluation, but standard GRPO applies sequence-level feedback uniformly across generated tokens. RCCA addresses this mismatch with rubric-driven tasks, hierarchical rewards, and localized token-level credit assignment, yielding gains across two benchmarks.
- Interactive applications are judged by whether they implement explicit user-facing behaviors, such as click-triggered panels, input-driven updates, and state transitions.
- Standard GRPO collapses multiple functional outcomes into one sequence-level reward and applies the resulting advantage uniformly, even when only localized code causes failure.
- RCCA converts concrete functional rubrics into localized optimization signals over responsible generated-code regions.
- RCCA’s hierarchical reward separates output-format, source-code, runtime, and rubric-level functional validity to improve reward discrimination.
- 41.25 on MiniAppBench improved Ling-3.0-Flash by 32.20 points, while 76.19 on ArtifactsBench improved the SFT model by 4.48 points.
2 Related Work
Related work moves from isolated code evaluation toward interactive, user-facing artifacts and finer-grained learning signals. RCCA focuses on converting requirement-level functional evaluation into code-localized reinforcement-learning credit.
- Traditional benchmarks measure isolated coding ability through programming problems, unit tests, or repository-level edits, but incompletely capture user-facing software quality.
- Recent benchmarks evaluate web applications ranging from static interfaces to richer interactive outputs using visual, functional, preference, and dynamic-interaction criteria.
- Interactive applications are increasingly evaluated as end-to-end user-facing artifacts rather than isolated programs.
- Standard GRPO uses a group-relative sequence-level reward but suffers a granularity mismatch when multiple functional requirements are collapsed and advantages are uniformly applied.
- RCCA contributes code-localized credit assignment by aligning attributed implementation regions with generated tokens for GRPO weighting.
3 Method
RCCA structures interactive web-application reinforcement learning around explicit rubrics, hierarchical rewards, and localized token-level credit assignment. It separates failure stages at the response level and traces evaluator diagnostics from functional outcomes to responsible code regions and tokens.
- Problem addressed: Standard GRPO applies one sequence-level advantage uniformly across tokens, even when only a localized code region causes a failure.This loses supervision at both sample and token levels and can weaken discrimination among applications with different failure modes.
- Rubric-driven training data: RCCA constructs training examples around explicit rubrics for independently checkable initial-state and interaction requirements.The same rubric set supports task construction, response evaluation, hierarchical reward computation, and code localization.
- Hierarchical reward: RCCA uses a gated hierarchy that separates output-format, source-code, runtime, and rubric-level functional validity.Only applications passing the validity checks proceed to rubric evaluation, while the gates place fundamentally invalid responses below those reaching later stages.
- Hierarchical reward: Rubric-level scoring weights violations by rubric importance and severity, with score ceilings for severe failures of essential requirements.The penalty increases with both importance and violation severity, preventing less important successes from offsetting severe essential failures.
- Rubric-to-code localization: Evaluator diagnostics identify implementation evidence, expand it through source-level relations, and produce localized source spans for each issue.Expansion can include enclosing functions, event bindings, state reads and writes, and function calls.
- RCCA training objective: RCCA maps localized spans to generated tokens and applies asymmetric weights that concentrate negative updates on diagnosed error regions.For positive-advantage responses, diagnosed regions are not further reinforced, while unaffected code receives a slightly stronger positive update.
4 Experiment
The experiments compare Ling-3.0-Flash, its SFT variant, and Ling-RCCA-Flash on MiniAppBench and ArtifactsBench. RCCA improves performance on both benchmarks, including the broader ArtifactsBench evaluation.
- MiniAppBench: 41.25% average pass rate on MiniAppBench places Ling-RCCA-Flash slightly above Claude-Opus-4.5 at 41.14%.The model is also competitive with the strongest evaluated open-source models.
- MiniAppBench: 14.40 points is the additional MiniAppBench improvement from SFT to RCCA, after the base model’s 9.05% and SFT’s 26.85%.
- ArtifactsBench: 76.19 on ArtifactsBench improves the pre-RL model by 4.48 points and establishes a new top score under the official leaderboard setting.
- ArtifactsBench: 3.64 points is Ling-RCCA-Flash’s margin over GPT-5’s official ArtifactsBench score of 72.55.
- Generalization: Consistent gains on ArtifactsBench suggest RCCA benefits extend beyond the target MiniAppBench evaluation.The paper attributes this broader pattern to optimization signals tied to implementation behavior and requirement-level feedback.
5 Conclusion
The conclusion presents RCCA as a reinforcement learning approach that addresses standard GRPO’s credit-assignment granularity mismatch. Results suggest improvements in transferable application-generation behavior across interactive and visual artifact tasks, while also linking agent infrastructure to model training.
- RCCA combines hierarchical application-level rewards with token weights derived from rubric-level diagnostic feedback.
- 41.25 on MiniAppBench and a state-of-the-art result on ArtifactsBench summarize Ling-RCCA-Flash’s reported benchmark performance.
- The results suggest that crediting responsible implementation regions improves transferable behavior across interactive and visual artifact tasks.
- AWorld-style harnesses expose interaction traces, requirement-level outcomes, and localized diagnostics that RCCA converts into optimization targets.
Limitations
The evaluation focuses on interactive HTML/CSS/JavaScript application generation and does not fully cover several production-oriented application settings. RCCA also depends on evaluator-generated judgments and attributions that can misassign credit when failures involve distant code regions.
- The evaluation does not fully cover large multipage applications, backend services, persistent storage, authentication flows, or production deployment constraints.
- Incorrect evaluator diagnostics may assign credit to the wrong implementation spans when failures arise from interactions among distant code regions.
Ethics Statement
The paper states that its training and evaluation data come from benchmark tasks and synthesized rubrics rather than private user data. It also limits the intended use of generated applications to prototyping and research evaluation because defects may remain.
- Data Privacy: Training and evaluation data are constructed from benchmark tasks and synthesized rubrics rather than private user data.The paper also says it avoids intentionally including personally identifiable information.
- Responsible Application Generation: Generated applications may contain functional, security, or accessibility defects and should be inspected before deployment.Ling-RCCA-Flash is intended for prototyping and research evaluation.