Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback

Motivation

• Numerical-feedback RL plateaus despite 8× training-data scaling or extended training, rarely benefits from spontaneous “Aha moments,” and repeatedly fails on specific training problems.Dense intermediate rewards can partially mitigate plateaus and persistent failures, but numerical feedback does not explain why responses fail or how to correct them.Source: §1 · p. 1
• Existing critique-based methods mainly imitate static, pre-collected critiques or refinements through supervised fine-tuning, leaving their integration into online RL unexplored.These offline approaches lack active exploration and real-time adaptation.Source: §1 · p. 1
• Critiques enable plateaued RL models to correct persistent failures, with verbal credit assignment providing access to refinement trajectories through in-context learning.The reported efficacy extends from simple indicators to detailed chain-of-thought evaluations.Source: §1 · pp. 1–2

Critique-GRPO

• Critique-GRPO learns from initial responses and critique-guided refinements while retaining standard exploration.It prioritizes initial responses and selectively incorporates refinements to mitigate entropy explosion and performance degradation from distribution shifts; a shaping function reinforces successful unfamiliar refinements and penalizes incorrect attempts.Source: §1 · p. 2

Results

• Across eight tasks, Critique-GRPO achieves average Pass@1 improvements of +15.0-21.6% on Qwen models and +7.3% on Llama-3.2-3B-Instruct over supervised and RL-based fine-tuning methods.The experiments use five different models.Source: §1 · p. 2
• Self-critiques yield a +16.7% Pass@1 gain over GRPO on AIME 2024.Source: §1 · p. 2

Enhancing LLM Reasoning with RL

• Numerical-feedback RL has improved LLM reasoning, but sparse scalar rewards reveal little about where or why reasoning errors occur (OpenAI et al., 2024; DeepSeek-AI et al., 2025; Fatemi et al., 2025; Li et al., 2025; Liu et al., 2025a; Yu et al., 2025; Xi et al., 2024; Gandhi et al., 2025).The limited diagnostic information makes it difficult for models to identify and correct specific failures.Source: §2 · p. 2
• Recent approaches combine online RL with expert demonstrations, but depend on curated, high-quality data (Yan et al., 2025; Lv et al., 2025).Source: §2 · p. 2
• Critique-GRPO integrates natural-language critiques into online RL loops, avoiding dependence on expert demonstrations.Source: §2 · p. 2

3.1. Three Limitations of Learning with Solely Numerical Feedback

• Qwen2.5-7B-Base performance saturated after 120 steps, with neither eight times more prompts nor extended training yielding significant gains.Training prompts increased from 4k to 32k.Source: §3.1 · p. 3
• Self-reflection behaviors contributed minimally to success on problems solved only by the RL-finetuned model across both reasoning tasks.The analyzed behaviors included verification, backtracking, backward chaining, and anticipation.Source: §3.1 · p. 3
• Even the best RL-finetuned Qwen2.5-7B-Base consistently failed on approximately 29% of training questions, with Pass@4 = 0.The passage describes these persistent failures as a limitation of numerical-only feedback.Source: §3.1 · p. 3

3.2. Promise of Learning from Natural Language Feedback

• All three critique types enabled refinement of previously unsolvable problems, where spontaneous self-reflection had failed.The critiques were applied to training questions with Pass@4 = 0 and prompted refinement of initial responses.Source: §3.2 · p. 3
• 36.47% valid refinement and 55.37% of persistently failed questions were achieved by CoT critiques for Qwen2.5-7B-Base.The authors attribute CoT critiques’ effectiveness to their step-by-step explanations.Source: §3.2 · p. 3

4.1. From GRPO to Critique-GRPO

• GRPO provides the base online optimization: it estimates advantages from the relative performance of sampled responses and normalizes rewards using the group mean and standard deviation.The paper introduces Critique-GRPO as an extension of this framework.Source: §4.1 · pp. 3–4, §4 · p. 3

4.2. Online RL with Critique-GRPO

• Critique-GRPO combines initial responses with critique-guided refinements, then optimizes the policy using both kinds of trajectories.The framework is built on GRPO and integrates natural-language and numerical feedback.Source: §4 · p. 3, §4.2 · p. 4
• Refinement runs only when the initial response set has no correct solution; a reward system critiques responses, and the system combines selected refinements with initial responses.It prioritizes correct refinements when selecting a subset; if none are correct, it samples randomly.Source: §4.2 · p. 4, Figure 2 · p. 4
• Critiques can prune the search space; when they localize the first error, constructive feedback reduces exploration complexity to linear, or to O(L) when it suggests the correction.The analysis contrasts this with exponentially scaling reward-only exploration and states that critique-guided exploration has higher success probability under a specified fixed computational budget.Source: §4.2 · p. 5
• Critique-GRPO uses a unified reward baseline and shapes refined-response policy ratios to emphasize low-probability refinement tokens, while removing the KL penalty.This is intended to facilitate learning from valid but unfamiliar refinements.Source: §4.2 · p. 5

5.2. Main Results

• Critique-GRPO improves average pass@1 over R1-GRPO and R1-Dr.GRPO by +4.4 points (42.66% →47.08%) on Qwen2.5-7B-Base and +3.8 points (64.46% →68.26%) on Qwen3-8B.It also exceeds Refinement FT by +11.9 points (47.08% vs. 35.21%) and +8.81 points (68.26% vs. 59.45%), and CITL-FT by +11.4 points (47.08% vs. 35.66%) and +12.4 points (68.26% vs. 55.84%), respectively. CoT critiques outperform indicative critiques by +1.8–2.4% on Qwen2.5-7B-Base and +2.0–2.3% on Qwen3-8B.Source: Table 2 · p. 6, §5.2 · p. 6

5.3. Investigation on Math-Centric Backbone Models

• Critique-GRPO improves average pass@1 by 21.6% over Qwen2.5-Math-7B-Base using 4k RL prompts, outperforming numerical-feedback approaches that use 46k prompts.With 4k prompts, it also consistently outperforms PRIME-Zero, which uses 46k prompts and dense intermediate rewards.Source: Table 3 · p. 7, §5.3 · p. 7

5.4. Self-Improvement via Self-Critiquing

• Critique-GRPO with self-critiquing improves average pass@1 by 4.5% over R1-GRPO and 12.0% over SFT.It also outperforms both across pass@k metrics on AIME24 and AIME25 for k ∈[1, 256].Source: Table 4 · p. 7, §5.4 · p. 7, Figure 3 · p. 7

5.5. Exploration with Varying Models

• Critique-GRPO outperforms GRPO on all eight reasoning tasks, with average gains of 5.1 points for Llama-3.2-3B-Instruct and 4.0 points for Qwen3-32B.Alternative proprietary and fine-tuned critique models yield average improvements of 5.6–6.8 points over GRPO.Source: §5.5 · p. 7, Table 5 · p. 8

5.7. Fine-Grained Ablation Studies

• Ablations show cumulative gains from removing KL (+1.5%), adding language feedback (+0.6% to 43.26%), selecting higher-quality refinements (+0.7% to 43.95%), and policy shaping (+3.1% to 47.08%).The authors attribute policy shaping’s gain to amplifying rare reasoning patterns with low initial probability but high success rates.Source: Table 6 · p. 8, §5.7 · p. 8

5.6. Investigation of Policy Exploration

• Critique-GRPO maintains higher policy entropy than R1-GRPO and R1-Dr.GRPO; early peaks reflect divergent self-generated refinements, followed by entropy decreases as the policy internalizes them.The authors connect higher entropy and beneficial distributional shifts with Critique-GRPO’s superior performance.Source: §5.6 · p. 8, Figure 4 · p. 8

5.8. Exploration of Weak-to-Strong Generalization

• Using weaker-model refinements with indicative critiques and ground-truth answers, Critique-GRPO improves average pass@1 by +12.3% over Qwen3-8B and outperforms R1-GRPO (65.55% v.s. 63.75%).The weaker teacher is Qwen3-8B-Base, and the trained strong model is Qwen3-8B.Source: Table 7 · p. 9, §5.8 · p. 9

5.9. Online Joint Optimization vs. Sequential Baseline

• Critique-GRPO significantly outperforms the sequential R1-GRPO plus Refinement-SFT baseline across in-distribution math and out-of-distribution science and general benchmarks.It reaches 77.80% on MATH-500 and 62.50% on AMC23; on GPQA-Diamond, it scores 37.88% versus the baseline’s 34.34%.Source: Table 8 · p. 9, §5.9 · p. 9

• Across eight reasoning tasks, Critique-GRPO consistently outperforms existing SFT and numerical-feedback-only RL methods.Source: §6 · p. 9
• Critique-GRPO is an online RL framework that learns from natural language and numerical feedback simultaneously.Source: §6 · p. 9
• The framework enables learning from initial responses and critique-guided self-refinements simultaneously.Source: §6 · p. 9

Impact Statement

• The authors report following the ICML Code of Ethics, using research-only datasets without personally identifiable information and checking responses for offensive content, misinformation, and personal data.They also say prompts were designed to reduce bias and discriminatory language and potential negative impacts.Source: Impact Statement · p. 10

A. Additional Discussion on Related Work

• Prior natural-language-feedback methods translate critiques into scalar rewards (Kim et al., 2023; Liu et al., 2025b; Lightman et al., 2024; Ouyang et al., 2022; Casper et al., 2023; Rafailov et al., 2024) or use SFT to imitate critiques or refinements (Xi et al., 2024), mostly offline.The authors contrast these approaches with Critique-GRPO’s integration of real-time natural-language feedback into online RL.Source: A. Additional Discussion on Related Work · p. 16
• Numerical rewards are often sparse and can limit progress beyond a model’s current knowledge, while Critique-GRPO combines textual feedback with online exploration to refine outputs.The passage describes expert demonstrations as another response to this limitation (Yan et al., 2025).Source: A. Additional Discussion on Related Work · p. 16

B. Analysis of Training Computational Cost

• 5 GPU-hours for Qwen2.5-7B-Base and 19 for Qwen3-8B (about 2% and 6.5% over GRPO) accompany average gains of +6 and +4.5 percentage points across eight tasks using 4K prompts.The training-time figures are for 300 steps on one NVIDIA A800 GPU; asynchronous rollouts minimize added compute.Source: Table 9 · p. 16, B. Analysis of Training Computational… · p. 16

C. Impact of Critique Length on Critique-GRPO

• 47.08 to 46.00: increasing critique length from about 683 to 2079 tokens slightly lowers performance, though all tested Critique-GRPO settings surpass GRPO.The authors suggest longer contexts make critical information harder to extract.Source: Table 10 · p. 17, C. Impact of Critique Length on… · p. 17
• Inverting critique conclusions, such as swapping “correct” and “incorrect,” triggers entropy explosion and subsequent performance degradation.The test checks whether gains depend on valid guidance rather than arbitrary natural-language signals.Source: C. Impact of Critique Length on… · p. 17

D. Scalability of Critique-GRPO to Large-Scale Models

• Critique-GRPO surpasses GRPO on all eight reasoning tasks with Qwen3-32B.The passage identifies the model and task coverage but gives no per-task values.Source: Table 11 · p. 17, D. Scalability of Critique-GRPO to… · p. 17

E.1. Three Limitations of Learning with Numerical Feedback

• Qwen2.5-7B-Base plateaued at approximately 45-46% accuracy after 120 steps and Qwen3-8B at 65-67% after 200 steps, with negligible peak gains despite 8-fold more prompts or 420 steps.The results indicate a scaling bottleneck: additional data and training compute did not drive further reasoning gains.Source: Figure 5 · p. 18, §E.1 · pp. 18–19
• Planning behaviors contributed more to successful problem-solving than self-reflection behaviors, while spontaneous self-reflection was unreliable for self-correction.Self-reflection contributed noticeably for reasoning-model Qwen3-8B, but backtracking, backward chaining, and anticipation contributed considerably less; it barely contributed for Qwen2.5-7B-Base in mathematics.Source: §E.1 · pp. 19–20, Figure 6 · p. 20
• Approximately 29% of Qwen2.5-7B-Base problems and 3.75% of Qwen3-8B problems remained unsolved with pass@4 = 0 despite trial-and-error fine-tuning.Qwen3-8B had fewer persistent failures but still struggled with some problems under numerical feedback alone.Source: §E.1 · p. 20

E.2. Promise of Learning from Natural Language Feedback

• Indicative critiques with or without ground truth yielded approximately 2%-4% valid refinements, and adding the ground-truth answer without explanation provided little additional guidance.Even a binary correctness signal produced some successful refinements, suggesting natural-language feedback can augment numerical signals.Source: §E.2 · p. 22

E.3. Leveraging Textual Critiques for Refining LLM Responses

• Critique-guided refinement used a reasoning-based reward model to generate CoT critiques and derive binary-label-only or ground-truth-answer critiques, then refined responses to questions whose initial samples were all incorrect.The section defines a persistently failed question as one for which all k initial responses are incorrect.Source: §E.3 · pp. 22–23

E.2. Promise of Learning from Natural Language Feedback

• 36.47% and 10.63% valid refinement rates and 55.37% and 20.00% successfully refined questions were achieved with CoT critiques on Qwen2.5-7B-Base and Qwen3-8B, respectively.These rates surpass those from indicative critiques and critiques with ground truth, although CoT critiques were valid for only 60.06% and 50.17% of erroneous responses, respectively.Source: §E.2 · p. 22, Table 12 · p. 23

E.3. Leveraging Textual Critiques for Refining LLM Responses

• For each selected incorrect response, the original LLM generates a refinement conditioned on a refinement instruction, the question, the failed response, and its associated critique.The full self-refinement process is summarized in Algorithm 1.Source: §E.3 · p. 23

E.4. Investigation on Qwen3-8B-Base

• The best-performing RL-finetuned Qwen3-8B-Base persistently failed on 17.18% of training problems, all three critique types facilitated self-refinement, and self-reflection contributed minimally to solving previously failed questions.The investigation covered 50 previously unsolved Minerva-Math problems and 15 GPQA-Diamond problems.Source: §E.4 · p. 23, Figure 7 · p. 23

F.1. The Critique-GRPO Algorithm

• Critique-GRPO samples initial responses, generates CoT critiques, produces and scores refinements, then samples k refinements to mitigate distribution shift.The algorithm uses a reward model to label critiques and refinements.Source: §F.1 · p. 24

F.2. The shaping Function in Critique-GRPO

• With γ=0.1, f(x)=x/(x+γ) amplifies low token probabilities more strongly, helping learn from unfamiliar correct refinements while penalizing unfamiliar incorrect ones.The shaping function is bounded between 0 and 1; larger γ values produce less pronounced scaling.Source: §F.2 · p. 24, Figure 8 · p. 24

F.3. Ratio of Responses per Prompt in Critique-GRPO

• A 7:1 initial-response-to-refinement ratio produced stable training and optimal performance; lower ratios degraded performance as refinement-induced distribution shifts sharply increased entropy loss late in training.Source: §F.3 · p. 24

F.4. Theoretical Analysis of Sample Efficiency; 1. Derivation of Effective Dimension (d)

• Constructive critiques that localize errors reduce reasoning-search complexity from exponential O(|S|^L) to O(L|S|), or O(L) when they suggest corrections.Reward-only feedback is described as a sparse needle-in-a-haystack signal; indicative critiques prune the search space by a constant factor, while constructive feedback decomposes it into sequential subproblems.Source: §F.4 · p. 26

2. Probability of Success

• For a fixed budget L|S| ≪ M ≪ |S|^L, critique-guided exploration has a strictly higher success probability than reward-only learning.The analysis compares searching the full action space with searching the smaller effective space induced by critiques.Source: §F.4 · p. 26, §2 · p. 27

F.5.1. INEFFICIENCY OF STANDARD EXPLORATION

• In standard GRPO, when p* < 1/G, many batches have no positive reward and produce wasted updates with Δθ = 0.The expected number of positive rewards in a batch of size G is Gp*; even when a positive signal occurs, the update toward the optimal policy is proportional to p*.Source: §F.5.1 · p. 27

F.5. Theoretical Analysis of Convergence and Policy Improvement; F.5.2. POLICY IMPROVEMENT VIA CRITIQUE-GUIDED REFINEMENT

• When Pcrit ≫ p*, including refined outputs makes optimal trajectories more frequent, reduces advantage variance, and yields a higher lower bound on policy improvement.The effective probability of an optimal trajectory in a batch is approximately Pcrit.Source: §F.5.2 · p. 28

F.5.2. POLICY IMPROVEMENT VIA CRITIQUE-GUIDED REFINEMENT

• Critique-GRPO’s convergence rate is accelerated by O(Pcrit/p*) over standard GRPO, assuming sufficient policy expressiveness and a Robbins-Monro learning rate.The analysis attributes the gain to critique-refined trajectories increasing the probability of observing optimal solutions in training batches.Source: §F.5.2 · p. 28

F.6.1. GRADIENT DYNAMICS OF THE SHAPING FUNCTION; 2. Peak Efficiency in the Learning Zone: Solving dΨ

• The shaping function dampens gradients for easy tokens and, with γ ≈ 0.1, maximizes them for unlikely but plausible correction tokens.This focuses updates on the reasoning gap between initial and refined responses while limiting overfitting to tokens the policy has mastered.Source: §F.6.1 · p. 29, §2 · p. 29

F.6.2. THE INSUFFICIENCY OF CLIPPING FOR OFF-POLICY DATA

• For off-policy refinements, PPO/GRPO clipping cannot resolve the often-orthogonal update direction, and unregularized refinement updates can pull the policy off its manifold and cause entropy collapse.Refinement updates shift probability toward regions disjoint from the initial response mode, conflicting with updates that reinforce the current mode.Source: §F.6.2 · p. 29

F.6.3. MANIFOLD PRESERVATION VIA THE ANCHOR HYPOTHESIS

• Maintaining Ninit ≫ Nref keeps the update within the policy’s trust region by anchoring it to the on-policy manifold’s curvature while treating refined-sample updates as perturbations.The anchor limits the influence of high-variance, off-distribution refinement gradients.Source: §F.6.3 · p. 30

G. Implementation Details

• Evaluation covers mathematical, scientific, and general reasoning benchmarks, using greedy decoding and reporting pass@1 over three runs.The mathematical benchmarks are MATH-500, Minerva-Math, OlympiadBench, AIME 2024, AIME 2025, and AMC 2023; broader evaluation uses TheoremQA, GPQA-Diamond, and MMLU-Pro.Source: G. Implementation Details · p. 31
• RL fine-tuning uses model-based critique generation and rule-based binary scalar rewards; critiques inconsistent with rule-based evaluations are regenerated.For pass@k evaluation, the section specifies temperature 0.6 and top-p 0.95.Source: G. Implementation Details · pp. 31–32
• Supervised comparisons include SFT, RAFT (Dong et al., 2023), Refinement FT, Critique FT, and CITL-FT (Xi et al., 2024).All reported differences are considered significant at p < 0.01.Source: G. Implementation Details · pp. 31–32
• RL comparisons include R1-GRPO (DeepSeek-AI et al., 2025), R1-Dr.GRPO, and Critique-GRPO variants using indicative critiques or ground-truth-assisted refinements.R1-GRPO uses binary scalar rewards, while R1-Dr.GRPO removes terms that cause biased optimization and also uses binary scalar rewards.Source: G. Implementation Details · p. 32
• Experiments use Qwen2.5-7B-Base, Qwen2.5-Math-7B-Base, and Qwen3-8B, with GPT-4o as the reward model; RL runs for 400 steps, while supervised baselines train to convergence.For R1-GRPO, eight responses are sampled per prompt; Critique-GRPO samples seven responses plus one refined response. Experiments use 40 NVIDIA A800 80G GPUs.Source: G. Implementation Details · p. 32
• For Qwen2.5-Math-7B-Base, the RL comparison includes SimpleRL-Zero, PRIME-Zero, and Oat-Zero.PRIME-Zero uses outcome and process binary rewards; Oat-Zero uses Dr.GRPO with outcome binary rewards.Source: G. Implementation Details · p. 32

H. Detailed Results and Analysis of Self-Critiquing Mechanisms

• +4.5% pass@1 over R1-GRPO and +12.0% over SFT on average for Critique-GRPO (self-critique).Critique-GRPO (self-critique & self-evaluation), which uses no reference answer, improves average pass@1 by 2.3% over R1-GRPO.Source: Table 14 · p. 33, H. Detailed Results and Analysis of… · p. 33
• 10–16.7% improvements over R1-GRPO on AIME24 pass@k for k = 1 to 4 show gains from self-critiquing.Critique-GRPO (self-critique) performs consistently better across pass@k metrics from k = 1 to 256.Source: Figure 9 · p. 33, H. Detailed Results and Analysis of… · p. 33
• Self-critique evaluates responses with CoT critiques and ground-truth answers as reference, while self-critique & self-evaluation uses no reference.The model itself serves as a reasoning-based reward model in both variants.Source: H. Detailed Results and Analysis of… · p. 33

• On Qwen2.5-7B-Base, Critique-GRPO generally maintains higher policy entropy than R1-GRPO and R1-Dr.GRPO, whose entropy falls to near zero.Critique-GRPO’s early entropy peaks likely reflect self-generated refinements that differ from initial responses; later decline suggests the model internalizes them.Source: Figure 10 · p. 34, §I · p. 34
• On Qwen3-8B, Critique-GRPO self-critique avoids entropy collapse and maintains higher entropy than R1-GRPO.The passage associates this greater exploration with improved performance.Source: Figure 10 · p. 34, §I · p. 34
• Although ground-truth critique refinement has higher entropy than self-critique, its average pass@1 is 65.55% versus 68.13% for self-critique.The authors suggest weaker refinements cause larger distribution shifts but are lower quality, making exploration-signal quality more important than entropy alone.Source: §I · p. 34, Figure 10 · p. 34
• Critique-GRPO improves performance with only a minimal response-length increase on Qwen2.5-7B-Base and tends to reduce response length on Qwen3-8B.The authors attribute this to precise error correction and, for Qwen3-8B, reducing redundant and ineffective self-reflection.Source: §I · p. 34, Figure 11 · p. 35

Fine-Grained Analysis

• On Minerva-MATH, Critique-GRPO achieves the best average and scores across factuality, conciseness, correctness, and logicality, outperforming R1-GRPO in factuality and correctness.The analysis covers 100 generated responses.Source: J. Qualitative Analysis · p. 36, Figure 12 · p. 36
• SFT's weaker logicality and conciseness may reflect redundant, sometimes illogical self-reflective reasoning in its expert demonstrations.Source: J. Qualitative Analysis · p. 36

Case Study

• In a MATH-500 example, Critique-GRPO corrects Qwen3-8B’s variable formulation, yielding a correct concise answer where the base model's ineffective self-reflection exceeds 6,000 tokens.The RL-finetuned model also uses effective self-reflection to validate its answer.Source: J. Qualitative Analysis · p. 36

K. Limitations

• Policy models sometimes fail to follow CoT critiques, which the authors attribute to a lack of deliberate self-refinement training.They suggest improving refinement capabilities or training a specialized refinement model.Source: K. Limitations · p. 39
• CoT critiques provide the greatest refinement benefits, but expert demonstrations may be ineffective when refinements favor conclusive answers over step-by-step reasoning.The authors use GPT-4o for reasoning-based critiques rather than expert-knowledge distillation, and suggest exploring other reward models and critique types.Source: K. Limitations · p. 39

L. Analysis of Cognitive Behaviors

• The analysis tracks six cognitive behaviors: subgoal setting, summarization, verification, backtracking, backward chaining, and anticipation.The paper defines these as behaviors that can contribute to self-improving reasoning during RL fine-tuning.Source: L. Analysis of Cognitive Behaviors · p. 40
• Behavior analysis counts each behavior once per response and prompts GPT-4o to count instances and assess whether behaviors contributed to correct answers.The prompt covers verification, backtracking, subgoal setting, backward chaining, anticipation, and summarization.Source: L. Analysis of Cognitive Behaviors · pp. 40–41

M. Prompts

• The RL training prompt requests step-by-step reasoning, while the refinement prompt asks the model to address critique-identified errors and provide a corrected solution.GPT-4o-generated CoT critiques are regenerated when their correct/incorrect judgments disagree with rule-based verification.Source: M. Prompts · pp. 42–43
• The qualitative-analysis prompt rates answer correctness, verbosity, factual accuracy, and logical coherence on a 1–5 scale.Future work could design prompts for higher-quality CoT critiques (Zhang et al., 2023).Source: M. Prompts · p. 43

N. An Example of Successful Refinement using a CoT Critique

• In the example, a CoT critique identifies the initial solution’s flawed similarity-triangle relation and angle calculation, then redirects it to the axial triangle’s inradius formula.The refined response begins applying that alternative approach; the supplied passages do not provide its final calculated answer.Source: N. An Example of Successful Refinement… · pp. 44, 46, 47

O. Responses Utilized in Qualitative Analysis

• A cited calculation gives sin(A+B)·sin(A−B)=sin²A−sin²B=−9, while another reports the original expression as 9/100 (0.09), confirmed by 0.805×0.1118≈0.09.Source: O. Responses Utilized in Qualitative… · pp. 50, 53

P. An Example of Failed Refinement

• In Example 15, Critique-GRPO fails to follow the CoT critique, which flags flawed substitutions, identities, derivatives, constraint checks, and the unsupported maximum 105/32 (correct: 10/3).The critique concludes that the initial solution is wrong.Source: P. An Example of Failed Refinement · p. 54
• The refinement uses positivity to require a positive denominator when isolating b, restricting the possible values of a and c.It rewrites the constraint as b(1 − ac) = a + c and hence b = (a + c)/(1 − ac).Source: P. An Example of Failed Refinement · p. 57
• Using a = tan α and c = tan γ, the response rewrites the objective as p = 2 cos^2 α − 2 cos^2(α + γ) + 3 cos^2 γ.It also sets θ = α + γ and expresses γ as θ − α.Source: P. An Example of Failed Refinement · p. 57
• The response assumes α = γ without deriving that condition, reducing p to 5 cos^2 α − 2 cos^2(2α) and then a quadratic in cos^2 α.The earlier displayed reduction gives p = −8x^2 + 13x − 2, but the supplied passages do not show a completed maximum calculation.Source: P. An Example of Failed Refinement · pp. 56, 58