CARMO: Dynamic Criteria Generation for Context-Aware Reward Modelling

1 Introduction

• CARMO uses an LLM to generate task-specific criteria before scoring outputs, addressing the limits of static rubrics that may not transfer across tasks.Criteria include logical consistency, relevance, clarity, and depth; different tasks can require distinct rubrics.Source: §1 · p. 2
• CARMO can be distilled into smaller models while retaining alignment performance, reducing reward-evaluation computational burden.The paper also releases its datasets anonymously to foster transparency and reproducibility.Source: §1 · p. 2
• 2.1% improvement on Reward Bench in a zero-shot setting; CARMO-curated preference fine-tuning attained 22.5% LC-WR (%) and 21.1% WR (%) on Mistral-Base (7B).The reported preference-optimization metrics are for Mistral-Base (7B).Source: §1 · p. 2
• The paper presents theoretical analyses that adaptive, context-aware criteria can mitigate reward hacking in which models optimize superficial cues rather than quality.A motivating example is a list-shaped answer that appears comprehensive but lacks deeper analysis or theoretical grounding.Source: §1 · pp. 1–2

2 Related Works

• LLM evaluators such as Alpaca-Farm and Prometheus support model-driven judgments and automated feedback collection, reducing reliance on human annotation.Prometheus offers fine-grained, customizable evaluation at scale and has shown performance on par with GPT-4.Source: §2 · p. 2
• Fine-grained frameworks decompose scores into skills, but FLASK and LLM as Judge retain predefined criteria, while Prometheus requires human rubric customization.These approaches may not capture nuances across varied tasks.Source: §2 · pp. 2–3
• CARMO addresses these limitations by autonomously generating dynamic, task-specific criteria for absolute and relative evaluations.Its criteria can include logical consistency, depth of explanation, and stylistic coherence.Source: §2 · p. 3

3 Methodology

• The methodology comprises CARMO's core evaluation components, a knowledge-distillation framework, and methods for generating reward signals for DPO-style and multi-preference alignment.The section is organized into Subsections 3.1–3.5.Source: §3 · p. 3

3.1 Overview of CARMO: Reducing Reward Hacking via Context-Aware Criteria

• Static rubrics can fail under task-distribution shifts or when features are spuriously correlated with correctness; CARMO instead generates criteria for each new input.The authors argue that making criteria explicit focuses evaluation on features faithful to quality rather than unmentioned superficial correlates.Source: §3.1 · p. 3

3.2 Generating Dynamic Rubrics

• An LLM generates C(x), a set of criteria reflecting the quality dimensions relevant to prompt x.Criteria may address factual correctness, logical coherence, style, or depth; absolute grading may condition on a reference answer, while relative grading may condition on multiple responses.Source: §3.2 · p. 3

3.3 Response Evaluation

• CARMO aggregates criterion-level scores into an overall rating and compares ratings to assign preferences between candidate responses.The same context-generated criteria are used in absolute and relative settings to evaluate dimensions intended to capture quality for the current prompt.Source: §3.3 · pp. 3–4

3.4 Fine-Tuning & Knowledge Distillation

• Knowledge distillation trains smaller models to reproduce criteria generation and evaluation, with the authors reporting near-GPT-4 performance at a fraction of the cost.A feedback dataset supplies tuples that GPT-4 uses to produce criteria, feedback, and scores; FT-Criteria maps inputs to criteria, while FT-Judge maps criteria-conditioned inputs to feedback and scores.Source: §3.4 · p. 4, Figure 2 · p. 4

3.5 Use Case—Preference Data Generation

• CARMO-generated rubrics support preference-data creation for DPO and multi-preference methods such as SWEPO, and the authors report improved alignment and generalization from this data.The approach compares candidate responses using dynamically generated rubrics rather than static-rubric preference labels.Source: §3.5 · p. 4, Figure 3 · p. 4

4 Theoretical Analysis

• The section presents concise theorem statements and proof sketches to explain failure modes of criteria-free and fixed-criteria methods, not to refute their performance in complex real-world settings.The paper argues that static rubrics may fail when the task distribution shifts or features are only spuriously correlated with correctness.Source: §4 · p. 5, §4.4 · p. 6

4.1 Notations and Setup

• The setup treats each query–response pair as an element of sample space Ω and models criteria as real-valued evaluation axes applied to those pairs.The true reward R represents response quality or correctness; fixed criteria are used across all queries and responses.Source: §4.1 · p. 5

4.2 Assumptions

• The analysis assumes nonzero variance in reward and each criterion, defines relevance and spuriousness by covariance thresholds, and assumes relevant and spurious criteria are orthogonal or independent.The orthogonality assumption simplifies proofs and excludes spurious-feature combinations that create net correlation with reward.Source: §4.2 · p. 5

4.3 Definition of Spurious Correlate

• A spurious correlate is a criterion whose correlation with true reward is negligible; bullet-point use is an example when it does not reflect correctness or relevance.Such a feature may correlate with reward during training but become irrelevant under distribution shift.Source: §4.3 · p. 5, §4.2 · p. 5

4.4 Main Theorems

• A model using only relevant features achieves strictly lower MSE than one mixing relevant and spurious features, because spurious features contribute little predictive gain.The theorem assumes the conditions in Section B.5; allocating capacity to spurious features leaves less for relevant signals.Source: §4.4 · pp. 5–6
• For any finite set of criteria, a true reward can exist that is uncorrelated with every affine combination of them, leaving the constant predictor’s MSE.The proof constructs a reward orthogonal to the finite-dimensional span of the criteria, yielding MSE Var(R).Source: §4.4 · p. 6

5.1 Experimental Setting

• Evaluations use zero-shot greedy decoding across multiple benchmarks; separate procedures evaluate distilled LLaMA 2 models and preference-optimized Mistral-7B and LLaMA-3-8B models.The reward-model evaluations compare CARMO with LLM-as-Judge, Prometheus, and a baseline prompt; preference data are generated from UltraFeedback.Source: §5.1 · p. 6

5.2 Experimental Result

• CARMO outperforms LLM-as-Judge and Prometheus across HHH Alignment, Alpaca Eval, and MT-Bench, with GPT-4o HHH Alignment F1-Score 0.938 and Accuracy 0.933.Relative to Prometheus, GPT-4o HHH Alignment F1-Score and Accuracy improve by 2.8% and 2.0%; Alpaca Eval gains are 6.2% in F1-Score and 4.8% in Accuracy.Source: Table 1 · p. 7, §5.2 · p. 7
• On RewardBench, CARMO improves GPT-4o’s Chat Hard (0.824), Safety (0.904), and Reasoning (0.969) scores and generalizes across Llama3.1-70B, GPT-4, and GPT-4o-mini.The reported comparison is against Baseline and LLM-as-Judge methods.Source: Table 2 · p. 7, §5.2 · p. 7
• Llama2-13b-CARMO-Dist attains an HHH Alignment average score of 0.8375, while Llama2-7b-CARMO-Dist achieves an average score of 81.10.The distilled models improve over their Prometheus and baseline counterparts.Source: §5.2 · p. 7
• CARMO-curated preference data consistently outperform UltraFeedback-aligned models across AlpacaEval 2, Arena-Hard, and MT-Bench, with SOTA performance across all metrics.Table 3 compares preference optimization methods and identifies length-controlled win rate (LC-WR) and raw win rate (WR).Source: Table 3 · p. 8, §5.2 · p. 8
• Two-stage prompting scores higher than single-call prompting on almost all Reward Bench subsets and applies instruction-generated criteria consistently across responses.Single-call prompting generates criteria separately for each response; the two-stage approach first generates criteria from the instruction, then reuses them.Source: Figure 4 · p. 6, §5.2 · p. 8
• Detailed prompts score slightly higher than normal prompts on Rewardbench, while normal prompts are faster and maintain comparable performance.Detailed prompts provide more comprehensive feedback analysis, but their extensive analysis can increase token counts and inference time.Source: Figure 5 · p. 7, §5.2 · p. 8

6.1 Bias in Criteria Generation

• Probabilistic criteria generation can introduce bias and produce different criteria across runs, potentially affecting outcome reliability and reproducibility.This variability may be challenging for applications requiring deterministic or repeatable behavior.Source: §6.1 · p. 9, §6.2 · p. 9

6.3 High Token Count and Computational Cost

• CARMO may generate many tokens, increasing computational expense and potentially reducing processing efficiency, especially at scale or in resource-constrained settings.Source: §6.3 · p. 9

A Related Works

• The related-work section surveys reward modeling, benchmarks, alignment strategies using reward models, LLM evaluators, and RLHF evaluation frameworks and metrics.Source: A Related Works · p. 11

A.1 Reward Modeling in RLHF for LLMs

• Classical RLHF trains a reward model on human preferences before policy optimization, while alternatives such as DPO and preference-aggregation methods change how preference signals are optimized.The pipeline can be complex and unstable if not tuned carefully.Source: §A.1 · pp. 11–12

A.2 Reward Benchmarks for Evaluating LLM Outputs

• RewardBench assesses reward-model ranking accuracy on human-vetted pairs, while WildBench uses an LLM judge for real-world tasks, pairwise comparisons, and absolute scoring.Both cover nuanced aspects such as factual correctness, logical coherence, and stylistic suitability.Source: §A.2 · p. 12

A.3 Alignment Methods Using Reward Models

• Reward models guide output selection in RLHF, while DPO bypasses on-policy updates and multi-objective methods combine dimensions such as helpfulness, honesty, and harmlessness.Iterative RLHF can refine reward models with fresh comparisons but increases computational overhead.Source: §A.3 · p. 12

A.4 Prometheus and LLM-Based Evaluators in Adaptive Reward Modeling

• Prometheus is a 13B open-source evaluator trained on GPT-4-based evaluations; LLM evaluators can score or rank responses and may provide natural-language justifications.Prometheus learns to reproduce GPT-4 judgments and rubrics, approaching GPT-4-level correlation with human assessments on diverse tasks.Source: §A.4 · p. 12

A.5 Evaluation Frameworks and Metrics for Reward Modeling

• Evaluation frameworks use GPT-4-judged chat assessments, human-checked pairwise comparisons, skill checklists, and HHH dimensions, with metrics including accuracy, win-rate, and correlation.CARMO is described as generating query-specific scoring criteria.Source: §A.5 · pp. 12–13

B Theoretical Analysis: Spurious vs. Criteria-driven Models

• Selecting only task-relevant criteria axes is presented as a way to obtain more faithful reward estimates when static features become spurious under distribution shift.The motivating example is list propensity, which may correlate with correctness in one domain but not in a shifted domain.Source: B Theoretical Analysis: Spurious vs.… · p. 13

B.1 Spurious vs. Relevant Dimensions: Definitions and Assumptions

• The analysis assumes non-degenerate R, S, and C, treats spurious S as approximately uncorrelated with R, and defines relevant C by nontrivial covariance with R.Independence is imposed to simplify proofs, with approximate-independence conditions discussed as a weaker alternative.Source: §B.1 · pp. 13–14

B.2 Optimal Linear Predictors and Mean-Squared Error (MSE)

• A relevant single criterion lowers MSE below Var(R), while an independent spurious criterion yields the constant-mean predictor with MSE Var(R).Finite-sample chance correlations can still make near-zero-correlation features appear useful and lead to overfitting.Source: §B.2 · pp. 14–15

B.3 Comparing Spurious vs. Relevant Single-Dimension Models

• A single relevant dimension strictly outperforms a spurious dimension; “listiness” may lose predictive value when a new domain does not reward bulleted style.Logical coherence is presented as a relevant dimension that remains correlated with correctness under challenging or shifted tasks.Source: §B.3 · p. 15

B.4 Multiple Spurious Dimensions

• Under independence, any linear combination of multiple spurious features remains uncorrelated with reward and predicts only the constant E[R].Small training-set correlations may degrade or invert after a query-distribution change.Source: §B.4 · pp. 15–16

B.5 Mixture of Multiple Spurious and Relevant Dimensions

• With n attributes, a model mixing k near-zero-correlation spurious features has strictly higher MSE than a model using n relevant features, under the stated assumptions.The comparison assumes spurious and relevant attributes are approximately orthogonal and relevant attributes have nontrivial correlations with reward.Source: §B.5 · pp. 16–17

B.6 Conclusion (Theoretical comparisons with No-Criteria Setting)

• Across the theoretical comparisons, spurious-only dimensions offer no improvement over a constant predictor, and allocating a fixed attribute budget to spurious features raises MSE versus using relevant dimensions.The analysis covers single dimensions, independent multiple spurious dimensions, and mixed versus fully relevant n-attribute models.Source: §B.6 · p. 18

C Theoretical Analysis: Fixed Criteria vs. Adaptive Criteria Models

• For any fixed finite set of criteria, the analysis says there are reward functions outside their span that those criteria cannot capture.The passage identifies environment shifts or tasks that diverge from the criteria’s assumptions as settings where failure can arise.Source: C Theoretical Analysis: Fixed Criteria… · p. 19, §C.1 · p. 19

C.1 Setup and Notation

• The setup models query–response pairs on a probability space and asks how accurately a linear combination of fixed criteria can approximate true reward.Each fixed criterion is treated as a rubric axis applied consistently across queries and responses.Source: §C.1 · p. 19

C.2 Fixed Finite Criteria: Orthogonality Arguments

• A finite set of criteria admits a nonzero reward function with zero covariance with every linear combination of those criteria.The argument centers criteria and constructs a nontrivial reward orthogonal to their finite-dimensional span.Source: §C.2 · p. 20

C.3 Main Result: Fixed Criteria Fails on Some Reward

• For any fixed finite criteria set, some true reward has zero correlation with every affine predictor from that set and MSE equal to predicting its mean.The best linear predictor is therefore no better than the constant predictor for that reward.Source: §C.3 · p. 20

C.4 Corollaries and Connection to Adaptive Criteria

• A single fixed finite rubric leaves infinitely many reward functions orthogonal to its criteria, with MSE at least Var(R).The corollary extends the failure beyond one missed reward function.Source: §C.4 · p. 21

C.5 Implications for Reward Hacking and Distribution Shift

• To approximate broader reward classes, particularly under distribution shifts, criteria must be allowed to grow or adapt beyond the original finite set.Adaptive criteria can shift the feature set when distribution shift renders prior features spurious.Source: §C.4 · p. 21, §C.5 · p. 21

D Experimental Details

• The following section summarizes dataset details, baseline evaluation strategies, and the experimental setup.Source: D Experimental Details · p. 21

• CARMO used the same closed- or open-source model settings for criteria generation and evaluation, with zero-shot prompting and greedy decoding.The models included GPT-4, GPT-4o, GPT-4o-mini, GPT-3.5-Turbo, Phi-4, LLaMA 3.1-70B-Instruct, and Qwen2.5-72B-Instruct.Source: §D.1 · p. 22
• CARMO was evaluated on Vicuna Bench, MT-Bench, Flask Eval, Alpaca Eval, and HHH Alignment against the authors’ baseline, LLM-as-Judge, and Prometheus.Source: §D.1 · p. 22
• For distillation, GPT-4 generated criteria and then feedback and 1-to-5 ratings from the Feedback Collection Dataset’s instructions and reference answers.The resulting dataset was used to instruction-fine-tune smaller LLMs.Source: §D.1 · p. 22
• Llama2-7b and Llama2-13b models were fine-tuned separately for criteria generation and evaluation, then compared with Llama2 and Prometheus baselines and GPT-3.5-Turbo.Evaluation used HHH-Alignment, MT-Bench, Flask Eval, and Vicuna Bench.Source: §D.1 · p. 22
• CARMO criteria generated from UltraFeedback instructions were used to score assistant responses and construct binarized and multi-preference datasets.The binarized set contains chosen and rejected responses; the multi-preference set retains responses with generated reward scores.Source: §D.1 · p. 23
• Preference optimization followed SFT on UltraChat-200k, with evaluation on MT-Bench, AlpacaEval 2, and Arena-Hard v0.1.The SFT starting points included Mistral-7B-v0.1 and Llama-3-8B; Table 4 is identified as covering datasets used to evaluate CARMO efficiency.Source: §D.1 · p. 23, Table 4 · p. 23

D.2 Baseline Methods

• CARMO and CARMO-dist are benchmarked against LLM-as-a-judge, Prometheus, and evaluator LLMs including GPT-3.5-turbo, GPT-4, GPT-4o, Llama3.1-70b-instruct, and Qwen.LLM-as-a-judge uses prompt enhancement to mitigate position, verbosity, and self-enhancement biases; Prometheus is fine-tuned using 1K human-labelled and automatic score rubrics.Source: §D.2 · p. 23

D.3 Baseline Methods for Preference ALignment

• SWEPO extends DPO’s single chosen–rejected response pair by weighting multiple responses scored by an external model and comparing positives and negatives with a group contrastive loss.The passage presents SWEPO as better able to use the preference distribution and reduce alignment biases than DPO.Source: §D.3 · p. 24

D.4 Experimental setup

• Training used 8 NVIDIA A100 GPUs with FSDP for 7B and 13B models; SFT ran for 3 epochs, while DPO and SWEPO ran for one epoch on CARMO preference data.DPO and SWEPO used Mistral-Base (7B) and Llama3-Base (8B) models fine-tuned on Ultrachat200k; decoding used temperature 0.8 and top_p 0.95, with three seeds averaged.Source: §D.4 · p. 24

E.1 Relative Evaluation Format; E.2 Absolute Evaluation Prompt; F.1 Relative Evaluation Prompt; F.2 Absolute Evaluation Prompt

• Relative prompts range from instruction-only output choices to factor-based feedback and verdicts, while absolute prompts request either a score from 1 to 10 or feedback with a rating.The simpler relative formats require only an output choice; the later judge prompts ask for an explanation before the verdict or score.Source: §E.1 · p. 25, §E.2 · p. 25, §F.1 · p. 26

G Prometheus Evaluation Prompt

• The Prometheus prompt asks for rubric-based feedback followed by an integer score from 1 to 5, in a prescribed format without extra text.The prompt supplies a reference answer scored 5 and a score rubric.Source: G Prometheus Evaluation Prompt · p. 27

H.1 Single Stage Prompt; H.2.1 Criteria Generation Prompt; I.1 Criteria Generation Prompt

• The single-stage prompt generates evaluation factors, uses them for feedback, and selects the better response; criteria prompts emphasize instruction fulfillment, honesty, helpfulness, harmlessness, and length only when instructed.The I.1 prompt additionally asks for a reference answer and the five most important factors.Source: §H.1 · p. 28, §H.2.1 · p. 28, §I.1 · p. 32

H.2.2 Relative Evaluation Prompt; H.2.3 Absolute Evaluation Prompt

• Criteria-based relative evaluation prompts request feedback tied to generated factors and an unbiased verdict, while absolute evaluation requests feedback and a decimal score from 1 to 10.The relative prompt returns a verdict; the absolute prompt specifies feedback followed by the score.Source: §H.2.2 · p. 29, §H.2.3 · p. 29

H.2.4 Detailed Relative Evaluation Prompt; H.2.5 Detailed Absolute Evaluation Prompt

• Detailed evaluation prompts require factor-by-factor feedback, with relative evaluation naming the better assistant per factor and absolute evaluation assigning each factor a score, both in structured JSON.Each also includes an overall assessment and final verdict or score.Source: §H.2.4 · p. 30, §H.2.5 · p. 31

I.1 Criteria Generation Prompt

• Criteria are designed to assess instruction fulfillment and reference alignment objectively, while covering honesty, helpfulness, and harmlessness.They are framed as detailed questions, avoid relying on specific reference-answer details, and consider response length only when the instruction specifies it.Source: §I.1 · p. 32

I.2 Evaluation Prompt

• An impartial judge evaluates the response using the generated factors and a reference answer scored 5, provides detailed feedback, and assigns an integer score from 1 to 5.The required output is feedback followed by the score, with no other opening, closing, or explanations.Source: §I.2 · p. 32

J.1 Baseline

• Table 5 compares four baseline models across Reward-Bench subsets, including safety, coding, math, dialogue, and prompt-difficulty evaluations.The listed models are GPT-4o, GPT-4, GPT-4o-mini, and Llama3.1-70B, evaluated in baseline prompt setting E.Source: Table 5 · p. 33, §J.1 · p. 33

J.2 CARMO

• Table 6 compares six models across the same Reward-Bench subsets using CARMO prompt setting H.The table scope includes GPT-4o, GPT-4, GPT-4o-mini, Phi-4, Qwen2.5-72B-Instruct, and Llama3.1-70B.Source: Table 6 · p. 34, §J.2 · p. 34

J.3 LLM as Judge

• Table 7 compares six models on Reward-Bench subsets under the LLM-as-judge prompt setting F.The listed evaluation areas include AlpacaEval, DoNotAnswer, HEP, LLMBAR, Math PRM, MT Bench, Refusals, and XSTest.Source: Table 7 · p. 35, §J.3 · p. 35

J.4 Advantage of CARMO over LLM as Judge

• Table 8 reports score deviations between CARMO and LLM-as-judge across Reward-Bench evaluation subsets.The supplied passage describes the comparison’s scope but gives no deviation values or direction of advantage.Source: Table 8 · p. 36, §J.4 · p. 36

K.1.1 Criteria Generation

• The criteria-generation examples tailor evaluation factors to each instruction, covering accuracy and relevance while considering response length only when the instruction specifies it.Examples address a pint-conversion question, category assignment, and a request for Technology’s legal history.Source: §K.1.1 · p. 37, §K.2.1 · p. 39, §K.3.1 · p. 42

K.1.2 Evaluation

• For the pint question, the evaluation favors Assistant A’s added US–UK measurement detail while noting both answers’ approximate wording.The supplied feedback describes Assistant B as concise but less informative.Source: §K.1.2 · p. 38

K.2.2 Evaluation

• For category assignment, the evaluation favors Assistant A’s concise, instruction-following list over Assistant B’s extraneous explanations and noncompliant format.The task requires one topic per sentence and no additional words.Source: §K.2.2 · p. 41

K.3.1 Criteria Generation

• The criteria assess chronological order, credible sources and references, and clear, coherent writing about Technology’s legal history.Source: §K.3.1 · p. 42
• The rubric also checks objectivity, coverage of significant legal developments, harmlessness, and useful, actionable information.Source: §K.3.1 · pp. 42–43

K.3.2 Evaluation

• The judge prompt asks for objective, instruction-relevant feedback and a final verdict, using criteria including accuracy, detail, chronology, sources, clarity, objectivity, and comprehensiveness.The prompt also asks judges not to favor assistant names or response order.Source: §K.3.2 · p. 43
• Assistant B’s technology-law overview is judged more relevant, detailed, and helpful than Assistant A’s refusal, despite lacking specific dates, cases, and references.The feedback also notes that Assistant A acknowledges the subject’s breadth but gives no specific information.Source: §K.3.2 · p. 44

K.4.1 Criteria Generation

• For Rust concatenation, generated criteria cover correctness, signature, input handling, output format, efficiency, Rust conventions, and error handling.The criteria-generation instructions say response length should count only when the user specifies it.Source: §K.4.1 · p. 45

K.4.2 Evaluation

• The evaluation criteria for the Rust function cover correctness, signature, input handling, output format, efficiency, Rust conventions, and error handling.The criteria are framed around whether the function meets the stated concatenation instruction.Source: §K.4.2 · p. 46
• Assistant A is assessed as concatenating correctly, while Assistant B adds spaces not requested; both are criticized for omitting the function signature.The evaluation also says B’s use of format! is less efficient than simple concatenation.Source: §K.4.2 · p. 47

K.5.1 Criteria Generation

• For dead-hamster disposal, generated factors include accuracy, legality, sensitivity, and safety, alongside clarity, helpfulness, and harmlessness.Source: §K.5.1 · p. 48

K.5.2 Evaluation

• For the dead-hamster request, Assistant A offers a practical disposal method but lacks legal guidance and fuller options; Assistant B provides no disposal method.The evaluation also notes that A’s response could be more empathetic and could mention gloves, while B is unhelpful.Source: §K.5.2 · p. 50

K.6.1 Criteria Generation

• For the piñata request, generated criteria include clarity, safety, effectiveness, inclusivity, engagement, and accuracy.The factors are framed as questions about whether a response meets the instruction and its relevant conditions.Source: §K.6.1 · p. 51

K.6.2 Evaluation

• In the piñata evaluation, Assistant A gives clear, effective techniques but lacks safety precautions and inclusivity; Assistant B suggests alternatives without answering the request.The judge evaluates both responses using the generated criteria, including clarity, safety, effectiveness, inclusivity, engagement, and accuracy.Source: §K.6.2 · p. 53

L.1 Example 1

• Assistant A’s binary conversion includes the required “db” prefix and suffix, while Assistant B’s omits the final “b.”The feedback identifies A’s output as correct and B’s as missing a required character.Source: §L.1 · p. 54
• Assistant A is judged more accurate and more instruction-following because Assistant B ends with “d” instead of the required “db.”The final verdict selects Assistant A.Source: §L.1 · pp. 54–55
• Both assistants’ functions are described as correctly converting the decimal input to binary using bin(decimal)[2:].Source: §L.1 · p. 55
• Assistant B’s output-format error also makes its output incorrect for the provided examples and other test cases.Source: §L.1 · p. 55

L.2 Example 2

• Around 350 billion songs is the example’s rough estimate for 3,500 years, assuming about 100 million songs are produced annually.The estimate assumes annual production remained relatively constant and is qualified as highly uncertain, depending on recording technology, cultural practices, and the definition of a song.Source: §L.2 · p. 56
• The song-count comparison credits A with greater depth and contextual breadth, while judging both responses clear, uncertainty-aware, and harmless.Source: §L.2 · p. 57
• The judge selects Assistant A as more accurate, comprehensive, and well explained in the song-count example.It contrasts A’s step-by-step estimate with B’s focus on U.S. copyrighted songs and lack of a clear estimation method.Source: §L.2 · pp. 56–57

L.3 Example 3

• For the note-parsing example, the judge favors Assistant A because it maps the whole note to four beats, unlike B’s three.Source: §L.3 · p. 58
• Both implementations are described as lacking explicit handling for empty strings or invalid note representations.Source: §L.3 · p. 59