PaperlayerView exact revision on arXiv

Source-linked AI summary

Prometheus: Inducing Fine-grained Evaluation Capability in Language Models

Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, Minjoon Seo

arXiv:2310.08491v2cs.CLcs.LG

TL;DR

Proprietary LLM evaluation raises transparency and reproducibility concerns. Prometheus is an open-source evaluator trained on customized feedback data; on 45 customized rubrics, its 0.897 Pearson correlation with human evaluators was similar to GPT-4’s 0.882.

  • Problem

    Proprietary LLM evaluators raise transparency and reproducibility concerns because their workings are undisclosed and versions may change beyond users’ control.

    Source: §1 · p. 2

  • Method

    Prometheus is a 13B evaluator fine-tuned on Feedback Collection, which pairs instructions and responses with customized rubrics and reference answers.

    Source: §1 · p. 3, §3 · p. 5

  • Results

    0.897 Pearson correlation with human evaluators on 45 customized rubrics was similar to GPT-4’s 0.882 and exceeded GPT-3.5-Turbo’s 0.392.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The conclusion is that appropriate reference material can induce evaluation capability in an open-source LM, with Prometheus specialized for fine-grained evaluation.

    Source: §7 · p. 15

    The human-evaluation setup excluded coding and math questions, leaving performance in those domains unestablished.

    Source: §6.2 · p. 13

Abstract

from arXiv · show

Recently, using a powerful proprietary Large Language Model (LLM) (e.g., GPT-4) as an evaluator for long-form responses has become the de facto standard. However, for practitioners with large-scale evaluation tasks and custom criteria in consideration (e.g., child-readability), using proprietary LLMs as an evaluator is unreliable due to the closed-source nature, uncontrolled versioning, and prohibitive costs. In this work, we propose Prometheus, a fully open-source LLM that is on par with GPT-4's evaluation capabilities when the appropriate reference materials (reference answer, score rubric) are accompanied. We first construct the Feedback Collection, a new dataset that consists of 1K fine-grained score rubrics, 20K instructions, and 100K responses and language feedback generated by GPT-4. Using the Feedback Collection, we train Prometheus, a 13B evaluator LLM that can assess any given long-form text based on customized score rubric provided by the user. Experimental results show that Prometheus scores a Pearson correlation of 0.897 with human evaluators when evaluating with 45 customized score rubrics, which is on par with GPT-4 (0.882), and greatly outperforms ChatGPT (0.392). Furthermore, measuring correlation with GPT-4 with 1222 customized score rubrics across four benchmarks (MT Bench, Vicuna Bench, Feedback Bench, Flask Eval) shows similar trends, bolstering Prometheus's capability as an evaluator LLM. Lastly, Prometheus achieves the highest accuracy on two human preference benchmarks (HHH Alignment & MT Bench Human Judgment) compared to open-sourced reward models explicitly trained on human preference datasets, highlighting its potential as an universal reward model. We open-source our code, dataset, and model at https://kaistai.github.io/prometheus/.

1 INTRODUCTION

Prometheus is an open-source 13B evaluator trained to assess responses using user-defined fine-grained rubrics and reference materials; the introduction reports human-evaluation correlation similar to GPT-4.

  • Human evaluation captures nuanced qualities that conventional metrics miss, while generic or coarse-grained criteria do not meet varied user needs and an open-source 70B model was insufficient for customized rubrics.The introduction motivates evaluation criteria such as playfulness, humor, and cultural sensitivity.Source: §1 · pp. 1, 3
  • Proprietary LLM evaluators raise concerns about closed-source transparency and uncontrolled versioning that can undermine reproducibility, while API costs can constrain large-scale evaluation.The paper gives an example in which evaluating four model variants at four sizes on 1,000 instances with GPT-4 could cost over $2,000.Source: §1 · p. 2
  • The Feedback Collection provides customized scoring rubrics and reference answers alongside instructions, responses, and feedback, and is used to train Prometheus for diverse real-world scoring criteria.The paper presents Prometheus as an open-source 13B model specialized in fine-grained evaluation beyond single-dimensional preferences.Source: §1 · p. 3
  • 0.897 Pearson correlation with human evaluators on 45 customized rubrics was similar to GPT-4 (0.882) and above GPT-3.5-Turbo (0.392).The rubrics were sampled across MT Bench, Vicuna Bench, and Feedback Bench.Source: §1 · p. 3
  • Human evaluators preferred Prometheus over GPT-4 in 58.67% of pairwise comparisons, and it achieved a 79.57% win rate over GPT-3.5-Turbo.Across 1,222 customized rubrics and four test sets, its correlation with GPT-4 evaluations was higher than GPT-3.5-Turbo and Llama-2-Chat 70B; on two unseen human-preference datasets, it outperformed two state-of-the-art reward models and GPT-3.5-Turbo.Source: §1 · p. 3

2 RELATED WORK

Prior text evaluation moved from reference-based lexical metrics toward semantic models and LLM evaluators, but preference-based evaluation remains subjective and lacks fine-grained customization. The paper positions rubric and reference-answer conditioning as important for that capability.

Reference-based text evaluation

LLM-based text evaluation

  • The authors argue that their evaluator handles thousands of customized preference criteria, unlike prior uni-dimensional preference evaluators, by training on feedback with score rubrics and reference answers.They describe this as, to their knowledge, the first evaluator sensitive to thousands of unique preference criteria.Source: §2 · p. 4
  • Prior LLM evaluators focused on one-dimensional preference, but preferences vary with applications, cultures, and objectives, while GPT-4 was costly, closed-source, and subject to uncontrolled versioning.Examples of preference dimensions include brevity, formality, honesty, creativity, and political tone.Source: §2 · p. 4

3 THE FEEDBACK COLLECTION DATASET

The FEEDBACK COLLECTION is designed to train open-source evaluators on fine-grained criteria by pairing evaluation examples with reference materials and explanatory feedback. GPT-4 was used to build the dataset, and Prometheus was fine-tuned to generate feedback before a score.

3 THE FEEDBACK COLLECTION DATASET

  • The dataset construction targets broad reference coverage, uniform reference-answer lengths by score, balanced score frequencies, and realistic user–LLM interactions.Each instruction is accompanied by a rubric and reference answer, and the dataset includes 20K instances for each score from 1 to 5.Source: §3 · pp. 4–5
  • 20K instructions and 100K response-and-feedback examples form a dataset whose inputs pair an instruction and response with a customized rubric and reference answer, and whose outputs contain feedback and a 1–5 score.The reference answer is intended to represent a score of 5, while feedback explains the assigned score.Source: Table 1 · p. 5, §3 · p. 5

3.1 DATASET CONSTRUCTION PROCESS

  • The authors expanded 50 curated seed rubrics to 1,000 with GPT-4, using rubric brainstorming and paraphrasing over 10 rounds.Four randomly sampled seed rubrics served as in-context demonstrations for generating new rubrics.Source: §3.1 · p. 5
  • GPT-4 generated 20K rubric-relevant instructions and, for each rubric, 20 instructions with five scored responses and feedback per instruction, balancing the 20K responses at each score.For score-5 examples, two distinct responses were generated so one could serve as the reference answer.Source: §3.1 · p. 6

3.2 FINE-TUNING AN EVALUATOR LM

  • Prometheus was fine-tuned to generate feedback followed by a score, with a “[RESULT]” phrase between them to prevent degeneration during inference.The procedure was applied to Llama-2-Chat models at 7B and 13B scales.Source: §3.2 · p. 6

4 EXPERIMENTAL SETTING: EVALUATING AN EVALUATOR LM

The experiments assess how closely evaluator models match human and GPT-4 judgments, using absolute grading and ranking grading. They test correlation and feedback quality across customized-rubric benchmarks, and preference accuracy on human-preference benchmarks.

4.1 LIST OF EXPERIMENTS AND METRICS

  • Absolute grading requires a 1–5 score and feedback from an instruction, response, and reference materials; ranking grading tests whether absolute-grading models can generalize to preference judgments.Absolute grading is more practical because it does not require an opponent response, but the evaluator must score from its own judgment.Source: §4.1 · pp. 7–8
  • The setup measures correlation with human and GPT-4 evaluations using Pearson, Kendall-Tau, and Spearman, and compares feedback quality through human pairwise judgments.Nine crowdsource workers were split across Prometheus–GPT-4, Prometheus–ChatGPT, and GPT-4–ChatGPT comparisons; they judged feedback quality and explained rejections.Source: §4.1 · p. 7
  • The correlation benchmarks include Feedback Bench’s seen and unseen rubrics, customized-rubric Vicuna and MT Bench sets, and FLASK Eval; Figure 4 reports Pearson correlation across 45 rubrics from the first three.Feedback Bench uses 1K shared rubrics in Seen Rubric and 50 new rubrics across 1K instructions in Unseen Rubric; Vicuna and MT Bench use 80 prompts each, while FLASK Eval uses 200 prompts and 12 relatively coarse rubrics.Source: §4.1 · pp. 7–8, Figure 4 · p. 9
  • Ranking evaluation uses preference accuracy on MT Bench Human Judgement and HHH Alignment, scoring candidates independently at temperature 1.0 until there is a winner, without reference answers.Because this procedure can iterate to force a winner, the authors caution that it is not exactly fair against other ranking models and is intended to test generalization, not claim state-of-the-art status.Source: §4.1 · p. 8

4.2 BASELINES

  • Baselines include Llama-2-Chat models, a 13B model trained on 12 coarse rubrics, GPT-3.5-Turbo, GPT-4 variants, StanfordNLP Reward Model2, and Almost Reward Model.The comparisons cover open-source evaluator models, proprietary evaluators, and reward models trained in ranking settings.Source: §4.2 · p. 8
  • Figure 5 reports Prometheus feedback win rates of 58.62% over GPT-4 and 79.57% over GPT-3.5-Turbo.Annotators compared which feedback better assessed the response.Source: Figure 5 · p. 9

5 EXPERIMENTAL RESULTS

Prometheus shows strong agreement with human and GPT-4 evaluations, and its fine-grained evaluation training also improves performance on human-preference ranking tasks. Human annotators prefer its feedback to GPT-4’s, though they more often criticize it for being overly critical.

5.1 CAN PROMETHEUS CLOSELY SIMULATE HUMAN EVALUATION?

  • Prometheus achieves 0.897 Pearson correlation with human annotators, compared with 0.882 for GPT-4 and 0.392 for GPT-3.5-Turbo across three evaluation datasets.The evaluation uses 45 instances, each with a unique customized score rubric.Source: §5.1 · p. 9
  • Human annotators prefer Prometheus’s feedback over GPT-4’s 58.62% of the time and over GPT-3.5-Turbo’s 79.57% of the time.Annotators compared feedback quality in addition to scoring decisions.Source: §5.1 · p. 9
  • GPT-4 feedback is mainly rejected as too general or abstract, while Prometheus feedback is mainly rejected as too critical; the authors conjecture this reflects its fine-tuning specifically for fine-grained evaluation.The figure summarizes annotators’ reasons for rejecting each evaluator’s feedback.Source: Figure 6 · p. 10, §5.1 · p. 10

5.2 CAN PROMETHEUS CLOSELY SIMULATE GPT-4 EVALUATION?

  • On seen and unseen rubrics, Prometheus 13B improves Pearson correlation over Llama2-Chat 13B by +0.420 and +0.397, respectively, and outperforms larger Llama2-Chat, GPT-3.5-Turbo, and GPT-4 versions.The comparison covers 1222 rubrics across 2360 instances; the authors suggest that training and test instructions share a similar distribution, even when rubrics are unseen.Source: §5.2 · pp. 10–11, Table 3 · p. 11
  • On Vicuna Bench, MT Bench, and Flask Eval, Prometheus improves Pearson correlation over Llama2-Chat 13B by +0.255, +0.493, and +0.202, respectively, but still trails GPT-4.On Flask Eval, the model trained specifically on that dataset with coarse rubrics outperforms Prometheus, suggesting direct task-specific training may help.Source: Table 4 · p. 11, §5.2 · p. 11
  • Simply scaling Llama2-Chat does not greatly improve evaluation, while coarse-rubric training only slightly improves performance; the authors highlight broad rubric coverage as important for customized evaluation.On Table 3’s seen and unseen rubric sets, Llama2-Chat 13B performs worse than the 7B model and improves slightly at 70B.Source: §5.2 · pp. 10–11

5.3 CAN PROMETHEUS FUNCTION AS A REWARD MODEL?

  • On HHH Alignment and MT Bench Human Judgment, Prometheus 13B improves over Llama2-Chat 13B by +5.43% and +5.38%, respectively.Coarse-rubric training hurts performance; the results show gains on ranking tasks without training directly on ranking evaluation instances.Source: Table 5 · p. 12, §5.3 · p. 12

6 DISCUSSIONS AND ANALYSIS

The discussion argues that reference materials let Prometheus focus on judging responses, while its practical use involves trade-offs in preparing references and training for specific tasks. It also describes the model’s feedback and gives guidance for customizing evaluators.

6.1 WHY IS IT IMPORTANT TO INCLUDE REFERENCE MATERIALS?

  • Ablations show that each reference component contributes, with removing the reference answer causing the largest degradation and removing the rubric especially harming Vicuna Bench performance.The authors argue that reference answers let the evaluator assess a response without first solving the instruction, while rubrics specify which aspects to judge.Source: §6.1 · pp. 12–13, Table 6 · p. 13

6.5 A PRACTITIONER’S GUIDE FOR TRAINING A NEW EVALUATION MODEL

  • Different starting models do not significantly harm evaluation performance, though the instruction-tuned and RLHF-trained Llama-2-Chat performs best, while Code-Llama may help on code tasks.In a code-domain example, Code-Llama training produced more detailed feedback and a reasonable score decision; domain-specialized evaluators remain future work.Source: §6.1 · p. 13, §6.5 · p. 15

6.2 NARROWING PERFORMANCE GAP TO GPT-4 EVALUATION

  • Prometheus reportedly surpasses GPT-4 on Feedback Bench and human evaluations, but the comparison is qualified by dataset similarity and the exclusion of coding and math questions.The authors also note possible human-rating error and conjecture that Prometheus tends to be more critical than GPT-4.Source: §6.2 · p. 13

6.3 QUALITATIVE EXAMPLES OF FEEDBACK GENERATED BY PROMETHEUS

  • Qualitative examples show Prometheus identifying specific errors and suggesting improvements, though annotators also found its feedback overly critical in one example.The examples include cases where GPT-4’s feedback was too general and cases judged a tie.Source: §6.3 · p. 14

6.4 A PRACTITIONER’S GUIDE FOR DIRECTLY USING PROMETHEUS EVALUATION

  • Prometheus’s evaluation workflow takes instructions, a chosen score rubric, reference answers, and target-model responses, then returns a score and feedback for customized analysis.Rubrics can be generic or fine-grained, and the model can serve as an alternative to sending outputs to OpenAI API calls during model development.Source: §6.4 · p. 14

6.5 A PRACTITIONER’S GUIDE FOR TRAINING A NEW EVALUATION MODEL

  • Directly training on a target feedback dataset yields higher GPT-4 correlation on that dataset than using Prometheus trained on the Feedback Collection, at the cost of preparing new feedback data.The authors recommend including reference materials and balancing reference-answer length and score distribution to reduce length and decision bias.Source: §6.5 · pp. 14–15

7 CONCLUSION

The paper shows that reference materials can induce fine-grained evaluation capability in an open-source language model, and presents Prometheus as a potential universal reward model. Prometheus achieves human-evaluator correlation on par with GPT-4, while its feedback was preferred over GPT-4 58.62% of the time.

  • Prometheus achieves Pearson correlation with human evaluators on par with GPT-4, and its feedback was preferred over GPT-4 58.62% of the time.Source: §7 · p. 15
  • Prometheus shows the highest correlation with GPT-4, outperforming GPT-3.5-Turbo.Source: §7 · p. 15
  • The Feedback Collection contains thousands of customized score rubrics and supports training Prometheus, an open-source evaluator model.Source: §7 · p. 15
  • Prometheus performs better on human preference datasets than competing models, suggesting potential as a universal reward model.Source: §7 · p. 15

A ANALYSIS OF THE FEEDBACK COLLECTION DATASET

The dataset analysis finds distinct score criteria and smoothly increasing sentiment across score descriptions. Responses have similar lengths across score ranges, while instructions, responses, and feedback show moderate diversity.

Are the Score Criteria Diverse Enough?

Are the Score Descriptions Well Formulated?

  • Score descriptions show a smoothly increasing sentiment tendency, supporting their role in determining scores rather than rewarding or penalizing minor details.The descriptions provide a reason for each score and are intended to increase sentiment gradually, not dramatically.Source: A ANALYSIS OF THE FEEDBACK COLLECTION… · pp. 19–20

Is there a length bias among the Responses?

Are the Instructions, Responses, and Feedback Diverse as Well?

B ANALYSIS OF THE FEEDBACK BENCH EVALUATION DATASET

The analysis finds that Feedback Bench contains score rubrics with low overlap against Feedback Collection, supporting its use as an unseen-rubric test set.

Does the testset maintain Unseen Score Rubrics?

  • Figure 10 shows low ROUGE-L overlap between randomly sampled Feedback Collection and Feedback Bench rubrics, supporting Feedback Bench’s validity as an unseen-rubric test set.The analysis concerns generalization to unseen customized score rubrics.Source: B ANALYSIS OF THE FEEDBACK BENCH… · p. 21

C FINE-TUNING AND INFERENCE DETAILS OF PROMETHEUS · D TRAINING A EVALUATOR LM ON COARSE-GRAINED SCORE RUBRICS

These sections describe Prometheus’s fine-tuning and inference choices and a coarse-rubric training baseline built from Flask data. They emphasize structured score generation and balancing the baseline’s skewed training scores.

C FINE-TUNING AND INFERENCE DETAILS OF PROMETHEUS

  • Prometheus was trained on 8xA100 (80GB) GPUs with PyTorch FSDP, using batch sizes 28 for 7B models and 20 for 13B models.The setup otherwise followed Meta AI’s official Llama2 fine-tuning code; its loss covers feedback and score decisions, not instructions, which the authors found performed poorly when evaluating responses.Source: C FINE-TUNING AND INFERENCE DETAILS OF… · p. 21
  • Naive Llama-2-Chat struggled to produce easily parsed final scores, while verbalizers mapped outputs such as “[SCORE 5]” to score decisions.The authors report that even after verbalization, Llama-2-Chat’s decisions remained difficult to parse; Prometheus was trained to generate a structured format directly.Source: C FINE-TUNING AND INFERENCE DETAILS OF… · pp. 21–22, Table 9 · p. 22
  • Because most instances contained at most 3072 tokens, demonstrations for in-context learning could not be used during inference.The authors also found that evaluator LMs generated meaningful feedback at temperature 1.0, unlike at 0.0.Source: C FINE-TUNING AND INFERENCE DETAILS OF… · p. 22
  • With adept prompting, GPT-3.5-Turbo and GPT-4 generated structured outputs, and inference used the hyperparameters specified in Table 9.Table 9 distinguishes verbalizer-accepted score formats from the exact “[Result] 5” format.Source: C FINE-TUNING AND INFERENCE DETAILS OF… · pp. 21–22, Table 9 · p. 22

D TRAINING A EVALUATOR LM ON COARSE-GRAINED SCORE RUBRICS

  • The coarse-rubric baseline used Flask because Feedback Collection instructions were closely tied to their rubrics and could not simply be reused with coarser rubrics.The authors split Flask into training data and evaluation data, naming the latter Flask Eval.Source: D TRAINING A EVALUATOR LM ON… · p. 22
  • Flask contains 1.7K instructions, 76.5K responses from 15 response LMs, and rubrics drawn from 12 options, which are coarse relative to Feedback Collection’s thousands.The rubrics include criteria such as factuality, harmlessness, readability, completeness, and conciseness.Source: D TRAINING A EVALUATOR LM ON… · p. 22
  • From 1.5K training instructions and 67.5K responses, the authors balanced score ranges 1 to 5, yielding 30K instances for the LLAMA-2-CHAT + COARSE baseline.The original Flask training scores were heavily skewed toward 5; the resulting model was used as a baseline in Tables 3–6.Source: D TRAINING A EVALUATOR LM ON… · p. 22

E PAIRWISE COMPARISON OF THE QUALITY OF THE FEEDBACK

The section examines pairwise feedback quality and finds that Prometheus’s less abstract feedback comes with a tendency toward critical judgments.

E PAIRWISE COMPARISON OF THE QUALITY OF THE FEEDBACK

  • Prometheus’s feedback was less abstract and general than GPT-3.5-Turbo’s, but tended to be overly critical; GPT-4 showed the same pattern relative to GPT-3.5-Turbo.Source: Figure 11 · p. 23, Figure 12 · p. 23
  • The comparisons indicate that GPT-4 was more critical than GPT-3.5-Turbo, and the gap was wider between GPT-3.5-Turbo and Prometheus.The authors suggest criticality could help analyze LLM limitations or provide improvement feedback, but could also reflect a general bias against optimistic judgments; they leave this exploration to future work.Source: E PAIRWISE COMPARISON OF THE QUALITY… · p. 23

F IS THERE A LENGTH BIAS DURING EVALUATION? … I QUALITATIVE EXAMPLES OF GENERATED FEEDBACK

The paper reports no observed tendency for PROMETHEUS to favor longer responses, while noting that its test instances lacked adversarial verbosity examples. It also documents the prompts used to construct the Feedback Collection and train PROMETHEUS, and points to qualitative feedback examples.

F IS THERE A LENGTH BIAS DURING EVALUATION?

  • PROMETHEUS and GPT-4 showed similar score–length distributions, with no observed preference for longer responses in the tested instances.The test instances from Feedback Bench and Vicuna Bench lacked adversarial examples for verbosity bias, so whether fine-tuned evaluator LMs inherit that bias remains open; absolute scoring also eliminates position bias in this setup.Source: F IS THERE A LENGTH BIAS DURING… · p. 25

G PROMPT FOR FEEDBACK COLLECTION CREATION

  • The Feedback Collection prompts matched generated-response sentence counts to the reference and grouped instruction/reference generation separately from response/feedback generation.The authors say this improved generation quality and reduced cost versus generating every component at once, which had produced similar responses and feedback across score ranges.Source: G PROMPT FOR FEEDBACK COLLECTION… · p. 25
  • The authors report using GPT-4 augmentation to obtain negative and neutral responses, specifically scores 1–4, and suggest exploring this strategy in other use cases.The prompt directs responses to match the reference answer's sentence count and bases their quality on the score rubric rather than length.Source: G PROMPT FOR FEEDBACK COLLECTION… · pp. 25, 28

H PROMPT USED FOR PROMETHEUS

  • The PROMETHEUS prompt asks for detailed rubric-based feedback followed by an integer score from 1 to 5, using the instruction, response, score-5 reference, and rubric.The authors also apply Llama-2’s basic conversation template after their prompt template to reduce discrepancy with Llama-2 training.Source: H PROMPT USED FOR PROMETHEUS · p. 29

I QUALITATIVE EXAMPLES OF GENERATED FEEDBACK

  • Figures 16–19 present qualitative examples of feedback from GPT-4, PROMETHEUS (13B), and Code-Llama trained on the Feedback Collection.The supplied passage identifies the example-generating models but does not report comparative findings from those examples.Source: I QUALITATIVE EXAMPLES OF GENERATED… · p. 30

J EXPERIMENTAL DETAILS FOR HUMAN EVALUATION

Human evaluation used a staged annotation interface to collect scores, feedback preferences, and annotators’ reasons, with randomized instances and feedback order.

J EXPERIMENTAL DETAILS FOR HUMAN EVALUATION

  • The interface collected response scores, which feedback was better, and reasons in sequence, instructing annotators to complete higher questions before lower ones.Figure 20 shows the annotation interface.Source: J EXPERIMENTAL DETAILS FOR HUMAN… · p. 30, Figure 20 · p. 37
  • Nine annotators each labeled 45 instances for at least 3 hours, received $50, and were split among PROMETHEUS–GPT-4, PROMETHEUS–GPT-3.5-Turbo, and GPT-4–GPT-3.5-Turbo comparisons.The annotation was conducted on LabelStudio hosted on Huggingface Spaces; feedback order was randomized to prevent order bias.Source: J EXPERIMENTAL DETAILS FOR HUMAN… · p. 30
  • Test instances were randomly selected from FEEDBACK BENCH’s Customized Unseen Rubric set, Vicuna Bench, and MT Bench, excluding math and coding instances.Source: J EXPERIMENTAL DETAILS FOR HUMAN… · p. 30
  • Figures 16–18 illustrate comparison outcomes: abstract GPT-4 feedback, overly critical PROMETHEUS feedback, and a tie when both evaluators gave good feedback.Source: Figure 16 · p. 31, Figure 17 · p. 32, Figure 18 · p. 33

Related papers

Loading 2310.08491v2…