Human-Anchored Factuality Evaluation with Strategic Annotation

Yu Wang; Craig Erickson; Kevin Small

Source-linked AI summary

Human-Anchored Factuality Evaluation with Strategic Annotation

Yu Wang, Craig Erickson, Kevin Small

arXiv:2609.00494v1Aug 31, 2026cs.CL

TL;DR

Factuality judges scale evaluation but can be systematically biased relative to human judgments, while human annotation is costly. The paper combines full-dataset judge predictions with selectively sampled human labels using FSA-guided ASI, improving effective sample size on AutoFA and RAGTruth. Its efficiency depends on stable residual structure and coverage of relevant failure modes.

Problem
Judge-only factuality estimates can be systematically misaligned with human judgments, while obtaining enough human labels for reliable estimates is expensive and slow.
Method
FSA derives residual-risk signals from judge outputs, evidence quality, task and input strata, and answer-rubric alignment to guide ASI sampling.
Results
FSA-guided ASI achieves average effective-sample-size gains of 40.3% on AutoFA and 27.1% on RAGTruth.
Takeaways & Limitations
FSA-guided sampling improves annotation efficiency over uniform and confidence-based alternatives while recovering the human-defined evaluation target.
Takeaways & Limitations
Efficiency depends on residual structure remaining stable, while rare complex failures may escape the four FSA failure families.

Abstract

from arXiv · show

LLM-based factuality judges provide scalable evaluation signals, but their metrics are often systematically biased relative to human judgments. We study human-anchored factuality evaluation under limited annotation budgets, where judge predictions on the full dataset are combined with human labels on a small selectively sampled subset to obtain statistically valid estimates. The efficiency of this approach depends critically on which examples receive human annotation: in factuality evaluation, judge-human misalignment is not driven solely by low confidence, but also by structured failure modes such as incomplete evidence, temporal mismatch, unverifiable claims, and rubric misalignment. To exploit this structure, we introduce a factuality-specific annotation policy design pipeline that uses failure-space analysis (FSA) to derive diverse predictive signals for modeling human-judge misalignment. On an internal reference-based factuality evaluation system (AutoFA) and RAGTruth, where judge-predicted estimates substantially underestimate human-annotated factual accuracy, our FSA-guided policy improves annotation efficiency over uniform sampling and uncertainty-driven baselines, achieving effective-sample-size gains of 40.3% on AutoFA and 27.1% on RAGTruth.

1 Introduction

Factuality evaluation needs timely, statistically reliable human-anchored measurements, but human annotation is costly and judge predictions can be systematically misaligned with human judgments. The paper combines judge predictions with selectively sampled human labels and uses failure-space analysis to improve this process.

Motivation: Human annotation is trusted for factual accuracy but requires substantial budget and introduces operational latency.This creates a practical tension between evaluation quality and efficiency.
Problem: Judge errors can be systematic when references are incomplete, stale, conflicting, indirectly supportive, or evaluated with a more literal rubric.These conditions can lead judges to reject answers that humans consider reasonable.
Approach: ASI combines full-dataset judge predictions with human corrections from a sampled subset to estimate human-defined factual accuracy with valid uncertainty quantification.The target is the human-defined factual accuracy rate under a limited annotation budget.
Contribution: FSA derives sampling signals from judge outputs, evidence quality, task and input strata, and answer-rubric alignment.The policy addresses structured misalignment rather than relying only on judge confidence.
Results: 40.3% and 27.1% are the average effective-sample-size gains of FSA-guided ASI on AutoFA and RAGTruth, respectively.The framework improves annotation efficiency over uniform and confidence-based sampling while preserving desired coverage.
Contributions: The paper frames factuality evaluation as a human-anchored, judge-assisted inference problem and demonstrates improved efficiency over uniform and confidence-based alternatives.The evaluation covers both an internal reference-based system and a public hallucination benchmark.

2 Background

The paper defines factual accuracy over query-response items and motivates judge-assisted estimation because human labels are expensive while judge-only estimates can be biased. ASI retains the human target while using judge predictions to reduce variance under limited budgets.

Problem Setup: Each evaluation item contains a query, model response, and reference evidence, with human labels represented by verified claims or detected hallucinated spans.The observable input is X_i = {q_i, a_i, e_i}.
Problem Setup: Factual Accuracy Rate (FAR) measures item-level factual correctness in the reference-based evaluation setting.The paper denotes the correctness function for item i by ϕ(X_i, Y_i).
Judge-Assisted Evaluation: Human annotation is expensive and slow, whereas automatic factual evaluation scales but generally produces biased FAR estimates.This motivates combining abundant judge predictions with limited human labels.
Judge-Assisted Evaluation: ASI selects a human-labeled subset using an annotation policy from a much larger set of judge predictions.Only n_bgt ≪ N items receive human annotation.
Metric Scope: FAR can measure faithfulness rate when the judge’s evidence is the grounding supplied to the answer generator.This is a setting-specific interpretation of the metric.
Estimators: The human-only estimator is unbiased but noisy under small budgets, while the judge-only estimator has low variance but is biased under systematic judge-human misalignment.These trade-offs motivate a hybrid estimator.
Statistical Efficiency: ASI remains human-anchored, but its variance depends on the sampling policy and the squared judge-human residual.Efficient policies therefore prioritize examples with larger expected residuals.

3 FSA-guided Policy Design

FSA designs ASI policies by predicting judge-human residual risk from information available before annotation. It converts structured signals about evidence, task difficulty, and rubric alignment into sampling probabilities.

Policy Objective: The oracle ASI policy assigns higher sampling probabilities to examples with larger conditional residual magnitudes.Because human labels are unavailable before annotation, practical policies must predict residual risk.
Failure-Space Analysis: FSA derives structured features from judge output, evidence quality, task and input strata, and answer-rubric alignment.These families expose disagreement risks beyond judge uncertainty alone.
Policy Construction: FSA uses historical labeled data to compute residual targets, extract pre-labeling features, train a residual-risk model, and convert scores into sampling probabilities.The resulting policy concentrates annotation on examples where judge-human disagreement is most likely.

4 Experiments

The experiments evaluate FSA-guided ASI on AutoFA and RAGTruth, examining residual structure, feature selection, human-anchored coverage, and annotation efficiency. FSA-guided policies preserve valid inference while achieving the strongest efficiency gains.

Evaluation Settings: The study evaluates two distinct factuality settings: internal AutoFA for voice-assistant responses and public RAGTruth with human span-level annotations.AutoFA uses retrieved reference answers and claim-level verification, while RAGTruth evaluates hallucinated spans across several generation tasks.
Judge Residuals: Judge-predicted FAR underestimates human FAR by 13.3 percentage points on AutoFA and by 30.6 percentage points on RAGTruth.On RAGTruth, judge FAR is 55.6% versus human FAR of 86.2%.
Judge Residuals: Judge-human disagreements are dominated by overrejection: evidence gaps and strict verification on AutoFA, and overly literal grounding of valid paraphrases or inferences on RAGTruth.On AutoFA, 90% of disagreements involve human-correct responses marked FALSE or UNKNOWN; on RAGTruth, 94% involve hallucinations flagged by the judge but not humans.
FSA of Judge Residuals: Residual risk concentrates in observable strata, including 2.58× lift for AutoFA partial verification, 1.29× for time-sensitive queries, 2.24× for RAGTruth low confidence, and 2.02× for judge-flagged outputs.Lift is the ratio of stratum-level to population-level residual density.
Policy Features: FSA features differ by dataset: ref-support-strength is selected for AutoFA, while self-confidence and flag-reliability are selected for RAGTruth.Feature selection uses calibration-data validation to balance residual-ranking quality and parsimony.
Debiasing and Efficiency: The judge-only estimator has zero coverage on both datasets, whereas all ASI variants maintain near-nominal coverage across annotation budgets.ASI combines judge predictions with human-labeled corrections to estimate the human-anchored FAR.
Debiasing and Efficiency: FSA ASI achieves the largest effective-sample-size gains, improving ESS by +40.3% on AutoFA and +27.1% on RAGTruth across budgets.Uniform ASI gains +13.3% and +11.3%, while Confidence ASI gains +17.0% and +22.6%, respectively.

5 Related Work

Prior work develops automatic factuality evaluation and statistically valid inference with limited human labels. This paper extends confidence-driven annotation by adding factuality-specific residual-risk signals beyond judge uncertainty.

Automatic Factuality Evaluation: Automatic factuality methods decompose responses into claims, atomic facts, or hallucinated spans and assess their support from evidence.These methods provide scalable evaluation signals for factuality and hallucination detection.
Human-Anchored Inference: Prediction-Powered Inference and Active Statistical Inference combine abundant machine predictions with limited human labels while preserving statistically valid inference.Active Statistical Inference samples examples using policies designed to reduce estimator variance.
Factuality-Specific Policy Design: Confidence-driven inference uses verbalized LLM confidence to guide annotation, but factuality residuals can persist when judges are confidently misaligned.The paper adds FSA features derived from judge artifacts, evidence quality, task strata, and rubric alignment.

6 Limitations

FSA-guided ASI is effective within the studied settings but depends on stable residual structure, incomplete failure coverage, simplified annotation assumptions, and separate operational budget decisions.

Distribution shift: Residual structure learned from historical or calibration data may degrade under model, retrieval, query, or failure-mode shifts.Silent high-confidence errors under distribution shift may be under-sampled, increasing inverse-probability weights and variance.
Failure coverage: The four FSA failure families do not exhaustively cover all error categories, including rare multi-hop reasoning failures.The rubric may need adaptation or extension for different application settings.
Annotation model: The formulation targets a final adjudicated label and does not model annotator-level noise, disagreement, or heterogeneous annotator costs and reliabilities.Joint budget allocation across heterogeneous annotators remains future work.
Budget selection: ASI improves efficiency for a given annotation budget but does not determine the budget required for a particular production decision.Budget choice depends on cost, latency, acceptable uncertainty, and release-risk tolerance.

A Policy Learning Details

The policy learns disagreement risk from pre-labeling features, converts scores into mixed sampling probabilities, and tunes judge strength to reduce variance.

Regression target: The binary target |∆| equals 1 for judge-human disagreement and 0 for agreement, with disagreement rates of 28.2% on AutoFA and 34.7% on RAGTruth.Figure 3 shows agreement as the majority class in both datasets.
Score model: An XGBoost regressor predicts residual risk from selected features, producing a score used to construct the sampling policy.The model is trained on the calibration split.
Score normalization and mixing: Uniform mixing interpolates between learned and uniform sampling, improving stability by limiting extreme inverse-probability weights.Larger τ improves stability but reduces policy concentration.
Mixing coefficient selection: The ESS-gain curves are concave: small τ overfits, whereas large τ underconcentrates the policy; selected values are τ = 0.40 for AutoFA and τ = 0.55 for RAGTruth.The search evaluates ESS gain on the calibration split at nbgt = 300.
Power tuning: λ∗≈0.2 consistently reduces AutoFA variance, while λ = 1 can exceed the labeled baseline at small budgets.Power-tuning shrinks the judge contribution to a level suited to the residual structure.

B Feature Selection Details

Feature selection balances residual-ranking quality against overfitting, and simpler FSA-derived configurations can yield better effective sampling policies than models with more features.

Selection procedure: Candidate features are selected using cross-validated Spearman correlation and parsimony because limited calibration samples can make larger feature sets overfit.The selection procedure uses 5-fold cross-validation on the calibration set.
Joint feature importance: Ref-support-strength dominates AutoFA predictions, whereas self-confidence and flag-reliability jointly dominate RAGTruth predictions.Other feature families contribute marginally in the all-feature SHAP analysis.
Post-hoc validation: On AutoFA, ref-support-strength alone reaches |ρ| = 0.50 versus |ρ| = 0.12 for self-confidence alone; on RAGTruth, adding flag-reliability raises |ρ| from 0.63 to 0.69.Adding features beyond the selected sets does not improve ESS in either dataset.
Validation metrics: Feature selection validation reports CV Spearman correlation and held-out ESS gain at nbgt = 300, with added features potentially reducing ESS through overfitting.The table marks the selected configuration in bold.
Policy performance: The all-features AutoFA model achieves CV ρ = 0.52 but only +24.7% ESS, while the single-feature model achieves +40.5% ESS.More aggressive uniform mixing stabilizes the all-features model but dilutes policy concentration.

C Human Annotation Details

The benchmarks use human annotation procedures with independent labeling and adjudication to establish factuality judgments for evaluation.

RAGTruth: RAGTruth uses two independent annotators per response, third review for substantial disagreements, and reports 91.8% response-level and 78.8% span-level consistency.Annotators were recruited through a professional vendor and paid $25 per hour.
AutoFA: AutoFA annotates all records, gives 90% a second independent annotation, and uses a third moderator when final response verdicts disagree.Annotators extract verifiable claims, search the web, and judge claim-level and overall factual correctness.

D Qualitative Analysis

The qualitative examples show that the judge can flag well-supported paraphrases and aggregated facts as hallucinations, including when confidence is high. FSA identifies these structured residual risks for targeted human annotation.

D Qualitative Analysis: The judge flags a well-supported paraphrase as hallucinated when the reference does not state its temporal relation verbatim.In record 168, “shot whilst running” is supported by separate reference facts but receives a 0.85 self-confidence score from the judge.
D Qualitative Analysis: The judge also flags an accurate weekly-hours summary after failing to aggregate identical daily schedule entries.Record 5718 summarizes Monday–Sunday hours as “seven days a week from 12:00 to 17:00,” but the judge marks it incorrect.
D Qualitative Analysis: For record 5718, low self-confidence and maximum FSA unreliability jointly illustrate different signals for identifying judge risk.The judge reports self-confidence 0.30, while FSA assigns flag-reliability=3.

E Feature Extraction Prompts

The appendix documents prompts that extract auxiliary confidence and verification-difficulty features for AutoFA and RAGTruth without determining final human labels. These prompts use structured integer scores and dataset-specific inputs.

E Feature Extraction Prompts: The appendix prompts extract auxiliary features or confidence scores rather than determining the final human factuality label.All prompts are Jinja2 templates whose variables are replaced with record-specific content at inference time.
E.1 Self-Confidence Prompt: The self-confidence prompt elicits confidence in the judge’s previous verdict for both datasets, with minor task-specific wording changes.The verdict concerns factuality verification or hallucination detection depending on the dataset.
E.2 AutoFA Feature Extraction Prompt: AutoFA’s feature-extraction prompt assesses verification difficulty using the response, query, and three reference answers without making the final decision.It extracts five FSA features using integer scores from 0 to 3.
E.3 RAGTruth Feature Extraction Prompt: RAGTruth’s prompt extracts general response–reference verification features and separately evaluates the plausibility of flagged spans.The prompt is framed around groundedness assessment and uses only the provided information.
E.1 Self-Confidence Prompt: The self-confidence prompt is explicitly conditioned on the judge’s prior factuality verdict, distinguishing confidence elicitation from independent feature extraction.Its template presents the earlier conclusion as factually correct or factually incorrect.
E.3 RAGTruth Feature Extraction Prompt: Across feature prompts, integer scores range from 0 to 3, with higher values indicating more of the corresponding feature.The prompts require valid JSON and prohibit chain-of-thought output.