The Attribution-Compression Frontier in Retrieval-Augmented Generation

1 Introduction

• 0.86 precision against summaries versus 0.12 against source spans at nominal 0.25 (achieved ratio 0.08) illustrates the attribution gap for a RECOMP-style compressor on ASQA.The source-span score uses the paper’s recovery protocol; the automatic estimates share an NLI evaluator and lack independent calibration.Source: §1 · p. 1
• The study compares reranking, extractive selection, abstraction, token pruning, ECR, and an identity reference on 944 ASQA and 976 QASPER questions.The study reports achieved context fractions and paired comparisons on shared questions.Source: §1 · p. 1
• The re-attributability protocol recovers source snippets for units without declared provenance, enabling grounded scores alongside scores against compressed text.A unit counts as re-attributable when NLI predicts that a candidate source snippet entails it.Source: §1 · pp. 1–2
• Three verification modes distinguish consistency with summaries, declared-provenance availability, and claim support after source recovery.The study separates structural rejection of missing provenance from measured rejection after recovery.Source: §1 · p. 2

2 Related Work

• The study covers text compressors with declared source IDs in extractive and ECR implementations, while its LLMLingua-2 integration lacks token-to-source mapping; soft compressors are outside scope.Related provenance-oriented representations motivate source links but are not evaluated as compressors here.Source: §2 · p. 2
• Prior work reports citation-grounding degradation under compression; this study adds cross-family ASQA and QASPER comparisons and explicit source-mapping procedures, not priority for the divergence.Its NLI-based support scores do not test whether generators causally relied on cited evidence.Source: §2 · p. 2

3.1 Setup

• The pipeline uses a fixed generator and NLI evaluator, with compressors mapping sentence-segmented retrieved snippets to units that may declare source IDs or none.The generator receives numbered context and is prompted to cite units; primary settings share the same question IDs after identity-run failures are excluded.Source: §3.1 · pp. 2–3

3.2 Emitted vs. Grounded Citations

• Grounded scoring resolves citations to declared or recovered source spans, whereas emitted scoring checks the compressed unit texts the generator saw.For units without declared provenance, lexical retrieval selects up to 10 candidates and NLI retains snippets predicted to entail the unit; unrecovered citations receive no precision credit and cannot support recall.Source: §3.2 · p. 3
• The emitted–grounded gap is an operational diagnostic, but cannot distinguish missing source evidence from recovery or NLI errors.The same NLI model filters recovered spans and scores support, and failed recovery does not establish that source support is absent.Source: §3.2 · p. 3
• RECOMP’s nominal budgets realize achieved ratios of 0.05–0.08 on ASQA and about 0.015 on QASPER, so curves use achieved rather than nominal compression.Across its four nominal budgets, the QASPER achieved ratios collapse to approximately the same value.Source: §3.2 · p. 3

3.3 Three Verification Modes

• Summary-checked, declared-span, and recovered-source verification pool different evidence; declared-span checking treats claims without declared evidence as unsupported without an NLI call.Recovered-source checking resolves missing mappings using the grounded-scoring recovery protocol, deduplicates source IDs, and then verifies claims; its reported RECOMP and LLMLingua-2 cases use nominal 0.25 on both datasets.Source: §3.3 · pp. 3–4
• QASPER compressor attribution maps declared cited source IDs to full paragraph texts and scores set-F1 against annotator-marked gold evidence.Runs without declared provenance have no declared-evidence F1; recovered-span scores would measure a different attribution pathway.Source: §3.3 · p. 4

4.1 Experimental Setup

• The study compares six methods, including identity, in primary evaluations on the same successful questions: 944 ASQA questions and 976 QASPER questions from 271 papers.ASQA uses top-5 GTR-retrieved passages; QASPER uses each supplied paper’s abstract and full text. ECR emits the highest-relevance member sentence verbatim per near-duplicate cluster, citing all member sentence ids.Source: §4.1 · p. 4
• QASPER’s paragraph-level gold evidence is scored by mapping declared sentence IDs to paragraphs, while answer quality is measured by str-EM.For ASQA, answer quality is also measured by str-EM; identity uses document-level units on ASQA and paragraph-level units on QASPER.Source: §4.1 · p. 4

4.2 Statistical Analysis and Metric Audit

• Primary comparisons use the intersection of questions successful across identity and all compressed settings, with equal question weighting and undefined metrics left undefined.Verification means exclude zero-claim answers, and paired verification comparisons exclude questions when either member has zero claims.Source: §4.2 · pp. 4–5
• Paired bootstrap intervals and randomization tests use 10,000 replicates; Holm-adjusted support uses pH < 0.05, and non-significance does not establish equivalence.Inference is conditional on fixed generator outputs, recovery procedure, and evaluator; the expanded test family contains 148 contrasts and was not preregistered.Source: §4.2 · p. 5
• A code audit corrected the precision gate, reducing identity precision from 0.5622 to 0.5086 on ASQA and from 0.4992 to 0.4826 on QASPER.Scores were recomputed from saved answers with the same NLI checkpoint; the correction is not independent validation, and the 512-token NLI limit can truncate evidence.Source: §4.2 · p. 5

Emitted and grounded scores diverge

• 0.86 versus 0.12 precision against summaries and source spans, respectively, for ASQA RECOMP; QASPER scores are 0.55 versus 0.06.The ASQA setting is nominal budget 0.25 (achieved ratio 0.08); QASPER’s achieved ratio is 0.015. The paired gaps combine compression, bounded recovery, and NLI judgments.Source: Figure 1 · p. 6, Figure 2 · p. 6, §5 · p. 6

Extraction and reranking

• ASQA extraction grounded precision is 0.43, 0.49, 0.49, and 0.42 across achieved ratios 0.51, 0.26, 0.11, and 0.06, while str-EM falls from 0.34 to 0.15.At nominal 0.25, ASQA reranking raises precision from 0.51 to 0.69 but lowers str-EM from 0.34 to 0.25; QASPER’s corresponding precision difference is −0.089.Source: §5 · p. 6

Declared provenance and verification coverage

• For RECOMP, recovered-source unsupported rates are 0.88 on ASQA and 0.95 on QASPER, versus summary-checking rates of 0.17 and 0.50.The gaps persist when recovery is attempted, but remain conditional on bounded search and the shared NLI evaluator; declared-span failures for no-provenance methods are structural.Source: §5 · pp. 6–7

ECR’s setting-specific result

• ECR’s only adjusted-supported grounded-precision gain over extraction is on QASPER at nominal 0.50 (∆= 0.063); other tested gains lack adjusted support.Its tightest QASPER setting does not establish an advantage over identity, and its declared-evidence F1 is slightly lower than extraction at every budget.Source: §5 · p. 7

Re-attribution

• RECOMP re-attribution rates on ASQA were 0.41, 0.41, 0.42, and 0.39 across nominal budgets 0.5 to 0.05, with roughly three-fifths of units lacking a passing candidate.A failed candidate search does not establish that the source contains no supporting evidence.Source: §6 · p. 7
• LLMLingua-2’s ASQA re-attribution rate fell from 0.56 to 0.40, 0.25, and 0.20 as budgets tightened; the final adjacent decrease was unresolved (pH = 0.242).The first two adjacent decreases had pH = 0.0148.Source: §6 · p. 7

Verification cost

• Table 2’s input-token proxy combines generator and declared-span verifier input before NLI truncation, but excludes compressor inference, output, and recovery costs.For no-provenance compressors, zero declared-span input reflects policy rejection without an NLI call, not the cost of recovering and checking source evidence.Source: §6 · p. 7, Table 2 · p. 7

Evidence selection under pressure

• Extraction and ECR evidence-F1 increased from about 0.09 to 0.18 and 0.17 as budgets tightened; reranking means declined from 0.07 to 0.03, with the final decrease unresolved.The extraction and ECR adjacent increases were supported (pH = 0.0148); reranking’s final difference had pH = 1. These are evidence-F1, not evidence-precision results.Source: §6 · p. 7

Sensitivity to the re-attribution threshold

• Across recovery thresholds 0.3–0.8, RECOMP grounded precision ranged from 0.15 to 0.08 and LLMLingua-2 from 0.16 to 0.04, while recovery rates also fell.Emitted precision exceeded grounded precision at every tested threshold for both methods; the evaluator, top-10 lexical search, and single-snippet requirement remained fixed.Source: §6 · pp. 7–8

Second-evaluator sensitivity

• TRUE rescoring found RECOMP emitted/grounded precision of 0.78/0.13 on ASQA and 0.44/0.01 on QASPER with fixed source mappings.The paired gaps were 0.65 [0.55, 0.75] and 0.43 [0.33, 0.53], respectively.Source: §6 · p. 8

What laundering looks like

• The ASQA Virginia example receives full emitted precision because the answer matches a RECOMP summary, but that summary has no declared source link.A second cited summary repeats an assertion that a person is not mentioned; neither example alone establishes an independently verified hallucination.Source: §7 · p. 8

Implications for practice

• The authors recommend checking citations against source spans and reporting whether provenance is declared or recovered, alongside achieved compression and verification input costs.They also recommend explicit recovery evaluation and documentation of citable-unit granularity.Source: §7 · p. 8

What dominating the frontier would require

• ECR’s measured gains are limited, motivating independent calibration and replication before claims of general improvement.The proposed future direction is to merge redundant evidence while retaining validated source links and accounting for verification cost.Source: §7 · p. 8

Failure modes and excluded examples

• All 33 failures occurred in identity runs, so primary comparisons use the common subset conditional on identity success.The failures comprised four ASQA budget-check failures and 15 QASPER budget-check plus 14 prompt-overflow failures.Source: §7 · p. 8

8 Conclusion

• Emitted and grounded scores diverge, but their magnitudes depend on recovery and evaluation choices; the authors recommend reporting both alongside nominal and achieved compression.The TRUE subsample retains the gaps, while human calibration and broader replication remain necessary.Source: §8 · pp. 8–9

9 Limitations

• Primary scores use one NLI evaluator without human calibration, and the TRUE audit does not validate the primary scores or establish equivalence across evaluators.The audit retains DeBERTa span selection under fixed mappings, while TRUE recovery reuses one judge; input-window and decoder differences also limit numerical comparison.Source: §9 · p. 9
• All results use Qwen2.5-7B-Instruct with greedy decoding, and the two English datasets do not cover multilingual, open-web, or longer-context settings.Frontier shapes may differ across model families and scales.Source: §9 · p. 9
• Top-10 lexical recovery and the single-snippet entailment requirement can miss matching or distributed support, so failed recovery does not establish that sources lack support.Threshold sensitivity does not test evaluator validity, candidate-pool size, retrieval strategy, or multi-snippet recovery.Source: §9 · p. 9
• Declared-span verification rejects claims without declared evidence by construction, making its 1.00 unsupported rate for no-provenance runs a provenance diagnostic rather than a calibrated hallucination rate.Recovered-source verification is reported at nominal 0.25 for two no-provenance methods and still depends on bounded recovery and the evaluator.Source: §9 · p. 9
• Primary comparisons condition on identity success and omit its failed cases; sampling intervals also exclude systematic NLI error and variation across generation runs.The common subset preserves paired question comparisons across settings.Source: §9 · p. 9
• Shared nominal budgets do not establish matched achieved context sizes because abstractive compressors may emit far less context than requested.Achieved compression ratios are reported, but RECOMP’s realized context can be much smaller than its nominal budget.Source: §9 · p. 9

Reproducibility Statement

• Saved generation outputs, input hashes, and pinned evaluator revisions accompany per-question rescoring records.The records support reproduction of the reported analyses.Source: Reproducibility Statement · p. 9
• Fixed contrast plans and paired-analysis reports regenerate the numerical tables and frontier figures without further model calls.Source: Reproducibility Statement · p. 9
• The harness, configurations, protocol, and per-question records are slated for public release upon publication.Source: Reproducibility Statement · p. 9

• Tables 4–5 report question-weighted means on common cohorts of 944 ASQA and 976 QASPER questions, with metric-specific omissions identified.Source: A Results on Common Questions · p. 13
• No-provenance methods’ 1.00 declared-span rates are structural, and verification rates exclude zero-claim QASPER RECOMP answers at specified budgets.Other settings use the full cohort; undefined metrics remain undefined rather than being scored as zero.Source: A Results on Common Questions · p. 13
• RECOMP produces identical per-question contexts across nominal budgets 0.50 and 0.25 on ASQA and 0.50, 0.25, and 0.10 on QASPER.Separate generation passes remain distinct operating points, while exploratory slot-compressor runs are excluded.Source: A Results on Common Questions · p. 13

• The 148 planned contrasts are reported across Tables 6–10, with effects and intervals rounded to four decimals for EF1 and three for other metrics.Source: B Complete Paired Comparisons · p. 13, Table 6 · p. 15, Table 7 · p. 16
• Paired differences use pointwise 95% bootstrap intervals and Holm-adjusted randomization tests; QASPER resampling keeps papers together while retaining question-weighted estimates.The paired-test procedure and common-cohort estimands are described with the comparisons.Source: B Complete Paired Comparisons · p. 13
• Recovered-source verification reuses cached NLI scores and source IDs, while removing repeated source IDs before pooling evidence for claim verification.Thus its unsupported rate need not equal one minus grounded citation recall; zero-claim answers are excluded from both verification rates.Source: B Complete Paired Comparisons · p. 13
• Tables 4–5 cover all 21 primary settings on 944 common ASQA and 976 common QASPER questions.Source: Table 4 · p. 14, Table 5 · p. 14

• The audit rescores 100 questions per dataset using TRUE T5-XXL, full-context identity, and extraction, RECOMP, and LLMLingua-2 at nominal 0.25.The sample contains saved answers from 100 ASQA and 100 QASPER questions, including 84 QASPER papers.Source: C Second-Evaluator Sensitivity Audit · p. 19
• Fixed recovery retains declared provenance and DeBERTa-recovered source IDs, while TRUE recovery rechecks top-10 lexical candidates and scores with TRUE.Both recovery modes retain the corrected precision gate and neither removes bounded-search errors or provides human calibration.Source: C Second-Evaluator Sensitivity Audit · p. 19
• The cross-evaluator comparison does not isolate model family because the final input cap exceeds the original TRUE and DeBERTa limits.The cap was 4,096 tokens, compared with 2,048 for original TRUE and 512 for DeBERTa.Source: C Second-Evaluator Sensitivity Audit · p. 19
• All 16 planned emitted–grounded gaps remain positive after adjustment, while eight cross-evaluator precision differences remain unresolved.The audit covers precision means and contrasts; ECR and other budgets are outside its scope.Source: C Second-Evaluator Sensitivity Audit · p. 19, Table 11 · p. 20, Table 12 · p. 20
• After normalization, 122 of 6,015 scored pairs produced nonbinary strings, including 103 empty strings; assigning them zero does not establish reliable nonentailment.Source: C Second-Evaluator Sensitivity Audit · p. 19