Source-linked AI summary
ATTRICITE: Training an Open 4B Model for Citation Recovery toward Faithful Attribution
Yee Man Choi, Xuehang Guo, Songcheng Cai, Yimu Wang, Yi R. Fung, Qingyun Wang
TL;DR
Citation attribution depends on identifying intended sources, but inaccurate references and literature growth make that task challenging. The paper trains ATTRICITE for tool-using citation recovery and releases CITEALIGN. GRPO improves target-match accuracy over Qwen3-4B, while the evaluation treats recovery as a proxy for source identification rather than full claim–source verification.
Problem
Method
The paper trains an open 4B model with GRPO for tool-using citation recovery and releases CITEALIGN, a recent-literature dataset with temporally separated splits and a scalable collection pipeline.
Source: §1 · p. 2Results
10.37 percentage points: GRPO-trained ATTRICITE reaches 59.75%± 0.19% target-match accuracy versus 49.39% ± 1.54% for Qwen3-4B, and outperforms gpt-oss-20b.
Source: §5.1 · p. 7Takeaways & Limitations
ATTRICITE establishes a reproducible improvement in open-ended source identification, while claim–source support verification remains a complementary next stage.
Source: §8 · p. 10Target recovery measures agreement with the recorded citation as a proxy for intended attribution; broader evaluation can assess evidential support and alternative appropriate sources.
Source: §7 · p. 9
Abstract
from arXiv · showhide
Faithful citation attribution begins with identifying the intended source for a scientific claim. We study this source-identification capability through citation recovery: recovering the paper cited by the original author from a citation-bearing passage. Our evaluation adopts the published author's citation as an observable human attribution signal and uses target recovery as a proxy for progress toward faithful attribution. We introduce ATTRICITE, an open 4B-parameter model trained for tool-using citation recovery within the CiteGuard retrieval environment, together with CITEALIGN, a 7,607-instance computer-science dataset drawn from recent scientific literature. For controlled evaluation, we construct a 709-instance benchmark subset of CITEALIGN, comprising 410 development instances from 2024 publications and 299 temporally held-out test instances from 2025 publications. Across three runs at an inference temperature of 0.7, GRPO fine-tuning improves Qwen3-4B from 49.4%$\pm$1.5% to 59.8%$\pm$0.2% target-match accuracy, a gain of 10.4 percentage points. Despite using only 4B parameters, ATTRICITE outperforms gpt-oss-20b and comes within 3.9 points of GPT-5.4-mini, while Gemma 4 31B IT achieves the strongest overall performance at 72.0%$\pm$1.0%. We release the model and collection pipeline https://github.com/KathCYM/AttriCite to support reproducible research on citation recovery toward faithful attribution in a continually evolving scientific literature.
1 Introduction
Citations connect scientific claims to prior work and credit its researchers, but identifying intended sources is harder as literature grows and inaccurate or fabricated references become a concern. The paper introduces ATTRICITE and CITEALIGN for tool-using citation recovery and evaluation on recent literature.
1 Introduction
- LLM-generated fabricated or inaccurate references and nonexistent citations in scientific papers make intended-source identification increasingly important [12] [22].Citations also ground claims in prior work and credit researchers, while literature growth and increased manuscript production add pressure [3] [16] [14].Source: §1 · p. 1
- Citation recovery identifies the intended scholarly source, while claim–source verification assesses whether that source supports the claim.The paper uses agreement with a published author's citation as a proxy for the source-identification component of faithful attribution.Source: §1 · pp. 1–2
- 10.4 percentage points: GRPO fine-tuning improves Qwen3-4B target-match accuracy from 49.4% ± 1.5% to 59.8% ± 0.2%.ATTRICITE outperforms gpt-oss-20b and comes within 3.9 points of GPT-5.4-mini; Gemma 4 31B IT achieves 72.0%±1.0%.Source: §1 · p. 2
- ATTRICITE is an open 4B-parameter model trained with GRPO to search scholarly literature and select a source within a bounded interaction budget.The system recovers the paper cited by the original author from a passage with its citation masked.Source: §1 · pp. 1–2
- CITEALIGN is a recent-literature citation-recovery dataset with temporally separated splits, and its scalable collection pipeline supports extension to new years and venues.The splits evaluate recovery of recent recorded sources through search of an external Semantic Scholar index.Source: §1 · p. 2
2 Problem Formulation
The problem is to identify the paper cited by the original author from citation context, using target recovery as a proxy for source attribution. In the open-ended setting, the system searches an external scholarly index, and evaluation separates finding the target from selecting it.
2.1 Citation Recovery toward Faithful Attribution
- Citation recovery maps a citation context x_i to the paper p*_i cited in the source document, rather than generating a reference string.The context helps distinguish similar statements referring to different methods, results, or lines of work.Source: §2.1 · p. 3
- Target recovery uses the published citation as an observable attribution label and proxies source identification, not the broader assessment of evidential support or alternative valid citations [5].That broader pipeline also includes evidential-support assessment and consideration of alternative valid citations.Source: §2.1 · p. 3
- Open-ended recovery gives the system neither a candidate set nor the citing paper’s bibliography; it must search an external scholarly index, unlike verification of a proposed claim–source pair.Source: §2.1 · p. 3
2.2 Retrieval Interaction
- In the CiteGuard interaction, a policy chooses search, inspection, context-request, or selection actions from the passage and interaction history, subject to a budget [5].The trajectory ends when a paper is selected or the budget is exhausted.Source: §2.2 · p. 3
- Retrieval failure means the target never appears among retrieved candidates, whereas selection failure means it appears but the model chooses another paper.More search alone does not guarantee better scientific retrieval or more reliable citations.Source: §2.2 · p. 3
2.3 Evaluation
- End-to-end accuracy combines retrieval recall with conditional selection accuracy: Accuracy = Recallret × Accuracysel|ret.Retrieval recall measures whether the target appears in returned candidates; conditional selection accuracy measures recognition once retrieved.Source: §2.3 · p. 3
- Paper identities are matched first by persistent scholarly identifiers and, when needed, by a normalized-title fallback for metadata or publication-version variation.Source: §2.3 · p. 3
3 CITEALIGN: Dataset Construction
CITEALIGN addresses limited training data and random-split leakage risks with a dataset for open-ended citation recovery and temporally separated evaluation. Its collection pipeline, partitions, biomedical set, and quality checks support the reported experiments.
3 CITEALIGN: Dataset Construction
- CITEALIGN provides training data and temporally separated evaluation for open-ended citation recovery, addressing limits of existing benchmarks and random-split exposure risks.Source: §3 · p. 3
- The collection contains 7,607 instances, and the primary-experiment subset has 350 training, 60 validation, and 299 test instances with no source- or target-paper overlap.Source: Table 1 · p. 4, Table 2 · p. 4, §3.1 · p. 4
3.1 Collection Pipeline
- The pipeline extracts single-citation passages, masks the citation, resolves its bibliography entry to a Semantic Scholar record, and retains targets retrievable from passage-only queries.Source: §3.1 · p. 4
3.2 Biomedical Cross-Domain Set
- The separate biomedical test set contains 143 open-access 2025 articles in medical imaging and cancer genomics, with no biomedical examples used for training or model selection.Source: §3.2 · p. 4
3.3 Quality Control
- A separate 100-record holdout audit found 99 records passed all checks and excluded one extraction failure; no reviewed or automatically flagged case remained unresolved.Source: §3.3 · p. 5
4 ATTRICITE: Reinforcement Learning
ATTRICITE adapts Qwen3-4B for citation recovery through the CiteGuard retrieval interface and GRPO training. Evaluation selects a checkpoint on a small validation set and uses temporally separated test data, with controlled comparisons and explicit limits on cross-model interpretation.
4 ATTRICITE: Reinforcement Learning
- ATTRICITE uses the inherited CiteGuard interface and five-action budget to search, inspect, and select a retrieved paper resolved through Semantic Scholar.Source: §4 · p. 5
4.1 Reinforcement Learning
- GRPO fine-tunes all model parameters using eight sampled trajectories per passage and a binary reward for matching the recorded target.Source: §4.1 · p. 5
- Because reward evaluates only final attribution, it leaves search trajectories unspecified; step 475 is selected after 22 noisy validation-checkpoint evaluations, with claims based on untouched test data.Source: §4.1 · p. 5, Figure 2 · p. 6
4.2 Evaluation Setup
- The 299-item test set uses 2025 passages whose targets are absent from the eligible 2024 pool, but temporal separation cannot ensure pretrained models never encountered those papers or metadata.Source: §4.2 · p. 6
- Models are run three times at temperature 0.7, and the controlled base–ATTRICITE comparison uses paired Bayesian bootstrap; cross-family comparisons may reflect pretraining differences.Source: §4.2 · p. 6
5 Experimental Results and Analysis
ATTRICITE improves main-set target recovery over base Qwen3-4B and shows preliminary transfer to biomedical collections. Retrieval analysis associates most of the improvement with surfacing the target paper, with a smaller gain in selecting it once found.
5.1 Main Results
- 59.75% ± 0.19% accuracy for ATTRICITE versus 49.39% ± 1.54% for base Qwen3-4B; GPT-5.4-mini reaches 63.66%±1.84%, and Gemma 4 31B IT leads at 72.02% ± 0.97%.The paired Bayesian bootstrap estimates a 10.37-point gain, with a 95% credible interval of 6.36–14.49 points; Table 3 reports the main evaluation.Source: §5.1 · p. 7
5.2 Cross-Domain Generalization
- 54.08% ± 3.15% biomedical accuracy for ATTRICITE versus 42.89% ± 1.07% for base Qwen3-4B and 25.87% for the fixed First-8 baseline.The evaluation uses the Medicine filter and shared identifier-first evaluator; the authors characterize this as preliminary transfer evidence for the evaluated medical-imaging and cancer-genomics collections.Source: §5.2 · p. 8, Table 4 · p. 8
5.3 Retrieval and Final Selection
- The fixed-query top-1 baseline recovers 106 of 299 main-set targets, yielding 35.45% retrieval recall@1 and end-to-end target-match accuracy.It uses the first eight non-stopword tokens after removing the citation marker, then submits the first non-source-paper search result.Source: §5.3 · p. 8
- Fine-tuning raises pooled retrieval recall from 58.64% to 68.12% and conditional selection accuracy from 84.22% to 87.73%.The authors say the larger retrieval-recall change is consistent with the policy primarily improving its ability to surface the recorded target, alongside a smaller selection improvement.Source: §5.3 · p. 8
- ATTRICITE trails GPT-5.4-mini by 2.78 points in retrieval recall and 2.05 points in conditional selection, with a 3.90-point end-to-end difference.These comparisons are reported as remaining gaps in the retrieval and selection analysis.Source: §5.3 · p. 8
6 Related Work
Prior work studies citation recommendation over fixed corpora, citation verification, broader scientific-agent workflows, and tool-using search. This paper distinguishes open-search recovery of a single author-recorded citation from those settings.
Citation Recommendation and Attribution
- CiteME and CiteGuard evaluate recovery of an author-recorded citation from an open search space, with CiteGuard adding iterative search and candidate validation.Common citation recommendation approaches use fixed corpora or indexes, including manuscript or citation-context representations [2], citation-informed embeddings [6] [20], and retrieval-generation methods [9] [4].Source: §6 · pp. 8–9
Citation Verification
Scientific Literature Agents
- Broader scientific-agent benchmarks assess retrieval, coverage, organization, and support; CITEALIGN isolates recovery of one author-recorded citation for automatic target-match evaluation.The cited systems include scientific agents, STORM [24], and benchmarks for scientific retrieval and target-paper discovery.Source: §6 · p. 9
Learning to Use Search Tools
- This work uses GRPO with a terminal target-recovery reward, and its controlled base-model comparison does not separately compare against supervised trajectory learning.The policy searches, inspects results, and selects a scholarly record under a bounded action budget.Source: §6 · p. 9
7 Limitations
The paper’s conclusions are bounded by CITEALIGN’s construction and discoverability filter, its recorded-citation evaluation proxy, and the limits of its training comparison. The authors also caution that attribution errors carry risks and that users should inspect evidence before publication.
Dataset Scope and Construction
- CITEALIGN covers five computer-science venues across two years, while its biomedical evaluation is an initial test in two areas; automatic construction can introduce parsing, resolution, or citation-span errors.The discoverability filter further limits the benchmark population to targets reachable through Semantic Scholar.Source: §7 · p. 9
Evaluation Proxy
- Target recovery measures agreement with the published author’s recorded citation, not evidential support or alternative appropriate sources.The authors identify evidential-support judgments and alternative valid sources as possible components of broader evaluation.Source: §7 · p. 9
Resource and responsible use
- Incorrect attribution can deny credit or lend a claim unwarranted authority, so users should inspect retrieved evidence before publication.ATTRICITE is intended to assist source discovery; further qualifications concern pretraining exposure, the live retrieval backend, the reward, and computational resources.Source: §7 · p. 9
Learning objective
- The controlled comparison estimates the effect of the full GRPO procedure, not GRPO’s necessity or superiority; no matched supervised trajectory-learning baseline is included.Such a comparison is needed to distinguish outcome-based reinforcement learning from task-specific trajectory training more generally.Source: §7 · p. 9
8 Conclusion
The paper introduces ATTRICITE and CITEALIGN, reporting improved target recovery after GRPO fine-tuning and preliminary biomedical transfer evidence. It frames source identification as progress toward faithful attribution, while leaving claim–source verification as a complementary stage.
8 Conclusion
- GRPO fine-tuning improves Qwen3-4B by 10.4 target-match points, with a paired Bayesian 95% credible interval of 6.4–14.5 points.ATTRICITE outperforms gpt-oss-20b and comes within 3.9 points of GPT-5.4-mini; retrieval analysis associates most of the gain with surfacing the target paper.Source: §8 · pp. 9–10
- The biomedical evaluation provides preliminary evidence of transfer to medical imaging and cancer genomics.The authors describe ATTRICITE’s improvement as a reproducible gain in open-ended source identification.Source: §8 · p. 10
- Claim–source support verification remains a complementary next stage beyond source identification.Source: §8 · p. 10
A.1 CITEALIGN Construction Details
CITEALIGN construction collects proceedings papers, extracts citation-bearing passages, resolves their cited works, and retains instances discoverable through passage-only search. The resulting records are split by publication year and preserve audit information for reproducibility.
A.1 CITEALIGN Construction Details
- The pipeline collects proceedings metadata, extracts body sentences with one unambiguous citation, and replaces each citation marker with [CITATION].It supports author–year and numeric citations and records collection state and extraction failures.Source: §A.1 · p. 12
- Reference resolution scores candidate papers by title-token overlap, string similarity, and year agreement, retaining the top result at a score of at least 0.55.Searches are limited to computer-science papers published no later than the citing paper.Source: §A.1 · p. 12
- Instances are retained only when a passage-only query retrieves the resolved target within Semantic Scholar’s top 20 results.Three query variants are tested, and the successful query and rank are retained for auditing but hidden from evaluated models.Source: §A.1 · p. 12
- The test set uses 2025 targets absent from eligible 2024 instances, with overlapping passages or titles removed from the 2024 pool.The 2024 pool is sampled by venue, with 70 training and 12 validation instances per venue.Source: §A.1 · p. 12
- Each retained instance preserves its passage, citation and resolved-target metadata, plus resolution and discoverability queries and scores.Incremental records allow interrupted runs to resume and new venues or years to use the same downstream stages.Source: §A.1 · p. 13
A.2 Full-Collection Audit and Release Accounting
The release audit adjudicated automatically flagged records, correcting recoverable cases and rejecting unusable contexts. The final metadata contains 7,750 records, with the audit accounting explicitly limited to flagged cases.
A.2 Full-Collection Audit and Release Accounting
- The audit flagged extraction failures, citation–bibliography mismatches, self-resolutions, entity errors, malformed contexts, duplicates, and questionable claim–source relationships.Reviewers used source PDFs, bibliographies, and cross-index records, storing corrections as explicit adjudications.Source: §A.2 · p. 13
- 7,750 records remain in the final metadata: 7,607 computer-science and 143 biomedical records.After the separate held-out audit excluded one additional extraction failure, the release checks found no unresolved flagged cases, duplicate identifiers, or missing required fingerprints and source-PDF URLs.Source: §A.2 · p. 13
- The audit accounting covers automatically flagged cases; it does not claim manual inspection of every unflagged record.Recoverable records receive traceable corrections, while unrecoverable contexts are rejected.Source: §A.2 · p. 13
A.3 Development Manual Verification and Adjudication
The appendix details collection audits, the retrieval interface, training and evaluation procedures, and exploratory reward comparisons. It also reports qualitative cases showing distinct retrieval gains, selection gains, and residual version errors.
A.3 Development Manual Verification and Adjudication
- After pipeline revisions, 99 retained audit records passed quality control; one bibliography-entry context was excluded, yielding a 1/100 audit-sample exclusion rate.Before adjudication, ten audited examples had incorrect canonical entities; review findings informed pipeline checks and regression tests.Source: §A.3 · pp. 13–14
A.4 Finalized-Pipeline Holdout Audit
- In the finalized-pipeline holdout, 99 records passed review and one incomplete passage was excluded, for a 1/100 failure rate.The 100-record sample excluded experimental and development-audit records and followed the collection distribution rather than balancing venues.Source: §A.4 · p. 14
A.5 Biomedical Cross-Domain Set Construction
- The biomedical set contains 143 evaluation instances from 59 source papers and 141 target papers, after removing one duplicate.Candidates came from open-access 2025 medical-imaging and cancer-genomics articles and passed the same passage-only discoverability requirement as the computer-science set.Source: §A.5 · p. 14
A.7 Training Configuration
- Training optimizes all Qwen3-4B parameters with GRPO, using the same five-action budget as evaluation.The reported configuration uses veRL with FSDP and asynchronous vLLM rollouts.Source: Table 6 · p. 15, §A.7 · p. 15
A.6 Agent Interface and Prompts
- The agent receives a masked passage and searches through a bounded tool interface, terminating on paper selection or after five actions.Its prompt specifies JSON actions and search constraints; the environment restricts reading and snippet lookup to the latest search buffer.Source: §A.6 · pp. 14–15
A.8 Exploratory Target-Retrieval Shaping Comparison
- 7.80 points (95% interval 3.49–12.16) favored outcome over shaped reward at 350 instances; at 1,000 instances, the 0.67-point difference provided no evidence of superiority.Cross-scale contrasts vary more than dataset size, and the comparison tests only one shaping coefficient.Source: Table 7 · p. 16, §A.8 · pp. 16–17
A.9 Models, Services, and Licenses; A.10 Evaluation Details
- Provider terms govern hosted models, while CC BY 4.0 covers only created metadata and annotations, not source-paper text.Mean end-to-end latency was 4.87 seconds for base Qwen3-4B and 2.75 seconds for ATTRICITE, including generation, agent execution, and live search requests.Source: §A.9 · p. 17, §A.10 · p. 18
A.11 Qualitative Retrieval and Selection Examples
- Examples distinguish retrieval gains from candidate discrimination: ATTRICITE retrieves a missed target and selects the correct AdaFuse, but both models confuse FlashAttention versions.The AdaFuse target was retrieved by both systems; the FlashAttention target was also retrieved, but both selected the earlier version.Source: §A.11 · p. 18